Agile Innovation
August 18, 2015
Psst, this is your conscious, I am here to remind you about something you have thought about, but then hid away in the back of your mind. Lots of this agile stuff is hypocritical, it preaches evolution and change, but then we ask the same old three questions at standup every day. Also, why must we have standup every day, isn’t that kind of prescriptive? Agile methods are supposed to facilitate innovation through iterative development followed by inspection and adaption. They practice the scientific method of measurement and feedback on products and team work; so why are the agile practices themselves magically exempt from this precious evolution?
I believe there are two main reasons; first off, it is to protect inexperienced agile practitioners from themselves. With a free rein to morph product and process there is a strong likelihood that by six months into a project the practices followed by the team would have deviated from the proven and tested methods of most successful teams. The risk of failure would increase and every project in a company would be using a radically different approach making integration, scaling and team member transfers a major problem.
The other reason is a little more sinister. Most of the creators, proponents and promotors of agile methods have interests in keeping the methods pure vanilla. This is so they can create training courses, certifications and web sites for them. While scrum, as one example, has its specialized ceremony names and products you can neatly market services for it. If you allow or encourage people to change it then the result is not so proprietary and more difficult to defend, promote and assert ownership over.
I am not suggesting we should be changing agile methods willy-nilly, I think a basic suggestion to try them out-of-the-box for a couple of years is sound advice. However, beyond that I believe there are great opportunities for growth and deviation outside the standard agile models for stable teams who want to evolve further. This article tells the story of one team that did just that and what other people can learn from it.
First off, in the spirit of full disclosure, I should explain that I was involved with the creation of the agile method DSDM and have worked with agile groups like the Agile Alliance (I was a board member for a couple of years) and helped create several agile certifications (iCAgile, DSDM Leadership, PMI-ACP). So, I am not immune from the pulls of standardization, but hopefully that adds some credibility to my encouragement to rise up and evolve your methods. The biggest threat to agile I see is not dilution and confusion, it is obsolescence and abandonment because they are not keeping up with new demand of collaborative teams.
Agile teams often talk about “Shu, Ha, Ri” progression that describes a three-step process of increasing mastery that progresses as follows:
1. Shu: Obeying the rules (shu means to keep, protect, or maintain)
2. Ha: Consciously moving away from the rules (ha means to detach or break free)
3. Ri: Unconsciously finding an individual path (ri means to go beyond or transcend)
The Shu-Ha-Ri model is used to describe the appropriate approach to process tailoring. Put simply, people should know and practice the “plain-vanilla” process works before removing things or inventing new flavors. However that’s where the commonly available advice seems to end, there is a definite lack of Ha and Ri recommendations for teams practicing for a couple of years who want to improve.
When creating DSDM in the early 1990’s we were influenced by the Participatory Design approach pioneered by Enid Mumford. Mumford reported, after years of trials and studies, that getting the customer involved in design improves buy-in and engagement. This results in higher quality deliverables that are more suited for purpose and have higher satisfaction levels. This is old news today, but useful back then so we engaged the customer in design workshops and got great results.
Continuing this idea we then engaged the customer in architecture and technical design workshops. However, rather than bringing more benefits it brought a bunch of conflict and problems. Customers leapt to implementation ideas, disregarded IS strategy, and did not know enough about the architectural issues to be helpful in the discussions. We got frustrated, they got frustrated, and nobody seemed better off.
So we discussed it with other DSDM Consortium members and agreed business involvement should not extend to architecture. The DSDM framework was updated and we carried on with our experiments and evolution of the method. For me this was a transformational moment, it was my first time of witnessing a process failure and the associated adaptation of process within DSDM and how we learn and adapt. We just changed the methodology; there was no sacred cow, just good old scientific experimentation.
Business involvement in GUI design: Good
Business involvement in architecture design: Bad.
Therefore, involve them in GUI design, but not in architecture.
There were plenty of other learning's after this as the methods grew from collective lessons learned, and now I have no qualms adapting process based on circumstance and feedback. This of course is not unique to DSDM, all methods evolved this way. When I took Scrum master training with Ken Schwaber in 2004, Sprints were 30 days long, no discussion; now two weeks is more common. Good on the Scrum folks for seeing the light and adapting.
Agile methods have come a long way since 1994 (for a start we did not call them “agile” until 2001) and with their widespread use has come some widespread misuse and failure. I found this a little sad until I discovered Sarah Sheard’s “Universal Technology Adoption Lifecycle” and realised it is inevitable.
Figure 1 - Sarah Sheard’s “Universal Technology Adoption Lifecycle”
All ideas get carefully replicated at first, then copied, then misused, unfairly blamed, criticized and rediscovered, and we should get used to it. DSDM was enterprise friendly and scalable from day one, it was criticized by other agile method practitioners as too large and heavy weight. These people are now adopting SAFe, Less and DAD to scale agile in the enterprise. It is all good, it is just the cycle of technology adoption.
The Evolution of an Agile Team
As a case study illustration, I recently worked with a high performing team that remained stable for 8 years. The company recognized them as a highly productive group and had the foresight to keep the team intact and bring new projects to this stable team rather than disband it and go through the Tuckman stages of Forming, Storming and Norming all over again.
The team of 9 people was co-located originally and then two people moved away but continued to work remotely. The remote members used predominantly instant messaging to keep in touch. The team had prolonged and heated debates; sometimes I thought I should intervene, but when you listened to the topics they were progressing (slowly) and led to strong buy-in for the decisions generated.
When they suggested they drop the frequency of stand-up meetings I was cautious, but not alarmed. We tried having them three times a week and then just twice a week for a couple of weeks. Soon they had dropped stand-ups all together but their output and quality remained high. People were still communicating all the time and it was obvious stand-ups had become redundant.
A weekly “Show and Tell” meeting was introduced for everyone to demo what they have been working on and discuss the next set of features queued for development. After the Show and Tell that included the whole team the developers stayed for a “Tech Talk” session that was part code review, part refactoring discussion.
We switched from two week iterations to a continuous pull of features from the backlog. We retained a weekly demo cadence and monthly release cadence, but dropped the two week planning, estimating and review cadence. T-shirt sizing and throughput metrics replaced our story estimation in points and velocity tracking. Retrospective were stopped, I questioned the team about this one and the response I received was “Why would we wait until a planned meeting to discuss and implement an improvement?” Instead “Sense Pull” is the practice of pulling improvements into the process just in time, like features are pulled in a lean process.
Throughout these changes, productivity and engagement remained high. The business was very happy with the quality of the work delivered and continued to fund additional projects. These adaptations to process came from the team and suited their work practices and environment. I would not recommend they are copied piece-meal because 1) I simplified the descriptions to keep them short and 2) they were the product of one individual team’s reflection and adaptation. Different teams will most likely have different evolutionary needs.
Moving Forward
“If you want to make enemies, try to change something.” - Woodrow Wilson
Lots of people are very passionate about protecting agile methods. So now we see large camps of agile purists and ScrumBut bashers wanting to preserve the faithful reproduction of proven practices. They are in disagreement with agile pragmatists who are happier to substitute and adapt. It is a tricky subject, and as always, there is some truth in the extremes and more sense in the middle of the arguments.
As Ron Jefferies says “Agile ain’t just any damn thing” and I think most people would agree with this. There has to be some common framework, values, and principles present otherwise the boundaries blur indefinitely and the name becomes valueless. On July 1 2015 Jeff Sutherland tweeted "Scrum is the Minimum Viable Bureaucracy. Keep the framework whole or it will fail."
Kent Beck warned us that XP practices are a balanced, self-supporting network and removing some elements will cause the others to fail.
For instance, “Small Releases” support “Continuous Integration” remove one and the other will suffer. Again, great advice, people really must know how the plain vanilla process works before removing things or inventing new flavors.
Interestingly, Kanban takes a different stance; in David Anderson’s Kanban book he states:
”Kanban gives permission to create a tailored process optimized to a specific context... You have permission to try Kanban, You have permission to modify your process. You have permission to be different. Your situation is unique and you deserve to develop a unique process definition tailored and optimized to your domain, your value stream, the risks that you manage, the skills of your team, and the demands of your customers.”
“The heresy of one age becomes the orthodoxy of the next.” - Helen Keller
As agile becomes the default approach for IT projects we can be sure that many projects will fail using it. This is simply because many projects are ill conceived, doomed from the beginning, poorly executed, funded, or supported. Before we rush to defend the method we should remember that a fool with a tool is still just a fool and no method is likely to change that. Also, many projects are high risk and a certain percentage of these high risk endeavors will fail.
“Failure is not fatal, but failure to change might be.” - John Wooden
So the question becomes how do we allow good changes to occur, like Scrum’s move from 30 day Sprints to two weeks, and poor choices like collaborative architecture, or random acts of laziness to fail? I think the answer is pretty simple. Try the change, evaluate it, validate it, adopt successful practices noting circumstances where they work, and stop doing failing practices in your circumstances.
“Nothing is complete and thus nothing is exempt from criticism.” - James Luther Adams
The one thing we must not do is think that we are done with the development of agile methods and now all we have to do is execute them. That is when learning stops and methods stagnate, only to be replaced by the next set of forward thinkers. I had a good discussion with Jonathon Kohl a few years back about agile methods evolving to embody new practice vs remaining a static model to be replaced by the next better thing (Link Here). Whether under the “agile” banner or some other name, we both agreed that progress and change are essential.
“If you don't like change, you're going to like irrelevance even less. “ - General Eric Shinseki
I think it is worth reiterating that we have to keep on learning and improving our skills, since standing still is the recipe for becoming obsolete. Nobody wants a mummified manifesto or petrified principles that are no longer relevant
“Talent ... is most likely to be found among non-conformists, dissenters, and rebels.” - David Ogilvy
The future belongs to people solving the pressing problems, challenging the status que and mixing things up. In a community who practices experimentation and evolution towards a better state for solutions I am continuously amazed at the reluctance to truly take this to heart. Yes, when you venture off your scrum roadmap you will have a risk of going the wrong way, but also opportunities to find great benefits.
If done correctly we can capitalize on successes and step back from failed experiments. Using the techniques we employ for product adaptation we can drive the next generation of tools and techniques. And remember: “Change is inevitable, except perhaps from vending machines.”
(note: this post first appeared at ProjectManagement.com here)
Project Management Institute has just released a 2015 analysis report confirming Agile growth. Is this something we should expect as a traditional approach in the future for the Oil & Gas industry? Do you know any example of one of the "Majors" actually using it?
Below the link to the PMI media release.
http://www.pmi.org/~/media/PDF/learning/taking-pulse-infographic-2015.ashx
Posted by: Lucas Vergara | January 31, 2016 at 10:30 AM
Hi Lucas, The title of that report is a little misleading since it was based on data captured in October 2014 so it is getting old now. However, to your question, yes, O&G majors are definitely using it. I have personally worked with ConocoPhillips, Encana, Cernovus, Husky and Nexen who all use it in pockets at least.
Posted by: Mike Griffiths | January 31, 2016 at 11:15 AM