Previous month:
March 2019
Next month:
September 2019

Innovation: Running Experiments and Learning

Experiment DesignIn my last article on Incubating Innovation, we explored the culture and mindset of accountable experimentation. This article focuses on actionable tools and approaches.

Within agile frameworks, the team retrospective is the primary workshop for planning and evaluating experiments. Yet most team retrospectives I see are broken.

Teams spend too much time recording viewpoints and information—but not enough time reviewing or planning experiments. It is common to see the majority of the time spent gathering what went well, what did not go well, and appreciations. Yet where’s the focus on experiments, the learning process and trials for the next iteration?

To make things worse, some teams do not take the retrospective seriously. Maybe after the potential stress of the sprint review, the largely internal retrospective is a relief. A chance to chill out, maybe share some food, and pat each other on the back. However, innovation and learning take conscious effort, forward planning and accountability.

As I work with organizations, I often sit in on retrospectives. Of all the regular workshops/ceremonies, these sessions are typically the least prepared for and worst executed. I often see lazy retrospectives where a basic lessons-learned format is used, but timings are not managed and the recommendations for the next sprint get skimped as they run out of time.

The pie chart below shows a typical planned allocation of time—and the reality of how time is actually spent:

R1

In these lazy retrospectives, people are slow to start, spend longer on recording what went well than what could be improved, and then try to cram the recommendations for experiments (the most important part) into the last few minutes. As a result, experimentation suffers. Few experiments are scheduled for the next sprint, and those that are run are not evaluated properly.

This is not how agile retrospectives are supposed to operate. An excellent guide to running effective retrospectives is Agile Retrospectives: Making Good Teams Great by Esther Derby and Diana Larsen. In it they describe a five-step process:

  1. Set the stage – Help people focus on the task at hand; check that people are ready and willing to contribute. Outline and gain consensus on the process we will use. Techniques we can use include: check-in, working agreements, focus-on/focus-off (see the book for full descriptions of these techniques).
  2. Gather data – Create a shared view of what happened during the sprint/iteration. When completed, we should have a common understanding of the observations and facts. Team activities we can use include: timeline, mad-sad-glad, team radar.
  3. Generate insights – This focuses on understanding the implications of our findings and discussions. We need to see the impacts of the problems we are faced with before trying to solve them. Techniques we can use include: five whys, fishbone analysis, dot voting.
  4. Decide what to do – Now we move from thinking about the iteration that just ended to what we should try next to improve things. We identify the highest-priority action items, create plans for experiments and set measurable goals to achieve the desired results. Techniques we can use include: SMART goals, circle of questions, short subjects.
  5. Close the retrospective – Here we reflect on the retrospective process and express our appreciations. We may summarize what we have decided to keep or change and what we are thankful for. Team-based activities we can use include: plus/delta, return of time invested (ROTI), appreciations.

This is a more useful format. However, despite people having access to good retrospective advice, poor implementations are still common. Teams continue to attend late, start slowly and run out of time or rush the agreement on what experiments to run.

R2

The recurring theme is poor experimentation design and restricted learning. Gunther Verheyen summarized the problem nicely in his recent post entitled “Inspection Without Adaptation Is Pointless.” Gathering data and deciding what to do is pointless if it is not acted upon. We are doing most of the preparation work but not getting any of the benefits.

Experiment Design to the Rescue
Fortunately, there are some good models we can use. We need to manage time and effort more effectively and use retrospectives to plan and evaluate more experiments. We should spend only 50% of the available time on gathering information and the remainder reviewing the results of past experiments, making wins part of our process, designing new experiments and learning from inevitable failures.

We can help the time management problem by assigning work to be done in advance. People should be thinking about issues and potential solutions independently. There are benefits of group discussion and consensus gathering on agreed experiment design, but observation and idea generation is best done individually.

The New Yorker magazine [3] describes numerous studies that show how brainstorming groups think of fewer, lower-quality ideas than the same number of people who work alone and later pool their ideas. There have been numerous reports on the downsides of brainstorming ideas as a group. Groupthink and the halo effect stifle idea generation. So, ask for people to come with ideas, then use the group setting to vet and vote for them.

Visualizing the ideas and experiments is an effective way to bring collective attention to them. Trent Hone and Andrew Jarding developed the “Ideas and Experiments board” pictured below. It shows the progression of ideas through experiments and their success or abandonment:

Experiment Board

Ideas and Experiments Board (Image Credit: Trent Hone and Andrew Jarding, MindSettlers)

As discussed in the last article, by design, 50% of our experiments should fail since we are trying to maximize our learning, not validate things we already know. So I would expect to see an equal number of abandoned experiments as successful ones.

However, this format (or a slightly modified version that represents an experiments Kanban board) is a useful tool to bring the focus for retrospectives to the experiments being run and considered. With some pre-work on idea generation and an increased focus on experiments, we can structure more effective retrospectives.

R3

This retrospective format saves some time by assigning idea generation as pre-work; this also helps avoid the groupthink pitfalls. It furthermore places emphasis on the experiments—the inputs for learning and innovation.

I have experienced pushback from teams about the goal of 50% experiment failure. People understand it optimizes learning—but say it sets people up for too much failure. I understand the sentiment but counter with two perspectives.

First, these are experiments; they should be dispassionate explorations, not evaluations of the people undertaking the work. We need to be professional and try to overcome habits of internalizing results. I know this is easier said than done, so also offer a second reason: We need to kill bad ideas early to save time and money for better ones.

In the article “The Hard Truth About Innovative Cultures,” Gary Pisano describes how killing bad ideas is critical. He profiles Flagship Pioneering, a Massachusetts-based R&D company. It uses a disciplined exploration approach to run small experiments minimizing expenditure. Instead of running experiments to validate ideas, it designs “killer experiments” to maximize the probability of exposing an idea’s flaws. The goal is to learn what went wrong early and move in a more promising direction.

Other useful ideas from the paper include:

  • Tolerance for failure, but no tolerance for incompetence – Hire the best people you can. Explain the goals clearly and let go of those that do not perform.
  • Psychologically safe, but brutally candid – Encourage frank but respectful two-way dialog. It may feel uncomfortable, but it can prevent issues or concerns from going unreported.
  • Collaboration—but with individual accountability – Encourage discussions, but avoid groupthink and hold people accountable for decisions and outcomes.

These are all great concepts and align with the frustrations I experience when I see teams not taking retrospectives seriously—or following through on conducting experiments. I realized I needed a better model for discussing the problem. Fortunately, I found the field of collaborative problem solving (CPS).

CPS is the study of how we work together in groups to solve new problems, innovate and build products. The innovation process and retrospective workshop fall squarely within its scope. CPS skills are quite separate from individual task-focused skills, meaning people can be great at working individually but poor at working together.

A good introduction to CPS frameworks can be found in the article “Advancing the Science of Collaborative Problem Solving.” One model they feature is the “PISA 2015 Collaborative Problem-Solving Assessment.” Unfortunately, like many academic models, the degree of difficulty goes downward, which may make sense as you read down through more advanced stages. However, I think graphically, so I have redrawn the model to show degrees of completeness and difficulty radiating up and out from a 0,0 origin, as shown below:

PISA 1

Along the X-axis, we see three categories of collaborative problem-solving competencies. These are:

  1. Establishing a shared understanding
  2. Taking action to solve the problem
  3. Establishing and maintaining team organization

Up the Y-axis, we have four categories of problem-solving:

  1. Understanding the problem
  2. Representing the problem
  3. Planning and executing
  4. Monitoring and reflecting

Within the body of the model, each square is labeled with the column number and row letter, and describes the tasks that occur in that space.

The model provides a diagnostic tool for identifying broken and lazy retrospectives. The poor engagement and weak follow-through I see in many Scrum teams is characterized by an incomplete execution of column 1 and only half-completion of columns 2 and 3 (as shown by the red outline below):

PISA 2

Teams are not spending time in “(D1) – Monitor and repairing the shared understanding,” nor are they getting to the “(C2) Enacting plans,” (D2), (D3) and (C3) areas to follow through on plans and hold each other accountable for actions and results.

What we want is a complete execution of all the collaborative problem-solving competencies; only then is the framework complete (along with the feedback mechanisms to keep things in check and moving in the right direction):

PISA 3

Summary
Innovation involves combining the right mindset and philosophy with tools and practical steps to ensure its execution. Motivation and attitude are paramount; people have got to want to do this work, enjoy it and create a pull demand for the tools and process that enable it. Trying to foster innovation with demotivated teams is like trying to push a rope.

When motivated and happy people create a strong pull demand for innovation, we need to be ready with the right tools to support the process and keep the momentum going. This includes designing experiments to maximize learning and killing bad ideas quickly—all while demanding competence, accountability and candor.

It is not easy to master the combination of soft skills and techniques required for successful improvement and innovation. However, organizations that succeed can respond to market changes faster and are poised to exploit new technologies and opportunities. Ideas and inventions are spreading quicker than ever. Learning how to build collaborative, innovative teams has become a critical skill.

References

  1. Book: Agile Retrospectives: Making Good Teams Great by Esther Derby and Diana Larsen
  2. Article and video: “Inspection Without Adaptation Is Pointless” by Gunther Verheyen
  3. Article: “Groupthink: the Brainstorming Myth” by Jonah Lehrer
  4. Article: “The Hard Truth About Innovative Cultures” by Gary Pisano
  5. Article: “Advancing the Science of Collaborative Problem Solving” by Arthur Graesser, et al.

[Note: I first wrote this article for projectmanagement.com here]


Let’s Rewrite the PMBOK

Future PMBOK
Phew, the wait is over! I have been wanting to talk about this for what seems like ages and now the official announcement is out! If you have ever been frustrated by the PMBOK Guide now here’s your chance to fix it.

We are looking for volunteers to write and review the next edition of the PMBOK Guide. However, this will not be just an update, instead a radical departure from all previous editions aligned with PMI’s new digital transformation strategy. That’s all I can explain for now, but more details will be announced when I can say more.

Meanwhile, we would like people with knowledge of the full value delivery spectrum (waterfall, hybrid, agile, lean, etc.) to participate.

The full details of volunteer opportunities and entry requirements can be viewed at the PMI VRMS site Here.

I will be acting as Co-Lead for the initiative, which is like a co-chair role. However, Chair and Co-Chair sounds too hierarchical so we switched to Lead and Co-Lead role to match the new structures we will be embracing.

If we want to change the future of project management I believe the best way to do that is from the inside outwards by doing the work - not from the outside inwards just criticizing. Longtime readers may recall my 2010 post Raise A Little Hell when the PMBOK v5 Update was being commissioned. Since then we developed the PMI-ACP, PMBOK Agile Appendices, and the Agile Practice Guide.

This is going to be different!

Click here to see full volunteer role details.


Incubating Innovation

InnovationIf success goes to those who can innovate the fastest, how do we nurture innovation? The basics are simple to understand—but difficult to implement and stick with in the face of adversity. We need to create an environment that encourages experimentation while also tolerating, investigating and learning from the inevitable failures.

It may sound easy, but executives and shareholders demand results, not “learning opportunities.” We need an approach that fosters experimentation and learning in a defendable way, with a bias for results. To innovate faster than our competitors, we need to maximize our learning potential. This means that by design, 50% of our experiments should fail since we are seeking knowledge expansion, not validation of things we already know. The trick is keeping people engaged and motivated when half of their experiment time is spent failing.

It starts with leadership and cascades down to a shared mindset of team members. Leaders need to be transparent about their own mistakes and learning moments. They need to model the desired behavior, share what they have learned and their new plan of action.

These can be strategic learnings (“Our European market testing has been poor, we are reworking the price options”) or personal (“Feedback on my presentation to investors indicated I was too technical; I need to find simpler ways to describe our technology”). Until team members see transparency in common use, they will be reluctant to practice it themselves for fear of reprisal or criticism.

We Are in the Maze-Solving Business

Maze 1
Developing new products or services is a maze-solving exercise. Nowadays, it is also a race. We need to find a workable solution faster than our competitors. There will be obstacles and dead-ends along the way, and that is okay. We must not let them demoralize us; we just need to learn from them, not repeat them, and keep going.

The process of learning starts with understanding the knowable and then adding to this through experiments and new learnings. So, we start with smart people who understand their industry domain, technology and the business goals. We then need to create an environment with a dual track of product development and experimental learning.

Organizations that run more experiments and iterate faster also learn faster. Scientists studying inheritance use mayflies because they reproduce and provide experimentation results so quickly. The more experiments you can run in a year, the more you can learn.

Our Mazes are Really Big

Maze 2

Developing a new product does not happen overnight. Outside of movies, rarely do lone geniuses develop a marketable product themselves. Instead, it takes teams of subject matter experts months to create proof-of-concepts and multiple iterations of tweaks to complete a viable product. These teams need support and coordination services throughout the process.

Sponsors, executives and shareholders need plans, projections and updates. Product managers, project managers and team leaders all play an important role in keeping the maze-solving teams motivated and moving in the right direction. They also need to keep funding and support going while providing inputs about changing market demands and conditions.

It sounds a difficult balancing act, but approaches such as design thinking, lean startup and agile provide stewardship models for development with inbuilt experimentation, observation and learning. What gets less attention is motivating teams to persevere despite the many failures encountered when experimenting for learning, not just validation.

The “Success Leads to Happiness” Fallacy
Most people start their lives with the mistaken view that success leads to happiness. Our internal dialog creates a series of “if/then” scenarios:

  • IF I pass this exam, THEN I will be happy.”
  • “IF I get this job, THEN I will be happy.”
  • “IF we finish this project on schedule, THEN I will be happy.”

However, the brain has a knack of moving the goalposts. We might be happy briefly, but then we quickly focus on the next exam, an even better job or a more ambitious project.

While it is good to progress in life, we should not connect achieving a goal with achieving happiness. Instead, we need to understand that happiness is only 10% extrinsic (external things that happen to us, like success) and 90% intrinsic (how we think and feel about things).

A Happy Brain is a Productive Brain

Dopamine

Happy workers are more productive and creative than stressed or unhappy workers. In our brains, the chemical dopamine is the neurotransmitter responsible for sending reward-motivated happiness signals. Put more simply, dopamine is a happiness chemical—it gets released when we are happy. Interestingly, dopamine also switches on more learning circuits in our brains.

Research [1,2,3] shows that happiness improves work performance. Happy people are 31% more productive, happy doctors are 19% more accurate at diagnosing correctly, and happy salespeople are 37% better at sales.

This intuitively makes sense. When we are unhappy or stressed, the brain prioritizes circuits for survival. If you spot a sabre-toothed tiger, it is probably best to focus on escape rather than contemplating the interplay of sunlight and shadows through the leaf canopy. Yet when searching for an innovative solution, we want all these extra brain circuits engaged. This brings us full circle on the “I’ll be happy when I am successful” logic. It turns out, being happy actually activates more of the brain to help us be successful.

Success Fallacy

Nurturing Happy Teams
So, if the smartest workers are happy workers, how do we make them happy? Well, we don’t “make them happy” at all—that would be trying to force it in externally, using weak extrinsic motivators. Instead, we equip them with the tools to help build intrinsic happiness themselves.

This might be sounding more touchy-feely than you are comfortable with. However, hard economics show that happy workers also persevere with problems longer, take less sick days, quit less and sue for wrongful dismissal much less, too…so let’s suspend the cynicism for a moment.

The good news is that with as little as 30 minutes a day, measurable increases in dopamine levels are achieved in three weeks. So, if we can encourage these behaviors and turn them into habits, we get happier, healthier, smarter and more productive workers.

These 30-minute exercises don’t even require expensive equipment or management consultants. They are simple, backed by research and include:

  1. 3 Gratitudes – a daily recording of three new things you are grateful for [4]
  2. Journaling – recording positive experiences from the past 24 hours [5]
  3. Exercise – increases blood flow to the brain and helps eliminate toxins [6]
  4. Meditation – resets multi-tasking fatigue and helps with concentration [7]
  5. Acts of Kindness - helping others and saying “thanks,” which makes us feel better [8]

Organizations spend vast sums of money hiring smart people and providing them with complex tools and training. In comparison, the cost/benefit potential in investing and encouraging team happiness is extremely compelling. As project managers, team leads and executives, we need to be conscious of the behaviors we model, because people are watching us.

If not already doing so, we can use these techniques ourselves to improve our own happiness and productivity—then share and encourage others to make use of them. A great benefit of intrinsic motivators is that they can be applied anywhere. It does not matter if you work in a toxic environment or report to a jerk. There is likely nothing stopping you making sure the first email of the day you send is to thank someone for their help or contribution. Also, no one will know if you make notes about positive experiences.

People with budgets and hiring authority do know that yoga instructors are much cheaper than labor relations consultants and HR lawyers. Longer lunch breaks and fun team activities may require some explanations, but improved problem solving, fewer sick days and better ideas are definitely worth it.

Don’t wait until project completion to celebrate team achievements. That’s too little, too late—and our brains have moved the goalposts to be thinking of the next project. Instead, celebrate the little things that recognize effort, persistence and displaying a good attitude.

Obviously, it is not as simple as “happy people are the perfect innovators.” There are some concerns that people characterized as “happy” might be less likely to spot certain types of risks. Optimism needs to be tempered with realism. However, given the variable success rates of helping people become happier (“leading horses to water,” etc.), there will still be some pessimists; but on the whole, it creates a positive change that is worth the risk.

Summary
Smart companies know the future belongs to the best innovators. Building and maintaining teams of productive innovators requires investment in tools, techniques and people. We need to have the right tools and be using today’s techniques such as the design thinking, lean startup and agile approaches. Then it comes down to our people. An appreciation of what truly makes us happy—and its effects on success—is a great starting point.

Most organizations are not R&D labs, so we need to balance innovation with everyday production and service. The mindset and changes described here may feel uncomfortable (or even unprofessional) at first. However, a quote from Eric Shinseki explains that “If you dislike change, you're going to dislike irrelevance even more.” It reminds us that the business landscape is changing faster than ever—and that we need to change with it to stay valuable.

References

  1. The Happiness Advantage
  2. The Benefits of Frequent Positive Affect: Does Happiness Lead to Success?
  3. Happiness and Productivity
  4. Counting Blessings Versus Burdens
  5. How Do I Love Thee? Let Me Count the Words, The Social Effects of Expressive Writing
  6. Behavior Matters
  7. One Second Ahead
  8. Interventions to Boost Happiness and Buttress Resilience

[Note: I first wrote this article for ProjectManagement.com here]


Review of Product Development Books

Product Development CycleNow that a software “Done” Milestone is more like a Tombstone

If you work in an industry that has digital products and services then the Product Development trend will impact you. As software becomes more critical to business operations and product offerings we are seeing that software projects do not end.

Many organizations are transitioning to become software focussed organizations that offer specialized services. Amazon is a software company with retail (and cloud) offerings. Banks are increasingly digital companies with financial services. The same with insurance, travel, music and even commercial goods. The cost of developing the software in new vehicles is now greater than the cost of the engine. It has become the single most expensive component, even in internal combustion engine vehicles with no autonomous driving features.

These websites and software services will only be “done” development when the company stops being competitive, offering new services or keeping up with technology evolution. At one time getting to "Done" on your software project was a relief, a goal, a milestone, now it is more of a tombstone. It means the product is no longer competing or actively being maintained as technology continues to evolve.

Switching from projects (that are temporary in nature) to products that are designed to be ongoing sounds easy enough - just keep funding the team, but for many organizations it is not that simple. Also, organizations that embrace the whole digital product view still need help governing the ongoing process.

This is where product development books can help. They describe the factors at play and provide ideas for guidance around planning, funding, staffing and governance. As I was working with clients experiencing the transition from projects to products I was lucky to engage with several authors of product-first, #NoProjects books and chat about the challenge areas and potential solutions.

I have written about #NoProjects a few times before:

 

Then when I recently read a third book about product development shift I thought it could be useful to review some of the books in the market. A neat aspect of these books is that they all present different views on the project to product mind-shift and journey.  

Continuous Digital

 “Continuous Digital: An agile alternative to projects for digital business

Allan Kelly, Software Strategy Ltd.; October 2018

Allan’s book was the first I read about switching to continuous product development. I had been following his blog for a number of years and was familiar with his work. However, it was not until reading the book that I saw his points laid out in order with full explanations.

It is a great read, it makes a compelling argument for why a project view of software projects is a flawed model. I loved the explanation on the diseconomies-of-scale for software and why it is actually cheaper in small batches – unlike physical goods.

It offers the best explanations I have heard into why a continuous delivery of features by a stable team is preferable to using conventional project models. It nicely describes the team aspects of knowledge work and offers some good suggestions around funding and governance models.  

Any change in mindset has to happen internally first before we can help others adopt it. Continuous Digital cemented my own thoughts about why good software projects never end. It explained the “Why?” questions at the heart of any shift in thinking and behavior. In the same way, we have to understand the agile mindset before generating any kind of commitment towards it. This book helps set the mindset and Why of #NoProjects so we can start our journey.

 

Noprojects book#noprojects: A Culture of Continuous Value

Evan Leybourn, Shane Hastie, lulu.com; July 2018

I read an early draft of Evan and Shane’s #noprojects about 6 months after I started reading Allan Kelly’s LeanPub drafts of Continuous Digital. While Kelly’s Continuous Digital and his spin-off book Project Myopia focus on explaining the Why with some How topics, #noprojects has more of a team focussed view.

It talks about the history of software development. It explains how we came to run software development with project structures and the inherent issues that came with them. It then outlines the case for continuous development with all the arguments for retaining knowledge, reducing handoff and dependences, etc.

I think it is a great follow-on from Continuous Digital. Obviously, it is designed to be a stand-alone text but, for me, Kelly explains the Why of product development better. Then #noprojects fills in some additional background information and focusses more on the team level implementation. It might just be because I read Kelly’s book first but if you are looking to convince others in your organization about the need for transitioning to product development it is the go-to source. Reading both will provide a great foundation to understand and transition from projects to products.

 

Project to ProductProject to Product: How to Survive and Thrive in the Age of Digital Disruption with the Flow Framework

Mik Kersten,  IT Revolution Press; November 2018

I first came across Mik’s work via a podcast he did with Shane Hastie. In the podcast, Mik explained that technologies often follow 50-year cycles and then transform through a tipping point into the next evolution in the way of working.

Organizations that try to continue operating with the old model flounder and fail. Moreover, it is now about 50 years since NATO held the first conference on software engineering and the age of software began. Mik explained he believes we are at that tipping point and transitioning to product delivery will be part of the differentiator for the next wave of successful organizations.

I have an interest in technology evolution stories and learning how ideas spread and then transform our lives. So, the 50-year cycle piqued my interest and I ordered the book. The book delivers and not only does it explain the tipping point we are living through right now, but it gives the best explanation of the digital revolution and need for digital transformation I have read to date. Kudos to Mik. If you want to explain why a digital transformation is necessary, and the implications of ignoring it, Project to Product is a brilliant source.

After this great introduction, the remainder of the book explains the Flow Framework Mik helped develop and promotes through his company Tasktop Technologies. The Flow Framework provides metrics and tools for tracking and managing product development. This is useful because while the project management world has a wealth of information about tracking and managing projects, organizations that switch to product management often experience a void or competing recommendations.

The Flow Framework is useful for explaining what we should pay most attention to tracking. Namely features, defects, risks and debts. It recommends a business outcome set of measures that include: value, cost, quality and happiness. The framework employs a lean inspired set of metrics that include flow velocity, flow efficiency, flow time, and flow load.

It is in the rebranding of tradition lean metrics that I struggle to recommend the book wholeheartedly. By prefixing the normal throughput based lean metrics with the word “Flow” Mik is able to define  specific versions of the terms that are often implemented slightly differently from organization to organization. I can see the advantage of that, but it seems an unnecessary name-grab or overloading of lean terms to create trademark-able terms.

That’s a minor quibble though and what Flow Framework does provide is a good mental model for organizing, executing and tracking your product development process. It nicely extends the pattern of books started with Continuous Digital that explains Why product development. Then #noprojects that provides some team-based and stewardship elements. Finally, Project to Product provides solid How To ideas for ongoing governance and improvement.

 

Summary

These three books form a useful progression for anyone wanting to learn about the product development trend. They each provide valuable ideas to help with understanding, practicing and then managing successful product delivery.

Product Development Books Progression

The books allow readers to understand and internalize the need for a product development mindset. Then how to practice on a small scale before encouraging others to try and providing ways to measure and manage it.

Obviously, you do not need to read all three of them, or in this sequence. Any one of them is a good read and source for practical ideas. However, I found that they build nicely upon each other and thought people would be interested to consider them as complementary.