I am excited to be presenting on the Enterprise Agile track at the Agile 2018 conference in San Diego, August 7. I have worked with several organizations this year that had issues with work dependencies between teams. My session called “Two-Pizza Team Heartburn Relief: Solutions to Team Dependencies” highlights the shift to global rather than local optimization.

We will investigate dependency problems through animations and simulations and then explore some candidate solutions. Each brings their own issues – if these problems were solvable they would have been already, but the suggestions do help considerably. Here is the description from the conference program:

Small teams are great - until they cause bigger problems than they solve. Small teams can communicate more effectively than large teams. They can leverage face-to-face communications more readily and share tacit knowledge without the need for so much written communication. However, for large endeavours, using many small teams present their own problems. Work dependencies between teams can cause major delays through costly hand-offs, mismatched priorities, and blocked tasks.

This workshop introduces strategies for structuring teams to reduce hand-offs and dependencies that create blocked work and delays. By investigating the (lack of) flow through multiple teams we can diagnose the cost of hand-offs and culprits of delays. We examine tools for making hand-offs and dependencies visible to highlight and bring collective attention to the problems. We then explore resolution patterns and work structures that maximize small team communications but limit negative aspects of managing multiple, inter-dependent project teams.

Learning Objectives

• Understand the time and cost penalties of team dependencies and hand-offs
• Gain tools for making dependencies, queues, and blocked work visible
• Learn how and when to balance small team benefits with more dependency issues
• Share implementation patterns and strategies to maximize team throughput
• Examine the pros and cons of larger teams, feature teams, and product vs. project development.

That probably sounds more technical than it really is. It is a workshop to show people how teams often get stuck with work items when they rely on work from other groups. It combines anecdotes and experiences from 20+ years of agile consulting along with some insights from Troy Magennis on dependency delays, and Dominica DeGrandis, author of Making Work Visible.

Through case studies and exercises, we explore the hidden impacts of well-intentioned small teams. First, we’ll explore the “mostly harmless” two and three team dependencies, and then see the impacts when five or six dependant teams try to get work done. Please come along if you are attending the conference and have issues with dependencies between teams.

Introduction

We know old concepts that govern agriculture do not apply to industry. Engineers do not consult the weather or growing seasons before designing machinery. Yet many project managers who work in the knowledge worker domain still apply project management approaches developed for the industrial era. This mismatch of approaches wastes effort and misses important new risks.

This article identifies the mismatch of applying industrial project management in today’s post-industrial marketplace. We first examine how to determine if your projects are: industrial, knowledge work, or hybrid. Then classify project management tools and techniques. Fortunately, for every industrial focused approach, there are modern knowledge worker equivalents. Using this information, we can apply the right tools for the job or at least identify the risks of mismatched projects and techniques.

How We Got Here

Work, like people, has evolved. Humans started out as nomadic hunter-gathers following the seasons and game. Then, when they discovered farming, they settled and built permanent home sites. This change was christened the Agricultural Revolution and heralded a huge shift in how people lived and worked.

Next came the Industrial Revolution. Farmers and craftsmen (craftspeople really) moved from distributed communities to live in expanding cities where the industrial mills and factories were booming. Again, this was a massive change for humanity. Schools focused on timekeeping, rigour, and repetition to prepare children to work in factories. Conformance to schedules and plans made the scaling of a workforce possible.

Concepts like Taylor’s Scientific Management provided tools for tackling big engineering endeavours and applying specialized labour. Progressive decomposition of work and detailed scheduling of tasks allowed complex projects to be planned and managed. Techniques like work breakdown structures, network diagrams, and Gantt charts were taught to project managers to tame and track engineering work.

These techniques work well for tangible, stable and mostly predictable projects. As long as an organization has a history of building a similar product, then the gap to a new design or bigger scale can be reasonably estimated and planned for. Difficulties arise when we try to use these approaches on intangible, unfamiliar, and new environments. Differences in understanding frequently occur when we lack physical reference points such as “I want a wooden door like this one, but a foot taller”. These differences result in more change requests, more reported defects, more uncertainties and risks.

In novel, intangible environments like software development or filmmaking things rarely progress predictably enough to follow the “Plan the work, work the plan” mantra of industrial projects. New technology evolution accelerates the rates of change. Demands to deliver faster worsen the situation. Many of today’s projects fit this new breed of project that were christened "Knowledge Work" projects by Peter Drucker.

Also, many traditional industrial projects have been automated or offshored to cheaper labour markets. This leaves a higher proportion of new projects developing largely invisible, intangible, difficult to reference, products and services – knowledge work.

I am not suggesting all project work has changed. Just as we still have farmers - and hopefully always will, we still have traditional industry and industrial projects. So, while not all work has changed, the fastest growing segment has. The increasing role of software in business also means a larger proportion of projects have at least some knowledge work component. 

To help diagnose your project types, answer the following questions about the nature of projects you execute.

If you scored more on the left-hand side of the table, you are engaged in mainly industrial type projects. This is good news for reliable execution, traditional project management tools and techniques should serve you well. If you scored more on the right-hand side, you are firmly in the knowledge worker domain. You should move from industrial project management approaches and adopt knowledge worker ones. If you scored equally from each column, you are in a hybrid environment. Here you likely need to draw on a combination of approaches to be successful.

New Territory, New Tools

The tools and approaches of the knowledge worker revolution address the complexity and ambiguity that accompany these projects. Let’s dig deeper to understand the characteristics and appreciate post-industrial project management techniques.

Knowledge work projects bring subject matter experts together to collaborate on new and unique products and services. This might involve scientists, teachers, doctors, lawyers, software developers, or web designers working with the business to build something new. Each of these groups has specialized knowledge, typically no single person knows everything needed to complete the project. What is being created is new or sufficiently different to the sponsoring organization that previous project’s plans and estimates are not particularly useful to predict progress.

Compared to traditional, predictable industrial engineering, complexity, uncertainty, risk and change rates seem very high. Without tangible reference work, it is necessary to use an iterative-and-incremental approach to determine fitness-for-business-purpose. Teams could attempt to analyze and predict all features and functions, but often initial use uncovers additional opportunities and requirements.

Trying to explain the nuances of iTunes or Netflix to someone who has never seen anything like it before is difficult. Incremental trial proves faster and more useful than speculative big-design-upfront that cannot anticipate every interaction with user behaviour or linked systems.

Tools rooted in big-design-upfront, predictable decomposition of tasks, linear progression of work, etc do not work well in these environments. These include detailed requirements documents, work breakdown structures, network diagrams, Gantt Charts and earned value management. That’s not to say you cannot use these approaches, just there are alternatives that better handle the high rates of change and uncertainty.

We still need to record requirements and the use of product backlogs containing user stories makes it easier to reprioritize when changes occur. We still need to break down work and help the business decide how to best divide a big project. Instead of looking at complex architectural component diagrams, the business can make better delivery decisions by using release roadmaps, and features lists.

In the face of high rates of change, averaging delivery rates to-date can give more reliable projections than estimating the durations for planned activities. Likewise, when work is creative or R&D type in nature, we often get nonlinear progression – in other words, some things go faster than anticipated while other items take longer. Approaches like earned value management that extrapolate performance to-date to predict likely completion schedules and costs assume a linear progression of work. Instead, tracking progress based on tested, accepted features only is a more reliable predictor of true progress.

Table 2 shows knowledge-work alternatives to industrial work project approaches

Traditional project management approaches are built on the realities of predictable, industrial work. Knowledge work projects defy these traditional laws of physics since they operate outside the physical domain. Instead, they deal with ideas, people and collaboration, which is intangible. Traditional resource management suggests if we are digging a ditch with 10 people, then adding 10 more people would complete the task in half the time. Fred Brooks’ law of software development tells us that adding more people to a project that is already late will increase its duration.

Traditional project management approaches are not flawed or broken. They work great for the industrial world. In these environments, the best way to run a project is with detailed upfront planning, clearly articulated tasks and schedules, and careful granular tracking. However, if your results from assessing Table 1 indicate a hybrid or knowledge work environment then use the appropriate tools.

Trying to use the recommendations from a previous work era is akin to waiting for a full moon before starting your kitchen reno. At best you are adding wasted activities and at worst you are ignoring the realities of your environment that carry the risk of overruns and failure. 

Transformations are flavor of the month. It is no longer enough to launch “initiatives,” “programs” or “projects” to undertake work. Instead, we launch agile transformations, digital transformations and productization transformations. They sound more revolutionary, more dramatic and further reaching. Our organizations will emerge reborn, uniquely positioned to compete in a new world of opportunities and growth. Like a larva transforming into a butterfly, we can now fly!

Well, that’s the idea and the promise of the consulting companies that sell transformation services. However, what really happens? Can the average organization actually become a disruptive leader just by adopting the structures, tools and processes from the real disruptive leaders? Or, is it like buying the same shoes as our basketball heroes wear hoping they will transform us into slam-dunking superstars? The reality is somewhere between these extremes. Any company can improve, but we should not expect to become something we are not.

Let’s look at some of the popular transformation services on sale and examine the promise and truths they hold.

Agile Transformations

The goal: Agile transformations move organizations from working with traditional project management approaches to using agile approaches. They also seek to change the way organizations are structured and run from a top-down, command-and-control model to a more business- and customer-led, value-driven approach.

They aim to instil lean concepts of respect for people, minimization of waste, and value delivery. They employ a more trusting Theory Y view of workers as willing contributors rather than the traditional Theory X view that workers need close supervision to work hard. They encourage workers through intrinsic motivators such as empowerment, autonomy of work, and belief in a worthy purpose rather than carrot-and-stick approaches.

The claimed benefits: Agility allows organizations to respond to change more quickly since plans and work are done in smaller batches with frequent checkpoints. This allows changes in direction to be made when feedback indicates it would be desirable. The evaluate-as-you-go and learn-as-you-go aspects of iterative and incremental development help organizations manage complexity and uncertainty.

Agile approaches allow for the delivery of value sooner since work is prioritized via business value. The empowerment and intrinsic rewards offered result in happier, more engaged employees. Allowing workers to design their own workplace and work practices results in a more loyal and productive workforce. A mantra of one agile approach is to “Change the world of work.”

The reality: Agile is much easier to implement at a team and project level than it is at an organizational level. Teams quickly see the benefits of frequent collaboration and business engagement. Tools like product backlogs, kanban boards and release roadmaps bring much-needed visibility to design work and problem solving that often manipulates invisible data and ideas. Iterative development of small batches of work, with frequent reviews, provides better insights into progress and issues than sequential, large-batch development. While some people find the “let’s try something” approach counter-intuitive to rigorous upfront planning and design, most understand the risk reduction and true requirements validation benefits.

At the organization level, it’s a tougher sell. The initial confusion and apparent chaos that comes with establishing empowered, self-organizing, self-managing teams can seem like the inmates are running the asylum. What happens to supervisors and managers? In some organizations, departments are built around functional silos. If I was head of the quality assurance group and now all my people report into individual teams, what’s left for me to do? How do I justify my yearly budget (and position) with my headcount down to zero?

Organizational structures often reflect their culture and decision-making style. This may be hierarchical, flat or distributed. Truly transforming the organization to be agile requires a change of structure, which means changing the culture. Not an easy task, and not something to be undertaken lightly. It requires sufficient buy-in from the very top through every layer to the bottom.

Agile transformations often stall at the organizational level. Instead, we see pockets of conversion and pockets of resistance. It often takes changes in roles for the transformation to occur. However, just changing the way teams operate brings many benefits. While not really an agile transformation, a “switch” to agile project operation within a traditional organization can still be very beneficial.

So, while true agile transformations are rare, agile implementations are common and still worthwhile. The organization may never grow wings and fly as promised by consultants—but if it learns to wiggle more efficiently, avoid danger, and eat faster, that might be all it needs.

Digital Transformations 

The goal: Digital transformations aim to convert and grow business in the self-serve digital domain. They do not have to involve websites, but many do. Rather than visiting offices or calling in for service, customers self-manage through apps and websites that greatly reduce labor costs and offer almost unlimited scaling opportunities.

The claimed benefits: Cost reduction and closer engagement are the main claimed benefits. They use websites and AI-powered chatbots to handle the majority of customer questions and interactions. This reduces the need to have as many people employed at physical locations and answering phone calls. Banks and insurance companies are undertaking digital transformations to offer services in convenient formats for customers (mobile phones) as well as reduce overheads.

Encouraging customers to manage their services via mobile apps also opens up options to ping, notify and promote upsell opportunities. It is cheaper and easier to push promotions and “exclusive member benefit offers” to people who install apps than compete for attention in traditional advertising channels. Apps also let companies gather additional marketing intelligence like location, contacts, spending habits, etc.—all additional fuel for promotions and potential sales.

The reality: Building compelling websites, apps and AI services is no small undertaking. Many organizations go through the significant expenditure to discover that only a portion of their customer base embraces the new options. Organizations then try carrot-and-stick paperless discounts or paper-based account fees to incentivize the desired behavior.

These new websites and app projects will never be finished or done. Since they now represent the organization's face and communications vehicle, expect ongoing investment in their upkeep and technology refresh cycles. When looking at the potential savings, do not underestimate the likelihood of initial build costs to spiral—and integration into existing back-end systems to be orders-of-magnitude more costly and time-consuming than anticipated.

However, it seems an inevitable trend. Using established content management systems and app frameworks can help rein in costs. Being at the forefront of technological capability is only paramount if your core business is selling technology services (Amazon, Apple, Google, Microsoft). For everyone else, fast-follower (or even majority-adopter) is probably fine. Digital transformations are real, already here and unlikely to be fading anytime soon.

Productization Transformations

The goal: This is the transition from using projects to build software systems to building and viewing software as long-term products. As organizations realize software represents a market differentiator, they recognize their systems will never be “done.” If they were to finish, it means they are no longer innovating, improving or competing.

So, they move from the start-stop world of software development through projects and instead adopt continuous development and drip-feed funding models. Historically, organizations staffed and funded projects through vendors and contractors using capital expenditure models. The switch to software as a long-lived product or service changes both staffing and budgeting. The increase in cloud-based hosting also raises the question of opex (operating expenditure) versus capex (capital expenditure) funding.

Organizations often reach out for help making these changes to development, staffing and funding models. This is the new and emerging world of productization or “continuous digital delivery.” It involves restructuring and forming long-lived product teams with everyone present to develop and maintain the software products over its entire lifespan.

The claimed benefits: By eliminating the handoffs between development teams and sustainment teams, more knowledge about the system, and how to extend it, is retained. Fewer handoffs in general is one of the biggest benefits of switching to products instead of projects. Handoffs are very wasteful—they contribute to the eight lean DOWNTIME wastes…

Defects
Overproduction
Waiting
Non-utilized talent
Transportation
Inventory excess
Motion waste
Extra processing

…all of which can occur when one group hands off work to another group.

By creating stable teams that are aligned with developing and sustaining long-term products, organizations can wean themselves off unpredictable vendor models. From a budgeting perspective, estimating the burn rates and capabilities of stable teams is much more reliable than estimating how long a new vendor-based team will take to complete some work.

Stability, continuous development and better knowledge retention are all compelling reasons to trade projects for products. The difficulties come in the transition and available support.

The reality: Switching from running traditional stop-and-start software projects to continuous product development is still a new idea. Usually, organizations have a suite of currently executing projects that still need to be delivered on schedule. Existing vendor contracts may make it difficult to switch to the onsite execution approach favored by continuous digital delivery.

Finance departments are typically set up to evaluate and approve requests for expenditures based on one- or multi-year ROI projections. In the continuous development world of productization, the spending never stops.

Instead of project-based funding, small teams create minimal viable products for evaluation. If they show promise, they get additional funding in more of a venture capital-style model. New metrics like customer market share and profit-to-funding ratios are used.

The benefits are real and don’t require a major upheaval to organizational culture or structures. However, experience is thin on the ground along with books and training courses. It’s like using what we today call agile approaches in the 1990s. There are early adopters, conference sessions and blog posts, but far to go before the idea even approaches the chasm (let alone crosses it).

Summary

“Transformation” is probably too grand a word for the degree of change most organizations achieve. However, as so many ideas compete for our limited attention spans, it would seem crazy to merely name a change initiative a “rollout” or “improvement” these days. People have become desensitised to reasonable names and seek revolution and excitement to generate the interest they need to participate.

We should not expect traditional organizations to truly match digital-first companies like Spotify. They were founded to disrupt existing businesses and came without the baggage of a traditional client base to support. (Also, they are led and staffed by people that share different values than most North American institutions.)

Their ideas may be great for other organizations to experiment with and adopt what works, but what makes them truly powerful is that the ideas were created internally and vetted through experiments. Copy the concept (internally generate new processes to solve local problems), but not Spotify’s actual procedures.

There is nothing wrong with buying the same kind of shoes as Michael Jordan wore (heck, if the placebo effect gets you exercising more, they were likely a good purchase). However, don’t leave your day job to sign up with a basketball team until you’re sure you are world class. Today’s transformations bring many benefits—as long as you take the “transformation” claim with a grain of salt.

[Note: I first wrote this article for ProjectManagement.com here]