Evolution or Revolution: Why Is Implementing Scrum so Tough?

Please don't excommunicate me!

If this article seems a little Machiavellian, it is. There's a lot of soft stuff when talking about business change. Anyone who's implementing scrum in a waterfall-centric organization will understand that the reality can be a lot tougher than they first imagined. For those organizations that are willing and ready to embrace change, the path may not be so tough. Those taking the long, rocky path may wonder why it was so easy for others. Maybe a more appropriate title for this paper should be "How to Kill the Prince II."

Having employed a few scrum masters and then sitting back and waiting, many executives wonder why the move to scrum is taking their people so long. Where are all the benefits they were expecting to see?

Implementing scrum involves the adoption of a new paradigm across the organization. 

In most instances, the severe level of culture shift and change aren't really appreciated. Once people get into the change process, they realize what they're facing, and try to backtrack. Very often, soft options such as scrum-but, scrum in a waterfall wrapper, or "our version of scrum" are inadvertently or purposely taken, and chaos ensues.

To give you a feeling for the change required, here is a list of the activities a pure waterfall organization could perform in order to transition to scrum:

• Implement a communication plan for scrum principles and processes to all employees.

• Stop advertising and hiring non-agile roles. These people will work in opposition to the transformation.

• Define a governance structure supporting scrum. Base this on fixed prices, self-managed teams, transparency, and inspection (i.e., burn-down charts, showcases, and velocity).

• Agree on a contract model that supports scrum, such as collaboration with your customers and suppliers as opposed to statements of work and arms-length negotiations.

• Assess the products you currently support and wish to develop, and agree on the resourcing levels you need to deliver this using scrum. Maybe agree on a product vision and road map?

• Create and formalize scrum job descriptions and remuneration packages. If you don't recognize scrum roles, it's unlikely you'll attract or retain the talent you require.

• Make non-scrum roles within your organization redundant — specifically project managers, business analysts, and technical architects who are no longer required.

• Create and advertise the scrum positions you wish to fill to deliver your products.

• Make training available and run workshops for those working in redundant positions so that you can develop the scrum master, scrum coach, scrum developer/tester, and scrum product owner roles.

• Set up your scrum teams and ensure that they understand the products they're taking ownership of.

• Reorganize your office seating to colocate your scrum teams.

They say a new broom sweeps cleanest. Did anyone say change was easy?

The above to-do list may seem a bit extreme, and I wouldn't advocate imposing this level of change on an organization without first understanding where it is on its journey to scrum and the impact these changes will have on its culture.

That said, if you are serious about scrum and really wish to build an agile organization, you probably need to consider each of the above activities.

The bigger the dog, the bigger the fleas!

Before embarking on a change, it's useful to understand the organization you're working with, from a political and contextual perspective. On the whole, larger organizations have a deeply embedded culture that makes transition management more challenging.

Big or small, it's worth understanding how political the organization is. It's best not to discuss politics openly, but try to gauge the culture as you engage with the various groups in your discussions about scrum.

• Are projects and project managers more important than the projects they deliver?

• Is "who" more important than "what"?

• Is the organization functionally segmented?

• Who holds and wields the real power?

• Who can and might stop you from implementing all or some of the above transitions?

• What sorts of things are seen as socially acceptable? For example, do people get a job for life here? Are regular redundancy rounds an occupational hazard and just business as usual?

Different strokes for different folks.

How receptive is your organization to change? 

This in itself isn't a simple question. Even within the same organization, not everyone will be on the same journey.

Faced with the transition to scrum, your development community will probably have heard of scrum and will want to give it a try. Your testers may be more sceptical. Your financial controllers may be scratching their heads, and your project and program managers may be in open revolt — although they will be clever enough not to say this openly (change managers aren't the only advocates of Machiavellian management).

So where should you start?

By now it's probably becoming clear that even in organizations that embrace change, implementing scrum is not always simple. Most scrum transitions fail due to cultural and organizational resistance, so obtaining top-level buy-in is essential but can be tricky.

There are probably as many ways to successfully implement scrum as there are organizations in the world. You could start at the bottom and work up (that is, employ some scrum masters and see what happens). But if you simply set up a scrum team in a project-centric organization, you're bound to run into organizational issues once the various camp followers realize their value in the new process and start interfering (or fail to get out of the way).

The following approach focuses on getting buy-in and understanding from the top and has a fairly high chance of success. Having said that, there are a lot of variables at play, not least your skills as change manager and scrum practitioner! And never underestimate the level of resistance or political skill of your adversaries, who will say one thing while doing another.

I hear and I forget. I see and I remember. I do and I learn.

Most of the resistance you'll encounter will be at a middle-management level, so teaching your senior stakeholders about scrum through doing scrum ensures that it's understood and adopted top down as well as bottom up. This approach helps neutralize your opposition, because once the senior people understand scrum they fear it less and build it into the business strategy. And, of course, if they can do it, why can't others?

The sooner you start, the sooner you know what you've gotten yourself into. This is also known as fail fast!

• Identify your key stakeholders. These people will be running projects and programs, sitting on project boards, and managing resources.

• Describe scrum to the senior stakeholder. Talk about the benefits and discuss where the organization is in its journey to scrum and what still has to be done. Talk about the process you want to follow and how you want to get buy-in by demonstrating to the senior team how simple scrum is.

Note: if you can't get an agreement at this level, you seriously need to reconsider your role as change manager and the likelihood of successfully implementing scrum.

• Organize a half-day workshop for your key stakeholders.

• Describe scrum to the team, talk about the benefits, and discuss where the organization is in its journey to scrum and what has yet to be done.

• Discuss the process you're going to use and invite the team to participate in the transition to scrum.

• Form a scrum team and identify who will be:

• The product owner (this will probably your senior stakeholder, but it could be delegated to the most powerful individual)

• The scrum master (this could be you, but it's best if you facilitate)

• The developers

• Agree on a vision statement, such as, "An organization that effectively employs scrum to deliver its products and projects."

• Create your user stories, describing the transitions that must occur in order for the implementation of scrum to be effective. Note that the product owner must agree to these.

• If you have a lot of user stories, consider agreeing on a minimum marketable subset. That is, agree on the vital few to achieve the transition.

• Ask the team to break down the user stories into tasks and estimate them.

• Agree on a Definition of Done.

• Agree on the length of the sprint, usually one to two weeks. (It's likely that you'll only engage the team part-time for a week — maybe less — but so long as the user stories are understood and eventually delivered, the exercise will have achieved its purpose.)

• Hold a sprint planning session and agree on the user stories that the team can commit to delivering in the sprint.

• Start the sprint. Hold daily stand-ups at an agreed time and track progress on a burn-down chart and a nice big scrum board. Possibly run a sprint review if you think it worthwhile.

• At the end of the sprint, hold a retrospective and see how things have gone. Discuss what went well and what could have been done better.

I do desire we may be better strangers.

What I have proposed here is a fairly simple but effective approach to change. Remember that before you start, you need to have performed some analysis of the organization, including understanding who your adversaries are likely to be and which stunts they're likely to l pull on you.

For example, opting out is a simple strategy to avoid engaging with change. Managers don't climb the pole and stay there by accident. They twist and turn like a twisty-turny thing (to quote Blackadder), so pinning down your stakeholders in large organizations for half a day can be a monumental task. Even if they attend, make sure they don't spend most of their time on their cellphones. If the organization is new to scrum, however, capitalize on this, as there will be plenty of enthusiasm and goodwill, so securing half a day from everyone won't be so difficult.

Delaying strategies is another well-known defense against change. Proposing that a budget is required before implementing scrum should slow things down. Alternatively, requesting a thorough analysis and creating a change plan is the ideal way in which to not do change. Better still, appoint a project manager to oversee the change!

Tweaking scrum, or "scrum-but," is another way to keep doing what you've always done while creating the illusion of change. One reason I love scrum is that there is less wiggle room compared with other agile frameworks. Do scrum or don't do scrum! You should be very clear to everyone that there is no such thing as "scrum-but."

A variation on "scrum-but" is to propose running scrum in your existing waterfall projects. What you usually end up within this situation is, unsurprisingly, a waterfall project — that is, one with all the downsides of waterfall complicated by mixing it with scrum.

Fortunately, most managers are a one-trick pony, so once you know the team, ask around and find out what they usually do to block change. Then think up viable counterstrategies.

What about the others?

Scrum masters are familiar with implementing the operational aspects of scrum, so I would expect that once you've kicked off the organizational and buy-in aspects of the change, you should find implementing scrum a relatively easy task.

That said, don't wait for the ink to dry on the transitional user stories; you're there to implement scrum. As long as you've initiated the change process, kick off your operational scrum implementation. The two should eventually meet up somewhere in the middle.

Related: Measuring Progress Toward Agility

Conclusion

I may have painted a bit of a bleak picture, and the execution of scrum in many organizations isn't that complex.

Running a high-performance scrum organization is every bit as hard as running a complex waterfall program; it's just that the challenges are different. The issues pop up sooner and the benefits come earlier, but without senior management support, it's unlikely that those benefits will be recognized.

This approach may not work in all organizations, but in the spirit of scrum it should provide fairly swift feedback, so you'll know early in the process whether it's working or not. If your scrum implementation has stalled, try this approach. If it doesn't work, remember that scrum is a learning system, so engage with your teams and try some new ideas.

Discover more resources here.