Chapter 14. The Myths and Challenges of Scrum
A studio’s development process is a reflection of its culture, and cultures change slowly. Change usually has to overcome resistance, but change happens—especially in the game industry—whether we like it or not.
Change requires commitment from a studio. It should avoid quick-fix solutions or ritual adherence to defined practices. With agile adoption, both of these extremes are often seen. Some fall in love with the values and principles and dive into adoption with full faith that the only possible outcome is success. Others sabotage agile adoption from the start. Saboteurs repeat “urban legends of agile failure” or label agile as “the latest management fad.” This is referred to as spreading fear, uncertainty, and doubt and is addressed later in the chapter.
There is a kernel of truth to many of the myths. This chapter identifies the major ones about agile and Scrum. The purpose of this chapter is to look behind the myths and explore the truths and falsehoods that they rely upon. It will also explore many of the barriers to Scrum adoption. Exposing these facts enables a studio to better judge the value of agile because the best possible path for adopting it is to understand the reasons behind every practice and leave little to faith. This is the benefit of an empirical system: to base what we do on what we know rather than on theory or conjecture.
Silver Bullet Myths
Common to the tales of vampire slayers, silver bullets possess magical properties that give the hero an advantage when facing certain disaster.
Often the adoption of Scrum is motivated by a project disaster. A studio ships a game that is a financial failure or has a project canceled because of budget, schedule, or quality problems. During these times, studio management is more open to change. This isn’t necessarily bad, but desperation often leads them to seek a project management silver bullet.
The problem is that Scrum doesn’t work miracles. It has no magical properties. Improvements in development by using it require an understanding of the underlying principles, not blind faith. Scrum is a framework to build a process that supports talent, great teams, and leaders. It does not replace them.
By avoiding the following silver bullet myths, you avoid falling into the trap of thinking a process can solve all your problems.
Scrum Will Solve All of Your Problems for You
Sometimes we have grand assumptions about what a process can accomplish. Perhaps if we create enough rules, then everyone will follow them, and problems will disappear. The truth is that if there are underlying problems, Scrum will only expose those problems. It doesn’t solve them.
Experience
A former California studio adopted Scrum following another project failure. From the start, the daily scrums unleashed a flood of complaints about studio operations. Management quickly decided that Scrum was the source of these problems and immediately halted its use. As expected, the level of complaints fell off, and management felt that they had averted a disaster.
The best that can be asked of Scrum is to facilitate problem solving through transparency. What is done with this transparency depends on a studio’s culture. This studio closed because they ignored the problems that existed for years.
Projects Using Scrum Can Always Ship on Time
Another myth is that Scrum teams always ship games on time. This is born out of desperation to avoid delays from impossible schedules, but Scrum won’t accomplish miracles.
Many projects attempt to fix budget, schedule, and scope simultaneously. A benefit of Scrum is that you measure velocity against the goal and know what is possible early on, make the right decisions and commitments, and then monitor and progressively refine those decisions.
Sometimes Scrum is blamed and abandoned when its empirical measures show that a goal is impossible. The goal was probably impossible to begin with, but Scrum reveals it sooner than a traditional project, which enables problems to be hidden.
Fear, Uncertainty, and Doubt
Some myths fall under the category of fear, uncertainty, and doubt (FUD).1 FUD myths are like urban legends; they are exaggerated stories that have a kernel of truth that make them stick.
1 http://en.wikipedia.org/wiki/Fear,_uncertainty_and_doubt
There are many reasons people use FUD to sabotage agile adoption. The main reason is that change alone creates FUD. Change threatens the status quo and therefore a person’s security in their position. Sometimes the memory of past successes creates the belief that the practices used then will continue to apply equally as well in the future. Problems are seen not with the process, which has proven itself, but with the developers using it now.
This section addresses some of the common FUD myths about agile. Chapter 15, “Working with a Publisher,” will address strategies of overcoming the status quo, which can be the greatest obstacle to adopting agile.
Endless Development
Agile teams never plan. They just iterate without a goal.
Sometimes a team new to Scrum assumes that being agile means they don’t need to plan. They start by iterating to discover the game that emerges. Eventually the urgency to ship intrudes, and the team has to scramble, often with enforced overtime, to finish something. This isn’t agile. It’s an iterative and incremental death march. It’s a danger for agile teams that don’t sufficiently plan.
It is also essential to have a shared vision for any project, whether it is agile or not. If the product owner on a Scrum team does not provide a vision, the team cannot be sure that their work creates the greatest stakeholder value.
Experience
I once knew a team that had no product owner. They were working on a first-person shooter. The product backlog was prioritized by the lead programmer. Since he was focused on physics, each sprint would show increasingly impressive physical effects. After their first release, buoyant rag-doll bodies were seen floating down a stream in front of the player. When we asked to see how enemies were shot and fell into the stream, we were told that shooting wasn’t implemented yet.
A few months later the stakeholders canceled the project.
Management Fad
Scrum is just another management fad. It will be replaced by something else next month.
Someone investigating agile will encounter labels such as Scrum, Extreme Programming, Evo, RUP, Scrum-#, Scrum Type-C, lean, feature-driven development, kanban, and so on. It’s a confusing process landscape. This was one motivation behind the agile manifesto. Decades of discovery about how people best work together to create new products are embodied in its simple values and principles. It is an umbrella “brand” name for any number of processes and frameworks that embrace them.
The expanding list of labels represents the various practices that are constantly emerging as agile becomes more common and accepted. This book has described numerous practices for game development that lie outside the core definition of Scrum. We could label them (perhaps Scrum-G), but the point is for each studio to evolve and adopt their own practices to best fit their needs. There will never be a single template for game development that applies to even a majority of game types or studios!
Scrum is a great starting place. For almost a decade, people such as Mike Cohn and Ken Schwaber have fought to keep it a simple framework for each team to adapt for the needs of their products:
Agile will go away, but it will most likely go away in the same way discussing the merits of object-oriented development went away. Agile will eventually become the accepted way of doing things, and it will just be what we do. In the same way no one says, “Gotta run, I’m late for the object-oriented design meeting” (they just say “design meeting”), we will stop talking about agile development but only after it is the norm.
—Mike Cohn
The Double Standard
We hear one of two things about Scrum. Either Scrum was successful, or if the project using it failed, it did so because they weren’t using Scrum correctly.
Scrum can’t be blamed for failure or even credited with success. Teams succeed or fail because of many factors: technology, capability, vision, communication, collaboration, talent, or even the underlying idea of the game. Scrum creates transparency into how well these elements are working, often in a measurable way, but that’s all. It doesn’t prescribe what to do when sprints repeatedly show the game isn’t fun or the velocity of the team is low.
Change Is Bad
Our process has worked in the past. If we change things, we’ll fail.
Studios have a certain level of resistance to change in the process they use to create games. This is not necessarily a bad thing. A studio’s process evolves over many years. It’s reinforced by its strengths and successes.
However, processes also embed cultural weaknesses. For example, many studios institute a one- or two-week “lockdown” before a milestone date. The lockdown ensures that no new features are allowed into the build that can destabilize it while the build is debugged and polished. This is a concession to a development culture that allows too many defects into the game during development.
Lockdowns slow development because the cost of fixing bugs increases the longer you wait. Also, the entire project staff is usually not engaged in bug fixing during a lockdown, and therefore they have to find other things to do. Lockdowns also end up becoming a dumping ground for checking in bug-ridden code or unpolished assets under the assumption that they will be fixed later. This creates a vicious cycle that can inflate the duration of future lockdowns!
Changing core practices, such as introducing unit testing to improve code quality, is more challenging than adding a quick fix, such as a lockdown. The problem is that quick-fix practices become a normal part of a development.
Over months and years, such quick fixes build up. Each of them adds a bit of drag to development. Resistance to change prevents corrections to the process until there is a catastrophic failure (such as a project cancellation over high cost or slow progress).
Normalization of Deviance
Two examples of how ignoring problems can lead to tragedy are the Challenger and Columbia space shuttle disasters that took the lives of 14 astronauts. Both incidents involved problems that were well known and documented with the shuttle system. These problems had occurred frequently but had not resulted in the loss of a shuttle. As a result, they became an acceptable part of operations until there was an accident.
Following the Challenger accident review, the phrase normalization of deviance was coined (Vaughan 1996) to describe how the attitude toward defects in the shuttle system resulted in its loss. Unfortunately, identifying this problem with NASA’s culture was not enough to prevent it from contributing to the loss of the Columbia years later.
Endless Meetings
Scrum consists of nothing but meetings!
As previously described, the meetings defined by Scrum within a sprint cycle are as follows:
• Daily scrum
• Sprint review
• Sprint planning
• Sprint retrospective
All of these meetings, except for the daily scrum, occur once a sprint. Many teams find that for a four-week sprint, these once-a-sprint meetings can be conducted in a single day. This represents one day of meetings for a 20-working-day sprint, or 5% of the available time. The daily scrum meeting is a 15-minute, timeboxed daily meeting. This uses about 3% of the working day. Totaling these percentages results in 8% of a team’s time, or a little more than three hours of meetings each week average.
Scrum meetings are optimized to create the highest bandwidth of necessary communication. A 15-minute daily scrum will often identify issues that are easily solved within a day. If not identified, the issue might grow to waste days and require hours of meetings to address weeks later.
The details in this book will help teams run these meetings as effectively as possible without losing their benefits. If any of these meetings do not engage everyone attending nearly 100% of the time, then there is room for improvement.
Note
Shorter sprints will use a greater percentage of time in meetings. For example, the sprint review, planning, and retrospectives might take six hours to complete for a two-week sprint rather than the eight that a four-week sprint might require. See Chapter 4, “Sprints.”
Scrum Challenges
The only thing harder than starting something new is stopping something old.
—Russell L. Ackoff
Scrum is not a “one-size-fits-all” solution for developers. A goal in adopting Scrum is to initiate a never-ending cycle of continual improvement customized to the needs of a studio’s culture, people, and products. Before this goal can be realized, a studio or team has to start practicing the fundamentals and navigating early challenges. This section describes a few of these challenges and some of the ways of navigating them.
Scrum as a Tool for Process and Culture Change
Any process can be used to make games, but no process is perfect. Even if we were to identify a perfect process, the constant change in our industry would quickly make it obsolete. This is a major motive for using Scrum. Scrum is not a process; it’s a framework for creating and evolving your own process. It can help any organization create transparency, which enables commonsense change.
Scrum will influence every part of your studio. Scrum pressures managers to focus on mentoring and coaching. It involves buy-in from departments such as HR, which can resist the emphasis on team performance. It demands people take ownership in areas in which they had no ownership and give up ownership in areas they once ruled over. It challenges marketing to participate with development far earlier in a project. It even puts pressure on facilities to provide open team areas and wall space.
Scrum adoption starts with creating transparency to expose cultural weaknesses and strengths. An example of this was a studio dominated by technology; games were seen as platforms to demonstrate technical achievement. Tools and pipelines to improve productivity for artists and designers were lower on the list of priorities because the programmers were always trying to accomplish nearly impossible challenges. The build was rarely working because of all the bugs left along the way, so the time it took to iterate on an asset change was very long.
How did Scrum influence change? First, it made the high iteration costs visible. Requiring a potentially shippable version of the game every two to three weeks exposed a lot of problems. At first, the teams spent half their sprints trying to cobble together a working build. Since velocity was measured by the value of the features working in the game, their initial velocity was low.
This simple measurement of velocity is important. Teams need clear performance measurements to evaluate themselves and make changes to improve that measurement. Our example team did this. They started to come up with ways to improve the reliability of the build. Because velocity derives from what is seen in the game, which includes art and design, they formed cross-discipline teams. They colocated to reduce the overhead of communication. Programmers focused on improving tools and pipelines. All of these things improved their velocity.
Before they adopted Scrum, they would have thought that such change was only possible through great technical effort. Scrum allowed cultural change, which resulted in huge performance gains.
Scrum Is About Adding Value, Not Task Tracking
Running a daily scrum is easy. Prioritizing a list of features to be implemented is straightforward. Creating a task board and burndown chart takes minutes. So, why do some teams struggle adopting Scrum?
One reason is that many studios that adopt Scrum come from a task-centric culture. Progress is measured against how well individuals complete their work against estimates. Task tracking is an important tool in Scrum, but value creation takes precedence. If value isn’t maximized, then the tasks were wrong.
Have you ever heard someone report “All of the tasks for feature X are completed” only to see that feature X is nowhere near being done? In Scrum, the conversation shifts to discuss a feature’s emerging value and the impact on the tasks for that feature. If this shift doesn’t occur, then the value of Scrum practices is greatly diminished. For example, in a task-centric culture, the daily scrum is seen as a task-reporting meeting that is easily replaced. In a culture focused on value and team commitment, the daily scrum takes on greater importance.
Cultural change is hard. The status quo will often fight tooth and nail against it. Scrum is a framework for this change, but it comes from real leadership, not from a book or a fixed set of practices.
Status Quo versus Continual Improvement
Building a culture of “continual improvement” is one of the ultimate goals of Scrum. Scrum practitioners use empirical measures to assess benefits of all practices. If, for example, a practice change improves velocity, then it’s a beneficial change.
This is a simple idea to introduce, but it can be challenging to embed. The status quo, or groundless resistance to change, is often a tremendous barrier to continual improvement.
Fear of change is not always baseless. Management fears that introducing profound change gambles a studio’s future. Introducing change requires bravery. For example, changing a process that may have worked for a PlayStation 2 game paints a big target on the person who changed the process for a PlayStation 3 game. There is enough risk in changing platforms alone to cause someone to hesitate over changing one more variable.
This is why it often takes utter project failure and crisis to usher in significant change. This isn’t ideal either since it often leads to the “silver bullet” adoption pattern described previously. Ideally, any method for introducing change needs to do it in ways that
• Make small, reversible steps
• Are measurable to ensure that any perceived improvement is real
Scrum supports change in this manner. At worst, a change in practices will impact a single sprint of two to four weeks. Metrics, such as task burndown slope, user story point velocity, or any number of metrics that the team has introduced, provide frequent measurement of the value of changes.
This change of culture can’t happen only from the bottom up. It has to be supported by company leadership. Management needs to understand the tools that Scrum provides them to ensure that teams are making progress and that leadership, vision, and commitment exist.
Lack of management commitment to Scrum and agile principles is a major challenge for studios attempting to adopt Scrum. Under pressure, a manager might find it easier to alter a team’s goal midsprint rather than fight to preserve the team’s commitment to the sprint goal. Educating management about the benefits of Scrum—especially for them—is an ongoing effort.
Cargo Cult Scrum
When teams first start using Scrum, they are encouraged to follow the practices “out of the book.” This enables them to become accustomed to the practices while the principles sink in. Once the team experiences the initial benefits, they begin altering practices to improve upon those benefits, but it’s important that changes to the practices still preserve the principles behind them.
However, some teams stick to the “out-of-the-book” practices and resist changing anything. These teams end up practicing what I call Cargo Cult Scrum. This refers to the infamous cargo cults of the Pacific.
Until World War II, many Pacific Islanders were never exposed to Westerners or their technology. This changed drastically during the war when the Navy arrived to occupy the islands and construct airfields. When the airfields were complete, cargo planes would land and bring in supplies. These planes carried many things including simple trinkets that were shared with the islanders to promote goodwill. The islanders loved these objects (such as steel knives and mirrors) and gathered around the airfield every time a large aircraft landed seeking further gifts.
When the war ended, most of these airfields were abandoned, and the cargo planes stopped coming. The islanders once again found themselves isolated from the Western world, but they did not forget the cargo planes and their treasures. They wanted them to return. Out of desperation, they tried to draw the cargo planes back by replicating the practices they saw at airfields during the war.
They constructed bamboo towers and manned them with natives wearing coconut headphones who spoke into pineapple microphones. They built bamboo mock-ups of small planes, lit fires next to the runway, and waved flags of cloth in the air. But the cargo planes did not return. The practices weren’t enough to bring them back.
This approach is similar to what is done on Cargo Cult Scrum teams. Following the practices isn’t enough to bring the full benefits of highly productive teams. Teams need to understand the principles behind the practices and improve those practices to best fit their product, people, and culture.
Scrum Is Not for Everyone
One of the challenges of Scrum adoption is that Scrum is not for everyone. The initial challenge will be that some people refuse to work in an agile environment and leave. Some of these people will be valuable. They leave because they are not comfortable with the change in their position.
Some developers reject participation in any team activity (such as a daily scrum). Some have grown comfortable from a career of working in relative isolation and being called upon to be heroes during crunch times. Others find a niche in a lax management environment where they are given a great deal of freedom to create technology or assets on their own. Joining a Scrum team and making daily commitments to a team of peers limits these individual freedoms.
Most studios find ways to accommodate these individuals, perhaps creating special “R&D” roles for them, but in many cases they eventually leave. The transparency Scrum introduces to a studio makes such positions stand out and not easily justifiable.
The benefits greatly outweigh the losses. Scrum grows leaders and outstanding contributors at a far greater rate than those who are lost.
Overtime
Scrum doesn’t limit teams to working 40-hour weeks. Its practices enable teams to find a sustainable pace of work. This pace is discovered as they commit to sprint goals and learn how much they can achieve sprint after sprint without overextending themselves.
Note
In everyday vocabulary, a sprint is something that isn’t sustainable. Calling it a jog might have been more accurate but less appealing.
A sprint can have many uncertainties. Unanticipated problems or unforeseen work slows progress. When this happens, teams sometimes put in a bit of overtime to fulfill their goal.
How much overtime should a Scrum team work? It’s up to them. If they find that they are working overtime too often, they need to address the problems that are causing it. Common examples for this are late commits or handoffs at the end of a sprint or committing to extra polishing and tuning work on the last sprint of a release.
When management doesn’t tell teams to work overtime, their attitude about it changes. When a team decides to work overtime, they do it as a team. Occasionally working a few extra hours shoulder to shoulder with your teammates is a team-building experience.
Crunch
Extended periods of enforced overtime are called crunch. Many studios that are new to Scrum continue to practice it until the empirical measure of velocity demonstrates its futility.
Studies have shown the impact of crunch on productivity and quality of life.2 For High Moon Studios, the proof came when management enforced companywide overtime early in our adoption of Scrum. The teams were told to work 10 hours a day for 6 days a week on a troubled project. The subsequent burndown charts told an interesting story. Figure 14.1 shows the hours the average team burned down per week from their sprint backlogs.
2 www.igda.org/quality-life-white-paper-info
Figure 14.1. Burndown during crunch
Week 1 was a normal workweek, before overtime. Weeks 2 to 5 were crunch weeks of 60 hours each. In the first week of crunch (week 2), velocity greatly increased; more work was being done because of the 50% overtime. However, as weeks passed, the velocity decreased until week 5, when the velocity was less than it was before crunch started!
How is this possible? The reasons are simple. People were tired. They made more mistakes. They lost their concentration.
This realization represented a huge benefit of Scrum for High Moon, providing simple empirical evidence about what works and what doesn’t. This is why there is no rule about overtime in Scrum. There doesn’t need to be a rule. If your teams are using Scrum to find the best way to work, they’ll quickly discover that after several weeks of overtime, any benefit from it is lost. It becomes common sense to maintain a sustainable pace.
Velocity in Hours
Although the term velocity usually refers to the number of story points accomplished per sprint, the velocity, or change, in hours of work remaining in a sprint per day or week provides interesting, though less stable, feedback about a team.
Summary
The best way to adopt Scrum is to do it with eyes wide open. Establishing useful metrics such as velocity and establishing practices such as the daily scrum provide immediate demonstration of its value.
Scrum is a framework. It doesn’t include practices to optimize code, create better art, or tune a mechanic. Those come from each studio’s development practices and culture. In this light, there is less to fear about the change Scrum introduces.
Keep the myths and challenges from this chapter in mind as you read the next chapter on launching Scrum. It will continue to introduce challenges and describe how to meet them and what to expect in a studio as Scrum principles take hold.
Additional Reading
Heath, C., and D. Heath. 2007. Made to Stick: Why Some Ideas Survive and Others Die. New York: Random House.
Pascale, R. T., M. Milleman, and L. Gioja. 2001. Surfing the Edge of Chaos: The Laws of Nature and the New Laws of Business. New York: Three Rivers Press.