The impetuous Scrum!

Originally Published May 12, 2018

 

Introduction

Jeff Sutherland and Ken Schwaber conceived the scrum process in the early 1990s. The term ‘Scrum’ was originally borrowed from rugby and referred to ‘team of individuals working toward a common goal’. The pair codified scrum in 1995 in order to present it at an object-oriented conference in Austin, Texas (during the OOPSLA 1995 conference) and published it in the form of a paper titled “SCRUM Software Development Process.” However, the development of Scrum is not so clear as needed to credit just one or two individuals.

Many publications and experts credit Hirotaka Takeuchi and Ikujiro Nonaka, two acknowledged management thinkers, as the proponents of first concepts related to Scrum. A paper published in the January 1986 issue of Harvard Business Review by the duo gave detailed principles of Scrum. Jeff Sutherland and Jeff McKenna drew on the inspiration of this 1986 article and developed it further. The first full implementation of Scrum occurred in 1993 when Jeff Sutherland along with John Scumniotales and Jeff McKenna implemented Scrum at the Easel Corporation.

Image Credit: Internet

 

Scrum was initially based on the notion that development of new, complex products, can be best achieved when small and self-organizing teams are given objectives rather than specific assignments. The team had the freedom (self-directing teams) to determine the best way of meeting those objectives. Scrum also defined time-boxed iterative development cycles whose goal was to deliver one or more features in each release, thereby delivering incrementally working software version with each release. Today, most teams that claim to practice an agile methodology mean that they’re using scrum.

The original Scrum definitions, purpose and uses are best found at Scrumguides.org

The best explanation of what Scrum is, in the words of Jeff Sutherland and Ken Schwaber is as follows:

Jeff Sutherland believes that Scrum is a framework with a set of best practices learned over fifty years. Ken agrees as well with this and gives a good analogy of comparing Scrum with playing chess: The word “framework” means that much is not specified and must be devised by those using the framework. I equate Scrum to the game of chess. You can read the official rulebook for chess. The moves, players, sequences, scoring, etc. are all specified. Learn them. Then you can play chess.

Image Credit: Internet

 

Core Practices in Scrum

There are core principles in Scrum that are widely discussed about and known. Based on these principles, there are multiple practices that are acceptable within Scrum, which teams may adapt based on their needs and requirements.

• Work is organized in short cycles, typically 2 weeks:
• The management takes backseat and doesn’t interrupt the Development team during a work cycle. Any interactions are through product owner.
• Role of Product Owner as the custodian of Product Backlog
• Product Backlog is the document where all requirements are recorded
• Product Backlog refers to all the work that remains to be done on the project
• The team reports to the client and product manager for work done
• Estimates are drawn up by the team
• The team decides how much work (how many story points, features, etc from the product backlog) it can do in an iteration.
• The team decides how to do the work in the iteration.
• The team measures its own performance.
• Define work goals before each cycle starts.
• Break down work in to small use cases, user stories.
• Systematically remove impediments.

 

The above are not exhaustive rules, but best practices modeled on Scrum rules which teams are free to adapt.

 

Growth of Scrum and its default usage with Agile

Scrum is now the default agile software development methodology. This management framework, which is “simple to understand but difficult to master”, is used by 66% of all agile companies.

VersionOne’s 2016 State of Agile Development survey puts the number of teams currently following a pure Scrum approach for their agile implementation at 58%. Furthermore, this number increases to 75% if you include hybrid approaches such as those which combine Scrum with another agile approach such as Kanban to create Scrumban.

Image Credit: VersionOne’s State of Agile Development survey

Why is Scrum so unpopular?

Before the Agile fad made it a lasting and widely-used process, many organizations used the term “Scrum” for what someone would commonly call a “Code Red” or a “War Room emergency”. If there was an uncontrollably fast spinning out-of-hand situation, or an unexpected failure, or rapidly-evolving problem, the management or the department leadership would call together their best people and form a cross-cutting team. This cross cutting team’s only mandate would be to resolve the problem as quickly as possible.

Of course, the team would have a leader for namesake, in other words as there is no clear manager as the team didn’t evolve but was rather plugged together, a leader would be ‘assigned’. This person is sometimes called Scrum master and his / her authority is limited to ‘removing impediments’ and ensuring that team members ‘meet’ daily. The management would not want him to be an official “people manager” (since he needs to be as impartial as possible). Since the crisis is short-term, the team members’ career goals and aspirations are put on hold. It’s considered a “sprint” because people are expected to work as fast as they can to solve the problem, and because they’ll be allowed to go back to their regular respective functions or teams once it’s over. The team members, many of them highly qualified engineers and thinkers pitch in whole-heartedly as they feel doing so is the best interests of the organization. The focus is on the mission and the mission only.

 

Image Credit: Internet

However as time wears on, and ‘Sprints’ progress, the never ending nature of ‘emergencies’ become clear. That these çode-red’ situations are now imbibed in to the organization’s DNA becomes clear when project after project falls in to the same framework.

Once a firm sees the dramatic benefits of small client-driven iterations in one area, it becomes natural to ask: Why not do all work in this fashion? The bigger question to ask is : Can all work be done in similar manner?  Can code-red situations exist forever or become the de-facto working style? What impacts does it have on the humans involved in the value chain? After all, an organization’s most important asset are its resources. So what about their perspective? Are these resources happy to work in an environment where they need to justify every hour of their day producing predictable results like a machine? Or is it not important at all to take their viewpoint and their idiosyncrasies in to consideration? More often than not, in the race to become Agile and implement Scrum, such questions get relegated to sidelines.

Image Credit: Internet

 

Here is a look at some of the reasons Developer communities and professionals in project management have ceased to be a fan of Scrum:

1. Use of Story Points -: The use of story points appears to be one of the defining features of Scrum. Each user story is given a certain count of story points, and a team is confronted with the number of points it has “achieved” at the end of a sprint. Some teams also follow their daily progress in terms of story points with a burndown chart that is consulted every day at stand-up. However, with teams working to gather points the focus gets shifted to completing tasks instead of delivering good software. The questions that seem to come under focus is how many stories are complete, and not the more important questions of how has the code base improved? Did it become more modular, simpler, habitable?
2. Barrage of meetings -: Scrum recommends the following type of meetings
   1. Planning meetings -: Planning always seems to be a regressive activity. Little attention is paid to the accuracy of estimates which may seem a pointless exercise after certain time. This is one area where teams can learn the most, because incorrect estimates point to misunderstanding of the code base and domain, but decent review of estimates is rarely, if ever done.
   2. Retrospective -: very little value add is achieved, as in Scrum, the retro is explicitly supposed to be about the Scrum process itself, not about the codebase, the technology stack or development patterns.
   3. Daily standup -: While sounding noble, I have seen more than one example where daily stand up meetings become more of a ritual than anything else. People talking in to voids, while others keep doing their stuff dreading their chance to speak, at the end of which same standard sentences get repeated.
   4. Scrum meetings aka Show and tell -: Points are awarded often for whether the particular feature works or not, irrespective of how much thought process went in to the work. Often there is extra work needed to set up the demo which doesn’t get counted in any estimates.

 

3. Scrum as Management Tool -: Scrum is seen by many as a Management tool rather than a Development process. When the management imposes process and demands one-sided transparency of all the workers, people feel watched. They get the sense that they’re being stalked and will be labelled “low performers” as soon as their week-by-week, individual fluctuations go the wrong way. So they become resentful.
4. Sprint, Sprint, Sprint -: Sprint is an iteration in which work is done; where usually selected user stories at the start of the Sprint are developed and released at the end of the sprint. Work to be done in the Sprint is selected before the start of the Sprint from product backlog. Once Sprint starts, the work cannot be changed, scope added or deleted unless in agreement with product owner and the team. The team retains the call to agree to scope changes during the sprint. However, this very idea of not deviating from planned work is like an anathema, as software development cannot be always predictable. Each project is different; new ways of solving even older problems can be found, and its entirely possible that team may discover new problems or new things about existing problem at hand which may require a course correction. These discoveries may affect not only the estimate but alter the entire path team is taking to resolve that problem. Hence assuming a linear trajectory for the project path in the sprint becomes really tough handoff between just getting the work done vs quality of work.
5. Oversimplification of Development processes -: the development process at the very least, is as much about humans as it is about work on hand. Its easier to assume that team would organize itself and the dependencies between work picked up by two or more than 2 individual team members get managed without any conflicts or creating further dependencies. Further, how should work be re-arranged when new discoveries are made?
6. Scrum inhibits deep thinking and innovation -: In Scrum, the gods of story points per sprint reign supreme. For anything that doesn’t bring in points, one needs to get permission of the product owner or scrum master or someone who has a say over them. Refactoring, reading code, researching a topic in detail are all seen as “not working on actual story points, which is what you are paid to do”. Specialization is frowned upon. Deep thinking, problem solving, research, are all considered wasteful activities when the same hours can be spent to code 2 more story points. Whatever technology one helps develop or is introduced to, s/he is not allowed to become an expert at it, because it is finishing a story that brings the points, not becoming an expert at it. These are all manifestations of the control mania of Scrum. Management uses Scrum to micromanage teams and individuals which leads to resentment and feeling of loss of autonomy
7. Unreasonable expectations -: Management unwilling to give the teams time to evolve around Scrum with the expectations of immediate results tied as a noose around the neck of team members. Team members are expected to move from story to story, from project to project with little time to develop expertise or finding that one area that truly interests an engineer.
8. Creating customer value -: Every feature and user story has to provide some kind of customer value or result in some benefit to customer. In reality though, there is a world of software beyond immediately recognizable value or benefit for the customer. Many software systems are complicated, integrated pieces of art which are working together as a cohesive unit, even when there are tens of things wrong that could break the code or stop the functionality. This is especially true in case of platform services which feature some kind of integration with Legacy systems. To keep the show going, there is behind the scenes work required to make the platform more reliable or resolve that one bug that has been carried on as technical debt, or tweaking some back end feature to display results faster. Does
9. Customer interaction -: Continuous interaction with customers is good but only to an extent. Not all team members are customer management experts, in fact very few technical people know much about managing expectations and are quite often not great communicators. This is not a negative, as its not a skill requirement for a great coder to be great communicator, however for someone dealing with clients and customers, good communication is must. Plus, when do you really draw a line between endless customer changes and delivering a final product.
10. Unnecessary cutoff from the world -: Submarine development (coding underwater for extended periods of time, surfacing only occasionally) is bad. Endless chasing of one feature list after other leads to burnout and lower morale.
11. Looking at the wrong details -: An aggressive focus on individual performance where it stifles creativity and detailed planning is often detrimental both to the individual and the team. When faced with such situations, people just try to finish the tasks to meet the sprint goals. The efficacy of those goals to meet customer expectations as well as quality takes a back seat.
12. Communication -: Each team member is supposed to articulate 3 things in the daily standups. It is a known fact that not all team members are equal communicators. Team members have varying communication skills – so how does everyone get an equal chance of being heard? And more importantly how do each team members’ point of view get voiced?
13. Lack of respect for team members’ seniority levels -: Respecting someone’s experience and skills is very different to creating hierarchy layers. Individual team member’s skill and experience can be rewarded in other ways than creating and awarding lofty titles. More often than not, such individuals with excellent skill, experience and track record of solid engineering background settle for autonomy to do their thing, getting involved in thought provoking research oriented work, higher end transactions, chance to lead client meetings, chance to train junior employees, etc. Basically anything which takes them away from daily grind and rewards them for their years of hard, superlative work. Scrum, on the other hand is often used as an excuse for lack of concern for people’s career objectives in work allocation. Imagine an engineer with 10 years’ worth of experience and an enviable track record, asked to code the same the same requirement as an engineer with an year’s experience. At any rate, that’s not a pretty scenario.

Image Credit: Internet

 

Is Scrum completely useless?

Most experts are not against every idea and practice in Scrum. For example, the principles of the whole team taking responsibility for the code base, or always having an integrated, working master are really noteworthy and something that all teams in all project settings can benefit from.

Software is research; its as much discovery and going along the unknown path as it is about following standardized, laid down paths. It’s a matter of discovering the solution, not rummaging through it.

Image Credit: Internet

 

Scrum, as many practitioners will tell you is far from useless. In fact, its simplicity makes it very popular among industry practitioners from all backgrounds. There are lots of other techniques that you can use with Scrum – user stories, continuous integration, burn-down charts, burn-up charts, estimation of tasks in hours, complexity points, etc. But none of these are mandated.

 

Image Credit: Slideshare

Conclusion

A lot of Scrum that is seen today is more along the lines of “This is a buzzword, I read a blog post and watched one of those YouTube videos…let’s go!” without understanding of the principles or how to implement those.

As one senior developer put it once: firms and teams are much better off and will have much higher chances of success if they attempt to answer these two questions honestly:

• Whats the team’s process from feature request to feature delivery?
• Why did the team choose that method?

 

Many in the development community particularly many old heads feel It’s time for this culture of terminal neglect of experience, low autonomy, and aggressive management intervention to pass over to something more adept at the growing needs. These folks feel that Scrum and Agile aren’t just bad ideas – they’re more dangerous than that, as an entire generation of new engineers and professionals using these techniques are getting their horizons restricted, having to absorb limiting methods such as Scrum and Agile. There are far too many young programmers being doomed to mediocrity by the idea that business-driven engineering and “user stories” are how things have always been done.

Image Credit: Dzone Agile

 

Agile and Scrum rose in prominence post 2001, when the growing requirement of immediate results just couldn’t be met through conventional waterfall project management techniques. Before that the software development field had persisted with same old conventional techniques for 3 or 4 decades.  Perhaps, just perhaps, the time for rollover to new methods has been shortened from 3 or 4 decades to 2 decades. Perhaps, Scrum has now become the new ‘old’ method of doing things.  Perhaps, its time already for newer methods to replace conventional Scrum.