Image Source: Internet

The Blitzkrieg

Major JFC Fuller, then 37 years of age, was posted at the Somme battlefield in France at the time of WWI in 1916. On that battlefield, Major Fuller observed for the first time the awesome power of the newest savagery in the war technology, known simply as the armored tank. Major Fuller seized immediately that this new machine, the tank, holds the answer to most perplexing tactical question in modern day warfare – how to cross an open muddy field, littered with trenches and barbed wire against a haze of blazing guns? No approach had worked so far, and even hundreds of thousands of brave men laying down their lives only had as much effect as millions of raindrops washing against stone façade. But the tank held the most promise. For, it seemed indestructible, carried more firepower and could march on undeterred in all kinds of weather and most all ground conditions. Major Fuller enthusiastically sent reports of the success of this new weapon to the English war leadership. To Major Fuller, the evidence of tank’s superiority was undeniable and hence there exists every reason for tanks to replace the archaic ways of horse mounted cavalry warfare.

All the major countries in World War I (1914–1918) entered in to the conflict with cavalry forces. German forces continued the use of horses on the Eastern Front well into the war while on the Allied side, the United Kingdom used mounted infantry and cavalry charges throughout the war.

The British war leadership was steeped thoroughly in tradition and failed to see the alternate methods, regardless of the pragmatism or inevitable tide of changing times. One British General compared the faces of soldiers riding horses to those riding tanks and quipped about the lack of intelligence on the faces of tank mounted soldiers. Not just the Leadership, many soldiers on the front lines who had never seen tanks in action were at best skeptical of the new beast.

Major Fuller sought transfer to the Tank division and went on to produce brilliant papers of how to break the German lines, destroy vital rail and road links, invade deep in to the territory and strike at the German war offices. A tactical approach aided by airstrikes and resting squarely on unarguably superior technology available to the British Army in form of tank will surely make quick work of the Germans, Major Fuller conceived. By striking suddenly at the German command, the Blitzkrieg will cause the German army to disintegrate and fall. Major Fuller didn’t give up hope and continued in his efforts undeterred. In late 1917, during the battle of Cambrie, the British war leadership finally gave in to Fuller’s persistent demands and decided to use 400 tanks to attack German front lines. Unsurprisingly the British tanks decimated German defense system and made quick work of the barbed wires and shrugged off lines of soldiers firing guns at the armored plating of the tanks. A measly top speed of 4 miles per hour was enough for the tanks to run through German war lines and trenches. The Germans were caught off-guard and outmaneuvered tactically and strategically. The soldiers who saw the power of tanks for the first time were awe-stuck. In what can only be dubbed as an irony, the British Army decided to send in horses to take advantage of the gaps created by the tanks. This non-sensical move allowed German forces to regroup and drive the British back. The momentum was lost. And so was the tactical and strategic opportunity. Major Fuller once again undeterred carefully documented the events, recording what worked well and what may be improved. His ideas were reluctantly adapted and dubbed Plan 1919, to be used in the year 1919.

Major Fuller’s work did not go completely unrewarded though. For his pioneering papers in strategy work, Major Fuller received many accolades and won the Gold Medal from a prestigious think tank of the day. The most important possible beneficiary of his careful and well documented work however remained cold. The British Army continued to give Major Fuller a cold shoulder. The most brilliant and accurate strategic work in modern warfare was seen more as a threat than an opportunity.

The beliefs of British war leadership were so deep rooted that the newly formed Tank core and rapid advances in tank technology throughout the war years amounted to exactly nothing. Before Major Fuller’s plan saw the light of the day, the war ended in 1918. However, that was not the end of tank warfare or for that matter the strategy of sudden, lightning paced attacks backed by airstrikes destroying vital road and rail links that Major Fuller had conceived. Exactly twenty years later, at the start of World War II, Germany used the same Blitzkrieg approach to effectively lap up entire Europe within a matter of weeks, almost unchallenged and nearly unstoppable. Despite possessing clear technological superiority and strategical advantage of having a brilliant war strategist in Fuller, the British squabbled away the technical and strategic momentum to German forces by late 1930s.  Major Fuller’s strategy proved right, not just right, in fact it was proven to be arguably the biggest breakthrough in war technology since the invention of guns.

Image Source: ideanote

How enterprises react to Innovation?

Major Fuller though is not alone, nor is the blissful ignorance of ground realities a trait reserved for British War Leadership. In 1970, the photocopying giant Xerox developed a state of the art research center in Palo Alto, California, called PARC, short for Palo Alto Research Center. PARC scientists quickly paid back Xerox by doing innovative work in laser printing that would establish Xerox as leader in printing technology for decades. Shortly thereafter Xerox scientists developed the first computer, truly ahead of its time. Steve Jobs during one of his visits to PARC was stunned by what he saw, the mouse and computer interface was truly revolutionary he felt. Xerox however had other ideas.  The same Xerox leadership team that led its PARC scientists to produce breakthrough in laser printing technology in 1971 and many other innovations, seemed equally capable of squandering away the strategic advantage held by true game-changer, the personal computer. Xerox was then dubbed as the company that fumbled the future.

In 1975, Steven Sasson invented the first self-contained digital camera at Eastman Kodak. Sasson’s patent claimed an arrangement that allowed the CCD to be read out quickly (“in real time”) into a temporary buffer of random-access memory, and then written to storage at the lower speed of the storage device; essentially all modern digital cameras still use such an arrangement. His was not the first camera that produced digital images, but was the first hand-held digital camera. 37 years later, in 2012, the digital camera technology became the prime reason for Kodak’s demise. Though Kodak did eventually market both professional and consumer cameras, it did not fully embrace digital photography until it was too late.

Image Source: NY Times

In 1999, Sony launched world’s first digital music player. Sony possessed the iconic and generation-defining brand Walkman and had endorsements of virtually every heavyweight in the music entertainment industry. Yet, within few years, Apple’s ipod defined the music industry, virtually destroying the Walkman promise. Sony worried about cannibalization and was slow to react, thoughtful and diligent at every turn. If it built a music player and service that made it easy for people to share digital songs, that might hurt sales of its own music records division, which had its own profit and loss statement. Apple on the other hand had one single profit and loss statement for the entire company. Steve Jobs’s business were simple and radical. Never be afraid of cannibalizing yourself. ‘If you don’t cannibalize yourself, someone else will,’ Steve Jobs said. Result is a bed of roses for Apple, while becoming thorns under the skin for Sony.

By 2013, Nokia had lost 4/5^th of its peak market capitalization in 2007. Customers were driving away in troves to competition. Nokia had ignored the glitz and glamor of Android, while it severely underestimated the new ecosystem. Microsoft lapped up Nokia’s handset business for a fraction of its value. However, unbeknownst to Microsoft, things had become so bad for Nokia that no amount of effort would be able to revive the brand. Microsoft’s own windows phone OS was in no way a challenger to Android – iOS domination. When Satya Nadella took over Microsoft from Balmer in 2014, he wrote off the entire $7.2 billion Microsoft investment in to Nokia, gave up efforts to review Microsoft’s 7 year old foray in to mobile phones and put the entire Microsoft mobile phone business on the chopping block, marking the end of a rather painful journey for Microsoft’s handheld devices business unit. This is a particularly hard pill to swallow as windows OS had won over many critics with its arguably superior interface when it launched in 2010. The initial success was short-lived and couldn’t be replicated to subsequent versions of both software and hardware.

Image Source: gsmarena

Could it all be a coincidence? The tank powered blitzkrieg, the underrated digital camera, the before-it’s-era personal computer and carry-in-your-pocket digital music player? Why do well-established, pioneering organizations lose out to maverick, new-comers? What powers the engine of growth in unconventional yet strategically sound products and technologies? Why does organizations get complacent and let upstarts overtake them? Why do leadership of these organizations fail to grasp the potential of emerging products and technologies in front of them? Isn’t guiding the organization through unknown times the primary purpose of bringing together individuals, otherwise known as leadership team? If the top organizations fail so miserably and so often, there must be some reason, some logical, rational explanation.

Answers to these questions are often hidden underneath layers of organization culture. Many modern business people, strategists and industry watchers coined the term ‘Disruptive’ and attached it to any new product, service or technology that sought to bring something new to the consumers. Disruption, in pure business terms is defined as an innovation that changes the business and industry dynamics in such a way that incumbent organizations must adapt to the change or fall by the wayside. In the face of disruptive technology or product, the incumbent organization needs to maintain its leadership status by embracing it as quickly as possible. If the organizations keep doing what worked for them in the past, they are more likely to fail as such disruptive forces demand disruption to the way of thinking and ethos of working.  “Disruption” in classic business parlance describes a process whereby a smaller company with fewer resources is able to successfully challenge established incumbent businesses. That sadly is not true of how market leaders work.

Disruptive Innovation and Architectural Innovation

The question is: why don’t organizations adapt? Its certainly not for lack of innovation. For kodak, Sony and Xerox were all highly innovative companies with zealous management teams. Then what made them lag behind and eventually lose the fight? This is where the theory of Disruptive Innovation pioneered by Clayton M. Christensen comes in. Briefly the theory of Disruptive Innovation suggests this: Specifically, as incumbents focus on improving their products and services for their most demanding (and usually most profitable) customers, they exceed the needs of some segments and ignore the needs of others. Entrants that prove disruptive begin by successfully targeting those overlooked segments, gaining a foothold by delivering more-suitable functionality—frequently at a lower price. Incumbents, chasing higher profitability in more-demanding segments, tend not to respond vigorously. Entrants then move upmarket, delivering the performance that incumbents’ mainstream customers require, while preserving the advantages that drove their early success. When mainstream customers start adopting the entrants’ offerings in volume, disruption has occurred.

While the incumbent leader organizations are looking elsewhere, the newcomers arrive, unburdened by legacy, take a half-baked product or technology and make rapid progress carving a niche market and gaining foothold in the industry to displace the incumbent.

Image Credit: HBR.ORG

The theory of Disruptive Innovation explains as much as it leaves out. The theory is certainly valid, and elegant for the most part. Christensen has a single clear idea of how disruption happens — and recommends a solution, too: disrupt yourself before you are disrupted by someone else. However, stretching it to fit all scenarios is at best a naïve attempt at explaining the why and how of how people work.

Kodak, Sony and Xerox were all highly innovative companies, each possessing an enviable track record. The technical teams at these organizations boasted of some of the sharpest minds, while the business leaders were equally brilliant. The leadership teams at these organizations could see what lay ahead.  As with innovation, the lack of vision could not be a factor. They could articulate the challenges of the times ahead and the promises of untested technologies. Yet, they were unable to put together a cohesive response strategy. It seemed no one at the helm could do the right thing. Where does this inability to lead the tanks in place of horses stem from?

The theory of Disruptive Innovation was not new in 1995 when it was first proposed, or over two decades when it was further developed. Perhaps the ideas were old, only the changing global nature of businesses made the traits ever so apparent. Or that disruption has been happening forever, we are just starting to recognize it now. Or perhaps, disruption is the normal.

When a company discovers or arrives at a successful business model often following years of painstaking work, management are given the explicit mandate to exploit that advantage to its fullest extent. This invariably means that most companies are structurally geared to manage, protect and nurture their currently successful business model. All the company’s assets – structures, operations, human resources, processes, tools and culture are geared towards doing what they have always done – protect, grow and nurture its current strengths.

It is no surprise then that swords are pulled out when there is even a shadow cast on the company’s current affairs. Any harbingers of change, which bring a radical suggestion or new idea, no matter how sound, or logically accurate, tend to be at odds with almost the entire company. This is not necessarily bad – companies do need to exploit their current positions as this is where their revenues and profits are coming from. The mistake organizations and leaders make is to focus exclusively on exploitation while ignoring most other ideas.

In the quest to understand the behavior of leaders better, theory of Disruptive Innovation does seem to fall short. Its true that Disruptive innovation changes the marketplace, however it doesn’t speak to why the incumbent organizations fail to take action? Or why the same organization with brilliant track record at innovation suddenly stops innovating?

Rebecca Henderson and Kim Clark postulated that unlike what is suggested in the theory of disruptive innovation, there are multiple points of failure where an organization fails to seize the opportunity. These points may exist all along, up and down the organization in no order. For example, in JFC Fuller’s case, almost every branch, company and division of the armed forces had little faith, mainly because many had not seen the tank in action. Questions on its size, slow pace, cramped insides, and unsightly presence were all valid, yet short-lived.

Then there are challenges about the financial viability of a product, or about its perceived value to the company, or about its future.  An architectural innovation challenges an old organization because it demands that the organization remake itself. The simple explanation is that a market leader in producing printers is much likely to accept breakthrough innovations in printer ink technology as there is no real organizational stress in pursuing that product line, and highly unlikely to accept the idea for a personal computer as there is little organizational mechanism for paying attention to the innovation and nurturing it along.

Within the camera business — Canon and Nikon made the transition to digital technology successfully while Kodak could not. Was that because Kodak was hidebound and clueless about digital technology? Not remotely as Kodak entered the digital market very early and with some early successes. What killed Kodak, though, was that it hadn’t really been a photography company for a long time, rather it was a film, photo paper, and chemical company.

The message of Henderson’s work with Kim Clark and others is that when companies or institutions are faced with an organizationally disruptive innovation, there is no simple solution. There may be no solution at all. “I’m sorry it’s not more management guru-ish,” she says. “But anybody who’s really any good at this will tell you that this is hard.”

Tesla’s solar powered vehicles, Tesla’s SpaceX, Tesla’s home solar program are all examples of what happens when organizations shun perfectly valid ideas or technologies for lack of viability, and lesser known players enter in to the market to fill that small gap of that niche product. Tesla started off as a ‘niche’ EV manufacturer, in a market segment which itself was considered a joke by several top automobile manufacturers, and in little more than 1 decade, managed to claim the spot of most valued US Automobile brand ever. And this feat is even more enviable considering that Tesla manufactures all of 3 vehicles today. That is 3 vehicle models competing against 100’s of competitors’ models, using technology which is barely two decades old against almost 120 years of development in conventional gas powered automobiles.

Oil Industry has dominated the game for almost a hundred years now. Yet, the implications for the Big Oil are really very straightforward. Adapt and invest in clean fuel or simply rollover. And that is not an exaggeration by any means. The writing has been on the wall for some time now and the Oil Industry is cognizant of the same.

Image Source: Forbes

Conclusion

The thing with new, game-changing technologies, products or ideas, is that to thrive it needs to find an organization that will accept it. Adaption of any new technology has severe implications; it not only changes the organization but often times creates a new industry or segment altogether. The tank changed the modern warfare forever; Netflix ushered in an era of online streaming; personal computers took computing out of huge air-conditioned rooms to homes; iPods created an entirely new marketplace for music; and the digital cameras brought photography to 3 year old and 80 years old alike. These are all game-changers, whose potential was not unknown to their parent organizations, yet it took alien organizations to realize their full potential.

Only those organizations which are truly willing to change themselves, reorganize and adapt, re-skin and lose its earlier identity, and let that idea, or technology or product guide it to the future, those organizations mature enough to understand that immaturity is a gift, those are the organizations that create unparalleled wealth and unimaginable success stories.

Introduction

Any IT product, application or software development effort is made up of stakeholders who are invested in the success of the product. These stakeholders typical comprise customers or users, executives, development team comprising architects, product owners, analysts, Ux/ UI developers, software developers, testers, etc.

In conventional waterfall projects, the teams above form strict structures and protocols for delivery of the desired work. Waterfall stresses on strict Finish to Start approach where end of a stage is prerequisite for subsequent stages to start. In Agile approach, the same structures are loosely defined and followed, while protocols are generally flexible and come with only ‘recommended’ status. Further, Agile is typically defined by iterative, small pieces of functionality delivered rapidly and in close collaboration with the customers and / or end users. Agile doesn’t follow Finish to start approach rather stressing on parallel tracks of work which can be finished separately or together.

In Agile projects, Product owner or product manager is responsible for maintaining the details of what is being built. The amount of work to be done in a project is recorded in what’s knows as a product backlog. Product backlog is a list of features and user stories required to complete the work in a typical Agile development. It is the product owner’s responsibility to work with Development team to organize the product backlog in to a list of prioritized features and user stories which dev teams then use to pull sprint backlog and deliver useful, working increments iteratively. Product backlogs features and user stories are continually revised, updated and modified as project proceeds along and as more clarity is achieved through a collaborative approach.

Image Credit: Internet

Scrum vs Kanban

Scrum and Kanban are two of the most popular approaches. Both the approaches have lot in common, yet both have points of divergence as well. Many experts consider Kanban as the methodology closest to the spirit of Agile principles. It is often contended that while Scrum does contain vital benefits, like continuous feedback loop and the ability for teams to self-organize independently, these benefits are effectively absorbed by the self-arrangement features of Kanban.

In Scrum and Kanban both, the focus is on efficiencies and transparency. Both the approaches are Agile and Lean, and favor using clear, most obstruction free path. Scrum and Kanban both target to deliver usable software early, efficiently and as seamlessly as possible, through breaking features into easily manageable pieces of work. Further, both Scrum and Kanban feature self-organizing, learning teams which evolve with time.

Scrum teams work in a series of sprints. Each Sprint is characterized by certain ceremonies and artifacts. Scrum team typically means the product owner, development team and scrum master. In many cases, depending upon the composition of the team, testers, Business Analysts and in few cases, devops personnel, are considered part of development team. Before each sprint, there is a sprint kickoff meeting, which is attended by scrum master, product owner and the development team. The development teams consult with the product owner to discuss the items on the backlog to identify and prioritize the work they can deliver at the end of the sprint. The selected items become the sprint backlog. And the sprint backlog becomes the sole focus of the development team for the next two weeks. This sprint backlog is held sacrosanct and as far as possible no new items are allowed unless under exigent circumstances. There are occasions when dev team is not able to complete all the prioritized work in the sprint. In such cases the incomplete items are transferred back to the product backlog at the end of the sprint to be taken up during subsequent sprint or re-prioritized for future. During the sprint, which is typically 2 – 4 weeks in duration, daily stand up call is organized by the Scrum Master where each development team member is invited to cover three things – what was done yesterday, what will be done today and any blockers / risks / issues. It is the Scrum Master’s responsibility to record all blockers / risks / issues and seek resolution to clear development team’s path. Scrum Master is also responsible to maintain the decorum by ensuring daily stand up meeting takes place daily, that most members participate and speak up and the duration of meeting doesn’t exceed 15 minutes.

At the end of the Sprint, Sprint Review (or Sprint demo or showcase of new functionality) meeting and Sprint Retrospective are held to present what the team has delivered and what are the lessons learnt respectively to ensure that next sprint is more efficient than the last. It is generally a good practice to document these sessions, especially where customer sign-offs need to be obtained.

Image Credit: Internet

In Kanban, there are no defined Sprints and there is no defined ‘prioritized backlog’ required to build an increment of working software at the end of iteration. The development team works in a continuous manner, as per its capacity. As the dev team works on the prioritized product backlog directly, there is no separate sprint backlog. Dev team keeps pulling items from the prioritized backlog as soon as one item is finished or any capacity becomes available. The capacity of dev team in each phase determines the number of user stories or features it can pull to work upon. During each phase of development, i.e. build (coding), testing, and marking items as done, movement of backlog items frees up capacity in the preceding phase. As capacity is freed up in one phase, pressure is on the preceding phase to move work up. This ‘pull mechanism’ exerted by each subsequent phase on the previous phase results in movement of work through phases.

Kanban has no time-boxed iterations or sprints, and as such doesn’t place a hard limit for each iterative delivery or increment of working software or any improvements that are being targeted as goal. Kanban teams generally work as per the workflow, which are defined as goals. The team expects to make continual improvements in an evolutionary manner as the teams get more mature.

Image Credit: Internet

In each Kanban board, there are specified columns. Under each column is a limited set of colored notes that signify the tasks assigned to the given column as per the workflow. As studies have noted, 80% of information is gathered visually, which makes the Kanban board a powerful tool for noticing and remembering the things that must be done.

Further, under Kanban, no set roles are defined. Practically speaking, it makes sense for someone to serve as a product owner, project manager or supervisor, especially for medium to large projects which are more complex, but the roles may also theoretically evolve with the needs of the project and the environment. A Kanban team is not required to be cross-functional since Kanban workflow is intended to be used by any and all teams involved in the project. Therefore, a team of specialists and a separate team of generalists may be working on different aspects of the same Kanban project from the same board at the same time, all in order to pursue the defined goals.

Unlike Scrum, where the focus is to produce an increment of working software, in Kanban, teams strive to achieve goals (complete workflow states) and reduce the amount of time to complete the entire process. A reduction in the average timecycle may be one of the indicators of success.

Under Scrum, work in progress is limited in each iteration. The team has committed to the number of tasks or user stories and has adapted that scope as sprint backlog. This is the scope that the team is ready to accomplish during the Sprint. All the items can theoretically move to the Work-in-progress section simultaneously. In Kanban however, the limit for each stage of the workflow is defined and noted on the Kanban board clearly. Kanban limits work in progress per workflow state to this number, which is nothing but the capacity of the project teams working on the Kanban board. For example, if there are 5 developers, and each developer has committed to 1 user story or task, then number 5 is denoted on the Kanban board to indicate the maximum capacity available in that phase. Consequentially, there shouldn’t be more than 5 items in the particular phase.

Under Scrum, only the development team can edit the sprint backlog once it has been committed. In Kanban, the Kanban board may be edited by a Product Owner. According to Essential Kanban Condensed Guide, Kanban has evolutionary defined two “hats” that the team members can wear: Service Request Manager and Service Delivery Manager. The “hat” of Service Request Manager is an alternative to the Product Owner. In Kanban, there is also a culture of slack resources, or free flowing resources who can don generalized or specialized hat to help resolve bottlenecks, as and when needed. For example if a resource has completed his / her tasks, s/he is free to move to help another team member on a task which is in blocked state or on critical path. Equally well, the free resource can choose to take up a fresh task or user story from the backlog queue.

In some Kanban teams, there is an additional section on the Kanban board, an Urgency section, which is typically represented as a swim lane. This section may be used for an unpredicted urgent task from the Backlog or a bottleneck task from the board. In such an event, the urgent or bottleneck task is moved to Urgency swim lane where it becomes the first priority for the team.

Image Credit: Internet

Conclusion

Scrum and Kanban are often used interchangeably or thought to be two sides of the same coin. As both approaches are used to drive efficiency and utilize product backlog and common terms, the different nature doesn’t become apparent to a layman. In reality however, there are significant differences between these two methodologies. An often raised complaint about Scrum is the length of its time-boxed sprints and the rigidity through which time-boxed iterations are imposed, which are considered too long when employed with startups. The main criticism stems from the fact that lengthy, rigid sprints lead to infrequent releases, which can cause work teams to drag their feet or become accustomed to slow pace when responding to the needs of customers. Similarly, in case of undersized sprints, where the scope of work for each release is large, the larger features need to be broken in to smaller iterations, which are unlikely to deliver full value to a customer and might end up confusing the customers and end users. The set time-boxed lengths of Scrum were designed to offer consistency, and may not always be useful in a world where technological innovations move at a faster rate than before. Kanban tackles problems raised in Scrum with a different scheduling protocol where instead of operating with time-boxed sprints, Kanban restricts the number of things that a collective can focus on during any particular time span. Hence, the benefits of Kanban are twofold: Organizations can get more response from the marketplace, and they’re able to adjust to the demands of that input with greater agility. In that sense, Kanban is often considered the most close to Agile spirit.

Understanding these differences is key to choosing the path that will work best in a given environment.

Introduction

A little more than a decade ago when the term Cloud, Cloud computing, and other terms associated with enterprise IT systems started to become part of technical discussions and mainstream IT topics, most IT professionals and managers, let aside outsiders, acted more in disbelief than with optimism. The initial reactions to Cloud computing and what it represented were mostly hostile. Instead of seeing Cloud computing for what it was and what it could become in the future, most IT managers were busy finding issues and raising concerns, mostly out of their self-induced paranoia of letting go of their little control areas. Most IT professionals viewed Cloud computing as a threat, something which would, if implemented, take away their most important assets – physical computing resources and put those in hands of an outside organization.

Fast forward 10 years, there has been a remarkable adaption of Cloud Computing services and its various sub-sections across IT industry. This remarkable adaption and emergence of market leaders AWS and Azure has fed a potential frenzy of activity around Cloud computing. The initial feelings that ranged from outright debunking Cloud’s potential at worst to careful cautious optimism at best have been left behind in dust long back. More than a decade or so later, Cloud computing has come a long way.

Image Source: World Informatix Cyber Security

Need for Security

The clouds surrounding Cloud Computing were as ominous then as they remain today, at least from security point of view. Many debated if it was possible that someone outside the organization be responsible to manage the IT infrastructure’s needs that the organization runs on? Could someone outside the organization be responsible to setup, operationalize, manage and provide security for organization’s IT and digital assets stored using IT infrastructure? The first and foremost concern expressed was around the security. As with any maturity model, the debate picked up and became a part of mainstream discussions as the industry moved towards gradual and in some cases scaled adoption of Cloud. More than a decade later, the debate is still raging. Security remains the primary concern, though the ecosystem itself has grown leaps and bounds and shifted in direction far, far away from the humble beginnings. In many ways, it is fair to say that while the ecosystem has completely transformed to the point that it bears little resemblance to what it was 10 years ago, the debate on Security has retained its position at the top of the concerns expressed by IT managers.

Image Source: Stratosphere Networks

Most organizations, IT included, recognize data as the most powerful resource today. Due to interconnected everything, every single swipe of finger or each step during the morning walks, or what one buys, how much, from where, at what price and at what time of day, every single aspect of life is being recorded and getting stored somewhere in deeper and deeper oceans of information. This massive amounts of data generated everyday by the myriad of systems in turn is being used to start controlling human decisions at first, providing intelligence and inputs to assist in human decision making process with the stated or unstated aim of taking over the decision-making process completely. Other than that, possession of the data has more immediate, financial and tangible benefits associated with it.

Image Source: IBM

The value of data has given rise to a host of platforms, systems, rules and standards aimed at keeping this treasure trove of information secure from falling in to wrong hands. With high-profile data breaches continuing to occur with alarming regularity across industries, IT security professionals are revamping their strategies to stay a step ahead. In fact, the security environment has become increasingly hostile, and the threat models so varied, that few in-house IT teams have the resources or bandwidth to keep ahead of the curve and protect their data systems.

Image Source: Sumologic

Security in Cloud

The security apparatus applied includes physical infrastructure like secure siting, resilience against natural disasters, and other specialized hardware; software and applications such as network stacks, applications exposed to internet, etc; logical assets like processes and policies, disaster recovery plans, etc; and non-tangibles like skills and experience of security professionals. Every single layer of the stack is a potential vulnerability, and responsibility of IT to secure it. A weakness at any point means that someone can gain a foothold into IT infrastructure and use that leverage to move around, find further vulnerabilities, and inflict damage.

Image Source: DataFlair

The attackers with malicious intent are getting savvier by the day. Any new advancements in security technologies is met or at times even exceeded by counter-measures from perpetrators of attacks on IT systems. This means that security measures and features must be constantly revamped and upgraded. Today, only the world’s most sophisticated businesses can deploy the technology and staff needed to counter today’s threat environment. And the deeper a company’s footprint across the technology stack, the more complex (and resource-intensive) this effort becomes.

In the case of a recent, well-known breach at Target, perpetrators got in due to low security measures implemented on the systems embedded in the air conditioning control systems and used that access to penetrate more valuable, vulnerable systems on the network.

The many advantages of adapting and using Cloud as the primary answer to meet IT needs have been discussed widely. These range from financial benefits – lowering cost of ownership and maintenance,  shifting infrastructure costs from capital to operating on balance sheet, lowering upfront expenditure, and better ROI moneys invested. These financial benefits provide much needed business flexibility that comes with Cloud computing. More benefits, perhaps equally significant, range from quicker response times, to elasticity, and to scalability that are intrinsic to on-demand computing resources. Finally, a significant benefit not often discussed in business case studies is that cloud infrastructure also delivers a much resilient and stronger security stack.

Image Source: DZone

The wide range of services offered by cloud computing companies can be categorized into three basic types:

• Infrastructure as a Service (IaaS). IaaS provides users access to raw computing resources such processing power, data storage capacity, and networking, in the context of a secure data center.
• Platform as a Service (PaaS). Geared toward software development teams, PaaS offerings provide computing and storage infrastructure and also a development platform layer, with components such as web servers, database management systems, and software development kits (SDKs) for various programming languages.
• Software as a Service (SaaS). SaaS providers offer application-level services tailored to a wide variety of business needs, such as customer relationship management (CRM), marketing automation, or business analytics.

Cloud infrastructure employs a critical approach better known as “Security by design”. This approach is intrinsic to modern cloud architectures. The logical controls and policy based separation among infrastructural components ensure a level of compartmentalization and segregation that few traditional data center platforms can offer. This segregation keeps compartments secure and agnostic to each other in the event of threats and potential attacks. Thus, organizations employing cloud computing begin with a much higher security standard and employ that standard as a baseline and not an aspirational level, across the entire perimeter.

Perhaps  most importantly, as cloud computing platforms are marketed and run as services, not just code, operational security — whether it’s resilience against DDoS (distributed denial of service) attacks, proactive detection of malicious code, or ensuring the integrity of messages in transit — has become a fundamental quality of any cloud computing business. Security is paramount and it has never been more in focus.  This requirement of providing higher levels of security alone enables the information security teams to move away from a primarily reactive (and losing) position of plugging the holes towards assuming a proactive, aggressive stance that’s fundamental to an organization’s overall IT strength and technology expertise.

Image Source: SDxCentral

The focus and shift towards security, making security the prime area of strength which defines almost all other parts of IT strategy alone represents a major shift in the quality of security of technology services. And, as cloud infrastructure providers now have the scale and importance to attract the best security and reliability engineering professionals in the world, the standards are continuously being refined, getting better with each iteration and with each attack. But there’s another factor at work, as well.

Security in Cloud – A shared Responsibility Model

The way cloud security is delivered largely depends on the individual cloud provider or the cloud security solutions enterprise has chosen. However, implementation of cloud security processes is always a joint responsibility between the business owner and solution provider.

Cloud service providers treat cloud security risks as a shared responsibility. In this model, the cloud service provider covers security of the cloud itself, and the customer covers security of what they put in it. In every cloud service—from software-as-a-service (SaaS) like Microsoft Office 365 to infrastructure-as-a-service (IaaS) like Amazon Web Services (AWS)—the cloud computing customer is always responsible for protecting their data from security threats and controlling access to it.

Image Source: McAfee

Conclusion

Its not difficult, nor it is far-fetched idea to imagine that Cloud is the way of the future. In fact, cloud is the future of IT. Cloud has already become a part of most enterprise’s tactical and strategic IT plans, while others are getting there in their adaption of Cloud. It’s not a far-fetched technical story any more requiring the attention of only IT department, it’s at the center stage of revamping each organization’s arrangement of IT infrastructure and the way IT is run and managed. Security has remained and is still the most basic yet the most important challenge, and the entire industry.

Image Source: PRNews

Today, most organizations view cloud-based services as a better way to deliver the services they need, including data protection and security. The risks and costs associated with maintaining self-managed, in-house IT security teams who manage organization’s IT infrastructure as a strategic answer to address security concerns is prohibitively high. Add to it the need to constantly upgrade skills and knowledge levels to respond to ever changing security landscape, changing and ever deepening nature of attacks, and redundancy of IT equipment at a faster pace than before, make it a luxury that many cannot afford. That’s precisely why most IT teams today understand that self-managed infrastructure — whether in-house or in third-party data centers — is almost always more tedious, less secure and more expensive than more modern, cloud-based alternatives.

When working on an Agile project, there is continuous discussion about development processes and timely, iterative delivery to customers. For the most part, Testing is construed, without much thought, to be an ‘included’ part of Agile Development process, just as design and requirements analysis are. Yet the domain of software testing is often ignored, or at the least not paid an equal attention.

There are multiple approaches or frameworks within Agile like SCRUM, Scaled Scrum, Kanban, XP and others. As the complexity of software development processes is increasing continuously, the software testing processes also need to evolve and mature to keep up with the development approaches. Agile testing is a relatively newer approach which focuses on testing smarter rather than investing considerable efforts, all the while delivering high-quality results.

The testers and developers are required to put in higher level of collaboration in Agile Testing. Agile testing is all about seamless integration between development and testing processes; the testers provide corrective feedback to the development team during the development cycle, while the development team reduces probability of high number of defects. Agile testing is a continuous process rather than being sequential. The testing begins at the start of the project and there is ongoing integration between testing and development.  The common objective of agile development and testing is to achieve higher quality product, free of defects and errors.

Agile Testing explained: What IS NOT Agile Testing?

Agile methods have been growing in popularity since the first formal adaptations in late 1990s to early 2000s. Agile has its own fan clubs and is fragmented in to practice areas, from simple, single team based Agile Scrum to complex models like SAFe Agile. Besides the more popular Agile methods, there are relatively lesser known yet equally effective methods like Crystal, DSDM and Feature Driven methodology.  Regardless of the method, all Agile methodologies share much of the same philosophy and features – regular feedback loops, speed, flexibility, and above all, regular communication and interactions between stakeholders, whose roles have been clearly defined and limited in all Agile methodologies.

Image Source: Clarion Technologies

Traditional testing practices have taken their lead from the Waterfall model of software development, where requirements gathering, software design, and implementation follow a deliberate and often lengthy path. Waterfall methodology has fallen out of favor recently, as Agile is becoming increasingly popular. Despite the tide of time in favor of Agile methods, waterfall testing still exists and in many cases is seen as flourishing. The ‘waterfall testing’ in an Agile implementation is particularly like an island in the middle of water which steadfastly holds its own.

The lack of Agile testing strategy is particularly characterized by two kind of practices. First, reliance on testing as an end point to development. Many development teams still consider testing as a conclusion to development stage. This promotes the view that testing must test the entire user story or feature or MBI or increment produced in its entirety. Secondly, testing at the end of development cycle is seen as merely a rubber stamp activity. Development teams often give impression that they have coded what needs to be coded and testing is merely a formality.

Today as seen in many Agile implementations, development may have moved to Agile however ‘testing at the end of development cycle’ continues to exist in its previous avatar, clearly meaning that Testing has lagged. This kind of hybrid Agile is being followed everywhere, where development stages include everything from high level design to business analysis to client collaboration, only excluding testing. Reasons can be many, perhaps out of convenience and / or ignorance, or perhaps due to simplicity.  Perhaps the people did not understand the difference between Agile testing and Waterfall testing, hence just carried over many of the same documents and processes as used in Waterfall. In most cases, testing is simply seen as a control point, or check-gate, rather than treating it as it should be, a value addition activity.

Agile Testing explained: What IS Agile Testing?

So what is Agile testing again? For some reason, people often treat Agile testing as a black box, not knowing what to expect. Agile testing is not a ready made approach, or a set of pre-defined test cases, or pre-determined set of processes. What Agile testing is, like all other Agile processes, an adaptive approach depending on the project and the functionalities being delivered.

There is inbuilt uncertainty in an Agile project. When an Agile project starts, what the final shape of product or what will the system look like, is not known. Agile projects begin with requirements which continue to change till the time product meets acceptance.  As a project moves forward, the risk profile of the project changes. At the beginning of a project, the risk profile is higher and as the project moves forward the risk profile grows conservative as there is more at stake during later stages. As the team learns to work together, the team and project dynamics change over time. The timescales change over time; WIP inventory and backlog continues to change over time. Similarly, Agile testing approach also must undergo a change to meet the changing, adaptive requirements of the Agile project. Whatever is the approach being adapted for development processes, testing should follow the same approach.

Image Source: Medium

The below are some rules that may be followed to define Agile Testing approach for the project or the function:

• An iterative approach to test software or product being built to aid meeting project goals
• Less focus on framework and rigid structure; higher focus on adapting to project requirements
• Risk profile -: Agile testing process must adapt to changing project risk profile. Test cases need to build in the amount of risk as per the risk profile at different stages of the project.
• # Defects -: Amount of testing work can change over time as the knowledge of developers increases with time. This may require changes to the testing approach.
• Systems architecture -: may change over time as project moves along. Testing team needs to build or import capabilities to meet this changing profile.
• Timescales -: The timescales themselves change over time. During certain stages in a project, a feature may span multiple sprints, while in other stages there may be multiple features delivered in a single sprint.
• Extra reporting -: another aspect of testing is the usual practice of publishing reports, test plans, templates, processes, etc. The reporting output must adapt to the project and its environment. If the project is unique, documentation may be more than other projects.
• Documentation -: By nature, testing is a process heavily reliant on documentation. This doesn’t mean however that documentation should be a rigid requirement. Changing documentation approach to adapt to project is a facet of Agile testing. For example, if the team uses JIRA or any other tools, then testing process needs to adapt to the same. If the team is using github to upload the test cases, then that’s the approach testing should follow.
• Communication -: there is greater responsibility to use variable communication as demanded by the project environment.

Agile test strategy Is a critical input for not only running effective Agile development processes but it provides excellent support to organization’s DevOps initiative, and the benefits of Agile testing extend beyond the immediate benefits of continuous testing. And continuous testing is vital to improving product quality.

As compared to testing at the end of finishing development, Agile development stresses on early and frequent testing of finished and unfinished code changes. In the world of Agile, testing needs to happen early and often. So, instead of waiting for development to be finished before testing begins, testing happens continuously as components or tasks on a particular user story or feature are completed.

Agile Testing Methodologies

Image Source: Infobeans

Test-Driven Development (TDD) − Test-Driven Development (TDD) is based on coding guided by tests. The word “test” in Test Driven Development may be misleading. The primary goal of Test-Driven Development is to make the code clearer, simple and bug-free. Test-Driven Development starts with designing and developing tests for every small functionality of an application. In TDD approach, first, the test is developed which specifies and validates the intended function of the code being written.

Acceptance Test-Driven Development (ATDD) − Acceptance Test-Driven Development (ATDD) is based on regular communications between end users or customers, developers and testers and driven by pre-defined Acceptance Criteria and Acceptance Test Cases. The thought behind Acceptance-test-driven-development is that user or customer perception of the system or product is just as important as its functionalities, so this perception is ideal to drive performance in order to help increase chances of adoption. To give this idea life, ATDD collects input from end users / customers, uses those input points to develop acceptance criteria, translates that criteria into manual and / or automated acceptance tests and then writes code against those tests. Like TDD and BDD, ATDD is a test-first methodology, not a requirements driven process. Acceptance-test-driven-development generally goes a step further than the regular Test-driven-development or Behavior-driven-development approaches, as it directly goes to the source or – end users and / or customers, to understand the rationale behind the product and how it will be used.

Behavior-Driven Development (BDD) − Behavior-Driven Development (BDD) testing is based on the expected behavior of the software envisaged by the project team. BDD approach is based on same principles as Test Driven Development approach, but instead of unit tests, it calls for testing at higher level, usually at business level. Instead of starting with a technical-facing unit test as in the case of Test driven development, BDD starts with an initial requirement based on end user or customer behavior and calls for tests that are “human readable” and can even replace some requirements documentation. This requirement is based on behaviors that the product should exhibit, creating an airtight guide for engineers to use as they develop tests.

Exploratory Testing -: Exploratory testing gives the testing team and testers an ownership over the code to test it in an organized, yet chaotic way. In exploratory testing, testers are not following documented test steps, but rather exploring the vulnerabilities in the software in imaginable and clever ways to try to break it. Testers document defects as usual, but detailed documentation of what and how the application was tested may not be always provided.

Exploratory testing doesn’t follow a set script or test cases. Rather, it’s about developing the most intuitive tests based on each unique functionality of the software. Because of its unconventional approach, exploratory testing often mimics how users or end customers will interact with the software in real world.

Session Based Testing – Session based testing is a step further than exploratory testing. Session based testing develops on the exploratory testing approach by providing more structure. As exploratory testing is completely unscripted, it relies heavily on the skills and experience of the testing team and testers involved, which is not the best fit always. Session based testing provides structure and accountability by developing a charter for testing and running testing in time-boxed intervals. Additionally, testers are required to report on the testing that took place during each session.

Conclusion

The test process is not a separate process that can be outsourced or left alone, it is merged with the development processes. The true benefits of Agile testing are revealed when it is used as part of Agile development and accorded an equal importance. The sooner testing team and testers get involved, better it is for the project. Ideally, testers should be part of the overall scheme of things from day one, simply because giving testers a seat at the table from the start provides a higher level of insight into requirements and goals, encourages collaboration and helps hammer home the need to conduct frequent, continuous testing.

Introduction

A mindset, or state of mind, is a set of beliefs, thoughts, biases, assumptions and values held by an individual or a group of individuals that leads the individual or the group respectively towards the choices they make. The Humans are most comfortable in choosing paths or making decisions that confirm with their beliefs, assumptions, biases and values. The existing mindset is made up of all these and rewards itself through positive reinforcement by guiding the individual or the group, in case of shared mindset, towards the path of least struggle. In other words, towards the path that is in conformance with the existing set of beliefs, thoughts, biases, assumptions and values. Simply put, it is a way of thinking about things and making decisions that those in a group share to the point that it becomes a way of life.

Flowing from the above definition, an Agile mindset can be defined as set of beliefs, thoughts, biases, assumptions and values that guide individuals and group members to make each decision to support an Agile way of working, or Agile work environment. Agile work environment at a broad level is made up of focus on collaboration, team work, flexibility to change, respect, communication, client-partnership, continuous feedback loop, learning cycle and fearlessness towards failures.

Lets not be fooled by the simplicity of the above terms. It is easier to define mindset and then attach Agile to it, or better still, define Agile terms around what should be done, or the ideal way.  In real world however, most people and teams practicing Agile struggle to understand or explain to others what constitutes an Agile mindset. Most of the times people hide behind thinly veiled excuses like, ‘Oh, I get it, but cannot explain it to others.’

Drivers of Agile mindset

The Agile mindset is the environment within which agile teams have their brightest chance to flourish. It is not laid out anywhere. It isn’t a prerequisite for an agile adoption, nor is it a standard which can be adapted for a functional agile team. This mindset is simply the best foundation on which Agile teams can propel themselves to success.

Specifically, an agile mindset is powered by 3 primary measures. First, an agile mindset always encourages early adaptation of Agile principles. Second step is continuous reinforcement and retention, while the third step is measuring metrics and driving compliance in day to day working environment and fine tuning approach as necessary. These three practices – adaptation, continuous reinforcement and retention and measurement of metrics are the key in any organization’s Agile journey.

Many enterprises begin their Agile journey with noble intentions. They want to improve their product offering to customers, improve their go-to-market or turnaround times, bring in internal efficiencies, reduce time between deployments, or all of these. However it’s quite common for many of Agile adapters to discover that they are not fully prepared for the Agile journey. This could happen due to one or combination of many factors. They may discover that employees do not fully understand Agile and any efforts to have them adapt Agile have not been fully successful. Or that the internal environment is not conducive for Agile way of working. This would primarily happen if one or more of key stakeholders or top level management is not a believer of benefits of Agile. In most cases however Agile adaptions fail because the organization tries to do too much in too little time. One of the great failure points of Agile is that Agile is not a magic wand. One cannot hope to implement few Agile practices or principles and expect overnight improvement. And when the improvements don’t come about, blame it on Agile. Agile is not a magic cure for years of inefficiencies or layers of bureaucracy in the organization or half-hearted efforts at implementation. Above all, Agile requires discipline and effort to commit to Agile principles and stay committed even in the face of less than expected benefits. In order to reap true benefits of Agile, it takes sustained and repeated training, high management focus, reinforcements and continuous practice of Agile principles. Remember: Agile is only as good as the people implementing it. So if you notice Agile is failing in your organization, guess what! Chances are something else is failing too.

Adaption of Agile methods is perhaps the easiest step. All it takes is starting implementation of Agile principles and the practices as per the chosen method. However, this is an inherent weak point as the adaptation needs to be accompanied with strong commitment as any halfhearted attempt will lead to disappointment.

The second most important aspect is continuous reinforcement and retention of Agile principles. Adapting agile in a Big Bang manner or as a quiet change is not worth much if not followed up with regular reinforcements through trainings, refreshers, communications and word of mouth messages. The Agile principles and practices need to be driven deep down to the deepest layers through untiring, unrelenting efforts by professionals like Agile coaches and leaders. Agile roles need to be strengthened and given enough autonomy to run things on the ground. Agile coaches must drive and ensure adherence to all Agile practices and ceremonies as per the chosen method. Above all, Agile must be made a part of, promoted and practiced as a ‘way of working’ and not as a special endeavor or initiative.

Whatever is the chosen method of Agile implementation– Scrum, scaled scrum, kanban, Lean or any other, the Agile coaches need to take the mantle of driving Agile principles. Constant reiterating is just one part of the journey. Teams and individuals must be constantly evaluated on their individual compliance with Agile principles and ceremonies depending on the chosen method. Individuals and teams lagging behind must be coached on the benefits of Agile and helped in order to bring them up to expectations. Remember: Agile adaptation is a disciplined team effort and reinforcement at individual and team level is necessary to enforce rigor.

And this brings us to the third part, specifically around measurement of metrics. Maintenance and adherence to product backlog, prioritizing sprint backlog, tracking efforts burned down and remaining during sprints, user stories closed and accepted by customer, are some of the metrics.

1. Sprint burn-down -: The sprint burn-down chart visualizes how many story points have been completed during the sprint and how many remain, and helps forecast if the sprint scope will be completed on time.

2. Lead time -: Lead time measures the total time from the moment a story enters the system (in the backlog), until it is completed as part of a sprint, or released to customers. It measures the total time for a requirement to be realized and start earning value – the speed of your value chain.

3. Agile velocity-: Velocity measures how many story points were completed by a team, on average, over the past few sprints. It can be used to predict the team’s output in the upcoming sprints.

4. Code coverage -: Code coverage measures the percentage of your code which is covered by unit tests. It can be measured by the number of methods, statements, branches or conditions which are executed as part of a unit test suite.

5. Cumulative flow -: Cumulative flow is a kanban metric which shows the status of tasks – in a sprint, a release or across software teams. It can visualize bottlenecks in the process – a disproportionately large number of tasks in any of the workflow stages indicates a problem. For example, a big “bubble” in the chart in a verification or testing stage indicates this stage has insufficient resources.

Image credit : Microtools

6. Failed deployments -: Measures the number of deployments (either to test, production environments, or both). Can help understand how solid environments are and whether teams are really building potentially shippable software.

Conclusion

Perhaps, what makes Agile difficult for lot of managers to grasp is that it’s not just a technology, methodology or process that gets implemented within an enterprise’s current culture and assumptions. The Agile way of running an organization seeks to fundamentally redefine the very concept of an enterprise which has prevailed for the last one hundred years.

Instead of an enterprise being conceived as an efficient, profitable steady-state machine aimed at exploiting its environment and existing business model and the industry its part of, the Agile organization is a growing, learning, adapting living organism that is in constant flux to exploit new opportunities and add new value for customers.

Introduction

Agile and Lean are some of the most commonly used terms in SDLC lifecycle. Often these terms are used interchangeably or as complimentary to each other. There are similarities as well as points of divergence between these approaches, and it takes years of practical hands on work to sort out the nuances.

Agile software development framework refers to 12 principles and 4 values that were put forth in Agile Manifesto which became the basis for an alternate method of software development to replace the waterfall methodology. Agile was primarily a reaction by the software community of engineers and sponsors who wanted to see faster progress and cut any bureaucracy, red tapism or flab in the system. The existing heavy-lifting style of waterfall development caused as many troubles as it resolved through delivery of the software systems.  Disconnected teams, uninterested project sponsors, disappointed customers, cost and time overrun, and a software system in disarray and out of sync with customers’ needs were quite often the remnants of a waterfall implementation.

Agile gained traction as a better answer for delivering software while Lean was adapted and refined on product success in conventional manufacturing. Design thinking is a relatively abstract approach for exploring problems and solutions with customer in mind.

In any business or services enterprise, blindly committing any model, framework, or methodology seldom delivers a positive result, let alone the desired result. Different LOBs or departments may have their own unique problems which go against the grain of ‘one solution fit all approach’. The need is to marry available choices with the needs of an organization in a way that each option selected returns the best possible ROI with acceptable levels of risk.

The intersection of Agile way, Lean manufacturing and Design thinking presents an interesting combination of techniques to progressively achieve improvements in software delivery. An unbiased examination of all three techniques in detail, potential and scalability reveals interesting areas of coupling, while there also exist enough reasons to adapt adequate cautions.

“Be stubborn on the vision, but flexible on the details.”

—Jeff Bezos

Agile, Lean and Design Thinking examined

Agile is our ticket to scalable, collaborative and customer-centric software development; Lean is the framework for achieving efficiencies and applying learning to achieve process improvements, and Design Thinking is an innovative approach to attacking problems through iterative exploration and application of learnings.

Design thinking is a way of getting to the solution of a problem through continuously questioning the previous outcomes. Design thinking often advocates going where the rules don’t exist and where precedents are unknown. Design thinking is more to do with learning ability than following a set path. A cook conjures up new dishes and a designer comes up with a new dress line in much the similar fashion. It’s through thinking about the user of the product and coming up with attributes  that would deliver the most value that designers come up with the most effective design elements.

Design Thinking often focuses on the subtleties of how deep look in to problems can lead to ingenious solutions. Abstract attributes like experimentation, dealing with ambiguities, empathetic learning and above all, a designer’s ability to make meaning out of otherwise ignorable facts, hold special place in Design Thinking. Simply put, Design Thinking is a study in to how problems are framed and analyzed and how an empathetic approach of looking at problems from customers’ or users’ point of view can often bring out subtle hints towards solutions. Design Thinking challenges assumptions, encourages innovative ideas, prototyping and experimentation, and defining users’ needs and pain points. Much of the Design Thinking process is similar to Agile, insofar as iterations, late changes, customer centricity and flexibility are concerned.

The heart of Agile way, irrespective of the chosen Agile method is to react to change. Agile is a dynamic approach to delivering software of value to the customer where the customers and engineers became collaborators to produce the software. Agile is highly effective in uncertainty and is the perfect answer to many professionals and designers’ worst nightmare of not being able to change the product design during later stages of development. Scaling Agile is a critical part of Agile implementation helping organizations address changing requirements without raising everything to ground.

Lean approach developed in the manufacturing world primarily as a cost saving tool. Based on cold, hardcore scientific methods of reducing waste through processes, rules, procedures and value addition, Lean was primarily control oriented. While Agile and Design Thinking advocate flexibility, less reliance on processes and empathetic learning, Lean often runs in opposite direction. One of the major drawbacks of Lean is that conventionally it often ends at process optimization, waste, and quality, and misses so much of what the Lean mindset offers potentially. However, Lean deals with uncertainty and advocates an adaptive approach, changing according to results of experimentation and learning. Lean framework also advocates empowering those closest to the work to decide the best approach of doing work under strict supervision and control. Hence, there are ample synergies between the Agile and Lean approaches.

The lean startup methodology was invented in Silicon Valley in the 90s, powered by the dot com boom but the conventional Lean has its roots in Toyota’s production system. Toyota’s lean manufacturing system was used to build things efficiently, yet it doesn’t tell what should be built.

Using Hybrid mix of Agile, Lean and Design Thinking

Many projects can and do fail despite best intentions. According to various studies, anywhere between 30-50% of IT projects fail to achieve their objectives. A study by IDC points out that 30-35 percent of IT projects still fail. Other research puts the figure upwards of 50 percent. Regardless of the study, the main reason consistently cited for the failure of software development projects to deliver their intended outcomes is misalignment with user and business needs.

Agile, Lean and Design Thinking place high focus on close collaboration with users and people closest to the work being done. While Design Thinking focuses on finding the right problem through empathetic approach and listening to the users, Agile places emphasis on collaboration and feedback loop, while Lean advocates placing power in the hands of those doing the work. Just like design thinking, a key tenet of effective agile application, irrespective of the Agile framework chosen, is to seek frequent inputs from end users and incorporate the feedback in iterations to achieve the right outcomes. Doing this during early stages of projects includes establishing project goals, defining business increments, writing user stories and / or features, establishing success criteria through definition of done and creating product backlogs from which iteration backlogs follow. Throughout the development process, the inputs through feedback process manifests form a crucial part of delivering the ‘right’ product .

While design thinking and agile can be applied separately, benefits’ realization can be higher if the two approaches are applied together, creating an environment where both approaches benefit off each other. This environment fosters a ‘customer/user first’ attitude and equips employees with means and methods to focus on user-centricity and rapid iteration as a means of reaching optimal outcomes. Design thinking brings a strong user focus while agile is an excellent way to deliver product incrementally, ensuring customer / user needs are kept at top and center throughout entire design and development phases.

For teams looking to leverage agile and design thinking for the first time, here are few recommendations to keep in mind:

• Fail early, fail often – No idea is a bad idea. During the brainstorming sessions or ideation phases, any criticism of any idea is simply not permissible. Each team member is free to put up their idea on the board so long as it’s a fair representation of users / customers’ needs. A critical part of Design Thinking is to come up with prototypes and working models, which are often very poor adaptation of users / customers’ needs however serve as a vital link.  The prototypes are shown to users / customers to get feedback and incorporate learnings. Poor prototypes have perhaps taught design teams more than any other source. The goal is to understand the needs of the users / customers empathetically.
• Low hanging fruits – By starting small and focusing on high-value, yet low-risk opportunities, confidence and experience both grow in applying principals of using design thinking and agile together. Then, as the capability matures, teams can take on more challenging initiatives.
• Create cross-functional teams – This is perhaps one of the easiest yet so-often-delivered-wrong concepts. By allowing team to leverage cross-functional viewpoints, for example, a developer may have critical feedback on UI, team leaders can tap in to experience from various domains. This not only fosters required creativity, but also creates a healthy understanding of each other’s challenges within the team. As far as possible, the team must be physically collocated with end users / customers to promote frequent collaboration.
• Balance design and development – Agile teams are often groomed to go in to an iteration mode, basically an all-guns-blazing mindset. And that’s not wrong. Typically, Agile teams learn and implement on the fly. Agile teams are often instructed to “just start coding”, fixing problems as they come along and gathering user / customer feedback along the way. Care needs to be taken while mixing Agile and Design Thinking as this may create tensions about how much time to spend on resolving problems before starting development. The team must understand that this isn’t a pure-play Agile implementation, or a pure-play Design phase which can take ages. The team must understand the value of the empathy, definition, and ideation phases, and that Design Thinking is not leveraged only at the front, rather it can crop up during any phase. In fact, the team must be prepared and encouraged to get back to the beginning at any point during project to uncover new user insights and / or reframe the problem, and then continue development with a renewed sense of “why”.

Conclusion

At a simplistic level, Design Thinking can essentially be described as a process that leverages divergent thinking (to generate ideas and get out of the box solutions), followed by convergent, team thinking to select solutions to pursue, all the while getting user / customer feedback. Design thinking is a mindset towards solving problems from human needs perspective rather than from a business perspective.

Agile is an umbrella approach encompassing variety of IT product development and business methodologies that has at its core deep, unwavering commitment to principles of highly collaborative, multi-functional, self-organizing teams. Agile isn’t limited to software or IT systems, but goes beyond that.  Agile’s minimal approach and focus upon working product or software as a measure of progress is in stark contrast with heavy lifting ways and upfront documentation before each stage of waterfall development.

Waterfall is called so because of the sequential, orderly tasks and downward flowing approach, where everything is supposed to be smooth, and one thing leads to the other. In real life, its rarely so.

Agile methods, of which Scrum is the most popular, bring in to calling flexibility, speed, and willingness to get things done. Many experts believe that Since agile was introduced in the development of software applications, both the overall quality and speed of delivery have significantly improved. It’s the quick gratification virtues of the Agile methods that have made the methods dear to many professionals in various industries.

However, Agile methodologies are focused primarily on developers and the iterative nature. The concept of Agile grew out of programmers’ attempts to resolve most common project management issues and pain points experienced during big software development projects. Unsurprisingly then, the Agile Manifesto (primary document delineating Agile principles) did not include the UX perspective, nor did it account for the time, resources, and research that UX professionals need in order to create trend-defining, excellent designs. For the much part, UX was taken to be part of development, as Design effort.

Customer Centricity, cannot and must not be equated with customer collaboration. Most people make this basic error, partly due to the fact that both terms contain the word customer, and partly due to the belief that involving customers in the development process takes care of all their needs and makes them delighted customers. The truth of the matter is that Customer Centricity is a far wider term which requires placing customers’ requirements at the center of all activities; while collaboration is just a tool available to implement the vision of Customer Centricity.

Image Source: SAP

Agile and UX: curving away from each other!

Agile is a mindset or approach about radical transparency and knowledge of the fact that the development team doesn’t know all the answers, or even most of the answers, at the beginning of a project.

An UX workflow outlines all the steps within the UX process, from UX research and gathering user feedback or behavior driven insights to defining design specifications, to low-fidelity wireframing and high-fidelity prototyping, UI / UX design, and performing customer or end-user usability testing prior to development.

Image Source: DIlbert

There is a tendency within Agile project teams to get ahead of the curve. At times, the developers run ahead of where overall team and the customer are in terms of discussion / elaborations / prototyping on requirements and how the final product will look and feel and subsequent sign offs. Perhaps the development team knows exactly what they’re making functionally, hence they feel comfortable, at times even overconfident, especially when re-writing a product that is already live (new technologies rather than a refactor). The customer may perhaps be expecting fine grained behaviors and focus on aesthetics, look and feel part, while unbeknownst to customers/ end users, the developers are trying to concurrently work on future functionalities of the system in order to get done as much as quickly as possible. Often this happens with the implicit knowledge of everyone on development team, with Business Analysts (and the Scrum master or PM) feeding back-end developers concise & defect-free stories to code. Developers hope to stay ahead of the curve in this manner, often working in a factory-manner, producing lines of code hoping that the customers’ expectations remain the same. And while by doing this, development teams may succeed, especially if the measure of customers’ success is through produced lines of code, where they often fail is meeting the more human expectations.

In fact, in projects where the Design teams are involved, the rigid way in which Agile is often implemented in medium to large organizations and / or projects (typically, projects driven using Scrum or similar methods) makes things even more challenging. Two-week iterations force tunnel vision on the design team, who are often tasked with making backlog items ready for development often at the cost of due thought process. The team runs risk of getting so focused by a particular feature or the user story that they may ignore the broader, often intangible product and design implications, such as integration, omnichannel consistency, or user-interface architecture.

Needless to say, such development efforts often go against the grain of Agile itself, as one of the core cornerstones of Agile development is feedback from customer and collaboration, which can be often sacrificed in the name of delivering incremental pieces of software, or iterations.

Image Source: Internet

Agile and UX: integrating towards and in to each other!

However, this sad state is not unmanageable. Over the years, many UX professionals who work in an Agile environment have developed their own ways of not only surviving, but thriving in this mad-paced, Agile environment. Once again, there is no set protocol or playbook here — most of the remedies that UX professionals employ are broad-based, strategic, and organizational factors in the fervent hope of marrying UX with Agile, keeping intact the benefits of both while coming together to make a better product.

Image Source: Adobe Stock

• Management buy-in and understanding the value of UX -: Organizations which are further along on UX curve inherently understand the value UX-focus brings especially to product development process. Focus on UX not only provides a competitive edge, but also is a key differentiator in the Go-to-Market strategy. This is yet another way of describing UX maturity, a concept long argued as the key to product success, and consequently, business success. Decision makers in such organizations realize that UX is not just an afterthought, or an extra coat of paint; rather it’s the careful and customer or end user-centered approach to product strategy, features, structure, interactions, content, and aesthetics. Project managers in these organizations and leadership don’t just ask UX designers to rush through features and user workflows, or merely rubber-stamp features that have already been decided upon by the rest of the team, instead, they expect the UX team to be involved from the very start and pay attention to nuances and subtleties in order to produce something which customers and / or end-users will cherish. Managers and leadership in such organizations embraces uncertainty and detaches its focus from merely quantitative measurement and understands that a “UX” approach means always hypothesizing, testing, and iterating on product design ideas. The focus is on qualitative methods to gather behavior-driven insights. The rich insights that UX teams need for their work and to make decisions often require time consuming and “putting-your-ear-to-the-ground” methods. On the other hand, organizations that are biased toward quantitative data as being the most important or most persuasive evidence for decision making often integrate Agile and UX poorly.

• UX People are critical Leaders -: While having managers and stakeholders that understand and support UX is extremely important, the reverse is also true: UX people that succeed in Agile environments exhibit leadership qualities. They point out assumptions about user behavior that could prove disastrous if wrong. They make time to reach out to colleagues to help explain UX process and principles, and why they produce better products. They bring their coworkers to usability tests to build empathy and observe users struggling. They fight for getting time to do user research, and to design something right. UX pros that thrive in Agile push the team back towards the bigger picture, and the context in which humans will actually use these products that are being built.

• Agile Process is natural fit for UX -: The original idea of Agile was to act as a flexible framework which can be used as a guidance, or a set of guiding principles to aid teams to develop quickly and effectively, where customers and end-users are as much a part of the process as developers are, where change and reacting to change is welcomed, and where speed and agility trumps the need to follow set protocols. By integrating Agile and UX, all of these principles are promoted. Customers’ or end-users’ feedback invokes behavior driven insights which are crucial to UX work, while readiness for change drives developers changing their approach in the face of radical decisions by the UX team.

• Agile development doesn’t limit user experience -: Users don’t interact only with small parts of a product’s design in isolation, they use many or all features of the product to accomplish larger goals. What good is one piece which works fabulously well, and is well thought out, while the overall application is a poor implementation? All pieces of a product design must work together seamlessly and harmoniously to provide a “beyond-good” user experience. In fact, in an omnichannel world, users expect their journeys to transition and perform seamlessly across multiple channels, even when developed by different teams.

• Rigidity drives away innovation-: UX works well when the Agile process isn’t strictly and mindlessly implemented. If an organization has rigid Agile-process guidelines that admonish UX teams for following due dilligence, and rewards UX professionals who merely rubber-stamp designs, it becomes very tough for UX people to become anything more than pixel-pushers. UX professionals who succeed in Agile work in environments that care more about responding gracefully to change than about how long a standup meeting can last. Once again, nuances and rules of Agile should be implemented in the spirit of producing a better product. Teams who incorporate UX well will figure out how to manage features and user stories so that UX has some time to get ahead of production and create validated, researched, and thoughtful designs.

• Learn when to accept UX debt-: UX teams, when working within an agile approach or workflow, need to compromise on the ideal or the best solution to get something built quickly, especially if that something delivers some, if not the most, value.  UX can’t be an impregnable fortress in the sea of Agile. Iterations in agile are planned to create a steady development velocity and UX design must keep up with this pace. Setting time limits for each of the team’s UX activities will help ensure that the team’s goals are realized on time, ideas are not overworked and the bad ones are thrown away early. UX teams must learn to limit the fidelity of design that is handed off to developers. This starts with the wireframing and prototyping phase of the UX design process. As an important lesson – UX teams must not waste time on details that could be worked out further down the line. It is essential to produce not the perfect design, but just the workable design, so that the same is communicated quickly and effectively to the developers.

• UX Professionals and Developers are on the same side-: Effective Agile teams communicate well and share a common understanding of the project’s goals (both large and small). Developers and UX professionals act in spirit of producing a truly better product. UX professionals step beyond pixel-pushing and decoration only, while developers are ready to make radical changes, often at the cost of abandoning their tested approaches and coming out of their comfort zones. UX professionals feel equally invested in the success of the product and take ownership for development issues. Often, such an ideal state of affairs is only achieved after working together and developing respect for each other’s work, something which is simply not possible in short runs.

• Its simply better ROI -: While many non-UX-minded project teams see UX as time consuming, UX-minded teams see the extra time used upfront as a factor which produces returns several orders of magnitude over the initial investment.

Collaboration is key for Agile-UX teams to deliver great software. If design and development teams do not keep an open-door policy, misunderstandings will grow and issues won’t be identified until it’s too late. Teams used to work a certain way, for example designers working head down for long stretches of time, while developers cranking away lines of code  often have to make comprises on quality or risk producing worthless code.

Image Source: Methodsandtools

Conclusion

It’s certainly difficult to make Agile and UX work together as two wheels of the same cart where one wheel is simply not enough to pull forward even an empty cart. Typical Agile processes tend to be biased against anything which is not measurable, and don’t take into account the time, resources, and efforts UX professionals need in order to deliver behavior driven attributes and user-centered products. Aside from its success in IT applications and software development, the agile processes are becoming popular in legal, marketing and other enterprise industries. At the same time, many are realizing the important and critical need to integrate iterative workflow into the user experience process. And despite issues, Agile UX teams agree that Agile and UX both help make a better product and deliver a better user experience.

Introduction

For any team and in any team setting, the role of a coach assumes centerstage in addressing the grooming, training and teaching requirements. The coach must possess skills about the primary reason of assembly of team, and must possess deep understanding of the nuances of the game. An effective coach knows how to groom the talent available and make the best within the given constraints. A successful coach understands not only the game, and how to win the game, but also the reasons why teams fail. That last part is often the sole difference between a coach able to inspire the team to higher successes and others. Beyond the training and teaching ability, the most significant trait a coach must possess is how to ignite the hunger for learning and subsequently winning in the team.

Agile values and principles are a tectonic shift from downward flowing command and control management practices to collaborative, team centric environments. To nurture an organization through this shift towards becoming Agile requires someone with depth of agile experience, knowledge of human behavior and strong influencing skills.

With the above in mind, let’s see what’s an Agile coach. An Agile coach is someone who is tasked with teaching and training the teams within an organization on Agile methods and the Agile way in order to spread the benefits of Agile. The purpose is to make sure the organization and individual teams excel in delivering better value to its customers and stakeholders by gaining efficiencies in how everyone in the organization works. The organization aims at gaining an edge through increased inner efficiencies, better ROI and lower costs. An Agile coach, by grooming the teams, training and teaching the values of Agile and inspiring team to relentlessly implement the chosen Agile methods, plays a valuable part in this endeavor.

What makes an Agile coach

So what skills an Agile coach must possess? In order to fulfill the above objectives, an Agile coach is someone who is an expert in theory and practice of Agile values. An Agile coach is someone who is willing and able to communicate freely to be able to teach others. An Agile coach is also someone who is able to take an outsiders view, is an Agile promoter within the organization, has the ability to mediate in case of shared responsibilities between projects, and guide large teams towards Agile methods. Above all, an Agile coach is someone with ability to work with multiple teams, projects and portfolio owners to drive Agile adaption and implement improvement efforts.

In addition, each organization may have its own preference of what skills it considers more important. Organization have diverse needs and may come up with their own specific desirable traits.

Image Source: Internet

1. Technical coaches: Agile coaches who are hired as technical coaches are primarily tasked with working closely with developers and typically have experience with coding and integration, since those are necessary skills when working with the dev teams.
2. Process/management coaches: Agile coaches working as Process or management consultants / coaches focus more on working with leadership and portfolio owners to educate agile teams and to oversee successful adoption of the agile method across teams. These Agile coaches play a significant role in monitoring adherence to Agile practices as per the chosen Agile methodology.
3. Non-directive coaches: Non-directive coaches offer selective, need-based, individualized support for people or organizations looking to solve specific agile-related problems. They are generally brought on for very short time periods with specific mandate.

Organizations choose their flavor of Agile coaching based on requirements, size of portfolios and / or projects, client demands, current state of Agile adaption and future desired state, as well as budget and other constraints.

Image Source: Internet

Responsibilities of an Agile Coach

An Agile Coach is typically a shared resource with multiple responsibilities on their agenda. Resolving roadblocks at an enterprise level or between projects, to coaching Scrum Masters and Agile teams, to custodian of portfolio level dependencies, risks and anything that may derail achievement of planned goals are the prime focus area. Most Agile coaches also track Agile metrics at team level and hygiene factors.

The roles played by an Agile coach vary from being a reflective observer: “You do it, I will just watch and tell you what to do”, to being a passive or an active partner: “I will show you once and then you can do it” or “We will do it together”.

Image Source: Credit Thinkstock

Organizational Responsibilities

1. Become the facilitator for the organizational culture change necessary for sustained agile implementation and success.
2. Helping the organization or Business Unit implement the chosen Agile methodology by working closely with teams and resources on the ground
3. Presenting data points, reports and extracts to leadership to support implementation of chosen Agile methodology
4. Becoming the bridge between leadership and Agile teams to enable communication and sharing of expectations, plans and goals.
5. Integrate related methodologies within the company to have an unified approach

Technical

1. Act as interlocutor to ensure right stakeholders are identified and issues are addressed with the right stakeholders
2. Leveraging previous experience to teach the basics to bring people up to speed with an agile way of working; organize and lead trainings at ground level and periodic refreshers
3. Guide scrum masters towards Agile implementation and coach on areas of weakness

Trains / Portfolio / Projects

1. Work with portfolio and project leaders to develop standards and requirements for the agile process implementation
2. Come up with plans and improvement measurement metrics and methodologies. For example, tracking of user stories, sprint burndown rate, etc.
3. Work with Product owners to monitor movement of items through backlog
4. Work with Product owners / project leaders / scrum masters to identify dependencies and risks and coordinate work towards resolution to ensure minimal impact
5. Organize discussions and track progress on removal of roadblocks
6. Help Scrum masters and projects to achieve goals together
7. Track team level Agile implementation and metrics
8. Bring all Agile teams under the same umbrella
9. Ensure transparency and fairness while dealing with teams and resources

Conclusion

Over the years as sports have become more competitive, sporting teams have realized that one coach for all aspects of the game is not enough. For example in Cricket, teams employ separate coaches for bowling, batting and fielding. In football, teams employ goalkeeping coaches, striking coaches, defensive coaches and more. Similarly, many large organizations today employ multiple Agile coaches with varying and diverse skill-sets, experience and backgrounds so as to derive maximum benefit and ROI.

Adaption of Agile methods can have more than just tangible benefits. The benefits of Agile are not limited to the organization only. As an organization pursues it’s endeavors on the path of Agile journey, it contributes to its ecosystem and to the field of Agile teachings. By playing an instrumental role in an organization’s Agile journey, an Agile coach can have a lasting impact beyond their expected role.

Agile Is Not Incomplete or No Documentation — Agile Means “Right” Documentation.

Introduction

Agile methods have been growing in popularity since the first formal adaptations in late 1990s to early 2000s. Agile has its own fan clubs and is fragmented in to practice areas, from simple, single team based Agile Scrum to complex models like SAFe Agile. Besides the more popular Agile methods, there are relatively lesser known yet equally effective methods like Crystal, DSDM and Feature Driven methodology.  Regardless of the method, all Agile methodologies share much of the same philosophy and features – regular feedback loops, speed, flexibility, and above all, regular communication and interactions between stakeholders, whose roles have been clearly defined and limited in all Agile methodologies.

As the focus has grown on speed, flexibility and continuous delivery, many Agile practitioners have come to regard Agile framework as highly flexible, evolving structure without rigid guidelines, rules, or methods. Agile teams have at times contended that documentation assumes lesser priority in Agile methods while some consider maintaining documentation a matter of convenience. Along with varying viewpoints, people have come up with their own adaptations of Agile practices and guidelines, merrily choosing one while ignoring other.

Why is documentation important?

All software systems are complex products developed and delivered through a series of steps and / or processes. Irrespective of the method used or the skillset involved, each software begins with an idea and translates into a product, prototype, document or piece of code to be used for the next project.  Whether a product, document, prototype, or working software, the artifact created in one step becomes the input to the next step.

The Agile Manifesto states as one of its core principals, “Working software over comprehensive documentation”, hence many people assume that Agile means little to no documentation. Nothing can be farther from the truth. In reality however, it is often a fallacy; and nothing more than a common misconception of those inexperienced with agile. Thinking that a project can be delivered more quickly and easily by avoiding documentation is often akin to not doing enough testing in order to save time.

Image Source: Mastering Business Analysis

Documentation serves an important purpose. In particular, project teams want to capture high-level information in documentation but not details, which is largely fine as long as documentation requirements are met. You will still need to capture important information permanently, and sometimes regulations require certain levels of documentation (yet another requirement)

1. Documentation is integral -: Documentation is as much a part of the system as the source code. In addition to working software, Agile teams need to minimally deliver user manuals, support documentation, operations documentation, and system overview documentation.
2. Initial stages and recording agreements -: Documentation plays very important part in recording understanding of everyone on the project team with regard to the overall scope of work and their individual wok areas and the points of integration. Documentation pertaining to requirements and recording definition of done, recording understanding of solution and design documents are integral part of project documentation. Without this documentation, most of the project work can come at risk and re-work can become the least of project team’s headaches.

3. Inputs to next work effort -: Building high-quality working software which meets the needs of the stakeholders is important, yet its equally important to ensure that the software can be maintained and enhanced throughout its useful life. Documentation helps regular operation and support functions and without adequate documentation there are higher chances of wasted work efforts and re-work.

4. Not all projects are created equal -: Projects vary in degree of complexity and time taken to complete. Complex projects demand more concise, well-detailed documentation than simpler projects. Similarly, working on new systems or new requirements means more documentation is needed as there is little pre-existing knowledge or resources to fall back upon. Working on complex and unknown systems means more documentation as interactions will need to be understood by the team.

5. Maintain transparency and communicate better -: documentation helps teams communicate better as well as maintain transparency in to the team’s activities. It’s a fool’s world to think that project teams exist and act in isolation. In varying degrees, whatever happens on a project team, impacts other teams around.

6. Ideas and tasks cannot just be all in people’s heads -: stuff that’s not documented is sometimes lost as soon as the project is over, and definitely lost over an extended period of time. Documentation however survives as long as it is considered relevant and maintained.

Image Source Scott W. Ambler

Agile Documentation

Unlike waterfall project management, where the focus is on clearly defined documentation, and documentation is a project phase unto itself, Agile views documentation effort as only what is bare minimum essential to the project.

Agile encourages “just enough” documentation as may be required for the project to meet the expectations of customers and stakeholders.  Among other things, the level of documentation needed for a particular project depends on newness of system and requirements, maturity of project teams, complexity and scale of the project, future needs, etc.

The documentation on an Agile project can be in the form of white boards or brain maps; or sticky notes or writing on flip-up chart papers. Pictures, rather than lengthy documents can be the flavor of the day. Similarly, videos can be created of project team members explaining requirements or detailing solution or just how any part of the solution would work. Agile does not believe or recommend in documentation just because some stakeholder feels that each and every detail of the project must be captured in writing to retain knowledge for future. The aim of Agile is release increments better and faster. “Just enough” documentation fits in the overall picture of efficiency.

While some information will always need to be captured in written form or documented clearly, there are techniques that may be used to reduce amount of writing, but will still give the customers and stakeholders what they want. One way to reduce the amount of writing is to create diagrams through brainstorming using a whiteboard and capturing those as images saved by individually or as part of a document.

Image 3: Source Internet

Agile – Documentation and changing requirements

The standard waterfall requirements documentation doesn’t hold much water in Agile world. First and foremost, for any medium to large project, obtaining stable requirements upfront is impossible. The business needs and customer preferences themselves change over time, and in many cases business  and customers aren’t even aware of what they want till they see a final working version of the product. In many cases customers realize they specified wrong requirements only when they see the final version of the product. The other problem with requirements identification is understanding the intent, or the unsaid needs of the customer. Perfectly well written, and word-by-word noted requirements turn out to be well short of the customers’ expectations because the team did not understand the unspoken requirements.

Image 4: Source Internet

The product backlog, where the requirement came from, and how it will be accepted by the customer (acceptance criteria or definition of done) are all various documentation that are essential and central to the success of the project. It cannot be optional, it cannot happen “whenever convenient.” It is a part of the development and acceptance process.

Documentation in and of itself doesn’t hold much value unless it is paired with an effective communication method. The focus in Agile world is, and should be, rightly on conversations first, and documentation later. In the above cases where traditional documentation of requirements will fail miserably, communicating frequently and holding conversations about product features regularly so that the customer and project team can build the product together holds the key to product acceptance.  The simple idea is to come to an agreement point at the end of discussion about each feature or story point (where features are broken in to story points) and documenting the same. Once again, remember! Conversation and agreement first, documentation later. The agreement between customer and business and project team centers on what is the acceptance criteria (definition, roles, acceptance tests), which is later documented and forms part of Agile project documentation. This final output, after the discussions are done and the dust is settled, is the key towards a successful project as without this final step, of documenting what’s agreed, months of work and planning can flow down the drain.

Acceptance criteria & acceptance test form the core of Agile requirements. These documented agreements – Acceptance criteria and acceptance tests – cut the ambiguity around the requirement as defined by the customer, thereby improving the chances that dev team/s build the right thing, that stakeholders will be happy with. Further, it affords developers some flexibility in how they implement or develop the requirements, as long as the final product meets the criteria/tests documented in previous steps.

Image 1: Credit SkepticalAgile

In addition, acceptance tests and acceptance criteria bring about an important change within the customer organization. In many cases there are more than one or two stakeholders. Entire customer teams from cross-functional areas can participate in requirement discussions and its very common to find divergent viewpoints among the same team. Discussing and documenting acceptance criteria lends a seriousness to the entire exercise and forces the entire customer team to come to shared understanding, which simply would not exist without documentation. Divergence among stakeholders is the single biggest pain point while working with medium to large projects and Agile documentation can very well nip the problem in the bud.

At implementation level, these criteria and tests become the common language between project team, developers and the stakeholders. Acceptance criteria and tests established earlier become the basis for discussing changes in requirements, as well as how progress is tracked. Of course the acceptance criteria and thereby the defined acceptance tests can themselves change if requirements change or even otherwise following the same original course of conversation and communication followed by agreement and documentation. Furthermore, these acceptance tests can be automated, using Acceptance Test Driven Development (ATDD) tools commonly available today.

Post the requirements’ documentation, another sticking point with many is the topic of Technical documentation. Technical documentation is meant to be a description of the software as it exists including all the changes to the original version thus far as on the date of recording. The purpose of technical documentation is to allow continuity, i.e. to allow a system to be maintained by future developers, long after the original team has left or as long as the system is deemed to be useful. In reality, however, technical documentation, such as UML diagrams, may not be very helpful in solving day-to-day development problems, and are often found to be out-of-sync with the actual behavior of the system.

At the core of Agile technical documentation is documentation through tests, in what is normally referred to as Test Driven Development. When the development team starts work on a story or feature, the testers and developers come together first to define and automate the acceptance tests. This helps on more than one front. First, this helps the team maintain their focus as they implement the acceptance criteria agreed earlier, by including only those factors that contribute towards meeting acceptance criteria and rejecting those that don’t. Second, this helps the team gain a common purpose and work towards the same goal in the most time efficient manner as the inconsistencies are resolved upfront. Thirdly, this acts as a final level of check to identify any tasks which can derail the entire project. Test driven development’s core feature is to only include the tests which are appreciated by the stakeholders, and which amply demonstrate the product features and acceptance of those features, rather than including mundane or repetitive tests.

Lastly, project teams’ maturity, whether it’s a new team or has worked on several projects in the past together, is an important consideration on the amount of documentation needed. It is extremely important to document, starting from requirements to acceptance tests, however the level of documentation needs to be managed at project level.

Image Source Dilbert

Conclusion

While it is true to a large extent that Agile methods advocate doing away with rigid guidelines, documentation and maintaining artifacts is not a rigid requirement to begin with.  Lets keep in mind, agile is first of all a set of values, not a settled and strict, rigid framework. All Agile practitioners should be confident on how to adjust these values to their environment. Remember that every company has its own pace, characteristics and values.

All Agile documentation must follow the same rule – it is not that agile eschews documentation, so much as it eschews up-front documentation.

Introduction

What does scaling Scrum mean? Scaling Scrum is a challenge that many enterprises face due to multitude of factors – growing complexity of the projects, changing needs of the organization, increase ROI, need to justify investment, or simply to appear to be following a truly Agile method. Whatever the motivation or the reason behind trying to scale Scrum, enterprises quickly discover along their journeys that scaling Scrum is actually quite different from initial implementation of Scrum. And, realize that it is quite difficult indeed. While the initial implementation of Scrum would be fairly simple and not demand much, the implementation of Scaled Scrum is anything but simple.

A single instance of Scrum implementation has a straightforward path – 1 scrum team that takes work from the product backlog, performs the necessary sprints in accordance with Sprint cadence to create an usable increment of product at the end of Sprint, adds unfinished or newly discovered tasks back to Product backlog as Technical debt and moves on along the same cycle.

Scaled Scrum, on the outside is exactly the same. Selected user stories from a product backlog provide the scope of work for a Sprint, where the goal is to produce an usable increment. The difference lies within. Within the Sprint/s however, the product owner or the project owner has chosen to implement multiple scrum teams, using any number of different compositions or structures.

All of the basic cadence of Scrum – the values, artifacts, roles, and meetings of Scrum apply, whether Scrum implementation is singular in nature or scaled up. These principles remain inviolate for the purposes of controlling risk, increasing predictability, generating creativity and productivity, and creating transparency.

Image Credit: K&C

Scrum – What to expect!

Scaling Scrum is a hands-on, get-dirty down on the floor, all-hands-on-deck, sleeves rolled up model where team members figure out one instance and nuances out of other instances of scaling Scrum that might work for their projects, releases, initiatives, or organizations.

As more Scrum teams come together to work on a Product Backlog, the number of people, their interactions, inter-working dynamics, complexities, and non-linear events increase. The situation is like too many vegetables simmering in a pot – you can guess but never accurately estimate how many flavors are one too many. For similar reasons, the relationship between productivity/creativity and the number of Scrum Teams is not clear. Product Owners employing multiple Scrum Teams need to carefully weigh the benefits and costs.

Techniques for organizing and selecting Product Backlog items, resolving dependencies and integrating work, and creating “done” increments are evaluated. Since every development situation is different (domain, people, technology), every set of techniques is unique and often has to change with time.

Scrum is designed to work best in an environment which fosters complex adaptive systems which exhibit intelligent goal seeking behavior. Scrum is designed to support an object-oriented component architecture.

A.      Early Implementations

One of the first implementations of Scaling Scrum  was studied in IDX systems (later known as GE Healthcare) in 1996 – 2000 period.

• Managers self-organized company into teams
• Managers became team leaders
• Directors ran Scrum of Scrums
• VPs became leaders of sites with multiple Scrum of Scrums
• Grew to over 600 developers
• Virtual architecture teams
• UX team and Integration team
• External experts verified that production doubled company-wide

Source: Scaling the business by scruminc.

Using Scrum at scale is another term coined at Microsoft following their extensive experience with Scrum here and here. At the same time, virtually all other top product and software application companies have realized the huge benefits of doing Scaled Scrum.

B.      Team Structure and reporting

The first step to scaling is to form teams focused on features in order to maximize the user experience and speed of iterating on working software.

• Every team has a product owner assigned and a professional Agile Coach who drive process improvement across the company working directly with senior management team.
• A very important factor to account for is location of the teams. Team co-location is crucial, however not always practical. If distributed teams cannot be avoided, extra steps need to be taken to induce the feeling of co-location by measures like periodic get-togethers, reliable and prompt communications, etc.
• Teams need to have clear autonomy and Scrum master must work with the team to remove impediments. Scrum master assigned to the team gets measured on process improvement and timely removal of impediments.
• Regular communications and meet-ups between the teams and are crucial to disseminate information and provide autonomy as well as relevance to teams. Cross dependencies can also be resolved better and faster this way.
• Cross functional teams are comprised to provide feedback to the community of practice they belong to. Communities of practice are generally the organization of resources according to their skillsets. For example, all Java resources may belong to the ‘Java practice’ in an organization, if the organization has such a setup, from where they are assigned to different projects. While in a project, resources report to project management as well as their respective community of practice.

Image Credit: Knowledgehut

Scrum – Frameworks

All SAFe scaling frameworks share some common patterns: Scrum implementations at team level, multiple teams sharing same product backlog, planning done as collaborative exercise across teams, and the general principles of pull and self-organization.

Scrum teams can be organized in many different ways, contingent on the framework used. There are number of frameworks that are available out there today. In this article we provide a brief on most of these frameworks for scaling Agile – LeSS, SAFe, and Scrum@Scale, DAD (Disciplined Agile Delivery) and Nexus

All Scaling Scrum frameworks start with cross-functional, self-organizing Scrum teams. The teams dissect and slice requirements into the smallest possible measurable tasks and increments that can be developed independently. Teams are expected to focus on self-learning, technical and process excellence such as doing continuous integration and automated regression testing to ensure that new code pushes don’t break any existing functionality. At the end of every sprint the teams should have a potentially deployable product, called an ‘usable increment’. The frameworks also encourage the use of Lean principles to optimize flow.

A.      LeSS (Large Scale Scrum)

LeSS is a framework for Scaling Scrum that comes from two practitioners – Craig Larman and Bas Vodde, based on their work in the financial and telecommunication industries.

LeSS is defined by approach of minimalistic nature in supervision as well as use of processes, i.e. use as little processes to get multiple Scrum teams to work with each other well enough. LeSS consists of a hand full of rules, and also has guides and examples for how to customize these rules as the organization grows.

Perhaps in some ways LeSS is a larger reflection of how an ideal Scrum implementation is supposed to work.  LeSS recommends that there be multiple Scrum teams with the same product owner and shared product backlog be maintained to promote consistency and predictability. Scrum teams should be long-term, cross-functional teams. Sprints should be synchronized to represent a product-level sprint leading to one usable integrated product increment at the end of the sprint.

A LeSS Scrum Master typically encounters complex large-scale problems and s/he needs to resist the urge to resolve them with equally complex solutions. Instead, the scrum master should leverage the spirit of Scrum and find simple ways to empower people to resolve their impediments. This approach leads to large-scale, yet simple, solutions.

Image Source: Less.works

LeSS is characterized by a single product owner with one product backlog, and several teams, each with its own sprint backlog. The Product owner is the owner of the complete product backlog and typically there are no divisions of the product backlog, unless using a methodology like LeSS Huge.

LeSS is the purest version of Scrum with only notable difference coming during the planning stage. As compared to Scrum which notes that there be 2 planning meetings for each scrum, LeSS recommends that the first sprint planning meeting be the joint meeting where user stories for product level usable increment is finalized (finalizing a product level Sprint Backlog), while the second part of the planning meeting is used by each team to produce their own specific Sprint Backlog.

B.      SAFe

In the words of Dean Leffingwell, SAFe® is a freely revealed knowledge base of proven, integrated patterns for implementing Lean-Agile development. It provides comprehensive guidance for work at the Portfolio, Large Solution, Program, and Team Levels.

SAFe approach gained prominence post 2012 and has had 4 major updates since its initial release in 2011.  Currently, the available approach is SAFe 4.6 launched in October 2018.

SAFe approach is a quintessential reorganization of the work with the aim of maximizing the speed of product or service delivery from initial idea to release and from customer feedback to enhancements and improvements. SAFe approach is based on organizing work in to value streams based on the repetitive nature of work performed. Each value stream has got its own organization of release team, called The Release Train. This ‘Release Train’ generally comprises multiple Scrum teams working together to deliver a component of ‘usable increment’ of the product. The ‘usable increments’ from all the Release Trains and Value streams then is combined to become a product increment.

SAFe methodology is further available in 4 different configurations dependent on the needs and size of the organization. The 4 different configurations vary considerably in terms of complexity and inter-play of components as well as the focus areas. These are:

1. Essental SAFe
2. Portfolio SAFe
3. Large Solution SAFe
4. Full SAFe

C.      Scrum@scale

Scrum@scale looks to tackle the problem of Scaling Scrum through questioning the ways of working and analyzing various choices to find the right approach to suit the context.

Scrum@scale is defined as follows:

Scrum@Scale (n): A framework within which networks of Scrum teams operating consistently with the Scrum Guide can address complex adaptive problems, while creatively delivering products of the highest possible value.

Scrum@scale imbibes all the principles of Scrum methodology – light structure, minimum bureaucracy and process controls, simple to understand however difficult to practice, needing regular and sustained cadence.

Scrum@scale contains two interlinked cycles: the Scrum Master cycle and the Product Owner cycle. These 2 cycles work in an interwoven way to provide a powerful framework to coordinate the efforts of multiple teams along a single, desired path.

Source: Scrum@scale guide

A set of the teams that operate within the project or have a need to coordinate comprise a “Scrum of Scrums”. The Scrum of Scrums is a “team of teams,” which hold a Scaled Daily Scrum (SDS) event with a representative from each team (usually the team’s Scrum Master, although any person or number of people may attend). The SDS exists to coordinate teams and remove impediments to delivering value.

The Scrum of Scrum must include all capabilities needed for product release at the level of Scrum of Scrums. That means that the Scrum of scrums should have the capability to look at the small goals and impediments while looking at the big picture all the time – to the point where it becomes ingrained in the team’s DNA.

Scrum of Srcums is at the heart of Scaling Scrum approach as laid out in Scrum@scale. The Scrum of Scrums team works independently as a release team and must be able to deliver value to its customers. To achieve this effectively, Scrum of Scrums has its own artifacts, roles and events – Scrum of Scrums daily standup, Scrum of Scrums retrospective, Scrum of Scrums review, etc where participants from all the Scrum teams attend, led by Scrum of Scrums master.

Scrum@scale also lays down the concept of Executive Action Team which is a super-body of Scrum of Scrums of Scrums operating within the organization. The purpose of establishing this super-body is to create an Organization level backlog of Agile initiatives (prioritized list of various initiatives) and aid in resolving the organization level impediments that hamper the achievement of these objectives.

Scrum@scale essentially views the entire organization as a super scrum team, something of the sort of a Scrum Master organization, transposing the agile operating system from project / program / product level to organization level.

D.      Disciplined Agile Delivery (DAD)

DAD or disciplined Agile Delivery is characterized as is a hybrid approach which extends Scrum with proven strategies from Agile Modeling (AM), Extreme Programming (XP), and Unified Process (UP), amongst other methods.  DAD extends the construction-focused lifecycle of Scrum to address the full, end-to-end delivery lifecycle from project initiation all the way to delivering the solution to its end users.

E.       Nexus

According to the definition available at Scrum.org Nexus is an exoskeleton that extends Scrum to guide multiple Scrum teams on how they need to work together to deliver working software in every Sprint. It shows the journey these teams take as they come together, how they share work between teams, and how they manage and minimize dependencies.

To start a Nexus, organizations should first:

• Identify the teams in their Nexus
• Form an initial Nexus Integration Team
• Have a single Product Backlog
• Have a definition of “Done”
• Identify a Sprint cadence

Just like Scrum@scale, the Nexus approach has its own terminologies, artifacts and resources. Nexus Daily Scrum, Nexus Retrospective, Nexus Review, Nexus Integration team, Nexus Sprint Backlog, Nexus Sprint Planning.

A representation of Agile Map

Conclusion: Scaling Scrum is a challenge!

As team sizes grow, organizations grow and their needs become more complex, bigger and more complex impediments arise along the road. The role of Scaling Scrum is to provide a framework to continually identify and remove dependencies created by increased complexity.

Scaling Scrum, like implementing Agile Scrum methodology is a matter of choice. Many organizations have reaped tremendous benefit and even more continue to do so. However, the primary integration point of Scaled Scrum is within the culture of an organization. The complexity of an organization’s hierarchy and culture defines the kind of products and applications it produces. In the words of programmer Melvin Conway:

“organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations.”

— M. Conway

An easy example of this can be seen through the website of a company, where it is often noted that the complexity, content and structure of the website depends on the demands of the internal teams rather than the needs of the users.

The Second challenge to successful implementation of Scaled Scrum comes from communications. As software projects grow in size and complexity, so do the teams of engineers that develop and maintain them. Brooks, in his seminal work, “The Mythical Man Month” discussed coordination as one of the key problems of running a software project with many developers. The coordination effort required to help each member of a team stay in sync and keep a project on schedule is enormous.

Early indicators of software quality are beneficial for software engineers and managers in determining the reliability of the system, estimating and prioritizing work items, focusing on areas that require more testing, inspections and in general identifying “problem-spots” to manage for unanticipated situations. Often such estimates are obtained from measures like code churn, code complexity, code coverage, code dependencies, etc. But most analysis often ignore one of the most influential factors in software development, specifically “people and organizational structure.” Read this paper for more information on the study conducted at Microsoft to analyze the above: Empirical software engineering at Microsoft Research

Generally speaking, LeSS is considered a good starting point for organizations who are relatively inexperienced in their Scrum adaptation. LeSS scales up with minimum addition of processes and leads to significant gain. On the other hand, implementation of an approach like SAFe demands a high organization maturity and experience of implementing Scrum. Implementing SAFe is generally for full scale Agile organizations and requires high degree of commitment and focus. Scrum@scale is a good approach for organizations with several scrum teams in place which want to focus on finding things and areas to improve.

In the end, choosing a methodology depends on the context and the situation, with the most important principle – ‘improving how to work together’.