February 03, 2014

Scrum and Anarchy - The path ahead for Enterprise Agile


Disclaimer: This article reflects my personal opinion, experiences and reading about how Scrum works in enterprises versus how it should work. These may not be the views of my organization and I have written this in my capacity as a "developer of the world".

After recently completing my Scrum training, a lot of thoughts have been floating in my mind about agile in general. There is overhaul required when enterprise teams are introduced to Scrum as Ken Schwaber described best "Scrum is now more mainstream than radical."

Let me start with where I come from. I started my career at an IT services behemoth in India. My employee number was in the low 60,000s. Within a few months of the outsourcing boom at its peak in 2006, the company suddenly grew to 110,000. With so many recent college grads and easy money to be made, the waterfall approach seemed great for mentoring new kids. The business model was mostly throwing a bunch of recent college grads who have gone through a crash course training into a team based on "domains" like Insurance, Healthcare, Banking and Manufacturing. Planning involved setting a tight deadline and apply as much pressure as possible to deliver within the deadline regardless of quality or whether requirements fully are met. There's always the on-site guy (poor soul) who will spend sleepless nights attending your conference calls and wake up early before you leave work so he can chat. This role had even more pressure of meeting the customer face to face and firefighting deployments that need to go on time. One of the main life lessons for me was to respect to people who knew better than me AND those knew who no better. Regardless, the experience taught me a lot about team structure, working towards deadlines and the general chain of command that is strictly enforced within the work culture in India (that I knew of).

Fast forward to 2014 and I have been working on a Scrum team for a couple of years. The passion to build, develop and create software has only grown. Everyday, I wake up with a sense of "What can I do today?","Oh boy, that feature I'm working on is going to be challenging AND fun!" and "I just want to do this everyday of my life". Building a product was revelatory in the Scrum world, where the focus was on customer relationship through the power of the product. It is the most fun I have had since playing with my brother's ZX spectrum. Or the time when I assembled my own PC. Or the time that I spent on learning Android development only to abandon it for another day.

In the waterfall world, customer relationship is mainly built by a sales team who sell snake-oil under-promising features and a development team over-working to delivering half-baked features with a boat load of documentation that will take a sane person two minutes to throw into the trash. The waterfall model promotes the establishment. You have structures, hierarchies, documenting everything (yes, even copyright banners into code that would anyway get stripped out when compiled) like a legally binding contract, following style guides that are 80 pages long and my favorite: slapping other people's wrist during code review for naming their variables in non-standard way. None of this has anything to with increasing value for a customer. A lot of the sweating and fretting is over non-standard process followed than actually delivering working software. I understand accountability is a must, but why not keep the whole team accountable? Live as a team and die as a team is a concept alien to most enterprises.

Although I am a still learning life in the agile world, the Scrum framework is possibly quasi-anarchy for the establishment. The establishment being "the enterprise". When someone has spent a very long career (we're talking 15-20+ years of following a set process like an automaton) doing mundane things while focusing less on the actual innovation, feature or customer feedback, you find yourself trying to tweak the process. One of the gems I've heard in an enterprise is "Process is more important than people" or "If a guy gets hit by a bus, I want all his work documented before hand, just in case". This mindset of settling to document pages and pages just for the heck of it when that time can be spent on learning a new stack or open source library, adopting new products, automating mundane tasks is something every enterprise is stuck with. Yes, you may have someone hit by a bus. That could happen even if he left a hardcover book detailing every hour of his work. Dealing with such situations is why you get a team to be "agile" not only in name but in pairing with each other to focus on a common goal and truly be cross-functional. The focus should be on learning each others work than one person typing away their (sometimes skewed) understanding of how things work. For the establishment, agile is anarchy. It disrupts everything that they have believed to be true. It is like telling a 5 year old Santa Claus doesn't exist.

Process and tools are necessary to help on-boarding new teams but the focus should be on individuals and interactions. As a Scrum team gets to know each other, should they decide a particular process is a waste of their time, they should be given the confidence and full-authority to throw it out the door. Processes as such should not impede you from interacting with your team or customers/stakeholders. I once had a friend telling me how one of the new team members who has never worked in a Scrum team went missing from his desk. For many hours a day, multiple times it appeared he just wasn't with his team. However, he followed the process of logging time, ensuring he attended standups and had something to say on what he was up to. An outsider analyzing his work using a tool or someone who attends the standup (a topic that has potential for exploring), they would not really find a difference between this person and someone else who is genuinely working towards achieving the sprint goal. Being a team player is more than ensuring your bottom is covered. Needless to say, this was identified and dealt with.

Scrum is being adopted by enterprises that are gung-ho about responding to change. The feeling that "Oh, I can keep changing requirements mid-way, the Scrum team will handle it" or "Adding work to the sprint backlog is good because the team is finishing things fast anyway" is the start of the slow march to a Scrum project's grave. This is a place where process creep is bound to come in. "Is the CR documented? I can only start working on it AFTER I get the document" or "I think that is bad customer data, the user is doing it wrong, they should not enter it that way" or "I don't think I'll spend any time on it unless the user confirms they followed the help manual" are some of the warning signs. Even though there maybe a clear steps-to-reproduce from the customer, you will hear things like "I don't know that feature, I would wait until X comes back from vacation in 2 weeks", "The customer is not really pushy, we can let the defect sit in the product until the next release" and so on (for more such quotes, check out Adam Weisbart's Agile Anti-Patterns). The delight of quick-turnaround response in delivering software is the same as the relief you get when a customer care representative talks to you in 1 minute even though the machine told you the average wait time is 30 minutes. Making sure the user will actually use the feature you are building and having customer feedback on regular demos bode well.

That brings us to the resourcing and staffing of a team. An enterprise generally does it on the skill-set of people who know the work or may have Subject Matter Expertise (SME) than people who are efficient at almost everything they do. In addition to Scrum teams that need to be cross-functional, a multiple-role AND cross-functional team is one I have seen have much success. A developer must learn to test too when and if it comes to it. A tester should know how to write automation scripts and always have their eyes and ears trained to be like users. A good product owner will learn how to sketch out interfaces to a designer. Building a Scrum team requires patience. There will be failure. There will be complacency. In an enterprise, you are mostly taking what is called the "fat, dumb and happy" crowd and throwing them into a fire to be "agile, smart and learning". Happiness is something that will come out the product you build or the service you deliver. It will not come from the mundane 9-to-5ness that is attached with an enterprise and the security that you will get good enough rating this year to fly under the radar. Performance reviews are a whole another animal. Don't be Milton Waddams (although I love that guy).

I had the good fortune of hearing someone say "Hey, how do I check in code using this tool? I don't want to get in trouble if code doesn't make it to the QA handoff deadline" and "The sprint burndown will look bad if I can't finish this task, so I'll just mark it as complete and work overtime". One of these was followed by a document written up over many days after review where a 5 step process for managing source code is written up. The scope for learning is killed and a repeatable process where "Thou process is less holier than mine" is built. This death by committee approach is very prevalent in an enterprise. Investing in talent or individual development will earn you workforce morale that will only benefit the organization. Forget scaling Scrum, review teams and committees are probably bigger impediments to an agile team than team members with less experience. "Waterfall within Scrum" or "Deadlines in a Sprint" is like saying "Get your scuba suits ready, this desert has lots of sand". To not fall short of more cliches, it is like taking a knife to a gun fight.

In case this "anarchy" doesn't play in to the "organization wide" practices, so be it. You might want to stick to following what works best for your organization's culture. Are they delivering quality features? Good. Are they building stuff with business value? Even better. Are they being honest in obtaining feedback and identifying areas to improve as a team? Excellent. Working software needs focus on problem solving and a "Git 'er done" attitude than somebody who spent an hour on referring to two 200 page manuals (one written in 2012 and one that is fairly newer), sending multiple emails to the original author to find out if he should use * or - as a comment border since the standards were changed.

Finally, an area that a lot of enterprises look at is to tweak existing processes to be "agile". I hope I'm not too harsh and I apologize to be the one to break it to you, the tweaking will be indefinite unless there is a change in mindset. A self-organizing team that has fully authority to police themselves and build great software will take more than "tweaking processes" and "enforcing rules". The tap on the shoulder to remind the team of the rules (until they are on track) is what a good ScrumMaster should do. Remember, Scrum does not have a project manager. As the establishment, if you half-adopt Scrum, it is bound to fail in a sense that you are not getting the best of what Scrum has to offer.

In conclusion, I leave you with this quote by Ken Schwaber that every enterprise should think of:
Scrum is like chess. You either play it as its rules state, or you don’t. Scrum and chess do not fail or succeed. They are either played, or not. Those who play both games and keep practicing may become very good at playing the games. In the case of chess, they may become Grand Masters. In the case of Scrum, they may become outstanding development organizations, cherished by their customers, loved by their users, and feared by their competitors.
P.S: If you found this article offensive, you might not be my target audience.