Because Things Go Wrong

Don't spend too much time on upfront planning because, no matter how well you do it, you'll be wrong, insists new GlobalLogic chief technology officer Jim Walsh. Here, the ardent Agile advocate talks about winning the development race the right - and perhaps subversive - way.

GlobalLogic, a leader in global software product development, did more than just solidify its commitment to Agile in naming Jim Walsh chief technology officer in April - they put the entire company behind it. Walsh is an Agile development expert with more than 26 years of industry experience, including 14 years in outsourced product development, responsible for more than 300 product releases.

Walsh came to Global Logic, based in Vienna, Va., from Chordiant Software, where he was vice president and general manager for offshore operations. Upon his appointment, Walsh stressed his commitment to delivering products to market faster, better and cheaper, saying: "Product development is a niche segment and companies possessing an 'IT project mindset' often fail to align their goals with the customer's end requirements."

Walsh chatted recently with ProjectsAtWork about how Agile is forcing companies to rethink their approaches to development in radical way.

When did you come into the whole Agile movement?

I can date it very clearly to … when I read Kent Beck's book in the year 2000. I got angry, I felt very challenged by what he was saying about software development. Not the command and control thing - I had always practiced that innovation comes from people who do the work rather than from the top down, that it needs to be driven by the individuals. But I also believed that you need an overarching architecture to work within, and then Beck was advocating, at least the way I understood it, that your architecture should evolve as you discover more and more about the product you are developing. I found that really radical and challenging.

It's sort of like building a house with one room and then you put the second room where you like it should go and the third room where you think is best and with no overall plan. I thought that would result in total chaos and to some extent I still hold that point of view. I do think you need a sketch of what the finished product is going to look like, and you can evolve the sketch but you should have a concept of this is where we are going.

But I think on a more fundamental level, Beck is right - that no matter how well you plan upfront, you are always wrong. By the year 2000, I had already been involved in shipping between 100 to 200 product releases; now it's way over 300 almost a decade later. And he's right, you never end up shipping what you originally design. So it's really wasted effort to spend too much time on upfront detail design, that you will always be wrong and Agile practices embrace reality, acknowledge reality.

Do you think some of the skepticism about Agile comes from that aspect of embracing the reality that things are going to wrong?

Yes, yes. At every level of an organization, program managers, team leads, managers, are often promoted because they do things right and can predict the future to some extent, to anticipate the future. The cultural and emotional challenge is getting them to admit, I really don't know what I'm doing!

It is difficult. It's difficult for people accept, it certainly was for me when it came to architecture. I spent up until that point doing about 10 years of work in architecting products and the fact that I was going to be wrong every time is very difficult to admit. And I think the loss of control as well, or the illusion of control, is another huge hurdle for people.

I imagine it's a pretty tough sell to the higher ups to say, 'Well we really have no idea how it's going to turn out.'

Yes - that's the difficult part. And the first time I did it, it was a major source of tension between me and my boss. On the other hand, though, it's a great risk mitigation strategy, right? You always have something to demonstrate. Every two weeks or whatever your sprint interval or iteration interval is, whatever you want to call it, you have something in hand that actually works and can be demonstrated to investors or customers or to your marketing team and you can get their input to say, is this what you had in mind? And in this particular company where I first used Agile, we were able to attract $5.5 million of funding on the basis of a demo that we put together in about six weeks.

When you do something new, the problem is often that no one can visualize what you are talking about. You can describe it verbally as much as you want but if it's a really new concept there is nothing like having a working product. They can't say, 'Oh that's impossible' or 'That's too far out there, you know it's just speculation.' You hand them a product and say, 'Here it doesn't have all the features it will eventually, but here it is.' You can't argue with that.

Where is Agile in its life cycle? Still on the cutting edge or are we beyond that?

We are kind of Geoffrey Moores, Crossing the Chasm. We are into the early majority adoption phase, where it's becoming mainstream now, it's no longer cutting edge. Companies are beginning to ask, 'What is your Agile strategy? How do you do Agile?' Rather than, 'What the heck is Agile?' So it's definitely getting more mainstream.

I would say it's still the best project management methodology, Agile and Scrum, which is the project management framework that usually wraps Agile, or at least in many companies, like ours. But in terms of the cutting edge, Agile is getting overlaid with concepts like Lean program development which comes out of the automobile industry. Toyota pioneered that with just-in-time inventory control, developing only the features customers actually value the most and eliminating waste from your development process. It's very compatible with Agile. It's not a replacement, it's a sharpening of the focus. Let's do only the things that are important to our customers. Let's eliminate waste by reducing the amount of defects that we put into our products and by focusing on the right outputs. Let's instrument our process so we know every motion and eliminate anything that doesn't contribute to delivering value to consumers. People who are most sophisticated about Agile are also developing Lean over Agile.

As for Scrum, people are also beginning to look at more sophisticated program management-level overlays. The Scrum process focuses a lot on the implementation phase and it kind of assumes the product owner is and expert on everything, a single source of authority that is always right. And in complex environments with multiple projects and multiple distributed teams where no one person can be the expert, the world doesn't map cleanly onto just one individual. So people are starting to do things like (creating) a team that supports the product owner. Still you have one person who makes the final decision, but a team that might include architects, quality architects, things like that, helping to do what is called 'elaborating the backlog.' They look ahead at future sprints and develop them into more detail so that when when the implementation team comes to that sprint, the product owner is in a position to answer all the detailed questions.

At GlobalLogic we have a high degree of tool support and process support so we can cope with more distributed environments. Those can be considered elaborations to Scrum, not violations, just that to more complex, real-world projects, the bare bones that they teach you at Scrum Master Class isn't really sufficient for release-level planning. There is more to it.

Is the Agile approach good for everyone?

I've yet to see a place where it wouldn't work. Whenever you talk to someone who is resistant to Agile they say, 'It's not going to work in my environment because you can't break it down into two-week sprints - our features take a long longer to be developed.' That sounds plausible and I haven't ruled out the possibility there may well be things can't be decomposed that way, but every time you probe deeper and you say, 'OK, well, describe one of the tasks you can't break down.' They tell you and you ask, 'Well how are you going to do that?' and then they instinctively break it down into smaller chunks of work.

Humans don't work in one-month increments. We have natural divisions of morning and afternoon, or basically four-hour chunks of work. I have yet to see a task that can't be broken down into smaller sub-tasks, a meaningful unit of work - demonstrable, measurable, shippable - that could be done in two weeks or whatever your sprint interval is.

Maybe they're like writers. We never want to show an editor anything before deadline - usually because we are finishing it at the last minute.

Yes. I think that's it, and it comes back to the point of facing reality, to the cultural barriers. I think the number one thing is just fear of what's going to happen when you have to show something at the end of a couple weeks. Is your management enlightened enough to say 'OK, I see that this partial, but this is good, this is good, this can be improved, this is missing,' without saying, 'My God, that's all you did!?'

There is a great deal of trust that is involved with Agile in terms of exposing your intermediate work product. And the trust has to occur at every level of the organization. We are being asked by a couple of large companies to come in - independent of our outsourced product development - to help with their cultural transformations internally. It's not an easy thing. In some organizations, profound changes have to happen in terms of trust up and trust down. It's subversive.

Read More