Sunday 13 May 2007

Exuberant programming

My colleague (Peter Merel) is the sort of person to get enthused at trying new stuff out. He is an Agile-evangelist, and has been trying all different ways to make this process fit into corporate-land. The main issues involved in this tend to be getting buy-in from the usual management-types who have never been exposed to this methodology, and finding ways to interpret it into the sorts of controls and reporting guidelines that they are used to.

Don't blow up the universe

The kind of "No, no! everything you're doing is wrong, get rid of it all and work our way!" approach tends to be frowned-upon as disruptive to standard operating practice. For good reason - it is.

Changing over an entire system of business practice (such as how to develop software) is a Big Thing for any company. Techie types will often scoff and refer to the inertia of corporate behemoths, entrenched dinosaurism and a corresponding lack of corporate flexibility, but the managers at these places actually have a good point. Changing systems involves time, effort and risk of failure (whether total failure of the system or simply setbacks that cause loss of market share).

A bigger company has more systems and people involved - and so more chance of breaking something valuable. They will generally have an entrenched and provably working system of practice. It may not be the best that is out there, but it has gotten them to where they are now. It's up to you to prove to them that what you're offering is not only better, but that it's worth more than the pain and effort involved in changing over.

Managerese

A complete re-write of corporate practice tends only to come onto the books once the proverbial hits the fan. At which point the company is desperately flailing around for any lifeline they can reach. But most companies actually aren't in this position, which means you have to actually, you know, convince them that what you're doing is a good idea (wierd huh?).

This requires an ability sadly lacking in most techie-types... the ability to speak with management in their own language.

There are a number of supporting tools that assist managers to get a handle on a project - tools they are used to and understand. The requirements spec, the gantt chart, todo lists and regular meetings. They not only understand thse, but know how to integrate them into a working whole. They use them to to juggle tasks and developers, to track project progress and momentum and to basically Get Things Done.

If suddenly you arrive on the scene like some animated furry-toothed geek enthusing wildly about "user stories" and index cards and stand-up meetings; sneering at the locals with their endless requirements specs, gantt charts and deliverables; refusing to work with management unless they change all their habits to match your own... you'll undoubtedly gather some resistance to your methodology ideas... even if they would be better off in the long run.

Bridging Mount Learning-curve

So what can you do to bridge the gap? Is there a way to implement all of our agile programming practices, yet still present the incumbents with the tools they are used to seeing? Can we apply the Principle of Least Surprise to our stakeholders too? Can both worlds co-exist in joyful harmony?

This is what we're currently working on. We call it "exuberant programming". It's a set of tools that allows us to be "abundantly fruitful" with our productivity (using the latest in agile methodology), while still giving joy to the more traditional managers.

The project managers are already productive and don't have the time to scale Mount Learning-curve. So let them have their requirements specs, but instruct them how to structure them using user stories, with each sentence a function point that can be fed into the priority planning-game. Let them have their Gantt charts - but use them to show User Stories. Update them at the stand-up-meetings and generate burn-down charts for the managers to let them see the project velocity.

We can work together. We can have peace in our time! We are the world... <waves lighter in air>

er, sorry, got a bit carried away there...

You can see the potential of what we're proposing. We're still working on the details. I'll post more about it as we develop it.

2 comments:

Peter Merel said...

Pete here

getting buy-in from the usual management-types who have never been exposed to this methodology, and finding ways to interpret it into the sorts of controls and reporting guidelines that they are used to.

That's sort of the idea but there's more to it. For a start for gorsake don't call it Exuberant Programming; we already have an XP and it works just fine, so don't fix it.

The problem is the rest of organizations don't work the way XP works. Yes there are plenty of joints where some mighty caryatid of a manager holds up the roof so XP will fit underneath; this works right up until caryatid falls beneath her stone, whereupon, just like the original XP project, a mess occurs.

Rather than "Exuberant Programming" then I've been thinking about Exuberant Enterprise. Basically I see no reason why XP should be an orphan child in an organization. I see the mismatch between agile and waterfall not as a matter of impedance, but of language.

I don't believe we need to reinvent XP or SCRUM; those are nice well-formed highly productive ways to turn the crank. I think there's a need to define a way to do these things so that waterfallen can grok 'em.

The two points of contact I'm looking at right now are Gantts and FitNesse. The Gantt ideas are described in brief at http://www.c2.com/cgi/wiki?AgileGantt . I'm producing the first of these critters in conjunction with some conservative stakeholders at my gig's (large) strategic partner and will put up an illustration once the sharp edges have been filed off.

The Fitnesse stuff is more a matter of integrating test processes. If you're doing Rails in a corporate context you've got yer built in programmer/functional/integration tests, yer WaTiR black box testing, yer FIT acceptance tests, yer continuous integration tools, and also some poor buggers in "testing" or "qa" promoting some proprietary testing tools to boot.

What I want is to connect these tests together so that their results can be automagically reflected in the Gantts. Yes, you heard it here, I want Gantts to be TRUE.

Don't blow up the universe

The kind of "No, no! everything you're doing is wrong, get rid of it all and work our way!" approach tends to be frowned-upon as disruptive to standard operating practice. For good reason - it is.

No, no! Blowing up the universe is wrong, get rid of it all and work our way! :-)

Yeah, when you've got all these Prince/PMBOK/TOGAF/Zachmann/CMMI/Proprietary-what-have-you button sorting methodologies arrayed against you it's very tempting to want to blow 'em all up.

At the end of the day, however, all of these things are subvertible illusions. They deal in big chunks of work and large organizational constructs. They're frustrating because they distinguish things that aren't actually distinct, and reduce processes that aren't actually reducible. Read Alexander's famous A City Is Not A Tree to get a feeling for why they're so completely illusory.

And I said subvertible. Exuberantly I have it in mind take the artifacts of the illusions and map them onto agile metrics. At least that's the programme - I don't have a complete approach to doing same yet.

A bigger company has more systems and people involved - and more chance of breaking something valuable. They will generally have an entrenched and provably working system of practice. It may not be the best that is out there, but it has gotten them to where they are now.

Actually, usually, it's not what's got them where they are now. What's got them there is the original innovative work that formed their core business generator - technological, political, or marketing innovation. The processes organizations evolve as they grow are the results of committees and policemen trying to tree-ify something that was once a flowing stream.

The metaphor then is to let the tree have it's water ... while keeping the stream flowing.

It's up to you to prove to them that what you're offering is not only better, but that it's worth more than the pain and effort involved in changing over.

Which you can never do. But town hall doesn't just exact property taxes. It also provides a huge amount of essential infrastructure. The idea with exuberant isn't just to hide agility in an organization; it's to provide a way for the organization to grow and use its agility to the maximum effect. Ra ra ra, etc.

Managerese. This requires an ability sadly lacking in most techie-types... the ability to speak with management in their own language.

I hate talking managerese. It's worse than COBOL. But it's a necessary part of my job so I want to learn to do it efficiently. So I just think it's part of the problem for any senior geek, architect, or engineer. We sure as darn tootin' are not and never will be managers. We just have to manage because it's how we get things done.

Getting Things Done personally is a liberating thing. It makes you calmer, happier, less concerned with the exigencies of life, more focused on the things you actually want to do, and more content with the way the world pushes and pulls at you.

Exuberant Enterprise should do the same thing for your business that GTD does for you as a person.

If suddenly you arrive on the scene like some animated furry-toothed geek enthusing wildly about "user stories" and index cards and stand-up meetings; sneering at the locals with their endless requirements specs, gantt charts and deliverables; refusing to work with management unless they change all their habits to match your own... you'll undoubtedly gather some resistance to your methodology ideas... even if they would be better off in the long run.

You can seduce managers into agile development - thousands of geeks have done that worldwide. But this just exports the impedance mismatch because then they have to explain the weirdness to their managers. At some point the buck stops with a caryatid. The notion with exuberance is to stop the buck right where it lives, in the translation layer between an agile development and its managers, so that they can keep right on managing using their traditional waterfall artifacts, and their managers will be none the wiser.

So what can you do to bridge the gap? Is there a way to implement all of our agile programming practices, yet still present the incumbents with the tools they are used to seeing? Can we apply the Principle of Least Surprise to our stakeholders too? Can both worlds co-exist in joyful harmony?

Yeah, I was thinking of calling it "Joyful Development" ...

This is what we're currently working on. We call it "exuberant development". It's a set of tools that allows us to be "abundantly fruitful" with our productivity (using the latest in agile methodology), while still giving joy to the more traditional managers.

Damn we are noble. Also, lazy, impatient, and proud. We are lazy so we don't want to do things the waterfall way. We are impatient so we don't want to waste cycles justifying agility. And we are proud, so we want to produce waterfall artifacts we could put on the fridge along with the childrens' paintings - Refrigerator Gantts - Hooray for Larry Wall!

The project managers are already productive and don't have the time to scale Mount Learning-curve.

Are they productive? Or are they just doing what they have to so the older managers don't box their ears?

So let them have their requirements specs, but instruct them how to structure them using user stories, with each sentence a function point that can be fed into the priority planning-game.

Having just played this ... and still playing it ... they don't want to prioritize. They want to draw lines in the sand. The trick is to prioritize anyway. Do it in Gantt form and they have no choice but to engage with you ...

Let them have their Gantt charts - but use them to show User Stories. Update them at the stand-up-meetings and generate burn-down charts for the managers to let them see the project velocity.

Nah, they won't understand burn-down or project velocity. But they understand baselines and resource efficiency on a Gantt. Of course baselines and resource efficiency don't tell you as much as a burn-down, or preferably burn-up. But hey, let the baby have it's bottle! The only trick we need to pull here is putting the milk in it - translating the burn-up into Gantt baseline variance and the project velocity into Gantt resource efficiency.

You can see the potential of what we're proposing. We're still working on the details. I'll post more about it as we develop it.

Hmm. Blogging live project politics is risky ... if our stakeholders start reading this stuff we're cooked. But we can say how we're doing what we're doing, and that's a fine subject for blogging ...

Taryn East said...

There's gotta be a rule about a comment being larger than the original post ;)