Everybody thinks they're above average...

A post inspired by 1.00 FTE:

This is a bit of an aha moment - I'd especially recommend reading the comments about what happens when the perceived capabilities clash.

I have left a job where my perceived capability was well below what I later found out to be reality. I definitely experienced a chafing-at-the-bit (or perhaps "crushed-beneath-the-boot") sensation at that workplace. I was constantly irritated that the mid-level manager didn't seem to trust me to do my job.... even when I tried valiantly to prove my capability, eventually bringing about a revolution to their help-desk system for which I still occasionally receive praise (six years after I've left).

Perhaps the manager was right - perhaps I wasn't as good as I thought I was... but given that I literally doubled my income when I walked out the door (and haven't looked back since), I doubt it.

I recognise it's difficult for non-IT-front-line bosses to accurately gauge the skill of their techies (see my old article on The economics of IT salaries for a discussion) but it's still not good when this sort of thing happens.
That being said - there is absolutely a real need to provide direction for those that need it, and to make sure that everybody's on the same page. So... what to do?

Is it better to downgrade everybody's skill level to make sure nobody incompetent accidentally gets through and influences strategy until they've "proven" they are capable? Or is it better to trust first?

I'd go for the latter... mainly because I agree with the old adage of "people will meet your expectations whatever they are"... but then I don't have to pay the bills. Opinions?

Who Cares About Forecast Accuracy?

Corporations and governments spend staggering amounts of money on forecasting, and one might think they would be keenly interested in determining the worth of their purchases and ensuring they are the very best available. But most aren’t. They spend little or nothing analyzing the accuracy of forecasts and not much more on research to develop and compare forecasting methods. Some even persist in using forecasts that are manifestly unreliable. … This widespread lack of curiosity … is a phenomenon worthy of investigation.

An eye-opening article by Robin Hanson, called Who Cares About Forecast Accuracy?, describes how and why corporations spend heaps of money on making predictions... but almost none on testing whether those predictions actually worked out.

The article discusses the fact that actually recording the accuracy of pie-in-the-sky forecasts means that it's much harder to hide a poor track-record for leadership. That a lot of forecasting is more about signaling affiliation to the company ("yeah, we're doing great, I predict we'll be done by tomorrow!") than actual accuracy. Quite possibly just another effect of bosses preferring overconfidence to accuracy.

From my own experience - I find that many "leaders" don't want you to say that the project might be in trouble... and the article discusses how this can happen because the middle-managers don't want somebody to go on record as having questioned their vision... because it'll reflect badly on them down the line should something go wrong. It's much harder to sweep losses under the carpet if nobody officially takes notice of how often it happens.

I think there's also another reason, not mentioned in the article. That being the long-term vs short-term payoffs. The strategic echelons of an organisation need your predictions so that they can make their own overall predictions of how long a plan will take to implement. But they don't want to bother with the long-term cost of implementing a full system with all the process-change and downtime that it will take. I've personally found it extremely difficult to "sell" both management and fellow-programmers on changing to to a methodology which will allow for recording and feedback of prediction accuracy. These are methodologies that aren't particularly heavy or difficult to implement (compared, say, with six-sigma), but which provide a quick and easy way to get better at providing an accurate estimate of your times.

The problem seems to be, that it takes more time and effort to implement, and people don't want to do the "work". They want to make their guesses *and* make t more accurate... but not to have it impact the "bottom ilne" in any way. This is a bit like saying "we'd like to buy faster computers, but won't give you any budget for their increased cost". You somehow have to just improve anyway.

What normally ends up happening is that you don't properly implement any system of accuracy, so you end up simply guessing and just hope for the best. Actually spending the effort to make those guesses meaningful is too much like hard work, and nobody wants to add to their workload. This is understandable... but not exactly helpful (or realistic). Instead of improving over time, we continually wallow in a world of wild guesstimates and "fudge-factors" that never seem to quite cover the extra time it always seems to take.

To get back to the article, Hanson suggests that we use a betting-system to literally "put your money where your mouth is"... the intention being that if you personally stand to win/lose by your prediction, then it would be in your best interest to start being more accurate.

I think I'd balk at my own actual money, but I'm sure some kind of system could be set up for "points" where you "win" various rewards (eg an extra day off, or even just the ability to work on "the fun project") if you do especially well.

Has anybody had any experience with estimates and measuring their accuracy? Either from the trenches or the strategic side of things?

Your customers *do* know what they want

I had a conversation with an old school friend last night; and we had a brief discussion about entrepreneurship as it relates to IT. He said he wanted to get out of the daily grind, and thought it'd be great to come up with a product and sell it. He was a bit leery, still, not sure about whether he could really "make it".

Having read a number of Paul Graham's fantastic articles I thought the idea was great, but I reminded him of how important it was to make something that your customers actually want. At which point he stated: "I speak to my customers all the time and in my experience they don't know what they want"

At the time I didn't answer. I could see he had a point, but it wasn't the one I was trying to make, and the conversation moved on. But now I think about it I think that he was simply wrong.

What question?

When you ask a user what content they want on their site or ask about styling or layout, you'll frequently bump up against a puzzled look and a lot of hemming and hawing as they try to figure it out. They don't have an answer ready for that and they probably didn't think about it before you asked. Often they have no idea what they want.

Many is the project where the requirements change like a fickle breeze as the scope creeps inexorably onward. It's obvious your customers don't know what they want. Right?

or do they? is it possible you're asking them the wrong questions?

What language?

Since I moved into freelancing, I've been studying up a whole lot more on business. One thing I was surprised to learn is that my customers are actually pretty savvy. They know their business, and they know exactly why they came to you to build their website.

Your customer knows they want a website that promotes their brand to get more customers, or to cross-fertilise their customer-base with their other product-lines, or to act as a tool that is a more efficient/less-expensive way of selling their products.

What they want has to do with their *business* goals, not with the specifics of the technology that will be used to achieve that goal. They know exactly what the goal is and whether they have reached it - and it has nothing to do with font-sizes or table layouts, or whether the "order" button should be on the product list-page or on the product's details-page.

If you don't know what your customer wants - it's most likely because you haven't asked them the right questions. But it's not enough to change what you're asking - you also have to change how you're asking it.

Your customer lives in a different world to you. You live in IT-world, but they live in the world of their business. Your customer uses a language that is different to IT language. They speak in the language of business - with market-penetration, conversion-rates and revenue. They don't operate in the realm of splash-pages, JavaScript or three-column-layouts. Business has its own language, its own way of organising and expressing ideas, and its own way of determining whether or not a set of actions will acheive a required result.

More than your job's worth

If you don't speak the language of business, then you'd better learn quick! These are your customers, and you already know they can't speak IT, and don't have the time or patience to learn. So it's up to you to do the translation service.

After all, that's your job. It's why you get paid the "big bucks" right?

It's your job to learn how to ask questions in such a way that your customer understands what you're asking and why. To translate their requirements into IT-jargon, and your questions back into business-speak. To express their choices in such a way that it gives them enough information to actually make the decision that you are asking of them.

Do they want flash or XHTML? does it matter to them if you use table or CSS-layouts?

No - what they want to know is "will this process of online ordering appeal more to my customers than the other, or will it be too difficult to learn?", "will my customers be able to find my products on this website, or will they get confused and I lose a customer?", "how can I use this software to give me better tracking of purchasing so I can promote or cross-promote appropriately?".

Tell them that if you place the "order now" button on the product-listing page, it could speed up the ordering process, at the expense of creating visual clutter. Explain that new users might get confused by this, but that it could aid frequent users to get through the process quicker. It then becomes a business decision: ie "are we aiming this site at new customers, or our returning/frequent customers?" - which is a much more meaningful basis for their decision-making.

So...

Translate your requirement questions into their language. Explain how their choice will impact them and their business goals. Explain the benefits in a way that they will understand. Make sure what you ask them leads back to their goals - not yours, and you'll find that they actually have a very good grasp of what they want.