At Agile 2008 James Surowiecki, the author of Wisdom of Crowds, gave the opening keynote address. If you have an opportunity to hear him speak, take it. He did a great job. No power point slides with a compelling style and message.

He told of some interesting findings. One of his stories is about a crowd at fair guessing the weight of an oxen on display. The original study was done a while ago, I think in the late 1800’s but don’t quote me on that. The person that organized the study was trying to prove something about eugenics. He wanted to prove that a crowd cannot be as smart as an expert. Paraphrased, his hypothesis was that the crowd is a bunch of dummies and their opinion could not be trusted.

Counter to his hypothesis, the crowds guesses were very good. When averaged, they came within one pound of the actual weight. No expert (butchers, farmers, cattlemen) did better. The average of the experts did no better either. The eugenicist was dumbfounded. James S. recounted additional stories and studies leading to his hypothesis that crowds can indeed have greater wisdom than any individual.

Some of the points I captured:

• Teams do better than experts.
• Diversity within a group is needed.
• The more diverse the smarter the group. He is referring to diversity of knowledge and opinion.
• A random group does better than an expert group.

But we develop software in small teams. How does his work on Wisdom of Crowds apply to small crowds?

He told a story of a small group experiment. Three different length line segments were projected on a wall. In another area a single line was drawn. One at a time, each of the eight participants made their selection. The first few rounds they all matched the single segment with the one of the same length. Ahhh… but the last person did not know that he was being setup. The next rounds of line length guessing had the first seven people intentionally pick wrong answers. They were in on the joke. By the time the last person went, they followed the crowd and choose the wrong answer. This happened over and over again. During some of these experiments the earlier guessers were told to heckle those that choose differently. I think I’ve been in that meeting.

I guess the conclusion is that people tend to follow the crowd. James also made these observations:

• The more a homogeneous group, the dumber it gets (this is good news for consultants)
• If one person always plays devil’s advocate, their opinion will eventually be ignored. (in my experience they DA becomes an annoyance and destructive to the team)
• Teams should get along, but if they get too chummy, they will start to get dumber, as they go along to get along.

Devil’s Advocate

He told us that the term “Devil’s Advocate” was originated in the Catholic Church. While vetting a candidate saint for canonization, the church would appoint a devil’s advocate, whose role was to argue against the canonization. Each investigation would get a new DA. The speaker told us that a devil’s advocate can be very effective to guard against unhealthy group-think. But cautioned that the DA must not always be the same person.

You’ve seen it yourself. Bart, a senior and skilled team member, always looks for and finds holes in any proposal on the table. He might think he is providing an important function for the team, warning of these problems. But, over time the team will stop listening to his endless negativity. Bart might go silent. Once this happens the team may invite Bart’s counter opinion, in an effort to be fair minded. When that formality is complete his input is ignored.

Maybe instead of having the natural devil’s advocate play that role, appoint a devils advocate to argue against the consensus position. Make sure it is not a permanent position. Make it rotational. Do you already have a natural devil’s advocate? Maybe its time for an understanding talk with the DA to warn of their potential ineffectiveness.

Planning Poker

I was wondering how Wisdom of Crowds would relate to people on agile teams doing estimation and planning. I was specifically interested in how his research applied to Planning Poker, a practice used throughout the world on agile teams. Did his message support planning poker? BTW, he was not speaking about Planning Poker, just about Wisdon of Crowds. I picked a few things he said that I think relate to Planning Poker.

• Look out for people that talk too much. talkative != smart && talkative != right.
• Leaders have to make sure they do not telegraph the right answer”.
• For groups to work people must act as individuals.
• The best group decisions take time. You have to be willing to have a good fight.

Planning poker came from a meeting where two guys were monopolizing the estimation discussion. We were getting nowhere. We were running out of time. Each estimate took forever, even where there was almost immediate agreement on effort between the two experts. They debated back and forth to be sure. The telegraph lines were singing. The group was getting dumber, or at least sleepier.

With blind estimation, where each Planning Poker Player holds thier cards close to their chest, the telegraph was shut down; each person got engaged. They had to think individually, for themselves, to arrive at their estimate. They may also have to be ready to defend their individual estimate. They were now engaged.

Someone else in the Agile 2008 crowd must have been thinking about Planning Poker too. They asked “what role does feedback play in arriving a good solution?” I was expecting that the practice of discussion and re-estimating in Planning Poker was about to get a hole shot through it. James had been talking more about statistical results with the big crowds. I was expecting to hear that the initial guesses should be averaged. Planning Poker, with its discussion and re-estimation rounds, is more of a technique to get to consensus, not averages. He replied that a 2 to 3 rounds of feedback and re-voting can help a small team come up with better results, converge on a better decision. In Planning Poker people have to stand up for their position, they have to listen to others, they have to vote again. It usually takes 1-3 rounds to get to consensus.

Something else just dawned on me. In a sense there is a rotating devil’s advocate built into Planning Poker. Whenever there are estimates outside the consensus, the person has to defend their position.

I did not see anything in the talk that argues against the practices in Planning Poker. It’s always good to see research supports what many people already believe.

Prevent your team from getting dumber

What can a well-gelled team do to avoid getting along too well and and getting dumber as time goes by?
Here’s some ideas to avoid institutional dumbness:

• Bring in new people, new ideas.
• Beware of too much agreement.
• Appoint a rotating devil’s advocate.
• Move people between groups.
• Use blind techniques like planning poker where opinions are expressed before hearing the right answer
• Beware when leaders telegraph the right answer.

I better order The Wisdom of Crowds. It sounds like sound research that can help software development teams.