Stack Overflow and the Tyranny of the Majority

As a reader of Joel, I was interested to try out Stack Overflow, which he has been talking about for some time, and just came out into a public beta.

It's a project developed in conjunction with Jeff Atwood that lets programmers ask questions, and other programmers answer them. The idea is that programmers vote up the questions and answers they think are good, and vote down the ones they think are bad.

In cases where there is a widely-known right answer, it works nicely. For example, the question, "how do you print information to the console in Java?" would be easy -- everyone knows to use System.out.println(), so answers that were way off base would be obvious to detect, and the question voted to the top would likely be the one to include extra information about redirecting output streams or whatever.

When I tried it out, I answered questions in the Java arena, since I know the subject well. Most things went smoothly, because many programmers know Java.

However, I also answered questions in the mathematics arena, since I'm ok at math. This is where things did not go as well. The right answers for the math questions seem to constantly lose out to the answers that would seem right to the average person who gives them a cursory glance, but are actually incorrect.

In some cases, the top voted answer contains some minor logical flaw. In other cases, the top voted answer is way off and comes to the exact wrong conclusion. No matter how hard I know and can argue for the right answer, I only have the same voting power as everyone else. If the majority of programmers think that you can reverse-project a 2D shape into a unique 3D shape, then that's what's going to show up as the "right" answer on stackoverflow.com.

Edit: Since I wrote this post, the right answer for the reverse-projection question has made it to the top, ruining my example. (Yes, I wrote it... how arrogant of me to assume my answer is right, eh?) But, if you browse around the "math" tagged questions, you'll probably see what I mean.

This phenomenon has a name: it's a tyranny of the majority. It's not the best answer that necessarily wins, it's the answer that most people are convinced is right.