This is exactly what always bothered me about Google's white board testing (and testing like it at other big firms). It lets them know that you can do pointers and O notation but it completely misses the specific kind of tenacity that seems to make great programmers great.
But it does filter out people who don't have that tenacity. Those people who do have that kind of tenacity would already have learned about pointers and O notation.
Personally, I've always (well, at least since I quit programming assembler for a living[72-79]) felt that pointers, especially C type pointers, were evil, and any language that has them available without some hoop jumping or big warning signs, is to be avoided.
I think Gladwell has a good point that trying to do a job is probably the best predictor of who will be good at it, and as a general principle moving that point up, via apprenticeships, internships, coop programs, and so on would probably be useful.
What troubles me is he seems to say that because people do a bad job of objectively identifying predictors of future success, that those predictors simply don't exist, or at least aren't strong enough to be useful. Did the "exhaustive survey" of law graduates he mentioned really find no correlations between something that could be measured before a career starts and eventual success at that career? Perhaps there are no good predictors in various fields, but I think I'd like to see some much stronger evidence before I buy that.
You're being skeptical for the wrong side. We humans have a long, inglorious history of inventing objective metrics that absolutely fail to be useful at predicting the things we want to predict. From the IQ tests to the NHL combine, we're traditionally quite awful at inventing predictive tests of performance.
Which is to say: if you're being intellectually honest, you should be far more skeptical that a test does what it claims to do, than you should be of the opposite argument.
Assuming that linked chart is accurate (which I don't automatically grant), pointing out that the average IQ for a janitor is lower than the average IQ for a doctor isn't the same thing as predicting someone's success as a doctor based on their IQ. There's a lot of overlap between those ranges.
We should be skeptical that any particular metric does what it claims to do, especially ones which are ad-hoc or based on intuition instead of evidence. I'm not suggesting that Gladwell is wrong about any of the examples he gave being 'mismatches', or that there are many of them, and exposing the misuse of metrics is certainly valuable. However, he seems to go further, suggesting that we can't find good predictors at all, which is a claim which I think deserves just as much skepticism. I doubt he would seriously make such a claim, and it is probably just an artifact of an early talk which won't be present in his book, but I have plenty of skepticism to go around, so I'm perfectly happy to deploy it wherever it is warranted at all.
It's a self selecting population so a lot of statistics ideas need to be tossed out. If you test random 5 year old people with IQ's under 80 are unlikely to become successful doctors / chemists etc. If you look at the IQ of people graduating from Medical School it's hard to correlate IQ's with job success because you are looking at a population that passed a lot of tests to get to that point. If you look at the median pay of Harvard Graduates its well above that of the average Collage graduate but it's also taken from the high end of the High School talent pool.
Gladwell is wrong about being able to fix the black-white performance gap in three years using better teachers. Not only does his argument go completely against all the prevailing research, but it doesn't even make sense because every white student would have to be assigned a bad teacher for the numbers to add up. (C.f. Equality & Achievement, Hart & Risley, etc.)
The point being that people make tradition-based voodoo over numbers-backed science all over the place. This is a hugely disruptive opportunity. For example a software company that hires people differently can propel itself to the very top in a span of just a few short years.
Alex, I do not see how the content of the links you posted addresses Malcolm Gladwell's contention that good teachers could close the Black-White gap. To refute that idea one would have to get contradictory data on a scale at least as large as Gladwell's.
I do wonder how much of the black-white gap is due to teachers: I would wager that the teachers(on average) that blacks have are worse than the ones whites get.
Almost everything he says needs a [citation needed] tag at the end of it. I guess that's one problem with public speaking--it's pretty hard to take apart in that manner. One of his comments highlights his overall problem,
...that ineffible, illusive gift, "Being a good teacher"
...in other words, he seems to think that teaching ability is "ineffable" and that therefore the best way to find the good ones is to (???) hire a large number, find the best ones, and fire the rest. Or something like that. One way or the other, objective measures are to be thrown away. That's his overall point, but that is pretty impractical. Ultimately, you have to measure people in some manner. You can't hire uncredentialled people on the chance that they might be good--it just wouldn't work that way.
I don't know why he went on about sports for so long. The "combines" that he talks about are actually the last measure used to weed out prospective hires. The sports agents actually do their most important work watching the kids perform athletics at a high school and college level. Likewise, smart employers of programmers will look at past projects and jobs, and only then subject them to brain-teasers and white-board coding. That's always how it's been.
ADDENDUM: Does it seem like Gladwell waxes a little too long about the physical beauty of the Russian hockey player at the beginning:
That argument makes no sense. We have to measure people? Really? Why? If his claims are correct, then Gladwell is right -- we're doing worse by screening people according to criteria that are useless, because we're essentially randomly filtering out a large percentage of the people who are good at the job.
You can attack the evidence for his claims, but if the evidence holds, then the conclusion Gladwell reaches is a logical consequence.
I'm sympathetic to his claims, as I went to a crap school and got a crap GPA. Okay. I'd love it if people had judged me based upon my winning personality, my good looks, or my tendency to point out everyone's foibles, but I'm not sure those are a superior measure of coding skill... Even if they were, they could be subjected to analysis and so become objective measures themselves.
The practical reality is that lots of jobs have 1000s of aspirants. How many kids want to play in the NBA? MILLIONS. You need something to divide them up, and if one objective measure is genuinely bad, it needs to be replaced by a better objective measure.
If the "objective metrics" are useless discriminators, then the only effect of using them is to make us feel as though we have control over a situation that we don't understand. In that scenario, it makes far more sense to restructure the game to allow more initial participants, and to cull the best players.
You seem to be arguing that we must have "objective" filtering mechanisms, even if they're random. Gladwell is saying that the whole paradigm is wrong.
"hire a large number, find the best ones, and fire the rest"
Actually, in many cases this is the only way to find the best candidates. The near impossibility of firing teachers is the biggest impediment to improving teaching quality in the United States.
I don't think the problem is finding great teachers it's finding any teachers. I was a great tutor in Collage but I would never become a teacher because the pay sucks. If we payed teachers 100k / year and fired the poor ones we could have a great talent pool but as long as they make 1/2 what I do there are just not going to be a huge pool to draw from.
I don't know if he's conscious of it, but Gladwell is parroting Socrates, right down to the impossibility of transmitting "true knowledge". Circular arguments anchored by a mystery. As a philosophy it has no practical purpose except to waste people's time while appearing sage.
Are those silly riddles about candles and ropes and bridges and shit that they ask you in Microsoft interviews our profession's version of The NHL Combine?
... except that Microsoft doesn't ask questions like that for software engineering positions, at least not when I interviewed there. I found the interview questions to be reasonably similar to Google's.
It's interesting that Gladwell doesn't bring up baseball in his examples.
If you read "Moneyball" by Michael Lewis, you come away with the impression that there is a definitive way to predict the probability of future success, albeit using untraditional metrics.
Would be interesting to know if software engineering has also a mismatch problem. Is there any information about that? But in any case it should be easier to find a good programmer than to find a good basketballer because its easier to test them while they are doing their actual job.
The mismatch problem should also be an important topic for YC - it's the daily work of an VC to predict success. Maybe they could provide Gladwell with some interesting statistics for his book ;)
