In the mid-2000s, Google hired software engineers at an extraordinary rate. Being the company that it was (and still is) it experimented with several unique ways to recruit talent. In one experiment, it posted the ad below on a billboard near the MIT campus.
Software engineers who saw the billboard, solved the puzzle, and went to the website, were given another puzzle. If they solved that one, then Google’s recruiters started the interview process in earnest.
Eventually, Google performed an internal study, and realized that puzzles like this one did not lead to better recruits. The kind of engineers who can solve puzzles are no more likely to be the kind of engineers who make valuable contributions to the company. Whether someone is good at solving puzzles provides little indication of whether or not they will be a good employee.
What’s more, becoming a good puzzle-solver does not make you good at much else. Most skills are not “transferable.” If you take an adult and teach them chess, they become no better at business. As this review puts it “Effects [of training] are mostly restricted to the specific tasks being trained and do not significantly generalize to other tasks or cognitive functions.”
On the one hand, this is bad news for those of us who want to become better at our fields of choice. There’s no shortcut to becoming better other than hard, deliberate practice in that field itself. There is no puzzle that will make you a better engineer.
On the other hand, this is good news to those early in the game. First-year graduate students spend a lot of time on problem sets and exams. But, in most fields, doing real research is very different from doing problem sets. Some of the same skills are involved, but, when it comes to actually publishing papers, many other skills matter.
You shouldn’t dismiss your talent until you’ve had a shot at the real game. In many fields, grades during your first couple years of graduate school are positively associated to eventual success. But the correlation is low. You don’t know how you’ll do until you are out there and you actually do it.