April 21, 2013

Can you think out of the blender? and other oddball questions (Part 2)

A month or so ago, I asked about oddball questions in job interviews. Some are related to job function but some are challengingly perverse. If you are, like me, disturbingly offended by a dark and fearful force within those questions, then it is to you I address this rant in three parts (I, II, III)

Where are these oddball interview questions coming from? Why are they so deliciously amusing yet so dangerous and so ... off-mission? I hope to clarify this in today's rant, but before that, a word of warning -- if you are in the HR industry, you will not be happy. If you are proud of your company's achievements in recruiting, you might be happier reading elsewhere. That warning given, press on!

Why do companies ask oddball questions in recruiting questions? And why does our ever-suffering foil, google, lead the world in this?

It's not because the employers are dumb, as indeed their strategy (and this goes for all high tech players) is often to find the smartest of the smart. So they ask smart questions. OK, I'll pay that as signal at least. Let's present another foil:

* "You are shrunk to the height of a nickel and thrown into a blender. Your mass is reduced so that your density is the same as usual. The blades start moving in 60 seconds. What do you do?"

Yeah. So, physics or biology or again, simply just brain teasers. What the questioner is trying to do is figure out if you are smart. Or at the minimum, whether you can whiteboard, draw in different ideas, whether you can think out of the blender, so to speak.

But, if they can truly do that, then they're onto something rather extraordinary. The world has been researching the alchemy of intelligence for a long long time. We've got IQ, EQ, psychometrics, Myers-Briggs, and a whole host of other things. None of them necessarily work to figure out if anyone is smart. Indeed, apparently, this is admitted:

"We'll get to the longer answer, but the short answer is that Google isn't looking for the smartest, or even the most technically capable, candidates. Google is looking for the candidates who will best fit Google."

Right. Thank you. Indeed, their HR mission bears repeating:

"Google is looking for the candidates who will best fit Google."

And who are they? That's a question akin to asking who's smart -- Alchemy! Let's employ occam's razor here and say it bluntly: people like the people who are already there. Look at this page and you'll see that there is an obsession with basic computer algorithms. So, if the founders liked brain teasers, then brain teasers they ask. If they liked economic problems such as "solve world hunger" then that's what is asked. Google's founders were academics in Computer Science, so they ask deep questions that are taught in CS school and recorded in Knuth. Etc.

In short, the HR mission of choice is:

People like us.

This is not really a new thing, indeed it could almost be considered best practice in hiring. The Economist distills an entire book entitled What It Takes: Seven Secrets of Success from the World's Greatest Professional Firms by Charles Ellis down to one line:

Above all, these firms are fanatical about recruiting new employees who are not just the most talented but also the best suited to a particular corporate culture .

The only problem is, this strategy has to be the greatest failing of any company... Indeed this is also known, in a perverse sense:

4. Only hire people who like you or think like you. Flattery feels good, but it doesn't pay the bills....

We can say it another way: /monoculture/. The good thing about the term of art /monoculture/ is that tech companies actually know what that means, courtesy of a famous or infamous paper by Dan Geer & friends entitled "CyberInsecurity: The Cost of Monopoly." It was famous because history proved it to be true, as it correctly identified the seeds of Microsoft's fall from grace; it was infamous because the lead author got fired within 24 hours for daring to question the powers that were.

Monoculture in hiring is pernicious because it is so attractive: I don't know how to find the best people because I don't understand the question, but I'm one of the best, so I'll just one look for people like me. Works for me.

The strategy sounds entirely logical, but it is based on several false assumptions. Let me rant a bit: For a start, there is research out there that suggests that people routinely rate themselves higher than others do. Geer et al suggests that diversity is better than monoculture for survival in an aggressive world, and wider research backs that up. Indeed, Belbin says diversity is critical for success, suggesting teams of 4 people minimum with 8 opposing roles. It's been known since 1980s (? Ed Yourdan?) that productivity varies as much as 20-fold. Mark Zuckerberg said somewhere it was a 100-fold difference. I'd say even a ratio is a hopeful figure because the difference between nothing and success in programming is far more than 20-fold. Divide 1 by 0 to see what I mean, I personally have suffered with programmers who could not do the job at all, and the result was zero, and often into the negative numbers if we add in indirect costs. Yet, for reasons that aren't easily explained today, we cannot easily apply the Star effect to the discipline of computing (tantalising clue), so the normal industry response fails. I could go on with what I know, but I can't easily cite the papers, and they're never checked anyway, so I'll stop here. End of rant.

Back to the subject, which is why the oddball question technique is so troubling: asking and answering smart questions may be a signal for productivity, but I'd reckon it's a false signal. Indeed, I'd suspect it is somewhat reversed. Really productive people don't have time for brain-teasers, whereas really unproductive people are surprisingly good at copying the signals of success. To succeed in an interview, you have to think quickly, but to succeed in productivity, you want to thing slowly -- the really difficult issues take some time to unravel.

So what happens when we run the reversely-correlated signal of /be like us/ up the flagpole? The Economist says:

In the past Mr Ellis would have praised Arthur Andersen, an audit firm that went bust in 2002 after getting embroiled in the collapse of Enron, a client. But the seeds of Andersen's demise were sown decades before, when the firm started to put its professional obligations to its clients second to the ambitions of its partners. Goldman began committing similar sins in the 1990s as its then leaders started to dream of awarding themselves fortunes by taking the firm public.

Echoes of Microsoft. The strategy works -- we ask for a lot of people like us. And we get them! I'll posit that this result is rational, expected and inevitable: copying the behaviour of rich partners of famous firms is somewhat easier than placing ones obligation to ones customers. Indeed, I'd lay good money on it: the job interview process is a race track of people to copy:

Each McKinsey applicant can be interviewed eight times before being offered a job; at Goldman, twice that is not unheard of. At Capital a serious candidate is likely to be seen by 20 people, some more than once.

Yet whoever heard of a customer at an interview? It's easy enough to meet people in bars, indeed we've even got a name for it: networking. But how do I get to meet McKinsey's clients? Google's customers? What's the name for showing you can meet your client's expectations? I don't have one on the tip of my toungue, do you? Certainly, there isn't a word as familiar as /networking/ or "pretending to be like you."

So what's really going on here?

The deep, dark secret of human resources is that traditional job interviews don't work very well.

Thank you WSJ. This is the big picture, actually written a bit too narrowly. Traditional HR doesn't work well at all. Job interviews included. Indeed, all of the recruitment process looks like a shoe-in for Spence's Market Signaling syndrome where signals can be pursued as essential, and have zero relevance to the business. With stability, but more on that later.

Not convinced? Let's consider more tantalising evidence from the external recruiting angle. If the company itself has no better signal than /people like us/ what then does the headhunter do? The evidence suggests that the process is best labelled as Star Gazing:

The study's "gaze tracking" technology showed that recruiters spent almost 80% of their resume review time on the following data points:

* Name
* Current title/company
* Current position start and end dates
* Previous title/company
* Previous position start and end dates
* Education

See what recruiters are looking at? What another company looks at. In short, if you're employed at a competitor, you're golden. If not, you're dead. Which is pretty convincing evidence that the most important thought a recruiter has is not his own client (nor his client's client) but the thoughts of a close competitor!

More on star gazing:

The best time to contact a headhunter is when you are employed.

"Headhunters don't typically work with job candidates that are unemployed," says Terri Lee Ryan, a career coach and author.

"Companies don't pay them big money to present workers that aren't gainfully employed. In this market there are many good workers on the sidelines, yet companies still want to see candidates that are gainfully employed and on the 'top of their game.' This is why I tell workers to never quit their job until they have a new one."

"These days, you never know if your job could disappear tomorrow," says Erik M. Tomasi, Chief Operating Officer of DTG Consulting Solutions Inc. "Anticipate the problem before it happens by networking and responding to headhunters, even when you're happy with your current job."

This suggests that your current employer is the primary signal that matters to the headhunter. Not you yourself, not the recruiter's client, and certainly not the client's client. This makes perfect sense if and only if the buying company has a strategy that only it can interpret, and cannot therefore explain it measurably to the recruiter, *and* if the recruiter cannot otherwise measure the quality of you, yourself. Check -- /people like us/ is almost by definition internally rich and vacuous to outsiders. Check -- techniques used are susceptible to copying and preparation.

In order to deal with this, the recruiter elevates the signal of competitor company to the top, and other things that don't deliver to the bottom.

Hence, /people like us/ leads to /companies like us/ which becomes the zero-sum game: we don't recruit, we poach. It's as if the ghost of IBM has come to haunt us -- nobody ever got fired for poaching (from IBM?)

Smart companies sometimes spot this and decline to enter the zero-sum game. Kudos to them. One controversial approach is the infamous no-poaching agreements. Another is smallprint notices on its websites to effect of "offers from headhunters will be plonked." A company may not be able to figure out the trap of /people like us/ but a smart company should be able to see the destructive pathetic lowest-common-denominator results of its sister strategy /companies like us/ .

What goes around, comes around. The late, great Steve Jobs was right in principle if not in method. To paraphrase his message, "You do business with me? No poaching the people you interact with in our partnership. I do constructive business with partners, not destructive business with opportunists."

In summary, we should now see why google leads the world in oddball questions. Google is a company chock full of people who like brain teasers, so they ask brain teasers in interviews. It does computer science at its core, a world full of oddball people. As the mix of CS and brain-teasers is impenetrable to other mere mortals, it appears to be the smartest of the smarts. On the outside, to the HR industry, at least. Who could see wrong in this?

However, the practice traces directly back to its HR mission of /people like us/ and is thus revealed as that humblest of concepts, best practices. Unfortunately, the so-called best practices are also a proof of mediocrity: if you cannot figure it out yourself, copy what others do. Like all best practices, it is a strategy that is weak enough for all to implement, more satisfying than doing nothing and allows you to talk loudly and confidently at the bar.

By asking the smartest CS brain teasers, google may appear to be the smartest company on the planet, but this is an accolade of Pyrrhic proportions to any firm that really cares.

What to do about this terrible state of affairs -- if one isn't Steve Jobs or Eric Schmidt or Mark Zuckerberg that is -- I'll leave to another post.

(This is part 2 of 3 parts: I, II, III)

Posted by iang at April 21, 2013 11:09 PM | TrackBack

"This is the big picture, actually written a bit too narrowly. Traditional HR doesn't work well at all."

Is this proposition also too narrow? Perhaps we could say the modern corporation doesn't do well at all. After all, the Director of HR must be making someone happy, and that person reports to the CEO. None of them care to make any "line" employees happy. They only care to make shareholders happy in limited, specific ways, and with modern Golden Parachute Syndrome only for limited periods.

But that wider indictment is contradicted by reality: if corporations are so bad why do we have so many of them? Why do so many people work at them? For a full accounting, we'd probably have to condemn most of society. Instead, let's identify the specific tasks at which corporate HR does excel, and...

...well I'm drawing a blank there. This area could use more study.

Posted by: Jess at April 23, 2013 11:16 AM

You're not the only one grappling with recruitement:

How Big Data Is Playing Recruiter for Specialized Workers
Jim Wilson/The New York Times

Jade Dominguez, 26, never went to college, but was hired by Gild, a start-up, after its algorithm put him atop a list of programmers.
Published: April 27, 2013

WHEN the e-mail came out of the blue last summer, offering a shot as a programmer at a San Francisco start-up, Jade Dominguez, 26, was living off credit card debt in a rental in South Pasadena, Calif., while he taught himself programming. He had been an average student in high school and hadn't bothered with college, but someone, somewhere out there in the cloud, thought that he might be brilliant, or at least a diamond in the rough.

"The traditional markers people use for hiring can be wrong, profoundly wrong," says Vivienne Ming, the chief scientist at Gild since late last year.

That someone was Luca Bonmassar. He had discovered Mr. Dominguez by using a technology that raises important questions about how people are recruited and hired, and whether great talent is being overlooked along the way. The concept is to focus less than recruiters might on traditional talent markers -- a degree from M.I.T., a previous job at Google, a recommendation from a friend or colleague -- and more on simple notions: How well does the person perform? What can the person do? And can it be quantified?

The technology is the product of Gild, the 18-month-old start-up company of which Mr. Bonmassar is a co-founder. His is one of a handful of young businesses aiming to automate the discovery of talented programmers -- a group that is in enormous demand. These efforts fall in the category of Big Data, using computers to gather and crunch all kinds of information to perform many tasks, whether recommending books, putting targeted ads onto Web sites or predicting health care outcomes or stock prices.


kr, Twan

Posted by: How Big Data Is Playing Recruiter for Specialized Workers (from Twan) at April 29, 2013 03:38 AM

David Lewin, a professor at the University of California, Los Angeles, and an expert in management of human resources, said that asking what someone could do was an important question, but so was asking whether the person could accomplish it with other people. Of all the efforts to predict whether someone will perform well in an organization, the most proven method, Dr. Lewin said, is a referral from someone already working there. Current employees know the culture, he said, and have their reputations and their work environment on the line. A recent study from the Yale School of Management that uses Big Data offers a refinement to the notion, finding that employee referrals are a great way to find good hires but that the method tends to work much better if the employee making the referral is highly productive.

For his part, Dr. Lewin is skeptical that an algorithm would be a good substitute for a good referral from a trusted employee.

Posted by: People like us! at April 29, 2013 04:01 AM


Employee referrals are a very common means by which firms hire new workers. Past work suggests that workers hired via referrals often perform better than non-referred workers, but we have little understanding why this may be. In this paper, we demonstrate this is because referrals allow firms to select workers better-suited for particular jobs. To test our model, we use novel and detailed productivity and survey data from nine large fi rms in three industries: call-centers, trucking, and high-tech. Referred workers are 10-30% less likely to quit and have substantially higher performance on rare "high-impact metrics" (e.g. creating patents and avoiding work accidents), despite having similar characteristics and similar performance on non-rare metrics. To identify the source of these behavioral differences, we develop four new statistical tests, all of which indicate that referrals benefit firms primarily by selecting workers with a better fit for the job, as opposed to referrals selecting workers with higher overall quality; to referrals enabling monitoring or coaching; or to it being more enjoyable to work with friends. We document that workers refer others like themselves, not only in characteristics but in behavior (e.g. unsafe workers refer other unsafe workers), suggesting that fi rms may gain by incentivizing referrals most from their highest quality workers. Referred workers achieve substantially higher profi ts per worker and the diff erence is driven by referrals from high productivity workers.

Posted by: "The Value of Hiring through Referrals," April 24, 2013, Burks, Cowgill, Homan, Housman at April 29, 2013 04:12 AM

The entire notion that a college degree "signals" something valuable to employers is breaking down. In the good old days, earning a college degree proved that a student was hard-working and conformist--just what hierarchical corporations and government agencies want in employees. (The "signaling" value of a diploma is based on work by economist Michael Spence in the 1970s. In general, the signal indicates an attribute whose value is correlated with the difficulty and cost of the signal: the harder it is to get a degree, the greater the value of the signal it sends.)

But in an economy in which education credentials are in over-supply, that signaling mechanism is running up against a basic reality: a degree accredits very little about the student's knowledge, problem-solving skills or professionalism. A degree is simply a proxy of knowledge, not evidence of knowledge or useful skills.

Indeed, the study Academically Adrift: Limited Learning on College Campuses concluded that "American higher education is characterized by limited or no learning for a large proportion of students."

Signaling an ability to grind though four or five years of institutional coursework is no longer enough; the signaling needed to indicate an ability to create value must be much richer in information density and more persuasive than a factory model diploma.

A resume is equally thin on information that accredits a worker's knowledge, useful skills and professionalism. A resume is a public-relations summary that everyone knows has been tailored to present the candidate in the best possible light. And precisely how useful and trustworthy is PR in any setting?

Put yourself in the shoes of a hiring manager or potential collaborator: there is precious little useful information in either a diploma or a resume. As a result, human resources departments have been tuned to eliminate as many candidates as possible by signal-based winnowing rather than by the collection of useful information on the skills, knowledge and professionalism of the potential employee/collaborator.

Posted by: How To Get a Job Despite the Economy - Charles Hugh Smith at April 14, 2014 05:31 PM

....... "This reminded me the years spent at large industry players like THALES in France or SPC Corp. in the US. Large companies often have a "postdoc" policy for recruiting R&D staff. University alumni often hire pals, people that look like them, or at least candidates with a common background."

Posted by: Trustleap at November 7, 2014 09:25 AM

I've been self-employed for the past three years. Though I did spend my first year out of college working for a three person, now-defunct startup, I've never had a typical 9-5 (or more like 10-8 nowadays) and to be honest, never really wanted one. Lara Schenck, LLC is a profitable business, and every day I do work that is enjoyable and challenging. I make my own hours, take vacations when I want to, and run everything on my terms.

While that's all awesome, what you don't get from working independently is the team experience. I base my work on teaching technical literacy to non-technical designers and content producers so that they can better communicate with developers. The theory is that if a designer understands why it's a bad idea to request 18 fonts, and if content producers know why it's not trivial to edit the titles of a set of related posts, life will be easier for everyone. At least that's my theory, and the assumption on which I've developed my business.

Lately though, in a bout of the good 'ol impostor syndrome, I've been feeling like, wait, how can I be telling people how to work on teams if I've never really worked on one? I've always been the 'Lead UI/UX/Visual/Web/Front-end Designer-person-thing', and have never worked for a larger company with separate teams for product, UX, marketing, content, frontend, backend, etc.

So I felt the urge to look for a job, and a seemingly perfect one fell into my lap. It was for an awesome company, and it sounded right up my alley skill-wise. The title was 'UX Engineer/Interaction Designer'. I usually balk at the the term "engineer" (perhaps for good reason) but considering the presence of "designer" and the nature of the job post, I wasn't too bothered.


Posted by: Tales of a Non-Unicorn: A Story About The Trouble with Job Titles and Descriptions at August 11, 2015 06:29 AM
Post a comment

Remember personal info?

Hit preview to see your comment as it would be displayed.