This is a pretty good manifesto. Something that I rarely (never) see mentioned is that the ability to quickly evaluate a technical hire is a skill managers can and should cultivate. I never give coding tests. I sit down with a candidate and within 20 minutes of verbally discussing their prior projects I know if I want to hire them or not. This does mean that the manager has to be technical and have the ability to deep dive in these conversations. It's by far the most efficient and candidate friendly (and interviewer friendly!) method I've seen. I've never had a bad hire doing this and I've hired dozens of people
I was hired with a very similar process. I was asked several technical questions that applied to the role and about my own history, etc. Three years later I'm still here and have survived several rounds of layoffs and a sale of the division of the company I work for. I still work with that manager.
Most interviews with tech screens that I've done have been fine, though. A couple of them have been bad where I've recognized the engineer testing me has been wrong but that never fares well for the person being interviewed.
Most calls I take are often an introduction before we even go to a tech screen, though, and I appreciate that chance usually. It's saved interviewers and myself a lot of time in the past.
Your story about surviving layoffs implies that you survived because the interview thoroughly vetted you. Wouldn't that be true of people that were laid off?
It doesn't imply anything except that I'm still working with that manager and we've worked well together. At the time I was the only technical hire under a technical manager. Much of the business isn't technical and weren't part of the same interview process.
I understand what you're trying to ask, but no—it doesn't apply in this case.
As a manager my goal is to hire the team I need to accomplish the tasks we have to deliver. This method has worked really well for me. It also means my relationship with my direct reports starts out based on trust, which has also worked out great for me as I often re-hire people from old teams when I move to a new company.
What does false negative matter for? As you said, you cannot know it, so why bring it up as a missing metric? What matters is if they hired for their demand, and their hires met or exceeded expectations
We cannot know it by anecdote. We can know it using a well designed longitudinal study. Why settle for the former? Why do so many proudly refuse to read the literature?
The only other thing I can compare to the anti-intellectualism around hiring is a similar phenomenon around nutrition and dieting.
I mean, I think you're planting your flag in some intellectual what aboutism. The confidence in the strategy of the process here is only as anti-intellectual as it is simultaneously conducive to that actual work trying to be accomplished.
Say a man marries a high caliber lady. Beautiful, smart, great personality. If he says, 'I did great here'. Does anyone say to him, 'Well what was your false negative rate?'. I guess you might
You ever read one of those interviews with a centenarian where they ask him what his secret is and he answers something like “I smoke a cigar every Sunday”?
I've often been surprised by what has come out in a coding test that hasn't emerged in a conversation. People who seemed dull and uninspired in conversation ended up being brilliant and vice versa. There's no substitute I think.
The trick is to get good at having those conversations so you can get to the bottom of someone's skills even if they're not outgoing, good at speaking... It's totally doable but it's on the manager to develop those skills.
There are a lot of excellent developers who do not code well in an interview context. Nerves, artificial restraints... coding is something that happens once you're in the flow. That's a state few can achieve in an interview.
It's rather like saying that you can interview a musician without asking them to play.
Yes, nerves play a part. That's why it's good to start easy and ramp up difficulty gradually, giving the interviewee a few easy wins to build up their confidence before getting them in to the harder stuff.
Frankly I find the most unnerving interviews (coding and questions) to be the ones where you have to second guess what the interviewer wants to hear. Where the question/task is difficult, but the desired outcome straightforward it's much less stressful.
It's closer to saying you can hire a songwriter without asking them to write a song in front of you. A musician gets paid to perform in front of people. Asking them to do that in the interview is exactly the skill they're looking for.
Spot on! Or take the Funnyman scene from Boondock Saints where a criminal boss tells a guy, "They tell me you're a Funny Guy. Tell me a funny joke." "Now? A joke? Uh... um, uh... A joke. Yeah, alright. Um... <awkwardly starts an offensive joke>... I'm not funny today. I-I know. I'm havin' a hard day. I-I-I- This joke sucks. It's-it's-it's a stupid joke."
That's kind of like how I code when sitting down for an interview, whereas I have no problem otherwise. My coding looks like an awkwardly told joke (oftentimes).
If all I had to do was pick some code I had memorized and practiced and "perform it", i.e. type it out, then yeah, sure, no problem. But I get asked to come up with the answer to something on the spot, something I usually take a little time to settle into, maybe do some Googling, etc, and always seems to be something I missed when spending several days to a week refreshing my memory on what I think I'm going to be asked ahead of the interview.
You’d probably ask a songwriter to share some prior songs, not ask “tell me about a time you had to work with a difficult co-worker” and trust your gut to tell you how that translates to song writing ability.
I don't know how hiring a songwriter actually works in the music business, but I would imagine it has the same issue that programming has. It's very difficult to objectively assess the quality of work in a vacuum. Discussing the process and mindset behind a work is often more helpful than only looking at the end product itself.
What kinds of things do you ask? I’ve interviewed and worked at FAANGs. I’ve done mostly coding and design interviews. I don’t doubt your ability but help me understand your model. Is it more open ended (tell me about your previous projects... candidate can speak to the problem domain, design, implementation choices, trade offs, or dive deeper into “Oh yeah we had to instrument events to feed into data processing pipelines for personalization...and XYZ failed so I had to do ABC.”) or is there something more critical?
I start pretty open ended, then ask specific questions about their choices. I almost always ask them to describe some of the schema of the db or how pieces of their project were communicating with each other... I also ask them what they like about their language of choice and why. I keep the questions specific to their projects and experience though.
I should add that I strongly favor candidate with open source experience, so most of the time I'll have also looked at their code myself and may have some questions about it.
The number one thing I look for is: can this person ship software? I'll ask a lot about the project lifecycle, what they did to get it across the finish line, how they supported and enhanced it... I try to do this in a non-leading way.
> "I should add that I strongly favor candidate with open source experience, so most of the time I'll have also looked at their code myself and may have some questions about it."
So, that's the most important part missing from your original comment. From that, I would assume that you never hired a developer without looking at their code before/after an interview. Is that right to assume?
Basically open sourced code (and not the interview) act as a substitute for coding tests in your process
No, I've hired plenty of people who haven't had open source contributions. It just makes the conversation easier when I can see their code beforehand. I also find that the really good people tend to do open source or personal projects, so there's a high overlap there.
So you so hired plenty of people without never looking on code they wrote?
How that variable affects the outcome? Is your hiring process any better/worse comparing when you can read their code and when you cannot read their code?
Don't send technical pre-screens to candidates before you talk to them. You're not Google. I'm not going to waste my time on something before I know anything about the company, the team, or the job itself.
I haven't really seen this interviewing for jobs on the west coast, but there's a lot of shops in Chicago (fintech) that seem to be under the impression that anyone applying for an engineering job is desperate.
Large number of recent grads are flooding the application channels. Half of them have skill level that probably shouldn’t be accepted anywhere.
At the same time established veterans are highly sought after and never really needed to ‘apply’.
As a result a massive proportion of application pipeline is filled with noise. And most of them are somewhat desperate for a job. Giving the illusion that qualified people are also desperate.
That's the key. It's very much two bubbles. Because some parts of the job is almost factory like, anyone with a bit of practice can do it, and the other part is extremely difficult, requires ton of experience, might need a computer science degree or more, etc.
Both groups barely understand the other. One is like "Lol I can write my email address on toilet paper and have a 500k/year job by Monday!", the other is "Ive applied at 150 companies and 149 of them didn't even reply back, the last 1 did to tell me there was a typo in my resume!".
But both groups have the same title (plus or minus seniority levels), work on the same projects in the same teams and follow the same career track.
I wonder how long until software engineering becomes more like other fields. The nurse, the general practitioner and the brain surgeon are not applying at the same job or even have the same backgrounds. The hvac technician and the engineer aren't either. Maybe the person who builds forms and the person who engineers a distributed system being on the same career track is the wrong model. It sets the wrong expectations. It can stunt career growth.
The "form" here is a metaphor for "simple things". If i have a large system and I have 3 forms, I'm not going to add (or worse, build) a dynamic form generator system. Worse, the forms could be completely different or all have very special ways of working that can't easily be generalized. At that point its a lot easier to build them by hand. However, you don't need someone with a degree in software engineering to do that. Repeat that for any other "simple but easier to do one off than to engineer around" programming tasks.
Not to different from factory automation. A lot of stuff done by hand COULD be done by robots. Its just cheaper to do it by hand.
I mean, sure, at a place with the engineering level or skill of the FAANGs. But at many places, developer effort is very manual and automation isn't considered.
> Maybe the person who builds forms and the person who engineers a distributed system being on the same career track is the wrong model.
Sometimes it feels like everyone wants to engineer a distributed system. And in the process people are losing the ability to recognize what parts of a system actually need to be engineered as a distributed system.
I think the nature of software being intangible and "black box magic" to some in the hiring process contributes to this (at least in companies that aren't software centric but still very much dependent on software).
It's a regular occurrence that non-software domain experts conflate all kinds of software development. Need a scalable CRUD app? Grab any ol' IT person. Need some safety critical code modified? Get that instrumentation engineer to cobble something up. After all, it's just software /s
I think if your channel is noisy you should check the input. If you're flooded by inexperienced or new grads for the role, the question isn't how to screen candidates but how to reduce the number of applications. For instance, don't put an ad out on Angel List or LinkedIn and take form applications with only a vague job description. Retain a recruiter and don't work with people on contingency.
This happened to me last week, and it wasn't a blind application. It was a recruiter who came to me about a job. I told them I wasn't interested in doing a technical screen for a job and company I knew nothing about, and in the past few months I've been soliciting interviews, not a single company I talked to tried that without a quick 15 minute phone call to see if everyone agreed that would be a good next step. It saves time and money for everyone.
FWIW in both my interview loops at Google, I've had a call with the recruiter before any kind of tech screen. I think that's standard practice. May vary when considering interns / new grads, I'm not sure.
Same here, but it's my understanding that they don't strictly hire for specific roles/teams and take a large number of unsolicited applications. The tech screen is also interactive with an engineer which is much better than automated assignments with a time limit.
Some companies use this kind of screen more as a way to filter out people who aren't adequately interested in the company/job to put in the effort on the testing -- so perhaps in some cases it is working as designed.
well there's the rub. How am I supposed to be "adequately" interested in the company/job if I don't know anything about the company or the job?
That's what I mean by talking to a candidate before the screen. Going through a recruiter isn't sufficient for most of the information I would like before deciding to apply for the job.
In this argument, it would seem that to be "adequately interested" in a company, I need to be willing to do unpaid labor for them as a first step. I would argue that most, if not all companies, should disabuse themselves of the thought that this is anything but exploitation.
Multiple hour coding assessments will work for people who are motivated for a new job. But every company wants "passive" candidates that are already working somewhere. If they're already (moderately) happy somewhere, they're not going to spend 3+ hours on your pre-process.
Companies that can identify talent without wasting the talent's time will be at a huge advantage
I would love a coding assignment. But make sure you have an style guide and tell me what paradigms to use and what patterns, and have an auto linter and style formatter. I would hate to be rejected just because I used tabs and you prefer tabs or vice versa. A could adapt to anything, even write the damn app in brainfuck if thats what you want. Instead you dismiss me after 5 minutes of soft talk with a group of people i never met.
My key point every time my team discusses hiring: If you think your take-home project takes N hours, add 1-1.5. You already know the context, and are underestimating the difficulty even if you've run through it before.
> ... especially if you've run through it before [edits mine]
We're doing a hackathon-style interview session this Friday, and I've already had to remind the team that one of our coding challenges is "easy" because we know the answer. You need to think back to the first time you saw it. Also, we're not actually interested in the solution, but how you arrived at it. Once you grok the problem, most software people hit on the solution pretty soon, and can begin coding - we want to see how you come to the realization, and then how you work to create the solution.
A lot of this is dependent on how reasonable the take home project is. But I've always assumed that my work needs to be professional. That means it needs to be readable, well-designed, and nicely formatted. Of course take-home projects should be short, and are basically a POC. But shortcuts should probably be discussed with the interviewer and should be well documented.
> Create a project based coding assessment that directly reflects the work the developer would be expected to deliver if hired, making sure to provide some open-ended flexibility that allows candidates to showcase their strengths. This might require the developer to digest an existing code base, gather requirements, communicate to your team a push request they’d like to make, or provide a detailed description of code that they’ve submitted before merging into production. These activities move your assessment process beyond the code, revealing insights into what a candidate would actually be like to work with.
That sounds like it'd take more than 3 hours, unless it's based on a very small project.
Yeah, I know some teams say that they discuss the project after, which would work if the candidate couldn't code at all. But if they're just a mediocre programmer who's been coached, it may be harder to detect, especially if they have a talent for BS'ing.
