Introduction

I've written a couple of articles on hiring now. It was just something I had to get out of my system and hopefully this is the final instalment.

Today's article is tips for how to run efficient technical interviews for software engineers. In this context a "technical interview" is an assessment of a potential candidate involving some technical challenges and discussions that would take somewhere from 30-90 minutes typically, usually over a video call. Generally it's just developers present.

The article will make more sense if you've read my previous articles about take home tests and hiring pipelines as they provides context for how I think about interviewing. But I'll try to make this self-contained.

In this article I'll outline some general principles and then provide tips for how to achieve them.

Goals for your technical interview

A good technical interview is:

• efficient for your team - you learn a lot about your candidate proportional to the time and effort you put in

• enjoyable for the candidate and creates a positive, truthful impression of your team

• low false positive rate, ie. avoids hiring bad people where possible

• low false negative rate, ie. avoids not hiring good people where possible

I discuss these a bit in my previous article.

Principles

With the goals set, these are some helpful general principles to help you achieve them:

• help them show their true best selves during the interview

• as much as possible, try to replicate a collaborative atmosphere

This lets you really get to know them properly and what they're like to work with, so you'll be able to make a well informed decision about whether to hire them or not.

It makes the process more enjoyable for your candidate which differentiates your team from other places they're interviewing with.

Below are some general tips and you'll see these ideas will keep popping up throughout them.

Tips

Relax your candidate

Unfortunately, some interviewers seem to take a perverse pleasure in watching their candidates squirm, or trying to trick them. This is a great way to lose good candidates from your hiring process.

Understand that there are a lot of good developers out there who don't do well at interviews. Maybe they struggle to think quickly on their feet, or they get nervous when people watch them code, or they struggle to articulate their thoughts, or they have some kind of social anxiety. These are attributes that don't affect them much once they're in a job and comfortable, but they can make a huge difference during an interview. Are you wanting to measure how well a candidate can perform in an artificial environment like a technical interview, or how much value they can bring to your team day to day with real software engineering?

From the very start of the interview, the goal is to make the candidate as relaxed and comfortable as possible. Then you get to see their true selves, the good and the bad.

Progress from easy to hard

Don't stack the most difficult aspects of your interview right at the start, but start off with the easier stuff first. When they can hit a few easy home runs at the start of the interview, that gives them more confidence and they'll be in a better frame of mind when the harder problems come.

One thing I'm often trying to assess is a candidate's ability to analyse pro's and con's and how they think. If I can see on their resume that they've got decent experience with two related technologies, I'll ask them something like: "I see you've used postgres and also cassandra. I'd be interested to hear your thoughts on them. Which one do you prefer? When would you recommend one over the other? Have you had any horror stories with either?"

The nice thing about a question like that is that there isn't a "right" answer. You're asking them about something they are already familiar with and they get a chance to share their experiences and talk without worrying that they're saying the wrong thing. At the same time, as an interviewer you're getting an insight into how deeply they understand the tools they use and what aspects of them they consider as important or unimportant. From that you can glean pretty quickly how senior they truly are.

These kinds of discussion questions that play to your candidate's strengths make more sense to put near the start.

Aside: Drilling in on their areas of expertise also helps quickly detect fraud. If they claim to have experience with something but can't answer basic questions about it, you know you've got someone who fills their resume with buzz words. Whereas if you ask them about things they're less familiar with, it's harder to know whether to attribute lack of knowledge to lack of skill vs just lack of experience.

Silly restrictions during the coding challenge

Chances are there will be some kind of coding challenge in your interview where they solve a problem in front of you. This is where some developers might get overly nervous and meltdown.

As an interviewer, there's some simple things you can do to communicate that you're a reasonable person and you're not looking to trip up or catch them out on some silly technicality.

Internet Access

I will usually tell candidates that I have no problem with them using the internet to look stuff up. After all, in a real software engineering environment, you're allowed to use the internet. Really the reason I do is to alleviate that fear they might have that they'll be expected to remember the name of some obscure method or the contract of some function and they'll be penalised if they don't. In practice they never need to do this because there's nothing so specific in my interview questions.

I just say this to communicate that I'm a reasonable person who understands what real software engineering looks like. It's also a good way to differentiate yourself from those silly technical interviews where you have to make some code compile and pass a test, but you can't use the internet to look up the name or contract of some function you really need but can't quite remember.

IDE

If practical, let them code in their favourite IDE.

I used to run interviews where we'd use a barebones online collaborative editor floobits directly in the browser. Often I'd see senior developers really struggle to cobble together code because they can't remember the finer points of basic syntax. I suspect this comes from being so dependent on the IDE to generate it for them.

In hindsight it probably would have been better to just let them use their IDE and share their screen. Developers are used to IDE support like code completion and jump-to-definition. They have often customised their IDE's with themes and keyboard shortcuts. When you pluck them out of that and put them in the equivalent of notepad++ they feel naked and disoriented.

You might say: "But they should know the finer points of the syntax if they're senior! They shouldn't need a fancy IDE!" Well.... if anyone should agree with this, it would be me: I'm a vim user who generally can't get LSP to work properly, which has forced me to learn the syntax of scala very thoroughly, plus all the fully qualified package names for many common libraries. When I type '.' nothing useful appears.

So I want to agree with you and make all those intellij users feel the same pain I do, but I can't. We're wanting to assess how productive they are with day to day development, and for that we can assume they will be pampered by their favourite overstuffed IDE.

Seeing them use their IDE also gives me a glimpse into how they develop - it's like observing them in their natural habitat. I can see if they're one of those people with a million customised shortcuts, if they use a shell separately or just stay within their IDE for everything, how they organise their files, do they use uppercase letters and spaces in their filenames etc...

"I just want the gist"

When I'm interviewing seniors in particular, it's common for them to be a bit rusty.

Seniors often get stuck in lots of meetings, or doing code review, or doing agile planning and might not get to code as much. Maybe their day job has them using a different language to the one you're testing them on and they're getting the syntax muddled up.

They're a bit older, their brains aren't as sharp and they're often a bit sleep deprived from a demanding job plus family commitments. Sometimes I'm interviewing them at weird times because of timezone differences or they have to work around their busy schedules0

If I can tell they've been out of the game for a while or are a bit groggy, I will try to reassure them by saying: "the code doesn't have to be perfect, it's okay if the syntax is off a bit or it doesn't compile, I just want the gist" - and they know what I mean.

I'm trying to communicate to them that I get it, and can understand if it's been 20 years since they've been at college and have forgotten all the jargon.

Ramping up the coding challenge

Above I mentioned how starting easy helps to relax candidates. For the coding challenge, I start easy and gradually ramp up the difficulty until I find their limit.

Start simple

I deliberately start with a very simple problem like implementing factorial or fibonacci. Candidates nervous about some tough obscure problem would probably breathe a sigh of relief and feel more confident at this point.

Also I want a problem that is very simple in order to minimise the advantage to candidates who happened to have seen it before.

To contrast this, imagine asking a much more difficult stereotypical problem like inverting a binary tree. If you'd never seen that before and were put on the spot, you might not do very well, even if you were actually a good engineer. Some candidates will have practiced that and some won't have and that will massively influence how well they do. We want a process that measures how good they will be at day to day software engineering, not preparing for technical interviews.

Starting with a dead simple question also means that if a recruiter of colleague gives the candidate a heads up about the test, it doesn't advantage them that much. Recruiters are incentivised to do this to get their commission.

Starting with a really simple problem also makes it quite clear if they can't code. If you started with a more difficult one which they struggled with, it would be harder to know if it's just nerves, they're struggling with the coding environment, they misunderstand the question etc... A simple question minimises those external factors so you can more easily correlate their coding to their ability. If they fail there, you can end the interview early and save everyone some time.

Ramping up

But of course I don't just stop after asking an easy question, or the test would be meaningless.

We'll do the usual thing of analysing the time and space complexity of the solution and then I'll gradually ramp up the challenge in little increments. Even seemingly simple problems have a surprising depth to them when you start to explore them more.

I'll start to ask them for different ways to implement it. If it uses recursion, we'll talk about the downsides and if we could use a loop or if we're in an FP context we'd discuss tail recursion and folding. We'll compare the time/space complexity for different approaches and discuss performance and readability.

Backend Engineering

I'm always interviewing backend developers (usually scala developers), so next I'll lift the problem from some isolated algorithmic task into an engineering context. For example: "Imagine we had a back end service responsible for computing factorial/fibonacci, and there's a front end which lets the user send it values, tell me how you'd implement that?"

Now we're talking about services, architecture, databases, caching, concurrency, queues etc...

The algorithm you've picked is usually O(n) running time or worse, so there's an engineering challenge of making the service run quickly when users put in massive numbers. But now that we're in an engineering context, there's other tools we can use outside of the algorithm itself to address this.

Free roaming

From here on there's much more discussion and much less coding. The problem statement is pretty open ended at this point, and I get to see where their mind goes which tells me a lot about them and their level of experience.

Some people might want to focus in on getting clear requirements: "Are we allowed to assume there is a largest number the user can enter? How many users are we expecting at any one time? Is there some requirement around maximum response time?". Some will go straight for the architecture or immediately think about optimising.

I go where they go and don't act like there is some kind of right approach. You might say: "They should get the requirements clear first" but sometimes they approach things in a non-linear way and you just need to give them space. Not everyone is the same. The goal is to get inside their heads and see their process. It's a very flexible "choose your own adventure" where I push them until I feel like we've hit the limit of their knowledge.

Collaboration

Through the entire coding challenge I'm trying to collaborate with them. I'm gradually changing the requirements and seeing how they iterate. I'm suggesting alternatives and sometimes challenging their ideas to see how they respond - do they dig in their heals, do they blindly follow it, are they open minded but willing to push back? I want to witness those critical points in the design process where there's a fork in the road and they have a decision to make. I want to understand what factors they consider.

This gives me a much more nuanced and deep understanding of who they are, how they're motivated, their abstract reasoning abilities and how they solve problems. And you just can't get this with take home tests because you only see the final product, not the process.

A good experience

Candidates learn new things from that collaboration and you feel a real energy from that, even if they know they didn't pass. That is such a contrast to the stereotypical interview where candidates feel judged and drained. You've had a genuinely human interaction with them, they're not just submitting code to some faceless grader.

I learn too. Even though I've done this question 100+ times, sometimes they mention an idea or approach I've never heard of, and that is exciting. When that happens we're learning together, as equals, and it's exciting.

That positive, shared experience will make your interview stand out in their minds and they're much more willing to join your team if you want to make an offer.

The candidate will also feel understood. You've really given them the space to show you who they are.

I think an anxiety some developers have is that they know they have some skills and knowledge, but they'll go into an interviewer and get asked questions that don't let them showcase that. I suspect developers don't mind failing an interview as long as they feel like they got to present their best selves.

A collaborative approach taps into that. It's about understanding what they do know, how they work, what experience they've had. It moves the focus to the process and away from some arbitrary "right answer".

Efficiency

This process is efficient too because I pick one problem and avoid context switching overhead.

I often see interviewers jump from one problem to another and it wastes time having to readjust and explain new problem domains and requirements. It's faster to iterate on something that is already familiar to the candidate. That familiarity removes a lot of psychological distractions for the candidate and lets us spend more time on the brass tacks of engineers.

Bad Questions To Avoid

"Bad questions" (explained below) will make less confident junior candidate feel nervous or uncomfortable and they will make experienced senior devs lose respect for you.

Gotchas questions

Avoid "gotcha" questions. These are trick questions where there seems like a right answer, but there's some hidden trap that you would only know if you've stumbled upon it. What are you really learning about your candidate when you ask them a gotcha question? Aren't you just testing whether they happen to have stumbled across this same thing that you did? That adds a lot of variance/bias into your interview. Often interviewers are asking these just to show how much smarter they are than the candidate to "put them in their place".

Overly specific technical questions

Avoid overly specific questions like: "what does the http status code 406 mean?" They might happen to know this, but what would that actually tell you about them? And if they don't know it, what does that actually tell you about them?

In real day to day development, if an engineer came across that status code and didn't know what it meant, they would just look it up. So practically it doesn't actually matter whether they have memorised that or not.

Hobby Horses

Another phenomenon I've seen is that interviewers can sometimes let their hobby horses creep in.

Imagine you're say a junior or mid level developer, and you had a revelatory moment where something suddenly clicked: "I finally understand FP!" or "I finally understand the garbage collector!".

Now that you understand it and think it's wonderful, it's easy for you to expect that everyone else on the planet should understand this too, after all it's so important to you.

It would be very easy for you to add a question about that in your interview, then punish candidates who don't get it and lavish with praise candidates who do. But if you'd been asked that question just prior to your revelation, you would have had no idea, and you're basically the same person now as you were then.

I was once interviewed by a guy who'd clearly had some kind of revelation about the benefits of data locality. Questions he asked in the interview seemed to keep coming back to data locality. The take home test was all about data locality. You were in, if you understood data locality.

Now maybe data locality is really important for that particular job because they're doing something specialised. But if it's not "industry standard" knowledge, it doesn't make sense to put so much emphasis on that for external candidates coming from roles where that isn't such a big deal. Measure them in areas where they've worked, not where they haven't.

They're useless...

These bad questions don't really tell you anything about the candidate and they create a negative experience for the candidate. It's fine for candidates to have gaps in their technical knowledge, if they're smart and good learners. So instead have discussions which surface things like:

• how good are they at solving problems?

• can they think abstractly?

• can they learn quickly?

• are they intellectually curious?

Basic Social Awareness

It's sometimes comical to me how socially unaware people in tech generally can be in interviews. It can make you come across as unfriendly or strange. There's something to be said for a bit of basic social awareness and common sense in how you engage with people.

I remember one time where I was the candidate and the interviewer sat opposite to me in a cafe. He opened his laptop directly between the two of us and would stare at it whilst he asked me questions. That was pretty bizarre.

I've had an online interview where I was solving an algorithmic problem and the interviewer was clearly doing something else. I've had an interview that was purely via text, I never saw the interviewer's face or heard their voice.

I can imagine software developers with mechanical keyboards typing notes about the candidate thumping into the microphone whilst the candidate is trying to explain something or solve a coding challenge. The candidate would probably be too polite to say anything, but it would be very distracting for them.

If you want to record feedback about the candidate during an interview, pen and paper is better than laptop, particularly when interviewing in person. When a laptop is present, the candidate doesn't really know what you're looking at on your screen and they might worry you're distracted. Physically it's a lot more intrusive because of how the lid pops up. With pen and paper, it's obvious that you're focused on the candidate. Maybe print out their resume and put some your notes in the margin.

Shadowing Interviews

Unbiased Feedback Sessions

If you have multiple interviewers, it's important not to bias each other when giving feedback after the interview.

A good practice is to get interviewers to write down their feedback prior to discussing it, so that their initial unbiased thoughts are captured.

Multiple Developers

Most interviews have a section where candidates can ask questions. This is where having shadows helps a lot.

Candidates often want a developer's perspective on the team. What's the culture like? Is it bureaucratic? Are there any issues? When I'm a candidate, I like to get multiple perspectives and get to know more of the team to reduce that risk that I'm getting a biased perspective or being deceived.

Smooth talking managers are more likely to gloss over negatives aspects of the team. Engineers are usually straight shooters.

Juniors Shadowing Interviews

I use technical interviews as a way to train more junior developers and also to get their unique perspective. They would usually just shadow and not ask technical questions, that way it's not intimidating for them.

From the training perspective, I want them to see how more senior developers discuss things and approach problems. I want them to then hear my feedback on the candidate afterwards. This helps them understand what it means to be "senior" and helps them grow. Later they can start running interviews themselves.

I also value their perspective on the candidate. They might say that they would feel nervous approaching this developer for help.

Often I'll have an interview question like: "I see you've used xyz technology. Can you explain it to me as if I was a junior developer?" I'm using this question to see how deeply they understand the technology (after all the deeper you know something, the more simply you can explain it), but also to see if they are an empathetic communicator. Having actual juniors there is useful because after the interview I'll ask them: "Did you actually understand the candidate's explanation? Are they someone you could collaborate with?"

Interestingly at my previous company, the higher ups said to stop having juniors shadow interviews for senior developers because it makes the wrong impression. I never got that feeling, and felt like it was a wasted opportunity to train them. But the higher ups were experienced in recruitment and would have had their reasons.

Wrapping Up

You can learn a lot about a candidate in a short space of time if you actually collaborate with them in solving a problem. That will tell you what they're like to work with, how they think, and they will learn a lot about you too.

Let them be their best selves. Be friendly and empathetic. Ask them about stuff they actually have experience with. Do simple things to relax their nerves to enable them to show you who they truly are.

Keep your interview efficient. You're looking for activities that will tell you a lot about your candidate in a short space of time. There should be a clearly articulated reason for everything you put into your interview - don't follow some old template if you don't understand its rationale.

If you do the above, it should be an enjoyable experience for your candidate, and when that great developer comes along who you desperately need, it might be the clincher that makes them choose your team over someone else.

Good luck!