TLDR: Put the time into articulating what makes a good hiring process and what you're looking for in your developers!

Introduction

A few weeks ago I wrote an article complaining about take home tests and all the problems I see with them as a means to measure candidates in a hiring process. That article was an example of something not to do, so I wanted to flip it around and talk about good things to do.

This is a big article/brain dump (sorry in advance)! I'm going to:

• set the scene

• discuss what makes a good hiring process

• discuss how to measure the skills of software engineers

• provide an example process that demonstrates the above 2 points

Setting the scene: When you're thrust into the action

You're a software dev sitting at your desk happily coding away in the zone, and then your manager suddenly says to you: "We've interviewing someone today, are you able to join the interview and assess their technical abilities?"

There are many reasons you might say yes.

This is your first chance to sit on the other side of the desk, you get to be through the looking glass. Maybe a part of you even secretly looks forward to making someone squirm in their chair the way you had to squirm under the spotlight in order to get your start.

Maybe you're protective of your team. You've got a nice team, a nice culture free of all the faff you hear other devs complain about, and you see a new hire as a wild card who might trample the flowers in your garden. You're going to be the gatekeeper who keeps out these toxic influences.

So you say "yes!". Now the tricky part: "How on earth do I actually assess someone?" What do we ask them? What if they're more senior than me? That is quite intimidating!

In a lot of small companies, I suspect this is how the "hiring pipeline" begins - an ad-hoc mix of devs and product/business stretching beyond their designated roles. They don't really know what they're doing, and are just figuring it out as they go. There are many traps waiting for these scrappy go-getters.

Compared to large companies, you might feel at a disadvantage, particularly if you're trying to hire a senior engineer:

• You can't compete on salary with big computers.

• You feel like your codebase and processes are a mess

• Your team is already stretched and lacks time to create a polished hiring process

But hiring the right devs is really important, particularly seniors. This article is slanted towards those small teams trying to hire more senior dev roles. But hopefully there's some more general principles you can take away.

A good starting point is to take a step back and think about the process itself. Articulate your goals and methodology - what are we wanting to measure and what is a good process to achieve that? This will help you avoid a lot of traps and wasted time.

A good process

It's helpful to zoom out and think about what makes a good hiring process. Here's some things that come to mind:

• low false positive rate (ie. avoid hiring bad devs)

• low false negative rate (ie. avoid not hiring good devs)

• efficient for your team

These are just some examples and in your context some things will matter more than others. There's probably a lot more!

Try to articulate what matters in a way that you have some ability to quantify and measure, otherwise it becomes too subjective.

False Positives

A false positive rate is fairly easy to define and measure. For example you could define a "false positive" as hiring a candidate who "leaves in under 6 months because they quit out of frustration or are fired". It will take you a while to accumulate enough data to measure this, but even without that data just having this as a guiding principle helps you make decisions about your pipeline.

That 6 month time frame was just an example, and it would depend on how much investment you need to put into team members before they start. If it takes 3 months to learn your product and stack, then someone leaving at 6 months means you didn't even get return on investment for the effort you put in. But if it takes 1 month to completely onboard someone (I wish!), you would probably still get a lot of value out them if they only stayed for 6 months, so you might be more willing to take risks.

Two important aspects to keeping team members for a long time are:

• Giving a truthful representation of your company, codebase, practices and what the candidate will realistically do day to day

• Making sure the role aligns with your candidates values and motivations

There's more on this in the longevity section below.

False Negatives

False negatives are good to keep in mind too. Teams can get so scared about hiring a bad candidate that they pass up genuinely good candidates who might just be a bit different to what you were expecting.

There's two kinds of false negatives:

• Candidates you wanted to hire but didn't

• Candidates you rejected but really shouldn't have

The first kind is usually because they didn't like you or got a better offer somewhere else. Keeping your hiring pipeline short so that you can make an offer to them earlier is a great way to reduce the chance of losing someone, and it's one of the downsides of take home tests or lots of technical interviews.

At the heart of the second one is often subconscious bias which can come from a lack of clear measurable definition about what skills you need. If you can't articulate what kind of person you want to hire, you're more likely to apply bias and prefer candidates who are similar to you. Then you miss out on candidates who could bring genuine value and offer different perspectives.

So the second one is one you can't really measure (by definition) because it's about removing unconscious biases from your process. The way to address it is to remove as much subjectivity as possible from how you measure developers. Software teams are getting increasingly distributed around the world across different cultures, so getting this right is increasingly important now.

So if your team is overly conservative, remember that every good candidate that you don't hire means:

• More interviews to keep looking which costs you more time

• Your team will spend longer under resourced

So don't let the good candidates slip through your fingers!

Efficient pipeline

By efficient here I mean "good bang for buck" in terms of time spent (on both sides) vs knowledge gained.

If I can learn 85% about a candidate with 2 hours of work, and it takes an extra 4 hours of time to get to 90%, it's probably not worth it, because that extra 5% is unlikely to tell you anything that would affect whether you hire that person or not.

My experience has been that people (particularly devs) see efficiency as "what's the minimum amount of work I can do?". Developers have a natural instinct to automate, but they will tend to automate processes to reduce their time spent, rather than optimising efficiency.

That's where you start to see take home tests and automated tech tests come in as they require almost 0 time investment from your devs. Another example is those weird interviews where candidates sit in silence on a video call solving an algorithm whilst a developer "supervises" them. The dev is probably just doing their own work in the background. From the candidates perspective:

• you're really just shifting the time burden onto them

• the process is becoming faceless and unenjoyable

• you're giving a bad impression of your company

• a lot of time is spent by them but you don't actually learn much about them

In contrast a well crafted technical interview where you solve problems together can give you a very good sense for how this developer thinks and operates in quite a short amount of time.

Overall an efficient process isn't about reducing the amount of time the developer spends, it's about finding that sweet spot on the diminishing returns curve. That might mean your devs do have to do some preparation or take time to talk to candidates, but it's time well spent.

What are we measuring?

What makes a good developer? What are the skills your team needs? Some of these characteristics will be pretty universal (e.g. "not a jerk") and some will be very specific to your team.

Sit down and think about this so that you're not purely assessing candidates on some subjective emotional gut feeling ("he seems like a nice bloke", "she's a bad culture fit"). Then try to break those broader attributes down into more specific measurable qualities.

Here's an example list of high level characteristics I often measure:

• technical ability

• team skills

• proactivity

• longevity

Let's break those down a bit.

Technical ability

This isn't just "can they use our stack", although that's definitely part of it. There's other aspects like:

• general intelligence

• general engineering experience

• architecture experience

• analytical/problem solving ability

• abstract thinking

• intellectual curiosity

• familiarity with your specific stack

General intelligence is just a measure of someone's mental horse power. How quickly can they wrap their head around a new concept? How quickly will they learn your product and stack?

General engineering is my way of describing those fairly universal engineering concepts, have they "been around the block?". Have they used a few different languages? Have they worked with databases? Have they worked with message queues? Have they worked with microservices? Have they had to fix performance problems? Do they have basic dev ops experience? This gives you confidence that they've been exposed to different ideas, are well rounded, generally understand "engineering" and will make sensible day to day decisions.

Familiarity with your stack is just what you'd think. It's unlikely you'll find someone who knows every single part of your stack. Maybe they haven't used some specific tool like kafka, but they've used something similar like rabbitmq. You're doing a mental calculation of how many weeks it's going to take them to fill the gaps in their knowledge to get up to speed with your team and be productive. That's a combination of intelligence, curiosity and current familiarity.

Team skills

Software development is a team sport. We're like artists all painting on the same canvas so you need to know how to communicate and get along. Some examples off the top of my head:

• empathetic communicator

• good written communication skills

• experience mentoring other devs

• assertive whilst still being a good listener

• easy to get on with

• willing to stretch a bit beyond their designated role when necessary

By "empathetic communicator" I mean someone who is able to tailor their language to their audience and anticipate their questions. It's putting yourself in someone else's shoes and it makes day to day operations run so much faster. In contrast it's common to see engineers explain concepts to non-technical managers or junior engineers as if they were senior engineers. An empathetic communicator is thinking: "What do they need from me in this conversation to do their job well? How can I get that across without burying them in irrelevant details? Is there anything they're completely unaware of that I should raise? I noticed something odd the other day which relates to what they're doing, I should probably mention that".

Good written communication skills are getting more important now that teams are remote and in different time zones. A lot of communication is in text form (chat, emails, updates on tickets, code review comments) and it's often async so a team is way more productive when it's clear.

Proactivity

Teams really thrive when you have developers who "manage up" and proactively lead the charge on things. You can't realistically expect your managers to lead highly technical initiatives around say big architectural changes, security improvements, technology migrations. Your managers will be managing that from the outside and always lagging behind. They don't see inside the codebase and the infrastructure, so they only become aware of these problems when an engineer tells them (or customers start complaining).

Proactive devs are the ones who notice something weird in the codebase, and instead of saying: "Yeah someone really should fix that", they start thinking about how to fix it. They're the ones who are champing at the bit to get some time allocated to fix "that thing" that really irritates them.

That proactivity really greases the wheels between the engineers and managers and it replaces fear with trust. Managers who trust the engineers to make good decisions are less likely to micromanage, and they'll give you latitude to refactor or do more personal development.

Specific examples of proactivity you can look for:

• examples on their resume where they've led the charge on some initiative

• experience managing other developers in some formal/informal way

• evidence of them stretching beyond their role (e.g. delivering talks, helping with interviews, mentoring)

• seems organised and prepared, has many questions and suggestions for your product/stack

Longevity

By "longevity" I mean the chances of this developer staying at your company for a decent period of time. Hiring and training a new engineer is expensive so you want good return on that investment, and don't want them to leave after a short time.

Likewise churn is very disruptive and expensive, particularly for small teams who don't have a lot of fat to absorb shocks in their resourcing and planning. Small teams are often more likely to develop knowledge silos and churn really hurts when you lose that key knowledge holder. Handover is often rushed and incomplete.

To measure longevity, you need to understand what fundamentally motivates them, and then honestly assess if the role is going to scratch that itch. If they are fundamentally driven by innovation and exploring new technologies, and your company is a conservative bank that uses a very bland "enterprise" java/C# stack everywhere, it's just not going to work.

Companies will often hide or paper over some of their weaknesses to better sell themselves, but that doesn't make sense to me. Once this person gets hired and looks around, they'll know exactly what it's like and there's nothing to stop them leaving. It's better to be honest so that they come in with the right expectations.

Every company has tech debt: that service that no one wants to touch, or that weird issue in the build we haven't figured out yet. Experienced devs have been around the block a few times and they don't expect things to be perfect. Some of them even enjoy the challenge of refactoring. So don't lie to them.

Aside: And stay away from meaningless aspirational statement like: "Yep we don't have any tests, but it's something we plan to work on." Unless you have something concrete planned for that in the next month, it's just weasel words. If it really mattered to you, you would have done something about it by now.

Good at everything?

It would be easy for someone to misunderstand me as saying that a dev needs to tick all the boxes above. No, that's not realistic. I'm just trying to set up a structure or language to help you articulate more clearly where devs are strong and weak and what value they can bring.

Maybe your team is already full of visionaries and leaders who are great at starting initiatives but struggle to follow them through. So you might put more value on a candidate who has a solid track record of executing, even if it's not their own vision.

Or perhaps your team is full of strong technical engineers, but they struggle to communicate with the managers and your planning suffers. An engineer with strong empathy and communication skills would add a lot of value there.

My point is that being able to articulate those things and measure them to some degree helps you think more clearly about what you actually need and how to measure it.

Intrinsic vs Learned Attributes

Some traits of a developer are much more intrinsic to their personality, and some are learnt.

For example, the desire to take initiative, step up and lead is tied quite closely to someone's personality, whereas their knowledge of react is something they learn.

It's important to recognise that you can't control or change someone's personality (nor should you!). So work on the assumption that the candidate in front of you is not going to change their personality. If they don't like to lead or manage, don't fool yourself into thinking that will change. If they're not great communicators, work on the assumption that this won't change.

I'm not saying that people don't change - they often do. But it's usually from random life events that you have no control over.

It's because of this that I put much more weight on intrinsic skills and leave a lot more wiggle room for learned skills. Maybe your team uses rust but the candidate hasn't used it before, but you can see that they're smart and motivated and have used many languages in the past. That is a skill gap that can be filled without much risk.

The process

Phew! So much setup...

We're now armed with some structure for what makes a good hiring pipeline and a basic framework for measuring a candidate.

So how do we translate that into an actual process? I'll give a concrete example below that will hopefully stimulate ideas that make sense in your context.

This is a simple process we used at a previous startup and it worked pretty well. At this point in time the company had about 20 engineers and was starting to mature from a chaotic startup to a more typically structured company.

Initial scouting

In that company we had a small team of scouts who would do outbound and inbound recruiting (across all roles). They weren't developers, but the devs had given them basic training and resources to know how to spot good developers.

The scouts would bring people into the top of the hiring funnel.

NOTE: We also had a developer who didn't like us calling it a "funnel" because "water doesn't leak out as it goes down a funnel". This was known internally as the "funnel controversy". Sorry Alan, everyone in the industry calls it a funnel nowadays, this is your cross to bear.

Written questions

When the scouts found a candidate who looked promising, they would forward their resumes on to the developers for a thumbs up/thumbs down. If the developers liked them, the candidate would get emailed a standard list of about 10 questions to try and get to know them better.

This list of questions would ask for more specific details around their experience with our stack (e.g. kafka, mongodb, scala, functional programming) as resumes often make that a bit vague. For example:

• How many years roughly have you used scala in an industry context?

• How comfortable are you with library xyz?

• What do you like/dislike about functional programming?

There'd also be some questions to understand their motivation for joining our team like "What is it about our company that interested you?". Sometimes the answer to that question was so bad that you could quickly eliminate them.

For the candidate, the survey wasn't onerous. It was about 10 questions and most were just simple 1-2 sentence answers and a few might require a short paragraph. I got the impression a lot of candidates enjoyed it because they got to express their opinions and feel heard. We'd often get quite long answers so some people really got into it.

On our side, we learnt a lot about the candidate:

• what is their written communication style

• how enthusiastic they are

• how do they feel about our stack

• what is their personality like

• what are some areas we can probe into more deeply during the technical interview

• why do they want to join our company

Quick introduction

A non-engineer from our hiring team would do a short catchup with the candidate just to run them through the basics.

They'd give them an overview of our product, get some general background, talk about salary expectations, talk about logistics like visa's, pick up any obvious red flags etc... All that boring stuff that devs don't want to do.

Technical interview

This is where most candidates would drop out of the funnel (sorry again Alan!).

By this point we've got a pretty good sense for our candidate. We've got their resume, their email answers and a basic understanding of their life situation and motivations. That gets used to tailor the technical interview.

I have so much to say about technical interviews and this article is already so overstuffed that I will do that in a more specialised article sometime in the future (fingers crossed). Instead I'll just give a summary of some main points:

• our interviews were usually around 50-90 minutes depending on how talkative the candidate was

• there would be some live coding, and a lot of discussion, including analysis

• the format was a fairly standard template designed to be catered to the candidate's skills and past experiences

• the interviews gave us a pretty well rounded sense for the candidates abilities across the major areas (technical ability, communication, initiative, longevity) in a short amount of time

• candidates gave us positive feedback about the interview and it seemed to be a welcome change for them

Final interview

If the candidate passed the technical interview, depending on their seniority, they might have another final interview with a higher up like the head of engineering, CEO or head of product.

Generally it's rare that this knocks out any candidates so it's probably more just to give higher ups peace of mind (managers need that feeling of control). If you want to shorten your pipeline even more you could often scrap this.

Summary

Once we got this running well, the total time for most candidates in the funnel (it's okay, Alan stopped reading long ago) was probably about a week or two. It felt lean to us, there wasn't much fat you could cut out. We started to get some good engineers hired into senior roles.

We didn't have any false positives (bad hires) that I know of, but I wasn't around long enough to get the long term data.

That's it!

The brain dump is complete!

Recapping, this article is arguing that it's good to take the time to articulate what makes a good hiring pipeline and what attributes you're looking for. Then you can design a hiring process that ticks those boxes. That seems obvious once you say it, but I suspect a lot of teams don't really do this, and the consequences can be very destructive for your team.

I've tried to give general principles here with examples to flesh out what I mean, but tailor it to your own context and what your team values.

I hope this is helpful for you, particularly if you're in a small company. Small companies often lack resources and can't necessarily compete on salaries which makes it tough to find good talent. But often big companies have hiring pipelines designed for scale, that tend to devolve to be somewhat faceless, automated and dehumanising. If you have a warm, human, efficient process, you can set yourself apart and attract the good talent that appreciate those things.

Good luck!