Interviewing.io studied over 20,000 technical interviews. Here’s what can make one great, or really really bad.

Don’t let nerves and lack of preparation keep you from finding the right engineers. A good technical interview allows you to evaluate a candidate based on what they do, not just how their credentials look on paper. The tricky part is to create an environment where the applicant can showcase their skills. If your questions are unclear or too rigid it throws off the whole interview and you miss out on great talent.

No one is saying technical recruiting is easy. It’s actually pretty nerve-racking. You want to appear smart and collected when you’re really just as nervous as the candidate. Even finding the right questions is a struggle. You may think you’ve found the perfect problem, but then the candidate takes it down some unexpected rabbit hole.

That’s why we talked to Aline Lerner, an engineer turned recruiter who recently founded interviewing.io – a platform for recruiters and candidates to practice technical interviewing skills to assure the best candidates get hired. After each interview, of which the platform’s hosted about 20,000 to date, both parties evaluate the other’s performance. From this heap of feedback Lerner distilled the four most important steps a recruiter or hiring manager can take to become an effective technical interviewer, with quotes from candidates about what works and what doesn’t.

• Sell your team: A technical interview isn’t just about vetting candidates, it’s a chance to sell your company! It’s a myth that you need a big brand behind you to nab great engineers. In fact, a brand is useful to get people to the door and not so important after that. So whether you work for a big-name company or a no-name startup, what matters is how you connect in the interview. First, put that candidate at ease. Start with some small talk, let them know the structure of the interview, and try to understand what they want. Share with the interviewee why you chose to work for this company. If possible, before the interview, gather all the hiring managers in your interview loop to practice sharing the story of why they work there too. The more you practice telling your story, the more organic it sounds.

“[A] little but of friendly banter – even if it’s just ‘how are you doing?’ –  at the beginning of the interview would probably help a bit with keeping the candidate calm and comfortable.” – Candidate 1

• Ask good questions: Create a brain dump for the engineers in your company to contribute interesting problems they come across in their work. These problems don’t have to be “cool” per say,  just something that made them think. Use this shared material as inspiration for creating interview questions that are unique and applicable to the job for which you are recruiting.

“Question was not straightforward…Spent a long time figuring out what the question was even asking…” –Candidate 2

• Grow like a pro: The first time you ask an interview question will always be bad, always!  You should practice asking a new question five to 10 times before using it on a real candidate. This helps you clarify your delivery and familiarize yourself with all possible solutions.

“The question seemed very overwhelming at first but the interviewer was good at helping to break it down into smaller problems…” –Candidate 3

• Collaborate: There is no correlation between how well the candidate did and how well they think they did. There is, however, a correlation between how well they think they did and whether or not they take the job. That means candidates are more likely to accept an offer when they feel good about their performance in the interview. Identify areas that may be confusing, and formulate hints to help candidates arrive at an answer or spot their own mistake. Remember, it’s not about tricking the interviewee, but seeing how you can work well together.

“I’m impressed by how quickly he identified the issue (typo) in my hash computation code and how gently he led me to locating it myself with two very high-level hints…”  –Candidate 4