In this guide 6 sections
Interview the engineer, not the sales lead. Run a short pairing task on your own stack, read a sample of their real code and ask about past projects and what went wrong. Confirm in writing that the engineer you met is the engineer who starts, with the same title, years of experience and stack.
Meet the engineer, not the account manager
Ask the vendor for a thirty to forty five minute call with each proposed engineer, and decline presentations by a delivery manager who speaks for them. You are hiring a mind. The account manager can wait outside.
Share the role beforehand: the stack, the team size, the codebase age and the first month’s work, and ask the vendor to confirm the engineer has read it. Then start on time.
Notice their questions. A good candidate asks about your deploy process, your tests, who reviews code and how a release gets rolled back when it fails. A weak one asks only about hours.
Run a short pairing task on your own stack
Set up a thirty minute session around a real problem from your repository, such as a failing test, a controller bug or a query that returns wrong rows. Trim it so nothing sensitive shows.
Share your screen, let the candidate drive and watch how they read an error, search the code and talk through choices. Process beats finishing. Watch the route, not the result.
Skip puzzle questions from a quiz website. They measure quiz preparation. A pairing task shows how they will behave in your standup, in a code review and when a deploy goes wrong late on a Friday.
- Reads the stack trace before changing code
- Runs the tests and explains the failure
- Asks to see related files, not just the broken one
- Says what they would check next if stuck
Read some of their real code
Ask for a sample they are allowed to share, such as a pull request they authored, an open source commit or a sanitised module. Then read it for naming, test coverage, error handling, commit messages and the size of each change, noting what a reviewer would send back.
If the vendor says all past work is confidential, ask for a live walkthrough of one past design, with the engineer sketching the structure on screen. Then ask what they would change today.
Beware of polished samples that look written by someone else, and ask why a particular choice was made in the code. Authors answer easily and others hesitate. You will hear the difference.
Ask about past projects and where they struggled
Probe three things: what they built, which part was theirs and what broke, because every engineer who shipped a real system remembers the outage. The awkward migration and the estimate that was wrong by a month stay sharp in memory too, and vague answers here are a signal.
Ask about limits, because a candidate who claims no weak area in your stack is either new or sales trained and neither helps you.
Check working hours and overlap with your team, the notice period they have with the vendor and whether they are shared across other clients.
- A project they owned end to end
- A production incident and their role in it
- A technology they chose not to use, and why
- Their weakest area in your stack
Confirm the profile survives the sales stage
Vendors sometimes present a strong engineer and then start a junior one on your account after the contract is signed and the first invoice is raised. Prevent that by naming the person in the contract with title, years of experience and the stack, because names bind and titles do not.
Add a clause that substitutions need your approval and an interview, and ask for a short overlap period when someone is replaced.
On the first day, compare the person who arrives with your interview notes, then keep the scorecard for the next hire and send the vendor a copy.
Candidate interview scorecard
The scorecard fits on one page and has five rows: pairing task, code sample, past project depth, communication and availability, so scoring ends with the call. Each row carries a score from one to four, a notes box and a yes or no on the question that matters. At the foot sit the named person, the stack and the start date, each copied into the contract and the vendor’s confirmation email.
Open a blank worksheet to printQuestions owners ask
How long should an interview with a vendor engineer be?
Forty five to sixty minutes works well, split into a short introduction, thirty minutes of pairing and questions on past work. Longer sessions add little.
Can we ask the vendor to run technical screening for us?
They can, but treat their screening as a filter only, because you still need to see the engineer work on your stack with your codebase’s own habits.
Should the candidate sign a trial period?
Many agreements allow a trial where either side may end the arrangement with brief notice, so ask for that term and a poor match can be corrected.
What if the engineer is good but the vendor lead is not?
Raise it early. A strong engineer under weak coordination can still stall, so ask who they report to and how often that person joins your standup.