I've had the most success doing a remote, one-hour pair programming exercise via screen sharing with the candidate. They're allowed to look at docs, do any google searches, and I don't penalize them if they don't finish. I'm more interested in the process and the way they navigate their environment to solve problems. I continually coach them to speak aloud whatever they're thinking, especially if stuck on something. My last hire was someone who didn't finish the exercise, but I really liked their approach to solving the problem. It's the closest thing to "what would it be like to work with this person" that I've found. This has been super effective for me after many years of making bad hires through "technical interviews" and take home assignments.
I encourage you to drop the whole "see how they think" process. You can't read minds, and it's not possible to see how they think.
I don't like complaining without a suggestion, so... consider letting a candidate code on their own, and then ask them to walk you through what they did. You won't be disturbing the process by staring at them, and you'll get to see a bit about how they work independently & articulate what they've done. Unless you're a pairing shop, that's really what you're looking for anyway, yes?
There is no “one true way” for hiring people. It depends on the position you’re hiring for, the team, the office environment (if there is one), the project, the budget, and so many other things.
The right hiring process for one position may be completely wrong for another. And that’s OK.
I have a strong dislike for the 'see how they think' or 'code like you would in production' style instructions for take-homes - they're not specific enough.
I don't think that necessarily applies to real-time pairing exercises though. The interviewer has significant latitude to interactively question and prompt to get at the skills and values a candidate possesses.
One time a guy asked me to bring in my laptop with some code I had written.
I showed him, we talked about it, he asked me to add a simple feature, I think I taught him something about the tech stack that he didn't know. It worked well for both of us.
Had one company paid me for my time developing their test interview app. That was the first and only experience I had. Took a day to do, but I made sure it was really polished.
We pay our applicants $50/hour for the later stages of skills tests because they can take 15+ hours to complete. More details on previous comment if interested: https://news.ycombinator.com/item?id=20710884
If you pay only for the later stages of skill tests that's 4-8 hours. You probably don't pay over $250.
But you do use the work to build your backend/office applications. In the end you are paying someone $250 for a 15 hour project + two days of in person development time.
Great idea. Many with follow.
"The costs of travel and lodging are compensated at this stage, but we are not paying them hourly. The work they do for us is not customer based work and does not generate revenue for us."
I read the linked comment. While I appreciate that the time commitment on your side is significant, and that a large part of the interview is paid, I don’t think I’d be willing to spend 16-18 hours, plus 1-2 workdays onsite to get a job. That’s basically 3-4 entire work days. Round up a little for all the preliminary tests before you even get to the 16-18 hours, and you might as well call it a full work week.
What’s so compelling about your company that people are willing to put in 5x the time commitment to apply as for most other companies? I can literally do 4 rounds of “phone screen -> on-site” with other companies in that time. If I’m working, why would I want to blow 1/3 of my yearly vacation to go through one interview process?
Paying for the interview is compelling. Almost nobody does that, so it's a differentiator. My first assumption on hearing that is "it sounds like they respect their engineers as professionals more than most companies do".
We often hire people with jobs. At most, it's three work days needed. If that's a problem, we can do the proctored skills test on a Saturday and we can cut the on-site down to 1 day. That's then one business day needed.
Most devs with existing jobs have good jobs with decent vacation/PTO policies. So they have days to invest if they want to.
I think, it ends up being different strokes for different folks. We have a very detailed, and for some, a very compelling job description and offer. The time commitment is relatively small up front and as they engage with us, they get a sense of who we are, how we operate, and they like it. So, by the time we ask if they want to put the time into the skills test, they've had a chance to evaluate us and ultimately feel like it's worth it.
Simplisticly, it might boil down to something like a preference for quality* interviewing over quantity.
* not saying our process is definitively better than others, just that some candidates want fewer interviews that they feel are a better potential fit for them.
I had a YC startup invite me to work for them for a week, for which they paid me a decent sum of money. I was able to push a decent amount of code, considering the onboarding process was “here’s a laptop with the repo and some basic tools on it.” They did not hire me.
Did you pay income tax on that interview? Did they withhold taxes on that payment and issue it via payroll? Genuinely curious, but I suspect the answer is ‘no’ on both counts which might explain why this approach is less popular.