Rendered at 22:47:20 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
cryptoboy2283 5 days ago [-]
Signals I've got from this post:
- Steve's company got acquihired by Amazon, granting him a free ticket in without all the torment of the multi-stage interview pipeline.
- It's a well known fact that it's easier to jump from one FAANG to another, so while interviewing at Google he had significant advantage, plus the blog gaining popularity.
- All of this has caused a deep down imposter syndrome, which resulted in an attempt to "improve" an interview process from the inside - but the wings were clipped pretty quickly by the corporate politics. It turns out that lawyers are not planning to reinvent anything there and hence are somewhat more important than engineers.
- The post itself is an self-applause over essentially a failed effort. "I've tried"
jumpkick 5 days ago [-]
Well, not exactly. He did say that the hiring process at Geoworks was incredibly rigorous, and that he passed that.
I know this author’s name but not his whole life story, but it reads like he got hired at Amazon before they acquired Geoworks and brought all of his former colleagues over.
disgruntledphd2 4 days ago [-]
Yeah, if you read the older blogposts he talks about interviewing at Amazon multiple times.
mgfist 4 days ago [-]
We must've read 2 different posts.
And even if you're interpretation is correct, you sound pretty rude. You're really gonna mock someone for not liking the interview process and trying to make it better?
Yhippa 4 days ago [-]
An equally rude and aggressive response.
mgfist 4 days ago [-]
In what way? I didn't attack his character like he did to the blog author.
s1ngular1ties 5 days ago [-]
Steve Yegge writing a self-applause post? Well, I never! =)
ok_dad 5 days ago [-]
Don’t worry he probably got an agent to do it. Maybe ten.
officialchicken 5 days ago [-]
- "This attracts strong candidates to you, because even your rejections are worth something to them."
That's some solid gaslighting right there! You're right Steve, I would rather be rejected by your team than wasting my time by doing something positive with my career.
BoiledCabbage 4 days ago [-]
Yah this was the part that really threw me as well.
"People would be pleased to have a rejection from us. They'd be proud to carry it sounds with them. Lucky them!"
It's funny, I see an article from Yegge and thought "I like that writer, I haven't read any of his stuff in a while, I'll see what he has to say." Then got to the end and see the links to gas town and gas city and remembered it was the same Yegge that while having accurate foresight about orchestration of agents also was a bit off the deep end in gas town.
But the biggest thing I see in this article is it really sounds like "here is the new company I landed at, and rather than make a post about its product, I'm going instead make a post about how terrible the problem it solves really is, and a post on a proposed solution. And the cues what I'll pop up in a few weeks and just coincidentally post about this new company that just happens to solve this problem in the way I've convinced everyone is the right solution."
While I don't have any evidence of this that's the feeling I left with. And if so, then "thought leaders" are a lot more interesting when not "talking their book."
iwontberude 4 days ago [-]
Counter: I failed a rigorous interview at Facebook years ago and the project lead of their mesh internet Aquila was one of my interviewers. I still was pleased to not have the job because I didn’t really want to be marked with being a Facebook employee.
mathisfun123 4 days ago [-]
I interviewed at Google last year and they said something similarly magnanimous: that they rejected people who wouldn't have been successful at Google and that the rejects actually thanked them for the wisdom. My eyes rolled all the way back in my head. I cancelled the rest of my loop and went to a different FAANG. When I sent the cancellation email I thanked the recruiter for sharing his wisdom.
4 days ago [-]
bluedevilzn 4 days ago [-]
He specifically mentioned that Geoworks got Acquired shortly after he was already at Amazon
So, he actually did go through the multi-stage interview pipeline.
Just because you can’t read well, you don’t have to be so cynical.
rpdillon 4 days ago [-]
This is correct. GP hid a lot of unfounded and speculative negativity behind "this is a signal I'm getting". Not only did Yegge interview at Amazon, he also interviewed at Google and was very successful at senior levels of engineering leadership for the better part of a decade in both roles.
As for imposter syndrome: Yegge graduated from high school at 14 and studied nuclear reactor engineering with the US Navy in addition to his roles at prominent Silicon Valley companies. He is not my pick for somebody with imposter syndrome.
I am a fan of his because I've been reading his blog since the Stevie's Drunken Blog Rants days, and he was quite active in the Emacs community for some time. I also worked at Amazon and so his experience there became particularly interesting to me during that time.
kristjansson 4 days ago [-]
Per TFA Steve got hired by Amazon and then kicked off the acquisition of his former company.
supriyo-biswas 5 days ago [-]
I read through it and that’s not the impression I got, it was a thought experiment and not much more.
However, a few companies do have a co-working day, where you work with various people on the team to solve a business problem. I wonder if he could have proposed that instead.
chihuahua 5 days ago [-]
The "provisional employment" idea sounds good at first, until you think about how it would actually work in practice. You have 100 applicants for 1 position. Which one do you provisionally hire?
Ah of course, we have to do a traditional interview loop to evaluate 10 candidates before we can pick one. So you do the traditional interview loop, and then you have 6 months of provisional employment.
You haven't replaced anything, you've just added another level of hassle for everyone.
shoopadoop 5 days ago [-]
It's an obstacle when hiring people who are currently enjoying stable employment. Probably fine for juniors, but not for experienced people with families and whatnot.
jb1991 5 days ago [-]
That is directly addressed in the article.
blizarre 5 days ago [-]
Not exactly, if I am thinking of leaving my job and I have responsibilities, I will not even entertain taking a leap of faith, resigning from my current position and grinding for 3 month in a super-competitive environment for 1/10 of a chance of getting a better job.
I'll talk to recruiters and interview at other companies on my spare time, as before.
5 days ago [-]
jb1991 5 days ago [-]
My point was that this perspective is not lost on the author of that article, as he directly addresses it.
blizarre 5 days ago [-]
What am I missing then, he talks about multi-days "campfires" contract work as a replacement for the interview process. Maybe I can get a week of holiday on a short notice but nobody will be able to do that 5 times in a month if they are actively looking.
saghm 4 days ago [-]
And even if someone could get time off drive times a month to work "provisionally" at a different job, I can't imagine they'd typically be as productive hopping back and forth at either of them compared to devoting all their time to one. Sure, maybe the prospective employer would be circumspect enough to judge someone based on the correct signals of stuff like whether they're asking the right questions, showing capacity to learn and grow into the role in the long run, etc. rather than strictly judging their output, but being smarter about what you're measuring instead of blindly checking off boxes is also an option in traditional interviews, and plenty of companies still don't do that, so why would the expectation that the lazy approach of only measuring results wouldn't end to happening with this as well?
5 days ago [-]
garretraziel 4 days ago [-]
He addressed it as “job market is in such a bad shape, so perhaps even seniors won’t have any other option”, which might be true, but is sad nonetheless.
pixelatedindex 4 days ago [-]
It’s awfully convenient for old heads who enjoyed the insane run up of the last decade and a half that are in plush situation to then say “oh it might suck for you now, too bad!”
Good for you to make all the hay when the sun shone? Fuck the future generation, let’s make it even harder for them with these insane interview ideas.
libraryofbabel 1 days ago [-]
Where? Please point it out! All he says is there will be more desperate people on the market in future and because they're desperate they'll have to accept trial work. But that's not answering the objection that people who do still have stable jobs in that world won't want to interview at your company if you require them to do a stint of trial work first. And those people include a lot of desirable candidates.
michaelt 5 days ago [-]
Yes, it says "hope your competitors randomly decide to fire their best engineers for some reason" (worded as 'ends up between gigs')
1123581321 4 days ago [-]
I’ve run a program like this. My answers to your objections are:
You don’t run it for everyone, just people who seem good enough and are willing to do it. A resume filter and a 30-minute conversation are usually enough.
You stop as soon as you’ve got your hire. If you’re bad at picking people for the screen, you learn the signals quickly because of your level of investment.
It does replace a longer traditional interview loop. You hardly talk to them up front. You tell the candidate you will extend an offer right after the work phase.
Another objection is not everyone can afford to spend that time with you. That’s true. We would pay them, and still got some refusals. You have to accept that every interview style works better for some candidates than others. You’ll also find candidates who love it.
tcdent 4 days ago [-]
This is my preferred proposal to new contacts as well (I set it up as a contract so there's a little less red tape, but even people that pursue me for traditional employment afterword usually land on an extended contract).
Two things it solves: You get to evaluate me, my ability to deliver, and how I interact with your team and I either bring real value within two weeks, or I don't. I can tell you verbally I am an indispensable asset or I can show you; other people have ruined the verbal trust layer which is why this whole debacle exists in the first place btw.
And more importantly, but less communicated, I get to evaluate you. How your team works, the level of talent present, management's ability to keep direction, and wether I genuinely enjoy what you have for me to work on.
sjducb 4 days ago [-]
My experience with this is that contract is usually better paid than perm, so when it comes time to convert to perm I am not willing to take the pay cut so I go contract somewhere else, or remain with them as a repeat contractor.
HR usually stops my clients from hiring me perm at 2X the rate they pay their full time employees, despite the strong demonstration of competence that I’ve shown over the last few months.
My clients are usually unable to make permanent employment with them more appealing than repeat contracting with them.
1123581321 4 days ago [-]
Yes, that last bit is important. A good employer should welcome being evaluated as much or more as they’re evaluating.
WorldMaker 4 days ago [-]
You could scale "provisional employment" by making use of "past employment" as a weighting mechanic. Let other companies do the "is employable" and "works on a team" checks. I believe that is called "a resume" and "references". "Let's add stamps and maybe a star system for these references" sounds like just reinventing LinkedIn with a slightly more Uber-like UX. LinkedIn already exists? Resumes already exist?
It seems to me like a lot these problems start and end at "we don't trust resumes and past employment history" and "we highly prize 'cultural fit' but think there needs to be a technical component to 'cultural fit' interviews rather than just doing proper 'soft skills' interviews". Maybe we all should just straight up stop pretending like we are looking for anything more than soft skills and vibes?
202508042147 4 days ago [-]
In a different life, I went through multiple interview stages at a FAANG company. In between the sessions, we got to talk to some of the (future) managers. One of them told me that this whole interview process is broken, we should all just be hired temporarily and see how that goes (akin to what Yegge is proposing). The batch had maybe 30 people. Probably not easy to place all of them, but certainly doable.
sgillen 5 days ago [-]
I think you've just lowered the risk of a bad hire for the company, which might allow them to "take a chance" on candidates they might otherwise pass on.
HarHarVeryFunny 4 days ago [-]
It seems the benefit would be one sided - that initial 100 -> 10 (ATS/interview/coin-flip) filtering processes would as usual have a large percentage of both false positive and false negatives.
The false negatives suck for both candidate and hiring company who have accidentally rejected someone they would have like to have hired.
The benefit of the "provisional employment" process would be that even if it doesn't help avoid the false negatives, it would weed out the false positives so that the company at least ends up hiring people who are qualified and a good fit (even though due to the false negatives they may have rejected even better candidates).
SOLAR_FIELDS 4 days ago [-]
Instead of having “provisional employment”, could we instead just have 30/60/90 day plans and evaluate the performance of the engineer and fire them if they don’t meet that criteria at the 90 day mark?
HarHarVeryFunny 4 days ago [-]
The only candidates who would be willing to sign up for that would be ones who were currently unemployed, and with the financial reserves to be able to risk being back out of work in 30/60/90 days.
Given how arbitrary in terms of talent mass layoffs are, there are of course tons of highly qualified out-of-work candidates (and due to age discrimination, maybe some of the best/most-experienced ones!), so this certainly might work to draw from that pool of candidates.
SOLAR_FIELDS 4 days ago [-]
I was being facetious, sorry if that wasn’t clear. That’s just how Regular Employment works. You don’t even need to tell the employee the implications of being on the 90 day plan. You tell them up front what the expectations are for the role, and if they don’t meet them in 90 days, you fire them. Does not need to be more complicated than that.
30/60/90 is pretty standard stuff. It’s literally the documentation and justification you need to fire the employee at the 90 day mark if you need to. Like a preemptive PIP if you will.
I don’t want to hire anyone who doesn’t want to be held accountable to what they say they can do anyway.
HarHarVeryFunny 4 days ago [-]
> You tell them up front what the expectations are for the role, and if they don’t meet them in 90 days, you fire them.
In the US at least, firing people for cause is difficult to do. More likely you just wait for next RIF and include your low-performers on the list.
So, yes, this is business as usual to some extent, but it's not a very cost effective or efficient way of hiring. If you really have work that needs doing, then you want to hire someone able to do it, not have a haphazard hiring process where it's a crapshoot and "if they don't work out we'll just fire them in our annual RIF, and try again".
SOLAR_FIELDS 4 days ago [-]
Limited sample size but I've not worked anywhere in the US that didn't have a 90 day carve out for terminating people from date of hire, MegaCorp or SMB.
Of course it's not cost effective, or efficient, but as the article points out, nothing really is, and this method is at least preferable to the weird quasi-employment situation the article proposes
musicale 4 days ago [-]
> You have 100 applicants for 1 position. Which one do you provisionally hire?
Sample size of 100 seems like overkill for finding a 6-month contract employee.
How about starting with a sample size of 1:
Sort the applicants (ether randomly or by criteria of your choice, pre-filtering as desired) and hire the first one who meets the minimum qualifications, based on a streamlined interview process of your choice.
You could experiment with a larger sample size if desired (perhaps large enough that you expect to find at least one qualified candidate.)
tcdent 4 days ago [-]
Losing two weeks while you try out a candidates fit has way less cost than bringing the wrong person on formally and spending the next year debating wether they were the right choice or not with all of the associated "soft" overhead.
musicale 4 days ago [-]
I think so. 6-month contract with 2-week (or however long) vetting period seems like it might work.
ryukoposting 4 days ago [-]
And that's besides the money problem.
You provisionally hire 10 candidates for 3 jobs, keeping each of them for 6 months:
Do you spend 5 years' salary over the course of those 6 months just to hire people? How do you financially justify ballooning your payroll for a bunch of people you're going to get rid of before you get any ROI out of them? You've (maybe) gotten better talent in the end, but have you actually improved the company's bottom line?
skybrian 5 days ago [-]
You could be less selective with a high-turnover pool of temporary workers, something like an internship program. But that doesn’t mean anyone gets to be an intern.
izacus 5 days ago [-]
To be honest, a lot of ideas are like that - coming from folks who want to be employed but haven't actually ran an employment process.
5 days ago [-]
digitallogic 4 days ago [-]
This kind of thinking is why a lot of market place startups fail.
You have two parties engaging with each other in a subpar way, and your solution is to make things better for one party (hiring side) and significantly worse for the other (candidates). Trying to convince candidates that this is good for them, won't make it so. Eg:
> Every stamp that you hand out, pass or fail, leaves a candidate richer than they showed up. This attracts strong candidates to you, because even your rejections are worth something to them.
Candidates don't want stamps. They want stable work.
MinimalAction 4 days ago [-]
Yes! At some point, the author even complains about the (weaker) US law for employee on how it's a mess to fire somebody, but easy to let go from an internship. So much about stability that employees seek.
jagged-chisel 4 days ago [-]
A mess to fire someone? Every state except Montana lets employers fire without giving a reason.
mooreds 4 days ago [-]
From the article:
> In the U.S. you generally can’t just up and fire someone, even if they are underperforming, even if it’s at-will employment, without risking a lawsuit and some sort of cash outlay. This is what makes hiring decisions so high-stakes.
I've been part of a few situations where someone had to be let go in the USA, and I can tell you it wasn't a trivial decision. There were legal aspects, sure, but there was also a personal aspect. As a boss said once "this is someone's life we're talking about here".
yawnxyz 4 days ago [-]
in most states, as long as it's not discriminatory, you CAN fire anyone for any reason
the article is wrong here
jagged-chisel 4 days ago [-]
> … for any reason.
If they give a reason, it had better be a legal reason. “Because you’re Mexican” - not legal. “Because we don’t like you/you don’t fit in” - are you in any way different than management? Differently gendered? Differently skin toned? Different faith? Pregnant? Likely not legal.
They are not required to give a reason. You can be fired without reason.
There’s also mitigating circumstances. If the employee can show retaliation by the employer (say the employee participates in protected activities, like espousing unionization or questioning managerial decisions), they stand a chance of recovering damages from the company.
yawnxyz 3 days ago [-]
yeah that's true, and it gets murky - because employers will just wait til an employee messes up (which everyone does), and use that as an excuse
convolvatron 4 days ago [-]
I guess that's the rub. every time I've been involved in firing someone that wasn't defensible as a 'broad structural layoff', there is a whole process, even in at will states. HR starts to collect documentation describing how the employees performance isn't acceptable, at that they were warned on such and such dates, and that all efforts were made in order to rectify the situation before an actual termination.
whether that's an excess of legal paranoia or not, that's how it seems to work
jagged-chisel 4 days ago [-]
> HR starts to collect documentation describing how the employees performance isn't acceptable, at that they were warned on such and such dates, and that all efforts were made in order to rectify the situation before an actual termination.
I think this is a response to inexperienced, immature, or thin-skinned managers. The company doesn’t want to be on the hook for some manager’s attempt to fire someone and have it be an illegal firing. Since there’s typically no day-to-day log of behavior, HR has to begin documenting things. The only thing they can be certain is “safe” for the company is to fire for underperforming. So let’s begin the paper trail…
Schnitz 4 days ago [-]
In practice, at any larger corporation with an HR department, despite the law saying so, you can absolutely not fire anyone for any reason. As a manager you’ll have to go through a 1+ month process of documented coaching, performance improvement plan, etc to gather evidence in case the employee sues after being let go. So not sure if Steve is technically correct here, but he is very much practically correct. It’s been like this in big tech, which is what Steve is talking about, for a long time.
cherry_tree 4 days ago [-]
>at any larger corporation
Like walmart, mcdonalds, amazon, target, home depot? It’s incredible how insulated the HN audience is to the real life day-to-day concerns of workers in the United States.
Schnitz 4 days ago [-]
These companies do PIPs for software engineers, which is what Steve is talking about. He isn’t talking about a warehouse or fast food job interview. I am not insulated, I am merely staying on topic.
yawnxyz 3 days ago [-]
layoffs happen, and they're firings "without a real reason" - and no PIPs exist for surviving layoffs
QuiEgo 4 days ago [-]
You can, but most places would rather spend months building a legal shield through a PiP rather than risk a lawsuit.
cherry_tree 4 days ago [-]
Most places? Are you a human? Have you been to the United States?
Do you genuinely think McDonald’s is building a months long case against a fry cook? That Amazon is building a months long case against a delivery driver or warehouse worker who refused to pee in a bottle? That target is putting a cashier on a pip for months and allowing lost sales or theft?
QuiEgo 4 days ago [-]
The article is about tech interviews (and tech workers with the resources to sue), not McDonald’s or blue collar?
And yes, I’m human?
Strange comment internet stranger.
mooreds 4 days ago [-]
There's the legal world and then there's the world of custom and ethics. They don't always diverge, but in my experience they do.
At least over the last 25 years in the tech spaces I've been part of, I rarely hear of people being dismissed/let go quickly. #anecdata
sitzkrieg 4 days ago [-]
is this sarcasm? tech industry has some of the worst layoffs ever. i have personally seen several different employers notify employees day of firing they no longer have a job
Cpoll 4 days ago [-]
No-one in this thread is talking about layoffs, they're talking about individual firings.
gruez 4 days ago [-]
I don't care about the notice period. I care about how much severance I'm getting. I'd be fine with no notice if the severance was generous.
mooreds 4 days ago [-]
Not sarcasm. I guess I've just had different experiences than you.
The larger point of the effort involved before the employees' last day still stands. But that's a hard one to speak to generally, since most folks in layoffs or firings don't talk about the process used to arrive at the decisions.
cherry_tree 4 days ago [-]
You haven’t experienced the last 2 years of layoffs across the tech industry? Do you have any friends in the industry? Do you avoid news about the industry?
spit2wind 4 days ago [-]
Dead wrong.
Because I see so many people who believe otherwise, here's a direct quote from a prior contract:
> I UNDERSTAND AND ACKNOWLEDGE THAT MY EMPLOYMENT WITH THE COMPANY IS FOR NO SPECIFIED TERM AND CONSTITUTES "AT-WILL" EMPLOYMENT. I ALSO UNDERSTAND THAT ANY REPRESENTATION TO THE CONTRARY IS UNAUTHORIZED AND NOT VALID UNLESS SIGNED BY THE PRESIDENT OR CEO OF THE COMPANY. ACCORDINGLY, I ACKNOWLEDGE THAT MY EMPLOYMENT RELATIONSHIP MAY BE TERMINATED AT ANY TIME, WITH OR WITHOUT GOOD CAUSE OR FOR ANY OR NO CAUSE, AT MY OPTION OR AT THE OPTION OF THE COMPANY WITH OR WITHOUT ME NOTICE. I FURTHER ACKNOWLEDGE THAT THE COMPANY MAY MODIFY JOB TITLES, SALARIES, AND BENEFITS FROM TIME TO TIME AS IT IS NECESSARY.
Every job I ever held but the unionized ones had some variation of that.
mooreds 4 days ago [-]
Have you known folks at those jobs to be notified of being let go and sent home on the same day?
Have you been involved with letting people go and seen it happen as quickly as the above legal disclaimer implies?
I agree that the legal justification is 100% there, but in my experience it is often buffered and takes more time.
yawnxyz 3 days ago [-]
yeah it happens at smaller companies - someone insulted the boss and got sent home (small agency)
kristjansson 4 days ago [-]
Whatever clause in whatever contract is just the starting point for negotiations if things go south.
paulryanrogers 4 days ago [-]
What did they say is dead wrong?
rustystump 4 days ago [-]
This is why severance even 2 weeks of it is common. To get it, gotta sign that giving up legal action for the money.
tptacek 5 days ago [-]
The gold standard in hiring qualification is work-sample testing. It works fine. You do not need to "make hiring a profit center" or "provisionally hire" or do internships. Work samples done correctly demand less time from candidates than interviews and scale better than interviews. They are standardizable and iterable.
What I feel like I'm reading here is someone who has been poisoned by FAANG hiring practices --- and they are terrible --- and has missed most of the work that's been done (outside of Google's admirable work in debunking their own processes).
I appreciate the "kitchen confidential" here, but with respect to Yegge, I think he's been working at the Olive Garden this whole time. Go stage at Gramercy Tavern! They're working at a different scale, yes, but you'll at least get a different perspective on the "gold standard".
enraged_camel 5 days ago [-]
The problem with work-sample testing (which is commonly administered as a take-home problem for the developer candidate to solve) is two-fold:
a) it discriminates against people who cannot spare 4+ hours of focused time on evenings/weekends to work on the problem. People with multiple jobs, single parents, etc.
b) in the age of AI it is no longer a reliable measure of someone's skill, for obvious reasons
Unlike Yegge, I haven't worked at FAANG, but the companies I have worked at all followed the same hiring practices and suffered from the same problems as he describes.
Provisional employment (or, if that's not possible, then well-paid internships) solve all of those issues. The candidate gets 3-6 months of stable employment, you as the employer get a large number of work-sample tests, and you can see how they use AI and how much.
dpark 5 days ago [-]
I think a very real problem is that these take home problems are often way more than 4 hours. And to that they often add the traditional 4-6 hour interview loop.
Provisional employment doesn’t work for most cases, though. It might attract people who have no job and it might attract people who have so much saved that they are okay with potentially being let go after 90
days. But I imagine the vast majority of the potential employment pool are not willing to quit their current job to accept a “maybe” job from another company.
tptacek 5 days ago [-]
Adding take-home problems to a traditional 4-6 hour interview loop is odious.
But the "way more than 4 hours" thing smuggles in a premise: that every candidate should be able to finish the challenge in the allotted time. But candidates with greater aptitude or conversance with the problem domain will complete work sample tests faster than candidates without, and selecting for those candidates is the point of hiring qualification.
michaelt 5 days ago [-]
It depends on the details of the work sample test.
If I ask you to write me a python function to convert OSGB easting/northing into WGS84 longitude/latitude the task has a very clearly defined scope. If you knock it out in a quarter of the allotted 4 hours, you've saved time. You can't use the remaining time to go further and demonstrate your mastery.
On the other hand if I ask you to write me a website for organising photos, there's no such thing as 'done' - no matter how good you are, after 4 hours you'll still be able to think of ways to make it faster, more beautiful, more featureful, more scalable, cheaper to operate, etc
Obviously, as a hiring manager I'll notice if you've spent 40 hours on the 4 hour task - but if you've spent 6 hours maybe I just think you're a fast worker with relevant experience and sharp tools. And my sense of how far you can get is calibrated by other prospective hires; if lots of people are spending 6 hours and claiming to have spent 4, my expectations will naturally be high.
tptacek 4 days ago [-]
Again, the premise is that you're exercising professional judgement. If you can't let a project go until it's perfect to a standard far past what's called for, that is itself signal. Either way: if the project budgets 4 hours, it is on you as a professional to stop at 4 hours.
lesam 4 days ago [-]
If you run an interview process where candidates who take 6-8 hours and claim to have taken 4 hours score highest, those are the candidates you will hire.
tptacek 4 days ago [-]
All these objections rely on removing agency from the professionals applying for jobs. You look at the work sample. You use your professional judgement. You decide if it's reasonable to execute it to what you think a professional standard would be in the time allotted. You make a decision.
This isn't a college application.
akerl_ 4 days ago [-]
I think you're answering a different objection than they're making. Their concern is that people will choose to spend 8 hours on your 4 hour problem but then tell you they only spent 4. Then you'll think they're a leet hacker because their solution is so awesome and they did it so fast.
tptacek 4 days ago [-]
Why would I think that? There's a rubric for the challenge defined up front, before anyone does it.
akerl_ 4 days ago [-]
No clue; it wasn’t my concern.
lesam 4 days ago [-]
Why would you design a hiring process that scores unprofessional people (by your own definition) higher than professional ones?
tptacek 4 days ago [-]
Again: I am responsive to the concern that hiring processes can demand too much of candidates, and particularly to the idea that work-sample challenges are unreasonably demanding compared to interviews. That's why hiring processes I've designed over the last 10 years have all been budgeted against the time typically allotted to an interview loop. And then, to people who say "the challenges take more time than the budget, so I'm forced to spend more time", I say "if you believe that to be the case, don't do the work sample challenge".
The rest of this I'm not interested in. For as long as we've been talking about hiring processes on HN, there have always been staunch defenders of interviews. Lots of people have spent time getting good at them, there are classes on it, there are books, there are drilling exercises. I don't anticipate talking those people out of their investment in interviewing.
4 days ago [-]
masfuerte 4 days ago [-]
> And then, to people who say "the challenges take more time than the budget, so I'm forced to spend more time"
People aren't saying that. They are saying that other candidates will put in more time. If I do a professional job in four hours, then an equally talented candidate who puts in eight hours will produce a much more polished effort, and they will get the job.
I guess the fix is to ask the candidate to pick a four hour window, and to ask them to complete the task in that time.
tptacek 4 days ago [-]
Again: the rubric is defined up front. You can actually lose points for doing too much.
I understand that some people are concerned that they're competing with candidates who will put in 12 hours to do what they should be doing in 4. But that's not their problem. Their problem as a professional is to evaluate whether they can do the challenge in 4 hours; that's the expectation the job is setting.
It is perfectly reasonable for someone to look at the hiring process we're running and say "no, this communicates to me that this job wouldn't be a good fit for me". That's a good outcome! Most jobs aren't a good fit for most people; that's the whole challenge of hiring.
masfuerte 4 days ago [-]
Imagine I'm an employer who wants to adopt this system. How can I distinguish the candidate who spent four hours from the candidate who spent twelve?
tptacek 4 days ago [-]
If you care, you enforce a time limit or have the work sample done on site. We simply don't care.
masfuerte 4 days ago [-]
Thanks for clearing that up. So if you employ someone who's actually really slow then I guess you just fire them?
I'm based in the UK, where firing people is harder, so this wouldn't fly. That said, I know interviews are basically useless so I would like something better.
tptacek 4 days ago [-]
I have no idea. It's never happened.
BoiledCabbage 4 days ago [-]
That's absurd thinking if putting in 6-8 hrs outta what everyone else is doing and what is needed to get you a job.
For all its flaws, part of the benefit of an interview is it's time bound and equal for everyone. Similar to a test.
tptacek 4 days ago [-]
Look, if you want to make people do work samples from an uncomfortable conference room at your office, be my guest. I am pretty confident I speak for the majority of candidates when I say that that my preference would strongly be for the ability to work on this stuff from wherever I want to.
ncallaway 4 days ago [-]
I mean, that doesn't have to be how it works. You can have a both fixed amount of time, and the ability for a candidate to work in whatever environment they want.
tptacek 4 days ago [-]
Of course, and if you want to do that, I've got no complaints. What we want is to eliminate pressure and scheduling inconvenience. We're also not unhappy to meet people who are not necessarily experts in our problem domain, but capable enough programmers that they can ramp up given a bit of extra time. I don't feel the slightest bit bad creating that affordance, so long as you can meet the rubric if you're an experienced professional in the time we allot.
dpark 5 days ago [-]
This is theoretically true but it’s also rife with misaligned priorities. The people putting together these take home assignments have little incentive to ensure that they can be completed by a competent engineer in the allotted time. The engineers completing these assignments are definitely incentivized to underreport how long they spent on the assignment.
With AI coding this is also largely useless. These “build this thing in 4 hours” assignments come with a literal prepared prompt so that they can be churned out in 10 minutes.
tptacek 5 days ago [-]
We don't ask or check how long candidates take. You're a professional, we give you a challenge, you can decide (up front, 30 minutes in, whatever) whether you're likely to be able to finish in the budgeted time. Maybe you can't because you've got a lunch date and don't have the contiguous block, and want to do it in chunks. Fine by us.
Again, the underlying smuggled premise here is that candidates have to finish the work sample. No they don't. In fact, that's a strong sign it's not an effective work sample. A test that everybody passes isn't a real test; it's just a hazing ritual.
jaggederest 5 days ago [-]
Just anecdotally, I can confirm that this method works great - it screened me out by showing me exactly what day to day work was like at your company, and that I was not nearly nerdy enough about specifically containerization to want to do it all day.
So you saved yourself/the team several possible hours of interviewing, and me quite a few hours - I think it took me about 10 to 15 minutes to see what you wanted in an engineer and that I was not it, and a total of 1 email which felt quite automated (whether it was or not) so there was a very low social cost as well.
dpark 5 days ago [-]
You are assuming the assignment is reasonable and the candidate is lacking if they cannot complete in the expected time. And for all I know that’s entirely true for you. What I know is that I’ve seen assignments from others where the assignment scope was unreasonable for the allotted time. And for those teams, the filter becomes not so much “is this person capable” but “is this person willing to put up with our shit”, and the teams likely don’t even realize that’s what they’ve done (because they also “don't ask or check how long candidates take”).
> Again, the underlying smuggled premise here is that candidates have to finish the work sample. No they don't. In fact, that's a strong sign it's not an effective work sample. A test that everybody passes isn't a real test; it's just a hazing ritual.
Now this is an interesting take. Usually when people talk about these take home assignments, they talk about assessing the quality of the work. How good is the design? How is the coding? Is it efficient/elegant/whatever?
Here you take a much different approach, saying that the completion itself is the filter. If one person completes your assignment in the allotted 2 hours and another needs 12 but never tells you that, do you not care about that discrepancy?
tptacek 5 days ago [-]
We do assess all of those things. Again: you're a professional. We give you a work sample test. You look at it, and use your best professional judgement to decide if it is (a) reasonable and (b) doable in the budgeted time given your capabilities. If either is untrue, you don't do the work sample.
I'm having a real hard time seeing how this isn't strictly better than an interview, which, as the article (and basically everything written in the last year about interviews) points out, is basically a random function.
tptacek 5 days ago [-]
A standard interview loop kills an entire work day, and is preceded by phone interviews that eat several hours. Properly budgeted work samples are strictly better from the candidate's time perspective, not to mention that you can do them from your couch rather than under flourescent lights in a confeence room.
The AI thing is an interesting problem, but a solvable one. We continue to hire resume-blind.
throwburn202605 5 days ago [-]
The question is whether companies would tolerate their own process if their employees did that recruiting process at a different company. They obviously do to some extent via plausible deniability; I have a 1 hour "doctors" appointment this afternoon, or I'm taking leave on Monday. Using it as cover to attend an interview.
Would this company permit an employee taking 3 months unpaid leave to provisional hire somewhere else and have free choice whether they stay or go at the end of it.
enraged_camel 5 days ago [-]
>> A standard interview loop kills an entire work day, and is preceded by phone interviews that eat several hours. Properly budgeted work samples are strictly better from the candidate's time perspective, not to mention that you can do them from your couch rather than under flourescent lights in a confeence room.
Yes, standard interview loops also discriminate, and the more time they take, the more discriminatory they are. Any on-site requirements compound the issues.
Like Yegge says: provisional employment/internships solve all of these issues. You get the best of all worlds: stable employment for the candidate where they get paid a regular wage and aren't under a stressful interview setting, and lots and lots of work samples for you, the employer. It's not perfect. For example, it can be difficult to entrust the provisional employee/intern with anything impactful if you don't know whether they'll be employed at the end. But it is significantly better than the alternatives in most contexts.
tptacek 5 days ago [-]
Provisional employment does not work. It requires candidates to leave their jobs before they know whether they have a secure job with your firm. I concede that provisional hires are higher-signal than work sample testing (or rather: that they're the platonic ideal of work sample testing), but the entire problem of hiring qualification is to make decisions in the context of a candidate doing a job search.
enraged_camel 5 days ago [-]
>> Provisional employment does not work. It requires candidates to leave their jobs before they know whether they have a secure job with your firm.
It works. I’ve seen it in two different places.
At the second one, the fundamental realization I came to was that it is virtually no different than “regular” employment, where the new employee can get fired for not meeting expectations within an arbitrary time period after being hired. This can be months, or even weeks. From the perspective of the candidate, regular employment and provisional employment have roughly the same level of risk: in both cases they take a job where they might be let go at some point. The benefit of provisional employment is that they know how long they will be evaluated for and against whom. It turns out a lot of candidates do in fact like the all-cards-on-the-table approach and enjoy being given the opportunity to prove themselves on the job.
tptacek 5 days ago [-]
Everything will "work" in the sense of you'll eventually get somebody in the job. To me, it's self-evident that a process that disqualifies everybody unwilling to work for weeks speculatively isn't actually working, at a minimum because it DQs most (probably all) competitive candidates. But yes, your mileage may vary.
pixelatedindex 4 days ago [-]
I’m curious though… these days I’ve to apply for n number of jobs to get 1.
Let’s say I get someone to do this provisional work. What about my existing job? I can’t just ignore it, and I can’t just causally say at my current company that I’m interviewing anywhere else.
And god forbid you like 2 potential gigs, and they both want you to do the campfire approach. Then my current job is really screwed!
esafak 4 days ago [-]
Provisional employment only makes sense if you work remotely and have full benefits (esp. healthcare). Moving is costly.
tptacek 4 days ago [-]
A committed or indefinite term of employment is itself a benefit, so if you're going to work on a temporary basis you should expect contractor returns, not FTE comp; revise your baseline up 150%-200%.
Nobody does this, of course, but then provisional employment is a silly idea to begin with.
kasey_junk 3 days ago [-]
It has to be more than that. 150-200 is what your rate is _if you have a pipeline for more work_. If you don’t, you need to account for building the pipeline. So if a non-contract job wants me to work like a contractor, they have to account for the risk they’ve asked me to assume.
FWIW I came up with a provisional employment scheme that I thought would work at a job where I was responsible for hiring pipelines. It involved making an annual bonus (in an industry that was bonus driven) garaunteed and accelerating it so it was 50% sign on bonus and 50% delivered at 6 months. The company balked. Because provisional screening is a very good mechanism. Its must more expensive than hire and fire, at least on paper.
littlexsparkee 5 days ago [-]
I feel like you could get around the AI bit by asking about components and what they do, rationale for decisions, etc. If someone can't speak to it, it should be a clear tell.
tptacek 5 days ago [-]
We hire entirely based on work sample testing, and there's a lot of stuff you can do to make it work in with AI-equipped candidates; I'm not prepared to write it up at the moment, but you start by recognizing that everybody is going to be using AI and designing the tests accordingly, and by relying on unassisted interactive challenges as a component of the process.
esafak 4 days ago [-]
As long as you are talking to them face to face; over the phone they will use AI with speech recognition and parrot its response, erasing all signal. Then the interview becomes all about AI detection.
tptacek 4 days ago [-]
I've been hearing these kinds of things since 2014 (when I wrote a long post about the work sample process we had used at Matasano). I've been hiring continuously since 2008, so 18 years, and in that entire time I have never come close to hiring a scammer.
It might be a more salient concern now, in the era of AI agents, and we are much warier today than I was at Matasano, but generally I think this risk is more talked about than experienced.
The_Fox 2 days ago [-]
I read that post back in the day and have remembered it ever since, and used it as a basis for the process at my current employment. Thank you!
Ive had people show up at onsites that were clearly not the same person I screened over the phone earlier as far back as 2013 at google. It’s real. Just ask any recruiters from well known companies and they’ll tell you. It’s happening to more and more startups too now because it’s gotten easier
tptacek 4 days ago [-]
I believe you, but again: I haven't experienced anything like this, ever. One of the clearest results we got at Matasano adopting this approach was a drastic reduction in turnover, which is exactly the opposite of what you expect to see if people are scamming their way into jobs they can't do. That's held up (I don't have the before/after comparison, but I do have the industry-wide comparison) over the years afterwards. We hire these people, then we work with them, and they're amazing.
mplanchard 4 days ago [-]
And coding interviews often bias against people who can’t spend days of time grinding on hackerrank or whatever to prepare for the interview. Provisional employment biases against people who currently have jobs.
There is no perfect interview process, so it’s important companies think about what they’re biasing for or against. A work sample gives a more realistic picture of someone’s ability than a coding interview, but less than provisional employment. But, it’s much easier for people to commit to finding the time for it than to spending 3-6 months in limbo.
Regarding AI and work samples, it seems like this problem is largely solved by having an in-depth conversation with the candidate about the assignment, no? They should be able to explain why they made the decisions they did, pros and cons of their approach, etc. If they can do that, and their solution is good, does it matter if it was LLM-assisted?
NoMoreNicksLeft 4 days ago [-]
>The candidate gets 3-6 months of stable employment
To me, "stable" implies I don't even worry about having a full-time job for the next 3-5 years. Anything less isn't stable, unless this would be my second job immediately after the McDonald's burgerflipper one where I was scheduled for random 4 hour shifts seemingly designed to maximize personal inconvenience.
If I knew that my job would only last 6 months, I would have to immediately begin conducting a job search and prepping for it. I put my odds at finding another job before the 6 months is up at less than 60%. Even that number seems naively optimistic now that I've typed it out. Offering me 3-6 months of employment only sounds like a deal if I'm currently unemployed and rent was due yesterday.
chollida1 4 days ago [-]
We do all our work samples in person at a our offices as part of the in person interviews. Takes 2-3 hours, never been a problem so far.
If you are going to take a day off to do 3-4 in person interviews at a company then this slots in well.
kevin_nisbet 5 days ago [-]
I'm a little bit more on the fence with the work sample interviews having designed them and also interviewed through them. I've also done my fair share of "traditional" tech interviews, all at startups, never FAANG.
As an interviewer, I much prefer the signals generated through a work-sample interview. I'm much more confident in the hiring recommendation than I get from a 1 hour zoom session. However, if I look at teams that were built through the work-sample and zoom interviews, I'm not sure the outcomes were that noticeably better.
As an interviewee, I think I understand the frustration being on the other side has. With an in-person interview, I often have a good sense that I bombed the interview or something to improve on or replay in my head, less surprising outcomes. On the work samples it's harder to know whether you're making mistakes, or are being out-competed by someone putting in 4 times the effort to polish the solution beyond what their regular work product would be. Although I had one really good outcome where the work-sample interview really flagged the internal dysfunction of a company.
And then with both interview processes, I still think there is a really big unknown on what the false no-hire rate is, how much effort is getting wasted rejecting candidates that would actually fit the team.
So having to choose a process as an interviewer, I'm with you and would always choose a work-sample interview. On whether it should be considered the "gold standard", I'm much more hesitant, I think there are some limitations that are still hard to control for.
I do wish Starfighter/Stockfighter model had gained more traction, would've been interesting to see a recruiting company specialize in this and then seeding the interview results to multiple companies model work out.
mplanchard 4 days ago [-]
Going through the interview process now for the first time in half a decade, and while I already would have said five years ago that I preferred work samples, that opinion is only growing stronger as I go through the process again.
Leetcode style interviews feel so stupid and divorced from the reality of the job, especially the “one weird trick” kind where you’re expected to discern the best possible solution to a problem on the spot and in a pressurized situation.
The reality of the job is usually that when you are under time pressure, a suboptimal solution that does the job is fine (to be fixed later), while if you’re working on something you know is important (hot loop code, core data structures) you have time to think about it and get it right. A leetcode interview doesn’t select for either of those things: it selects for people who have time to grind leetcode problems.
On the other hand, a work sample is realistic: a timeboxed task that you can approach in a familiar environment, without an interviewer breathing down your neck, expecting you to think out loud, railroading you to their preferred solution, etc.
As an interviewer, I always pushed for either work samples, which I quite liked, or coding interviews where we very explicitly said that we just want a working solution, which we could then talk through and look for potential improvements. We also explicitly viewed the coding interviews as being low signal, and tried to make the bar for passing low, so we could get candidates to higher signal conversations.
I do think the work sample route is a little more difficult in the LLM era, in that you are more likely to get a decent performance from a candidate who doesn’t actually know the domain, but a subsequent discussion asking them to explain their approach seems like it would be enough to ferret that out.
tptacek 4 days ago [-]
I shouldn't keep leaving this comment because it's not that useful but: AI-era work samples are a fun problem (you definitely can't just use your pre-agent default work samples!), and we've come up with a couple useful solutions.
If you're in an environment that is open to agents to begin with, one simple thing to do is just tell people to use agents, and ask for the prompts. Prompts are a high-signal artifact, and you can construct rubrics to evaluate them objectively.
I'll get around to writing a piece about this within the next couple months.
Ultimately work samples aren't a technology technique; they're a longstanding concept in management science. Long before the first coding take home was ever given, firms were using work samples to qualify salespeople, support professionals, factory workers, whatever. AI agents are a part of what the work of software development involves, and there's obviously no fundamental reason why you can't work sample them like anything else.
csh0 4 days ago [-]
In a similar vein, in the pre-LLM era I gave a few interviews where the candidate was asked to screenshare while solving the problem. The candidates were allowed to use any resource they wanted from the broader internet.
I often found that I learned more about the candidate by the way they phrased their Google searches and how they selected and explored sites for information than from the actual solution they produced.
musicale 4 days ago [-]
> the “one weird trick” kind where you’re expected to discern the best possible solution to a problem on the spot and in a pressurized situation.
Those were (and are) absolute rubbish; "in 20 minutes please come up with something that took [Knuth, Dijkstra et al.] years to come up with."
Fortunately AI seems to have killed them (leetcode interviews, that is - Knuth is very much alive
and seems to be happily coexisting with AI [1].)
Disclosure: I did enjoy Google Code Jam as a fun thing (even though AI killed it as well), but it's useless as a model or method for hiring.
As a recent interviewee, I much more prefer work samples. Less stressful, more in my control and less bound to whether I got lucky and clicked with the problem in a live interview. It's also just much more akin to what work is like, and therefore requires far less studying. The fact that live interviewing is a completely different skill to actual work is a really bad smell.
jghn 4 days ago [-]
> I'm not sure the outcomes were that noticeably better.
It's not just you. At the end of the day interviewing has been demonstrated to be close to a crapshoot in the best of circumstances, and very few interview schemes are the best of circumstances. Work samples are part of the optimal strategy [1] but even then the signal is quite low.
Work sample interviews don't have to be take home. We ran our technical interviews as close to work samples as possible and in person.
dilyevsky 4 days ago [-]
Ime take-homes was a good way to purge all the good candidates with options from your pipeline as they would simply not turn anything in and go with someone else. Maybe with AI and so many layoffs it’s better now I haven’t tried in last couple of years though i doubt it
tptacek 4 days ago [-]
People have been telling me that for 18 years and (a) we've hired our fair share of candidates with gold-plated resumes (and stopped hiring candidates that only had gold-plated resumes, acquired from failing up) and (b) the nontraditional background candidates we've hired have generally been our top performers.
Hiring in most of the tech industry is gatekept, and like every protectionist scheme, it isn't organized for productivity.
dalyons 4 days ago [-]
Take homes are much lower signal now with AI
dilyevsky 4 days ago [-]
Depends on what you ask, i suppose. I’m sure i can come up with something that can’t simply be one shot or the result would be bad if you do. A bit more difficult on logistics tho as you’d have to arrange for an environment with some prepaid llm access
HenriTEL 4 days ago [-]
Without a way to estimate "AI power" used for the task I don't see how you can fairly rate home assignments.
dilyevsky 4 days ago [-]
Like i said either use pre-baked env or give a candidate an auth token with something like $100-200 quota from the provider your company already uses
brennanpeterson 4 days ago [-]
It would be nice to have portfolios, but systems are broken enough that that becomes hard to see. I suspect one of the reasons for the bias to hiring PHds in fields where it really isn't necessary is at least you have a work portfolio.
tptacek 4 days ago [-]
Why do I care about your portfolio if I can just see how effectively you do the actual work?
nitwit005 5 days ago [-]
I gave the feedback at one Google interview that they should send Google employees through to see how many get hired. Good to see they basically tried that.
The conclusion at the end that bringing someone on board is the ideal method is true I'm sure, but even that runs into the issue that employee evaluation is an even worse situation than the interview process.
You can openly see some managers panic when they realize they have no idea what their employees have been doing for the last 6-12 months when they're asked to provide feedback.
kudokatz 5 days ago [-]
2 anecdotes ...
1) The worst interview I ever had (BY FAR) was at Google--disrespectful people, no respect for time, I could go on and on. And I went back to try again to get that money showered on me. Worth it in the long run.
2) Their new system for "performance management" is a hoax. Just like at all other places, it "documents" what you should do so they can fire you more easily with unspoken rules and all sorts of arbitrary causes as well. A friend literally hit EVERY pre-agreed target and still got pushed out for "not delivering".
Traubenfuchs 5 days ago [-]
I once failed all my goals agreed upon a year ago but got promoted anyways, because priorities had shifted and the director above me just really, really wanted me to continue building his reputation. (not at google)
LtWorf 5 days ago [-]
Oh I've had a terrible interview at Google too!
They told me it was all about pseudocode and how I think in advance, then on the actual interview they were being annoying about variable names and spaces after comma, while I was supposed to come up with some clever optimisation that boiled down to: "do you know this obscure theorem already? Cool you pass"
inerte 4 days ago [-]
Same for me. I applied for engineering manager. Was told it was going to be a code review. I found a bunch of things that it could be improved, detected a very slow part / quadratic loop, but couldn't come up with the best algo to use. It was just leetcode in disguise.
LPisGood 5 days ago [-]
Which theorem?
LtWorf 5 days ago [-]
I don't even know the name. I reviewed the question with a friend of mine who has a PhD in mathematics and teaches at university and he figured it out (after a way longer time than the duration of the interview itself) :)
dnnddidiej 5 days ago [-]
It is not necessarily bad if people hired cannot make it through. Reasons:
1. Standards got higher. Luckily if you got in early and proved yourself you are OK. But doesn't mean you would pass the current interview.
2. A marathon runner (with rare exceptions) can't run a marathon on a random day. They train for a specific date. Same with interview prep.
antiframe 4 days ago [-]
If I read the article correctly, they handed the hiring committee /their own/ anonymized packets and the committee chose to only hire 1/3 of themselves. That mean numer 2 is not the case. Presumably the hiring committee prepared for their interviews of they passed their rounds.
Number 1 can still be true, and likely is. But then what's the point of using dated packets to test the hiring committees calibration?
thaumasiotes 5 days ago [-]
> I gave the feedback at one Google interview that they should send Google employees through to see how many get hired. Good to see they basically tried that.
They did, but not with the intention of doing anything about the problem.
This is a question of reliability, the conceptual 'correlation' of a measurement instrument with itself when measuring the same thing.
Reliability is one of two major concepts in psychometrics, the other being validity, the conceptual correlation between a measurement instrument and that part of reality that you're hoping to measure.
The question behind validity is "I want to know X; if I measure Y, how helpful will that be?". And the question behind reliability is "if I measure Z, how accurate will that measurement be?"
Yegge calls out both concepts explicitly, though not by name, in this essay:
>> The outcomes from interviewing are statistically terrible. Google did wave upon wave of analysis over the years, and all the results were incredibly depressing.
>> [reliability] To name just a few off the top of my head: interviewers barely agreed with each other. Put the same candidate in front of two of our sharpest people and you’d routinely get a confident “strong hire” from one and a flat “no” from the other.
>> [validity, though the 'problem' here is strongly confounded by a restriction of range issue] And once people were actually on the job, their interview scores told you next to nothing about how they’d do
>> [reliability] Hell, some of our star performers failed their Google interviews four or five times, finally got in after 2+ years...
>> [validity] ...and then outshone everyone else.
The discussion of how interviewing outcomes are statistically terrible would benefit from naming the ways in which they're statistically terrible. Knowing the problem you have is an important step toward solving it.
(And as a side note, the last I heard from Google, you're not allowed to interview more often than once a year. Interviewing five times in two years would seem to violate that policy.)
It is a basic theorem that the validity of any instrument is bounded above by the square root of the reliability. It isn't possible for an unreliable instrument to be tightly correlated to reality, because it is, by definition, not tightly correlated with anything. That's what it means to be unreliable.
Thus, any company that wanted its hiring process to be good would necessarily be extremely concerned with making that process accurate; you need to come to the same decision when you assess the same person. This is something that interviews cannot achieve except at extreme cost. You'd need far more than five interviews to get a reliable assessment from them, despite the claim in this essay that "any more than four interviews and you're just playin' with your food". Of course, the Google interviews aren't supposed to be reliable anyway, so in that sense the claim is probably accurate.
The prescription Yegge offers is valid. Multi-month work assessments will give you a strong, reliable, and valid signal. They're also very expensive.
Another thing the essay completely glosses over is that this problem has been recognized for a long time, and we already know how to do assessments that are reliable, valid, and cheap to perform. They're called standardized tests.
Ferret7446 5 days ago [-]
At least historically, Google prioritized not hiring bad candidates over hiring good candidates. So it was neither a priority for interviews to be consistent (for good candidates) or for employees to be able to consistently pass interviews.
HarHarVeryFunny 4 days ago [-]
That certainly makes sense as a goal, given the cost of hiring someone bad and then not being able to get rid of them.
The problem is that companies like Google that have evaluated their own hiring process, by comparing candidates "hiring score" with subsequent on-the-job performance, have found that there is little correlation. So, while the goal (be more concerned about false positives than false negatives) makes sense, their process of trying to achieve this is broken.
esafak 4 days ago [-]
Big companies already have standardized tests; test banks that get rotated with grading rubrics. Examiners (employees) will ask their favorite questions over and over to calibrate where a given candidate stands.
syndacks 5 days ago [-]
Serious question, tell me what you think of using IQ tests to hire SWEs? Should we just do that instead?
HarHarVeryFunny 4 days ago [-]
Why not do that for all jobs. Forget resumes and work experience/accomplishments, and just hire based on a test score?
Perhaps we could administer this IQ test at age 12, so that the low-scoring individuals can go straight into the fast-food industry, and the rest can pick between the doctor/lawyer/SWE offers that will be showered upon them?
syndacks 4 days ago [-]
From what I understand, this isn’t so far from what Germany does with their secondary schools
dennis_jeeves2 4 days ago [-]
Yes I do think it has merit. I think some kind of specialized IQ test which measures for aptitude can be used for for screening. Of course it's not a be all and end all but it should significantly reduce the 'grinding-leetcode' situation.
tptacek 5 days ago [-]
He said standardized test, not standard general cognitive test.
thaumasiotes 5 days ago [-]
The tests that carry the particular branding "IQ" (Wechsler / Raven's / etc.) suffer from some problems in this regard - not very many questions exist and there are very large coaching effects. (Also, psychologists will tell you that getting an accurate result means you need the test to be administered by a trained psychologist. This is mostly nonsense, but to the extent you believe them, it's cost-prohibitive.)
Hiring from a test that measures IQ is a very good idea (and there is a test that's commonly used for hiring purposes, the Wonderlic†); hiring from "an IQ test" is a bit less good. Anyone who wants to subvert the Raven's test will be able to do that. High-stakes tests need more security.
The concept of "IQ" can be toxic in contemporary American politics, so there are many more tests that "happen" to test IQ than there are tests that advertise themselves as testing IQ.
† https://psycnet.apa.org/record/1982-00123-001 : "correlations between Wonderlic IQs and WAIS Full Scale IQs were [0.93] for the main group and [0.91] for the cross-validation group". Note that this test involves only basic math and takes 12 minutes of the candidate's time.
ludicrousdispla 5 days ago [-]
The decisions made by individual interviewers are extremely accurate if you realize they are just saying 'YES - I want this person hired' or 'NO - I don't want this person hired'. It's entirely subjective but likely very repeatable.
thaumasiotes 5 days ago [-]
It's not repeatable; that's the whole point of describing how the same person gets wildly different results when they interview on multiple occasions.
ludicrousdispla 4 days ago [-]
Candidates are getting different outcomes on separate paths through the process because different people are interviewing them.
thaumasiotes 3 days ago [-]
That gives the process low reliability. That's all that matters. If every interviewer is perfectly internally consistent (and they aren't even close), that wouldn't do anything to help the hiring process unless the hiring process took advantage of it.
sameers 5 days ago [-]
I couldn't find the text of this joke, attributed to Dirac. I'll paraphrase.
A man walks into a pet store. There's a parrot for $100, it says this parrot speaks perfect English . The one next to it is $1,000, and says, This parrot speaks 12 languages fluently.
Then there's a bedraggled looking, droopy, parrot, and its label simply says One Million Dollars.
Does it sing opera and has successfully run for President? the man asks with a sneer.
This parrot, says the store owner, _thinks_.
That's what this entire post is about - how to evaluate people with a series of attributes, score, correlate, blah blah blah.
Hire them, see if they think. If they don't, fire them. It's cheaper than this credential/signal rigmarole, most of which is about CYA legal b+llsh1t. Yes, it's a simplistic strategy and it doesn't work for Shoogle, Banthropic, Goober, whatever. You know what, boo f*cking hoo. You're a trillion dollar company, suck it up. You have a zombie horde at your doors and you're just upset the "true gems" are hard for you to spot amongst the slavering masses. You're going to heartlessly lay them off anyway in a few years. You SHOULD feel this pain and anguish of having to sort through them, constantly regretting all your choices. That's the only way to have balance in the Universe.
roxolotl 5 days ago [-]
There’s an old Malcom Gladwell podcast episode, I think the show was Revisionist History, where he says he’s an interview nihilist. As long as the person seems reasonably capable, and can probably do a bit of what you need, hire them. Interviews are so hard to get right that what you’re saying ends up being most effective.
That sounds like a corollary of the Diamond Paradox. If there's any cost at all to price-shopping, then once you're at one store that sells the thing you want, it's usually not worth continuing to shop around.
Tangent: people do a lot of stressing out lately about which LLM they should be using for a certain task. I bet this advice applies there, too.
bitwize 5 days ago [-]
Unfortunately from an organizational perspective, a bad hire could cause so much damage through incompetence let alone malice, that making no hire the default unless they're a perfect cinnamon roll of a fit, is actually a good strategy.
laserlight 5 days ago [-]
> a bad hire could cause so much damage through incompetence let alone malice
The fact that an organization cannot deal with such a case is a bigger problem in the first place. Eliminating incompetence and malice is among the basic skills of an organization.
esafak 4 days ago [-]
Trying not to hire them is dealing with it. What are you suggesting?
laserlight 4 days ago [-]
What if a bad apple is hired despite all the checks? The system should be able to detect and eliminate bad apples before they give “so much damage” regardless of when they are hired.
Of course the organization should do its best to avoid bad hires. It should do so because of the opportunity cost of not hiring the right person, not because of the damage that they might give to the organization.
bitwize 3 days ago [-]
I was actually hired at my current job to replace a worker who was let go because pretty much from day one, she was acting suspiciously, consistently with someone who was outsourcing their work to contractors and splitting their paycheck with them. Posing a significant risk to the confidentiality of corporate IP and data, from day one, despite putting on a convincing game face throughout the interview and selection process.
In light of this kind of threat, strongly favoring no hire, and even policies like mandatory on-prem work, start making a whole lot of sense.
laserlight 3 days ago [-]
The case supports my argument though. Trying not to hire a bad candidate is not a resilient strategy. Someone might fall through the cracks. You still have to screen the candidate after the hire. Heck, even not hiring is not a good strategy, because someone inside may turn bad.
not_the_fda 5 days ago [-]
Our hiring dis-function is because there a lot of people that dislike conflict and firing someone.
anon_e-moose 5 days ago [-]
ie - people that are incompetent at their role of people managers.
Why is incompetence so discussed for ICs but rarely for management?
Maybe if more managers were competent and less conflict-averse they would do their jobs better and cycle out incompetence and bad intentions faster.
Ah wait, but middle managers specifically choose conflict-averse, easy to control, domesticated people. That's why you find this particular type of personality in management so often.
So the issue is incentives as well and bigger than initially thought.
cindyllm 5 days ago [-]
[dead]
zipy124 5 days ago [-]
It's funny that the post states:
"None of these band-aids really help — we still all hire tons of false positives (unqualified) and turn away false negatives (actually qualified), despite every attempt to make the process perfect, or even good"
The post touches on it (regulation about firing causing legal issues) but the unfortunate truth is that turning away false negatives has little cost to a business as long as you make sure to turn away false positives. Bad hires are extremely expensive, where as the lost revenue of turning down a good hire isn't nearly as bad (it's easy to try to hire them again later).
The stamp idea is good, reputation based hiring is important, as shown by the use of referrals in hiring. I've always wondered why we don't keep track of referrals inside companies in order to figure out who is recommending their peers who legitametly are good fits, vs those who are just trying to help out friends (who might not be the best hires).
Further I think there is also the problem that anti-corruption/anti-nepotism often harms hiring. It's much easier when working in a start up to hire the most talented people you know without other people sitting in, compared to at a big company where I know people who actively know good people they want to hire, but who can't be arsed with the long interview process and high chance of rejection.
If it wasn't for the regulation around firing some sort of staking your job on the line would be effective. You can refer someone and have them skip most of the process but your job is on the line if that was a bad call.
__mharrison__ 5 days ago [-]
I don't understand how a failing stamp for a campfire is good for the interviewee. It signals that they weren't good enough to get hired. Why would they want to parade that around?
zeafoamrun 5 days ago [-]
I had the same thought. "I didn't get hired at X! I didn't get hired at Y!" It's like how people write ex-Google, ex-Amazon in their headlines but worse: failed-OpenAI, couldn't-get-in-to-Anthropic. Let's all get around the campfire and sing kumbaya
gramie 4 days ago [-]
It used to be a thing in India, I have read, that people would want to associate themselves with a prestigious university, even if they dropped out. It would be written like this (made-up example):
B.A. Political Science, Bangalore University (failed)
JimmyBuckets 5 days ago [-]
You dont need the failing stamps to hang around. All you need are the successful stamps. It's actually better from a human psychology perspective anyway. Less shame.
Everyone will have some successful stamps, and they accrue over time giving better signal. Hiring managers can make a rational decision about what number of stamps over what period of time is sufficient to evaluate a person, trading against the risk of misreading a profile by accepting someone with fewer stamps.
ludicrousdispla 5 days ago [-]
Reading that part of the post leads me to conclude that Yegge is founding a startup called 'StampTown' or something similar as a talent incubator. Each candidate would have a digital avatar with suspenders that would display their accumulated stamps.
__mharrison__ 4 days ago [-]
I mean, if you are really into this stamp thing, then it is actually a good use of crypto/blockchain.
__mharrison__ 4 days ago [-]
Isn't your resume/LI your successful stamps?
frigg 11 hours ago [-]
You can't undertand because the argument is nonsensical.
foreigner 5 days ago [-]
How about instead of a contrived take-home project or doing unpaid "real work" for the company, we ask candidates to do n hours of work on an open source project?
That's real useful work, if we don't hire them they can show it off to a different hiring company, and it makes the world a better place.
IshKebab 4 days ago [-]
Not a bad idea but the problem with real work is that you can't make it fair. You can't give everyone the same bit of real work to do. Homework problems are fine. You can make them non-contrived quite easily.
The problem with homework problems now is AI. But I think it would be quite easy to detect if someone just fed it in to Claude when you go through the code with them.
overgard 4 days ago [-]
I'm skeptical of the idea that work can be consistently decomposed in such a way that an outsider can be effective immediately on anything other than toy problems. Saying "they can point their agent at it" is handwavy.. it helps but any mature codebase is still going to take weeks to get up to speed on, and there's going to be gaps that an agent can't fill because the knowledge isn't in the codebase. This seems like it would only work for mostly green-field projects that don't require a lot of expertise. Even things like "how do I use the product" can take a couple days to get up to speed on.
I also think this is pretty bad for candidates. The last thing I want to do is a bunch of unstable gig work for fickle startups in one of the most expensive places to live in the world. Also, I really think this only works if you decouple things like health insurance from employment, otherwise the instability can be too much for a lot of people.
martythemaniak 5 days ago [-]
The most talent dense place I've worked had a dead simple process - two one hour chats, one with your potential manager/team and one with the CTO or CEO. If things didn't work out, well you got sent away, probably happened twice per month. There was a particular meeting the CTO/CEO used and if you saw someone meeting with them there on a Friday afternoon, you would not see that person on Monday.
The place was not big - never got much beyond 100 engineers, but produced dozens of founders, VPs, GMs etc at well known companies, as well as engineers with very notable OS projects and lots of high-placed engineering in FAANG companies.
akurilin 5 days ago [-]
Curious what those two hours would entail!
majestik 4 days ago [-]
I see this kind of “let’s make candidates work for free” proposal from time to time.
It doesn’t work for software engineers because nobody wants to do free work. Also, free work isn’t consistently, and fairly testable because project requirements can evolve or completely change over a hiring period. Not to mention IP issues.
What has worked in my experience is a synthetic take home exercise that isn’t easily LLM solvable, the same one given to all candidates with the same constraints, and offering to be available to answer their questions by email. Sometimes we can tell a lot about the candidate just from the questions they ask, or how they package the solution, without even looking at the code.
In my experience (FANG hiring) less than 10% of candidates refuse to do this exercise, so it’s worked well for us.
seanmcdirmid 4 days ago [-]
The drawback of this approach is that candidates will judge the work they will do based on the exercise. I’ve gotten these exercises before and just noped out, determining from it alone that this isn’t going to be a place I want to work at. Or maybe that’s a good thing? As long as the exercise is representative of the work the candidate will be doing, it gives them a chance to decide early if it won’t work out.
I received one of these when I applied at twitter (pre-musk) and it was so poorly worded and organized that it went badly for me and the interviewer. But I see that as dodging a bullet (google was a much better fit for me).
jaggederest 4 days ago [-]
I prefer that sample work be longer, substantially comprehensive and serious work as you say, but that the candidate's time be compensated on completion with either a direct honorarium in a fixed amount (something modest enough it's not worth applying as a job), or a larger donation to charity of their choice. That realigns the candidate and employer experience by showing that the employer cares enough about the output to invest in it.
petters 4 days ago [-]
It also means you'll have to onboard a lot of new employees all the time. That sounds exhausting.
dnnddidiej 5 days ago [-]
Seems a bit detached from the real world.
If I am doing a 6 month contract which is what he is proposing then yeah I want a great day rate and to earn 12 months salary in those 6.
It basically means I invest in a company that doesn't believe in me and probably wont try to help me succeed.
Also mortgages, car loans, life and health insurance etc. are harder to get on a short contract.
Hiring is broken (maybe) but 6 month paid interviews are not the solution.
QuiEgo 4 days ago [-]
The only way provisional works is if your survival (food, health insurance) is not tied to having a job at all times. Once you're past 26 and can't be on your parent's insurance it's a total non-starter in the USA.
wffurr 4 days ago [-]
This is an underrated point. Healthcare especially is tied directly to employment in the US and basically kills this idea all by itself without major reform away from employer provided health care.
I think it might actually be a good thing for both employers and employees if health care, housing, and food access was secure enough to give people the freedom to take six month internships, but I have no idea how we get there from here.
analog31 5 days ago [-]
The paradox that strikes me is that "hiring is broken" yet these companies are beyond successful. So there's still yet another layer of something in between observing that employees are capable / incapable, and the company successful / unsuccessful.
spaqin 4 days ago [-]
A personal tragedy of failing an interview has little effect on a multinational conglomerate. If the average employee is half decent, they will grow anyway...
drooby 4 days ago [-]
Lots of humans. Lots of companies. Random chance.
Of course, its not that simple. Some companies probably are great at scouting. Yegge mentioned a few ways in the post. Good internship programs, acquihiring, etc.
jasongi 4 days ago [-]
I don't know if it's the same in the US, but every job in Australia generally comes with a 3-6 month probationary period where employment can be terminated for basically any (non-protected) reason with little notice. I've observed that the option to do so is seldom used - probably because there is not usually much incentives for managers to decrease their number of reports unless there's serious problems.
Most places still interview. Because hiring someone for 3-6 months is still an expense. For large companies, onboarding can take many weeks before you're even thinking about being productive. Interviews don't need to be 100% accurate. They need to be time-efficient and an ok filter to prevent hiring the worst people.
I don't see how a bad job market is going to make skilled people be willing to intern in order to get employment. Employment is a market, if demand goes down, then the reaction will be that price goes down.
The reason students are willing to intern is because the supply is so high and demand so low that you can effectively hire them for nothing (or next to nothing). The interns know that on completion the internship will upgrade them to a new category with new supply/demand. It's the same reason they are willing to pay large amounts for a university degree.
So interviewing will stay the same. If demand collapses, we will see wages drop, and I imagine the supply of software people will react through early retirement, career switching and reduction of people choosing it as a career before we see an upheaval of hiring methods.
bluecheese452 4 days ago [-]
That probation period never ends in the us. You can be fired at any time for nearly any reason.
marshray 4 days ago [-]
Or no reason.
wood_spirit 5 days ago [-]
The thing is the internship or code campfire or probation or whatever all need the hiree to stop doing what they were doing. It’s incredibly intrusive. Lots of the best candidates have jobs and families and aren’t going to give up their current job to try out for a company that’s not committing. The whole post is so biased to the hirer.
whstl 5 days ago [-]
From my POV, the main thing that's really broken with interviewing right now is the filtering process, even before candidates do a take-home test.
In the last few years I was the main tech interviewer for a 300-employee fintech.
For a specific position, one recruiter got around 150 applicants, selected 5 great ones, who did take-home tests and mild-tech interviews. Offers were made to most.
For the same role/salary, but from another queue, a second recruiter got around 900 applicants, cherry-picked about around 70 of them. Out of those, only 40 completed the 1h take-home test. Only 20 delivered it, only 10 implemented the requirements. Of the 10, all were unable to answer even basic questions.
This was concurrently, so it wasn't "affected by AI".
I didn't changed my methods and in fact I didn't even got close to asking hardball questions to the second group.
The second recruiter didn't get their contract renewed and left.
YeGoblynQueenne 3 days ago [-]
>> None of these band-aids really help — we still all hire tons of false positives (unqualified) and turn away false negatives (actually qualified), despite every attempt to make the process perfect, or even good.
Yeah and you know why? Because like everyone in the industry you make the mistake of trying to hire engineers at the level of expertise and with the knowledge and skills you think they need to do the job. Except, any engineer worth their salary (i.e. their salt, I guess) is going to grow into any job and learn new skills and new technologies that they need to do the job, so the person you really want to hire is someone who is not yet ready to do the job you want them to do but has the potential to learn it really, really well. And the engineers you really hire, if they already have the knowledge and skills that you want them to learn, won't learn them because they already know them, will be bored to hell with the role, and as a result do a sloppy job and move on quickly to another role that is more challenging. That is, if they like to be challenged. If they don't like to be challenged- congratulations: you hired yourself a boring engineer.
That's what's wrong with your hiring practice. And you have no way to fix it because you keep telling yourself you want to hire "talent" but what you really try to hire is "experience". If you hired for talent, you'd get both talent and experience, but if you hire for experience, you often end up getting neither.
Incidentally, my pet theory is that this explains why 90% of modern software is crud.
ascotan 4 days ago [-]
here's the uncomfortable truth. most software engineers are good enough to hire.
i've seen a few interview types in my time:
1. technical interviews run by nontechnical or junior engineers who can't judge technical talent. this typically produces sub-par hires. imagine you are made the director of UX design and you have to hire someone but know little about the industry. what type of hires are you going to get?
2. technical interviews that focus on CS skills. can you write a red-black tree in Java? can you write a bubble sort in haskell? The problems here is that this has bias towards new CS grads that dont have much industry experience but just took tests on stuff like this in their degree. google pioneered this style of interview and it's taken off as leetcode. the problem is that experienced devs dont typically write bubblesort algorithms. this type of interview is biased toward younger out of college hires and codecamp hires that have little experience but gring on leetcode sites.
3. technical interviews that have too many opinions for a hire recommendation. this seems to be a new trend where you need 3 or 5 or 7 thumbs up to get hired and any negatives sink the candidate. this does raise the bar, but typically what i see is someone on this panel has a high opinion of his/her abilities and gives everyone they seem a thumbs down.
What i've found is that soft skills and team dynamics are > the technical accumen. I can teach you how to write go(lang) but its much harder to teach you how to influence your peers or to communicate effectively.
Experienced devs probably dont know how to ace your leetcode interview, but they do know how to influence and communicate and when to stay away from bad ideas. You dont get tested on these things. It's analogous to a being combat veteran. You've been there, you've done that you know how to survive. You might not know the new tech, but that can be taught. I have never seen an dev not be able to pickup a new platform/language/skill within 6 months of hire - anywhere.
Here's hoping this will get better one day but i'm not holding by breath.
0xbadcafebee 4 days ago [-]
> There is a material difference between the signal from an internship (~7 weeks of usable work time after ramp-up) vs a co-op (5 months actually working)
And that's why the trades require you to perform years of work as an apprentice before you're ever qualified to pass an exam to become a journeyman. Not only is it like a years long internship, but you're paired with someone who has already passed a high bar. You're learning directly from vetted people, and those people can vouch for you.
> University of Waterloo famously sends their Computer Science students through a total of six internships, giving them roughly 2 years of real-world work experience before they graduate.
Still less time than an apprentice is required to work, but it's better than nothing. Most trades require you to be an apprentice for 4 to 5 years.
> The reason is, hiring engineers has historically been so competitive that you couldn’t convince a senior engineer to do an internship [...] So the industry converged on not requiring it.
Which is one reason the trades made it the law that you have to follow the apprenticeship in order to become a journeyman. "The contracting companies don't feel like hiring licensed workers" isn't an option. Companies don't do the right thing unless they're forced to.
This is why we need professional engineer certification for software engineers. We need a rigorous, time-tested, reliable process to ensure engineers have actually done the job in the right way. Otherwise it's a bad guessing game like Steve is explaining.
This is also why we need a software building code. We need to explicitly define exactly how you're supposed to engineer, so that we can create a certification that people can pass. Otherwise designing and building software is completely subjective. Engineering should not be subjective.
This is not some mysterious experimental idea that nobody knows if it'll work. Trades have been doing this for decades. It's not perfect, but it's much better than the alternative.
overgard 4 days ago [-]
I guess my concern there is that there's such a high degree of diversity to software jobs that figuring out what "qualified" means is also very diverse. For instance, what it takes to be a frontend web dev vs working on embedded systems vs working as an SRE are practically different professions.
0xbadcafebee 4 days ago [-]
The trades are all different professions, and within them are sub-professions. There are apprentice electricians, journeymen electricians, master electricians, electrical engineers, industrial electricians, commercial electricians, residential electricians, new construction electricians, maintenance electricians, US-specific electricians, UK-specific electricians, etc. And we standardized them all. What we didn't do is say "let's not standardize professional electricians because it might take some effort."
It doesn't have to be perfect right off the bat. Start by writing down how you do your job. Get some other people who do your job to review that. And get some more people to review the review. The reviewers note down where there's some gap, something you're not doing, some considerations for doing it, etc. All of it gets updated, consolidated. In a few months you have a document that standardizes your profession.
There are millions of people around the world that write software. Get different groups together to document their own specialties. By parallelizing, we can get a basic set of standards for most specialties in ~1 year.
And don't forget there are already thousands of books on how to do each specialty. It's not like all the knowledge is tribal. It's not really that complicated to explain how software and systems work.
ok_dad 4 days ago [-]
> Engineering should not be subjective.
I think there’s a lot of ways to do engineering right. It’s very subjective in many ways. Every project has multiple possible paths to success.
kristjansson 4 days ago [-]
Internship etc. proposals miss that status quo job seeking is a parallel process for applicants. An internship model means each candidate can only “consider” one employer at a time. I think this is what Steve’s campfire proposal is trying to counter by making samples/internship outputs public?
I think the real solution is something like the Bar Exam. Apply the hard filter once, publicly, administered by a trusted neutral party. Anyone that gets through is assumed to be good enough, and interviews can assess only firm/team specifics
alright2565 4 days ago [-]
This is what Triplebyte tried to do, but unfortunately they failed in the market. Maybe they were ahead of their time and a similar concept would work now.
kristjansson 4 days ago [-]
Yeah I wrote “by a third party” then thought of them and revised to “trusted neutral party”. Has to be someone with ~no financial interest in the candidates success
danieltanfh95 4 days ago [-]
The real solution is stop proposing solutions like a bar exam. Software engineering evolves too quickly for anything but a work sample to work.
kristjansson 3 days ago [-]
The Bar is also a not-great signal of actual ability. It is a great signal for 1) base knowledge and reasoning capacity and 2) sticktoitiveness / willingness to meet ~arbitrary expectations to work in a given industry ... which is all these 'technical' interviews are really measuring. By revealed preferences that's what management wants at those companies, so better to save people time and stress and make it a one time thing?
andrewstuart 5 days ago [-]
Steve Yegge is one of my very favorite authors I love his work. But lots of things to say here.
FIRST - is that before you get to campfire anyone, you have to have done some sort of interview process to boil it down to the one person to do the campfire - so how does that work eh?
SECOND - campfire is deeply invasive to the candidate's life and time.
THIRD - you have to pretty damn sure someone is a hire before doing a campfire. You CANNOT do campfire as an evaluation step, after which there are more interviews.
FOURTH - this is effectively just a really really long version of the take home work test which is absolute bullshit.
FIFTH - there's STILL no science to the campfire. Don't give anyone a fucking test if you don't actually know how to scientifically evaluate the results. And campfire does NOT result in a scientific outcome, it still results in an arbitrary opinion.
SIXTH - any company that wants me to do a campfire - to commit days or even weeks of work as part of them trying to decide to offer me a job - can fuck off. Sorry, the party got spoiled by all the other companies who asked me to do something as part of the interview process and then either ghosted me or gave me some bullshit outcome like "they didn't like your work".
I can tell you how to recruit people and it does not require campfire.
You TALK to people about software development - you engage them in extended conversation about what they have done, what they know, what their interests are, what they have built, what projects they worked on, what went right, what went wrong. You look for people who have BUILT STUFF - this was true before AI and is 100X more true now - anyone who has not built anything today is not worth employing and anyone who has built something must be able to talk about it in depth. This interview processes worked before AI and it works after AI. And finally, you accept the limitations of recruiting which is that people are people and you won't find out how well someone performs until they have been on the job six months - live with it.
Sorry Steve - I love your work but I'd never work for any company that wanted me to do a stupid campfire because they don't know how to actually work out if I can do the job or not.
rhines 5 days ago [-]
I think you're largely right - but there are some challenges.
Talking requires that the interviewer be competent and care. Which seems like a low bar, but it easily degrades. When an engineer is burned out, or is interviewing for a project they have less stake in, or has business requirements forcing them to hire someone fast, the bar lowers. Those engineers now are the gatekeepers, and they let in even weaker engineers.
Even worse, any utility from this is predicated on the founder making good hiring decisions. If they don't understand how to hire a competent engineer and make them care about the company, then it's dead from the start. And a lot of founders or execs in general are awful at this.
Companies that maintain an excellent work culture such that engineers deeply care about the team and have stake in any project they conduct interviews for will do well with this model. I hope we see it used by them.
em-bee 5 days ago [-]
You CANNOT do campfire as an evaluation step
why not? to expensive? candidates don't have the time? i'd love to do that because it also gives me insights into the company. how they work, etc...
I can tell you how to recruit people
why is interviewing broken then?
You TALK to people about software development
didn't we try that and find out that the sweet-talkers master this and are able to fool everyone? or didn't we find that this leaves out good people who don't do well when questioned like that?
btw: my personal preference for a tech interview is 1 hour of pair programming. (not just live coding)
woeirua 2 days ago [-]
How are you going to do a campfire while gainfully employed? Job hopping becomes way harder in this world.
em-bee 2 days ago [-]
take a few days off. ok, in the US with only two weeks yearly vacation that is a bit harder. so this mostly works for people who are already out of work. but i don't think that's a downside.
these days there are so many people being laid off that you still get a large selection of potentially good candidates, also, for someone who has been out of work for some time i would like to test them more intensively. in fact from a social standpoint i should optimize hiring for people who don't already have a job.
mrwaffle 5 days ago [-]
This sandwich is huge but the bread layers (top and bottom) are moldy, love the empathetic writing style though. Agree on your TALK paragraph - mostly - as well. Make sure to consider juniors in that lumping too though, we can't have systemic failure over time. AI-outcomes are not guaranteed.
There is a way to make the campfire approach a little bit less utopian: California could pass a law that makes it legal to have a second temporary job. So that FAANG engineers for instance, would be allowed to campfire/interview at another company with legal protections. Put a bit of grease in the interview system is good for candidates.
cowpig 5 days ago [-]
A lot in here about Google, which hilariously has done a bunch of studies that concluded it's hiring process is awful (is sure is!) and the takeaway is that candidate evaluation is impossible, not that Google in particular does it badly.
HarHarVeryFunny 4 days ago [-]
Lots of replies here pointing out that that any kind of "provisional employment" wouldn't work for the majority of candidates who are currently working.
From the candidate's POV what is mostly broken about the application process is all of the ATS gaming, resume tweaking, etc, just to avoid getting filtered out before an interview, as well as the inane leetcode screening that some companies are doing.
A better process might be to replace all initial profile/resume-based screening with task-based evaluation (evaluation could be at least semi-automated - if a computer/AI is going to reject me, I'd prefer it to be based on task evaluation job skills rather than ATS-filter avoidance skills!).
Lengthier on-site interviews/evaluations could also be task-based - for a developer role perhaps a 2-4hr peer-programming or problem solving task. Far less signal that a 3 month provisional hire of course, but maybe a better use of everyone's time than a traditional talk and brief whiteboard challenge process which is clearly failing as a useful filter.
I wouldn't expect companies to forgo the traditional touchy-feely team/culture fit type of screening as well, but better if this came after they'd already determined, as best they can, whether you've got the chops to actually do the job well.
Maro 5 days ago [-]
Maybe I'm an oddball, but I've always thought this: if a cool company wants to hire me but isn't sure, give me a 3-6 month fixed contract and LFG. There's zero doubt in my mind it would work out. And if not, so what, I'll be okay.
Today I'm 45 with family and have a fancy VP title, but I would have no problem to do this for an interesting role at a cool company.
socratic_weeb 5 days ago [-]
The industry should get down from its egocentric ivory tower and start hiring like other industries. You are not special, tech. Just have a sort of bar exam you have to take every x amount of years. The actual interview should just be an in-person behavioral one, with your future boss. Period. Don't get enough signal from that? Sorry, that's life, taking risks. Imagine trying to open a business without taking risks, and only kickstart it once you are a 100% sure it will work. No other industry is as delusional. Here is a reality pill: what you are asking, a 100% safe investment, is impossible, as any economist will tell you. "Campfire", give me a break, grown up adults with families don't have time for that Silicon Valley wankery, and you have no evidence that the results will be as good and unbiased as you think they'll be...thanks for bringing attention to the subject though, and in passsing also confirming how garbage the Google (and FAANG, in general) hiring process really is.
Just one more thing: the industry should really put a bit more weight on measuring people's potential and the concept of long-term growing and learning on the job. Just saying. You know, like every other industry on this planet.
danieltanfh95 4 days ago [-]
This is why we need interviews to filter off people who are bad enough to think "bar exam" is the right abstraction for software engineer.
3 days ago [-]
aryehof 4 days ago [-]
The solution would seem obvious - decide what you want, then hire the first applicant that qualifies.
Of course, the real issue is that a prospective employer doesn’t know what they want. Our “engineering” industry runs on vibes, heroic effort, fashion and hype-cycles. Someone very smart, enthusiastic, quick learning, flexible and affable is the only real job description.
Combine this with the “supply” side entirely overwhelming demand, having to compete with AI, and experience having no value, and for most software developers the future is bleak until a new more equitable equilibrium is found.
Nevertheless in the meantime, at least one can have solace in collecting the authors “failure stamps”.
mrwaffle 5 days ago [-]
Seconded, stop the theatrics and gatekeeping and let's keep a growth mindset while training / retraining those who 'pass the buck' overtime. At least everyone can get skills and talented outliers will find themselves with more structure and collectively we'll produce better outcomes for more engineers at multiple levels of experience.
akurilin 5 days ago [-]
> Another reason is that on the supply side, nobody wants to sign up to do a bunch of free work just to be rejected. If you just put up work, the candidates incur all the risk, meaning they walk away with nothing if you don’t hire them.
It's true, but prepping for a typical senior+ onsite loop in big tech still requires weeks of grinding leetcode, re-learning the latest system interview questions and the system interview answer framework, refreshing and rehearsing STAR stories, studying the company and its unique quirks that you're expected to know to pass the culture filter, remembering how to do all of this speedrun-style since you only get 40ish minutes per session, etc.
While that knowledge is more reusable across onsites, it's likely even more work than doing real or pretend-work for the company for a couple of days.
> When candidates get to walk away with something of lasting value that they can keep forever
I'm curious why them getting rejected from the position, even with the work sample they can carry away with them, wouldn't be still interpreted as a negative from future employers. "The other co passed on them, am I the fool for thinking they're good?" type of herd mentality which is often unavoidable.
Won't that "work sample guest book" be treated as the list of all companies that rejected you, a net negative for your personal brand you're projecting?
> (Me paraphrasing what Steve was implying) Take-homes are impacted by AI one-shotting them for candidates
I've been pleasantly surprised by how much you can glean from having the candidate upload their conversation log with the coding agent for whatever take-home you give them.
zeafoamrun 5 days ago [-]
I was going to scoff at the amount of interview prep you said is required but then I remembered I read the book written by the founders of my current company before the interview.
eikenberry 5 days ago [-]
His idea has some merit but will require the old system to completely crash out before anything new will be considered and I'm not sure if it will crash or just keep limping along. If it really does crash out hopefully we will see multiple new strategies emerge as there are many possible options once the current one is off the table.
zffr 5 days ago [-]
As yegge mentioned, there might be more appetite for trying out this idea now because there are many engineers who are currently unemployed. Offering them short co-op could be beneficial to both the engineer and prospective employer
piokoch 4 days ago [-]
Well, campfire method sounds easy in the blog post only.
You need an NDA from a candidate (many would not like to risk - what if someone gets job on the competing company and is sued for revealing some secrets from the campfire interview process).
In many cases interviewee would need to get a laptop from the company, as there are specific requirements about data security (disc encryption, usage of solutions like Fortinet or Zscaler), be added to company SAP to get access to the resources (Office, Teams, etc.), company need to purchase licenses for Office, Github, Jira, etc.
Surely hiring is hard, surely there are false positives and false negatives, but fixing this requires hell a lot of resources and organizational changes and costs.
airocker 4 days ago [-]
We have been working on https://talentpulsar.ai for exactly this. Hope to find some good collaborations with the hn community with some little self promotion, hope this is okay.
siliconc0w 4 days ago [-]
I like take home projects the candidates then need to present and answer questions about. LLMs just mean you can be more ambitious here (though you should pay for their tokens).
The problem with provisional employment is that it can take quite awhile for a new hire to be productive. In a complex FAANG environment I'd wager it's about six months before they're not a net drag on the team and maybe 1-2 years until they're close to fully productive. These are complex not just technologically but organizationally and there are tons of hidden rules and micro-decisions that are the difference between a project stalling vs moving forward.
stephbook 4 days ago [-]
This just generally over-estimates whether you can "measure" good work on scale, and make your org produce more of it. Google didn't manage to measure and predict its own employees' work [0]. Even with 6 months probation – the default in Germany – it's just guess work.
If you want good employees, poach those that others are paying millions already.
1. For juniors, any sort of proof you have passion is enough.
2. Treat mid-levels as seniors.
3. Seniors have to show proof of passion (with longevity and intensity, AI and consultants exist for expertise and menial work) and competence (Take home problems with a long time scale with whatever tools they need. Use AI. I don't care, but be expected to be scrutinized for your design decisions, depth of exploration, and architectural write-ups.)
lz400 4 days ago [-]
I don't think this works, and other comments have also pointed out the same. You open a position, you receive 200 CVs. Ok, then what, get them all to come in for a couple days work? A company that needs only one position can't possible handle that.
Now you're amazon/google. You have 200 open positions in a certain site/country. You receive 10000 CVs. Same problem, different scale.
So ok, you need to filter CVs, mmm, which sucks, right, so perhaps we do a screen interview? mmm, low signal, maybe....
CuriouslyC 4 days ago [-]
Even smaller and niche companies are getting 700-1000 CVs now for an open position. Every company I've talked to says hiring is completely broken.
greatgib 4 days ago [-]
The guy looks like to be a kind of cancer of all that is going back with hiring in FAANG.
He is indirectly responsible or at least part of everything that was terrible in most of the big techs he was part off. By his own admission, each time things were evaluated, the scientific conclusion was that it was "horse shit" and totally randomly hiring results in the end despite having candidates go through an awful process. And now he is full of ideas about what kind of new nightmare we can create to reach the same stupid result...
What I see in his post, and that I find despicable is that it looks like that he has this "superiority" syndrom making him consider candidates and employees as disposable resources.
Instead of considering the process as a "mutual" process, where it costs and the investment on joining the company is also from the candidate side, he is considering that just the employee is a liability for the corp and that the candidate should submit fully to it. Even wasting weeks, month or years without guarantee of stability or not wasting his time.
m3kw9 4 days ago [-]
Most interviews are just referrals then confirming the bias, even if you fail some questions, you are likely in. If you came from no where, they are not gonna like you unless you get all the questions right.
If you want to know if someone is good at your company in 3-4 interviews, it’s tough, the best they can do is ask these technical questions. Talk to you about your past work, ask you technical what ifs. Most dumb ass companies will ask you to do trick coding leet code crap.
autaut 4 days ago [-]
I’ve seen this before. The result is a revolving door of temporary work. Because soon companies will be “you already put 6 months in, we don’t have enough signal work 6 more months” and at the year mark they swap you for a new candidate.
And before you say that this is inefficient consider that despite being terrible for morale and efficiency (proven in un’etica studies) companies still maintain the bottom 10% out or up or out policies.
Companies always love their power on their employee over efficiency.
esafak 4 days ago [-]
The proposal does not make sense. Candidates do not benefit from accumulating and parading rejections until they reach a company that accepts them. If they have offers, they can and already do tell recruiters. Furthermore, this practice of spending days at a company is expensive in opportunity cost; you will have to burn your vacation time, and be able to interview at fewer companies. In sum, the proposal benefits companies, not candidates.
justincormack 4 days ago [-]
This paper (from the same research as his book on high end culinary organizations) is worth a read https://journals.sagepub.com/doi/10.1177/0001839214557638 - he talks about "negotiated joining" which is a similar scheme for placements that works well with undefined job roles.
SOLAR_FIELDS 5 days ago [-]
> One day, the recruiters gave us a special round of packets to review. In these special packets, we were able to read the interviewer notes and candidate responses. All personal details were stripped out, and we were told it was a “calibration exercise.” We had to do our regular voting job with these special packets, and see how it went. I think we may have assumed they were from another site, since cross-site calibration was common.
Our group did our job, and voted not to hire about 2/3 of the packets. This was about par for the course.
But surprise surprise, this time, those were our own packets from when we had all interviewed at Google. The recruiters had tricked us into reviewing our own interview packets, and we had voted not to hire most of our own group.
For that brief moment, we all had a glimpse into how utterly broken our process was. The people-team had rubbed our noses in it.
Or maybe the company changed in the 10 years or so since everyone in that room was hired and the employee needed 10 years ago is not the same as the ones needed now?
sethammons 4 days ago [-]
The idea is that if you are already in and doing a good job but can't pass the interview, then the interview is bad. Specialists hiring is a bit different if no specialist are on the team yet.
ReptileMan 4 days ago [-]
Does anyone knows what happened from his loudly proclaimed stint at Grab? He went there with huge expectations couple of years ago.
commandersaki 5 days ago [-]
Everyone is annoyed when Google or whatever has a gruelling high bar to get in, but nobody bats a lid when say Jane Street does it. Both wildly successful.
choppaface 5 days ago [-]
Because Jane Street is unabashed about greed, and yet they do not let that greed lead them into attempting to pollute and derive us of our attention every day.
citizenpaul 5 days ago [-]
[dead]
osigurdson 5 days ago [-]
Maybe three days is enough for some code bases, but if you have millions of lines of code, agents aren't going to help you that much.
i_am_a_peasant 4 days ago [-]
you don’t need to understand millions of lines of code to be effective at a job. you just need to identify the boundaries and the subsystems you need to touch. I’ve always only worked on huge codebases, and i tend to ramp up within the first 2 weeks at every job
randomdev123 5 days ago [-]
I think interview is part of tech life where I feel very, very bitter at.
Warning, sad boi rant ahead.
I was from a programming bootcamp. I entered the tech because I was the only person in my bigger family members who have the skills/education/chance to earn a big salary. I still remember, I paid $12k back then, and it was me and my mom's only savings. I started tech in my 30s, and mostly worked physical labor work before that.
I was the weakest at that cohort, but I studied really, really hard, until the bootcamp noticed my progress and hired me, albeit at a very low salary. That was my first programming job.
I eventually learned about big tech, and liked the fact that they didn't care about credentials, whether I came from a good university or not, unlike the YC combinator startups who mostly cared about credentials. I learned that they pay really well, and I did, read Steve's blog, "Get that job at Google". It motivated me to study DS&A, and also to get CompSci degree.
Overtime, I lost count, how many interviews I have failed at big tech. I gained more experience, I became older, but I kept studying, and studying, and studying, but I keep failing. I also wasn't sure which area to study, so I ended up studying for everything, from frontend to backend. It was a lot, really, a lot of things to study, from leetcode to JS specific to DOM to backend system design to frontend system design, to behavioral, and frontend interviews back then was still a crapshoot, some companies ask for deep JS/DOM questions, some companies just ask for leetcode questions.
But I kept failing.
I think I'm pretty good, or at least, that's what I thought. I learned quickly, I have no problems for clearing mid level leetcode questions in under 20 mins. Overtime, the bar gets really hard and it became hard level leetcode questions in under 20 mins.
But what made me really bitter over all of these interviews, was that I saw my friends, people I knew, from programming bootcamp, from my CompSci degree, got a job at big tech. I knew, or at least, what's what I thought, that they had less skills than me in programming and in DS&A. But they got into big tech. Some of them were minorities, and during that time, diversity hiring was a thing, and maybe that's why they got in, I thought to myself.
Sometimes I asked them what the interview questions were, and to my surprise, it was easier than my questions. Idk why.
Sometimes I wonder if my luck in interviewing is really bad. I tried everything I could. I bought courses and devour those materials. It's been years and years and years and years. It did wear down on me. I want to cry, which I did sometimes due to keep failing big tech interviews. But it won't do me anything, I can only keep my head down and keep trying.
I ended up making pretty good money in this field, and able to help my family members. I did work with some of those big tech engineers. I realized that those big tech engineers were just average, and I don't think I am less capable than them. That made it sucked even more, because I don't understand why I kept failing, and why the people I know are succeeding.
I do some freelance now, but will finish some of my contract. The job market is scary, and one of my contract will finish soon. I am having hard time getting even recruiters to contact me, maybe because I don't have big tech credentials.
I also did mostly fullstack/frontend leaning lately so I ended up studying frontend interviews. But frontend job market is kinda dying lately, since most companies don't really respect frontend and don't think its worth it anymore. These days, I don't even know what to study anymore, so I need to study everything again, with the addition of all the AI stuffs.
I can't help but to think, and always think even today, especially during this tough economic times, what if I was able to get the big tech job. My family would be proud of me. I would have better companies in my resume, and could stand out more in the job market. I could've done some cool highly scalable projects that I can boast of in my resume. I could've joined some good team, mentored by good engineers. I would have more savings than now.
These days it doesn't matter anymore. The big tech salary era already closed. Things are way way more competitive than before. With my average background, I don't stand even a chance at being asked to interview.
Life is really a mystery. Things that you really need or want, you don't get it no matter how hard you try. But it is wasted away on those who got it.
I usually don't post comments, but seeing this post, I just needed to vent.
Thanks for reading.
SoftTalker 5 days ago [-]
Honestly, it was probably just your age. Being in your 30s, an entry-level coding job at Google or other big tech company just isn't going to happen. Those companies all have a lot of ageism especially in the coding ranks.
randomdev123 5 days ago [-]
Not sure why HN doesn't display my edited post. So I'll just reply here.
Thanks for this article Steve. I need to read this. Maybe it will finally give me closure, after years of failing.
horns4lyfe 5 days ago [-]
The current interview process isn’t meant to get the best talent, its intention is to give managers on visas an easy way to bring in other visa workers (via sharing questions and scaling difficulty for no-visa applicants)
IshKebab 4 days ago [-]
> Bringing in outside contributors is pretty easy, as they’ll have agents and should be able to come up to speed very quickly. If they can’t, there’s your signal.
Sorry what.
da_grift_shift 4 days ago [-]
What's with the fox performing phrenology on a sheep in the generated image captioned "The moment we realized it was us"?
QuiEgo 4 days ago [-]
The strongest signal I've seen is recommendations from people who have worked with a candidate before.
marshray 5 days ago [-]
This reads as if he's really struggling not to say "Now that the applicants are desperate we can begin to interview them properly mwahahahaaaa!"
BoneShard 5 days ago [-]
I came to conclusion that a lot of these interviews just nerds revenge lol (20+ yoe in faangs).
poppafuze 2 days ago [-]
when you glaze yourself that hard for that long, those are exactly the kinds of ideas you'll come up with.
bg24 5 days ago [-]
Very well thought and written. Provisional employee or intern. Or having the candidate to come and do real work for a couple of days. The challenge imo is the big company culture vs startups. Do the things move at the pace in big companies where the teams have the ability to evaluate? Startups are a different beast however.
rbbydotdev 4 days ago [-]
Steve Yegge is the creator of gastown, a multi-agent workspace manager. It's an open source 21st century master piece of ai psychosis, slop, and most especially redundant processes, orchestration and code (MILLIONS OF LINES) I'm finding no surprise and even laugh-out-loud hilarious, that the author of such abomination is also the progenitor of the equally inane and psychotic hiring process of FAANGs like google and amazon.
manuelz 4 days ago [-]
Mucho respect to the OP who was part of such elite fellowships as the "Bar Raisers" and "Hiring Committees" (and I'm sure he would totally have made it into the prestigious "As Appropriate" group at Micro$oft if he only applied for it).
My record is comparatively humble: I hired around 300 people in tech for small-to-medium size companies, in the small tech backwater of Vancouver, Canada.
My conclusions:
(1) Interviewing is an intractable problem. Start by recognizing that.
- You don't interview the best candidates, but the ones with the best resume.
- You don't hire the best candidates, but the ones that do best at interviews.
- Screening by HR (phone or zoom) is at best useless.
- Timed coding assignments are a waste of time. They are used because they're cheap + provide a [generally wrong] "quantitative signal". Noone's job will consist of solving 8 "leetcode" riddles/day.
(2) "Technology fit" is a dangerous illusion:
- It is very, very, very unlikely that any candidate will be able to pick up right away your tech environment. (ok; exception: you're hiring permanent an existing community contributor for your open source project)
- Your best new hires will be 0% productive in their first month (negative; training will use resources); 15% in their second; 50% in their third.
- Rejecting e.g. "Java" when screening for "C-sharp" is stupid.
(3) The interview process is about building relationships.
- People you don't hire will remember your company from the interview + disseminate.
- Someone you didn't hire today (one of your top rejections) may be super-attractive 2 months later, or next week if your top candidate accepted another position.
- Ownership of the process or at least buy-in from the team (vs. just the hiring manager + opaque "corporate committees") is the first step in a working relationship. Your "superstar" may end up being toxic in the team and you could find that out in the interview.
(4) Five simple rules:
- Treat phone/zoom screening like an advertisement for your company. Ideally do it yourself (hiring manager). Largely ignore feedback from HR :)
- Hire candidates who are: smart + hungry. Programming languages; frameworks; environments are secondary.
- Try to get a sense of the fit with the other humans in the team they'll be working with.
- Take "3-month probation" seriously. Explain it to the candidate + team. Sell it internally. Candidate compensation for a botched probation is reasonable + just money, after all.
- Treat candidates as humans: Send a personalized rejection (from you, the hiring manager _not HR_) to everyone who made it to the interview. Call or zoom everyone who made it to the final round. If you can, provide them actionable feedback on ways to improve their interview process. Leave a "human" door open.
MagicMoonlight 5 days ago [-]
It’s interesting, this guy has a Wikipedia page but no achievements on it. It seems like his whole career has just been moving around and bullshitting about how great he is. I kind of expected to actually see an achievement on there, like inventing a programming language or developing something important.
lanstin 4 days ago [-]
I normally get jobs thru people I have worked with already. It reduces everyone’s risk quite a lot. Obviously I had to bootstrap, and that was in the far easier hiring time of the 1990s when all the people who wanted to be rich weren’t trying to do programming. I still interview and have people look at my public git hub and so on, but the context is much more “do you understand the problems we are trying to solve and do you have ideas or experience that relate” than “can you do this programming puzzle” (which no, I can’t, my particular forte in software is identifying where systems or solutions almost line up and a little bit or torqueing with a glue layer or two will enable this awesome work to be used in this novel problem space; even larger green fields is just taking open source libraries and frameworks and pipelines and combining them into a useful thing designed for exactly the use case at hand, which will never not be a far more useful product than some big generic product, quicker to change to meet the customers shifting sense of delight and more fun.)
neilv 4 days ago [-]
> Mostly I just want you to come around on the diagnosis, and agree that we need to fix it properly for the first time in 50 years. [...] At Google, rather than putting a little black desk on every interview loop, they just assumed all the interviewers had their heads up their arses (an evidence-based assumption), [...] People are not allowed to question whether the interview process is valid. Challenging it is akin to casting aspersions on the entire engineering staff. First of all, how dare you?
A scoping/guidance suggestion to start: Is the problem of improving hiring more tractable if you don't try to solve it for Google?
What if Google is hopelessly tainted with biases, from having run almost every single person there through a process that you acknowledge is a horrific load of poo. Which not only means the process has become a religion/frathouse that people are proud to have been accepted into, and will defend fiercely, but also greatly determines the strengths and weaknesses you have to work with, if you want to get them to do anything else.
What if you instead start with smaller companies that don't have the baggage of FAANG, and look at their requirements, which may include some mix of (off the top of my head): prior experience, learns whatever is needed, creative/innovative, works as a team member, aligned with company success, professionalism, good for morale, good for catalyst, will have some loyalty during the harder stretches or when tempted by a little more money.
It would be nice if, instead of many 2-person startups making their first hire with the assumption that they should mimic the known-bad processes of megacorps (often based on their own FAANG-oriented interview prep in college), the megacorps were desperately trying to figure out how to hire as smartly and genuinely as SMBs, but at FAANG scale and with all the FAANG legacy workforce biases clawing them down.
wiseowise 5 days ago [-]
Man do those slop images look ugly. I’d take child scribbles over the garbage picture in the article.
Mon0t0n 5 days ago [-]
kitchen confidential? in your dreams, nerd.
drdrey 5 days ago [-]
> And it can pay for itself twice: once in real work shipped, and once again in something else you could probably use more of, which is gravity.
what?
alfiedotwtf 4 days ago [-]
This article shows that Silicon Valley’s lust for the technical interview has been nothing but industrialised hazing, gate keeping, and pseudoscience.
neilv 4 days ago [-]
> And then, importantly, each work item also counts once for the candidate: they walk away with a permanent, portable record of what they did and how well they did it, signed by you, whether or not you make an offer.
Why would this be useful signal?
I'll assert that you're in a professional environment of: dishonest posturing and oversized egos in startups, bureaucracy and butt-covering in established companies, and a culture of thin niceness veneer.
I'd think all of that would be barriers to a useful Permanent Record.
> This system is basically worth exactly what your honesty is worth. If you hand out gold stars to everyone, you’re just reinventing LinkedIn endorsements, which are worthless. Companies whose stamps mean anything will be the ones known for only handing them out for quality work. Being a hard, fair judge is an advantage for everyone.
What is the precedent of that working, in the environment I just characterized?
Given our current environment -- which isn't going to change easily; it works for the people who have most of the money now -- seems like a not very reliable new category of surveillance capitalism. But it gets worse... A lot of techbro founders will immediately see this as the next sketchy, hated tech company, which won't only charge companies a fee to participate, but the real business will be twisting it into off-label uses. For example, with whom and when a person has interviewed, and who which companies are interviewing, is valuable data to sell, no matter how useless the feedback. They could also see how close you could get it to a protection racket, by selling Pro individual memberships on the side.
So we get bad signal for the ostensible service, in exchange for additional techbro dystopia.
w10-1 4 days ago [-]
I like the evidence of how broken hiring is, but the treatment is worse than the disease, and the analysis is flawed.
The treatment: there's a long failed history of using contractors and hiring the best, which is a recipe for abuse and PTSD.
The wisdom now is that one should prioritize avoiding false positives, but the observed facts are that many people who later do great interviewed poorly.
For false positives, there's the legal risk (hard to fire), but the greater risk is the damage to the organization before they are fired.
The premise is that people are great or not at interview time, and the process just needs to be accurate.
But for both false positive and false negatives, the reality is that how people work out does not vary primarily with their skills (probably because all candidates have enough baseline skills) but is a function of both them and their work+environment - not f(emp) but f(emp, work, others).
For f(*, work, others) there are 2 troubling issues: 1) for underperforming groups, there's a massive incentive to blame someone, and blaming the new hire is the solution that doesn't disturb existing relations; and 2) even effective combinations f(emp, work, others) might not be able to perform as needed, so performance is only a qualified measure of effectiveness.
So the more interesting fact is when low-score hires end up doing well, which is signal for a high-functioning team being able to get the best out of people. And when emp goes on to other contexts successfully, the effect could be persistent (and they could be bringing it transitively to other groups).
Yes, companies need to hire good people. But sometimes it's more important for companies to make good people of those they hire.
I suspect the poster child for selection is Amazon, and for making the best of what you have is Apple. Amazon is great at pursuing new opportunities relentlessly, while Apple is good at managing a very complex supply chain and delivering consistently.
So on the "bundle" theory of companies (from recent discussions about Japanese company diversification), you'd want to optimize hiring to company capabilities and orientation.
If you make "accuracy" at interview time the standard and optimize against false positives, you're almost deliberately hiding from the details that determine what people will grow and contribute the most. (Here's where prior relationships, school networks have a positive effect that pragmatists accept (notwithstanding the social bias).)
Almost any work can be as formative as prior experience and education. I think that depends most on mutual consideration and a growth perspective, and you can interview for that. Like investments, you're looking for growth potential, not past performance.
Ironically, successful teams that make the most of their people don't need to hire so much, so you'll have less signal.
That answers why interviewing can not only be broken but stay broken: you're studying failures and optimizing for anti-failures, but there are a million ways to fail and a billion incentives to do so.
Better to deeply understand and replicate the few successes.
SadErn 4 days ago [-]
[flagged]
pslab 5 days ago [-]
[flagged]
osigurdson 5 days ago [-]
[dead]
dude250711 4 days ago [-]
[flagged]
Ancalagon 5 days ago [-]
[flagged]
throwaway_42353 5 days ago [-]
[flagged]
tomhow 5 days ago [-]
Please don't register alt accounts for the purpose of idealizing violence.
dpark 5 days ago [-]
So brave to create a new account to claim that Google employees should be executed. So brave.
- Steve's company got acquihired by Amazon, granting him a free ticket in without all the torment of the multi-stage interview pipeline.
- It's a well known fact that it's easier to jump from one FAANG to another, so while interviewing at Google he had significant advantage, plus the blog gaining popularity.
- All of this has caused a deep down imposter syndrome, which resulted in an attempt to "improve" an interview process from the inside - but the wings were clipped pretty quickly by the corporate politics. It turns out that lawyers are not planning to reinvent anything there and hence are somewhat more important than engineers.
- The post itself is an self-applause over essentially a failed effort. "I've tried"
I know this author’s name but not his whole life story, but it reads like he got hired at Amazon before they acquired Geoworks and brought all of his former colleagues over.
And even if you're interpretation is correct, you sound pretty rude. You're really gonna mock someone for not liking the interview process and trying to make it better?
That's some solid gaslighting right there! You're right Steve, I would rather be rejected by your team than wasting my time by doing something positive with my career.
"People would be pleased to have a rejection from us. They'd be proud to carry it sounds with them. Lucky them!"
It's funny, I see an article from Yegge and thought "I like that writer, I haven't read any of his stuff in a while, I'll see what he has to say." Then got to the end and see the links to gas town and gas city and remembered it was the same Yegge that while having accurate foresight about orchestration of agents also was a bit off the deep end in gas town.
But the biggest thing I see in this article is it really sounds like "here is the new company I landed at, and rather than make a post about its product, I'm going instead make a post about how terrible the problem it solves really is, and a post on a proposed solution. And the cues what I'll pop up in a few weeks and just coincidentally post about this new company that just happens to solve this problem in the way I've convinced everyone is the right solution."
While I don't have any evidence of this that's the feeling I left with. And if so, then "thought leaders" are a lot more interesting when not "talking their book."
So, he actually did go through the multi-stage interview pipeline.
Just because you can’t read well, you don’t have to be so cynical.
As for imposter syndrome: Yegge graduated from high school at 14 and studied nuclear reactor engineering with the US Navy in addition to his roles at prominent Silicon Valley companies. He is not my pick for somebody with imposter syndrome.
I am a fan of his because I've been reading his blog since the Stevie's Drunken Blog Rants days, and he was quite active in the Emacs community for some time. I also worked at Amazon and so his experience there became particularly interesting to me during that time.
However, a few companies do have a co-working day, where you work with various people on the team to solve a business problem. I wonder if he could have proposed that instead.
Ah of course, we have to do a traditional interview loop to evaluate 10 candidates before we can pick one. So you do the traditional interview loop, and then you have 6 months of provisional employment.
You haven't replaced anything, you've just added another level of hassle for everyone.
I'll talk to recruiters and interview at other companies on my spare time, as before.
Good for you to make all the hay when the sun shone? Fuck the future generation, let’s make it even harder for them with these insane interview ideas.
You don’t run it for everyone, just people who seem good enough and are willing to do it. A resume filter and a 30-minute conversation are usually enough.
You stop as soon as you’ve got your hire. If you’re bad at picking people for the screen, you learn the signals quickly because of your level of investment.
It does replace a longer traditional interview loop. You hardly talk to them up front. You tell the candidate you will extend an offer right after the work phase.
Another objection is not everyone can afford to spend that time with you. That’s true. We would pay them, and still got some refusals. You have to accept that every interview style works better for some candidates than others. You’ll also find candidates who love it.
Two things it solves: You get to evaluate me, my ability to deliver, and how I interact with your team and I either bring real value within two weeks, or I don't. I can tell you verbally I am an indispensable asset or I can show you; other people have ruined the verbal trust layer which is why this whole debacle exists in the first place btw.
And more importantly, but less communicated, I get to evaluate you. How your team works, the level of talent present, management's ability to keep direction, and wether I genuinely enjoy what you have for me to work on.
HR usually stops my clients from hiring me perm at 2X the rate they pay their full time employees, despite the strong demonstration of competence that I’ve shown over the last few months.
My clients are usually unable to make permanent employment with them more appealing than repeat contracting with them.
It seems to me like a lot these problems start and end at "we don't trust resumes and past employment history" and "we highly prize 'cultural fit' but think there needs to be a technical component to 'cultural fit' interviews rather than just doing proper 'soft skills' interviews". Maybe we all should just straight up stop pretending like we are looking for anything more than soft skills and vibes?
The false negatives suck for both candidate and hiring company who have accidentally rejected someone they would have like to have hired.
The benefit of the "provisional employment" process would be that even if it doesn't help avoid the false negatives, it would weed out the false positives so that the company at least ends up hiring people who are qualified and a good fit (even though due to the false negatives they may have rejected even better candidates).
Given how arbitrary in terms of talent mass layoffs are, there are of course tons of highly qualified out-of-work candidates (and due to age discrimination, maybe some of the best/most-experienced ones!), so this certainly might work to draw from that pool of candidates.
30/60/90 is pretty standard stuff. It’s literally the documentation and justification you need to fire the employee at the 90 day mark if you need to. Like a preemptive PIP if you will.
I don’t want to hire anyone who doesn’t want to be held accountable to what they say they can do anyway.
In the US at least, firing people for cause is difficult to do. More likely you just wait for next RIF and include your low-performers on the list.
So, yes, this is business as usual to some extent, but it's not a very cost effective or efficient way of hiring. If you really have work that needs doing, then you want to hire someone able to do it, not have a haphazard hiring process where it's a crapshoot and "if they don't work out we'll just fire them in our annual RIF, and try again".
Of course it's not cost effective, or efficient, but as the article points out, nothing really is, and this method is at least preferable to the weird quasi-employment situation the article proposes
Sample size of 100 seems like overkill for finding a 6-month contract employee.
How about starting with a sample size of 1:
Sort the applicants (ether randomly or by criteria of your choice, pre-filtering as desired) and hire the first one who meets the minimum qualifications, based on a streamlined interview process of your choice.
You could experiment with a larger sample size if desired (perhaps large enough that you expect to find at least one qualified candidate.)
You provisionally hire 10 candidates for 3 jobs, keeping each of them for 6 months:
Do you spend 5 years' salary over the course of those 6 months just to hire people? How do you financially justify ballooning your payroll for a bunch of people you're going to get rid of before you get any ROI out of them? You've (maybe) gotten better talent in the end, but have you actually improved the company's bottom line?
You have two parties engaging with each other in a subpar way, and your solution is to make things better for one party (hiring side) and significantly worse for the other (candidates). Trying to convince candidates that this is good for them, won't make it so. Eg:
> Every stamp that you hand out, pass or fail, leaves a candidate richer than they showed up. This attracts strong candidates to you, because even your rejections are worth something to them.
Candidates don't want stamps. They want stable work.
> In the U.S. you generally can’t just up and fire someone, even if they are underperforming, even if it’s at-will employment, without risking a lawsuit and some sort of cash outlay. This is what makes hiring decisions so high-stakes.
I've been part of a few situations where someone had to be let go in the USA, and I can tell you it wasn't a trivial decision. There were legal aspects, sure, but there was also a personal aspect. As a boss said once "this is someone's life we're talking about here".
the article is wrong here
If they give a reason, it had better be a legal reason. “Because you’re Mexican” - not legal. “Because we don’t like you/you don’t fit in” - are you in any way different than management? Differently gendered? Differently skin toned? Different faith? Pregnant? Likely not legal.
They are not required to give a reason. You can be fired without reason.
There’s also mitigating circumstances. If the employee can show retaliation by the employer (say the employee participates in protected activities, like espousing unionization or questioning managerial decisions), they stand a chance of recovering damages from the company.
whether that's an excess of legal paranoia or not, that's how it seems to work
I think this is a response to inexperienced, immature, or thin-skinned managers. The company doesn’t want to be on the hook for some manager’s attempt to fire someone and have it be an illegal firing. Since there’s typically no day-to-day log of behavior, HR has to begin documenting things. The only thing they can be certain is “safe” for the company is to fire for underperforming. So let’s begin the paper trail…
Like walmart, mcdonalds, amazon, target, home depot? It’s incredible how insulated the HN audience is to the real life day-to-day concerns of workers in the United States.
Do you genuinely think McDonald’s is building a months long case against a fry cook? That Amazon is building a months long case against a delivery driver or warehouse worker who refused to pee in a bottle? That target is putting a cashier on a pip for months and allowing lost sales or theft?
And yes, I’m human?
Strange comment internet stranger.
At least over the last 25 years in the tech spaces I've been part of, I rarely hear of people being dismissed/let go quickly. #anecdata
The larger point of the effort involved before the employees' last day still stands. But that's a hard one to speak to generally, since most folks in layoffs or firings don't talk about the process used to arrive at the decisions.
Because I see so many people who believe otherwise, here's a direct quote from a prior contract:
> I UNDERSTAND AND ACKNOWLEDGE THAT MY EMPLOYMENT WITH THE COMPANY IS FOR NO SPECIFIED TERM AND CONSTITUTES "AT-WILL" EMPLOYMENT. I ALSO UNDERSTAND THAT ANY REPRESENTATION TO THE CONTRARY IS UNAUTHORIZED AND NOT VALID UNLESS SIGNED BY THE PRESIDENT OR CEO OF THE COMPANY. ACCORDINGLY, I ACKNOWLEDGE THAT MY EMPLOYMENT RELATIONSHIP MAY BE TERMINATED AT ANY TIME, WITH OR WITHOUT GOOD CAUSE OR FOR ANY OR NO CAUSE, AT MY OPTION OR AT THE OPTION OF THE COMPANY WITH OR WITHOUT ME NOTICE. I FURTHER ACKNOWLEDGE THAT THE COMPANY MAY MODIFY JOB TITLES, SALARIES, AND BENEFITS FROM TIME TO TIME AS IT IS NECESSARY.
Every job I ever held but the unionized ones had some variation of that.
Have you been involved with letting people go and seen it happen as quickly as the above legal disclaimer implies?
I agree that the legal justification is 100% there, but in my experience it is often buffered and takes more time.
What I feel like I'm reading here is someone who has been poisoned by FAANG hiring practices --- and they are terrible --- and has missed most of the work that's been done (outside of Google's admirable work in debunking their own processes).
I appreciate the "kitchen confidential" here, but with respect to Yegge, I think he's been working at the Olive Garden this whole time. Go stage at Gramercy Tavern! They're working at a different scale, yes, but you'll at least get a different perspective on the "gold standard".
a) it discriminates against people who cannot spare 4+ hours of focused time on evenings/weekends to work on the problem. People with multiple jobs, single parents, etc.
b) in the age of AI it is no longer a reliable measure of someone's skill, for obvious reasons
Unlike Yegge, I haven't worked at FAANG, but the companies I have worked at all followed the same hiring practices and suffered from the same problems as he describes.
Provisional employment (or, if that's not possible, then well-paid internships) solve all of those issues. The candidate gets 3-6 months of stable employment, you as the employer get a large number of work-sample tests, and you can see how they use AI and how much.
Provisional employment doesn’t work for most cases, though. It might attract people who have no job and it might attract people who have so much saved that they are okay with potentially being let go after 90 days. But I imagine the vast majority of the potential employment pool are not willing to quit their current job to accept a “maybe” job from another company.
But the "way more than 4 hours" thing smuggles in a premise: that every candidate should be able to finish the challenge in the allotted time. But candidates with greater aptitude or conversance with the problem domain will complete work sample tests faster than candidates without, and selecting for those candidates is the point of hiring qualification.
If I ask you to write me a python function to convert OSGB easting/northing into WGS84 longitude/latitude the task has a very clearly defined scope. If you knock it out in a quarter of the allotted 4 hours, you've saved time. You can't use the remaining time to go further and demonstrate your mastery.
On the other hand if I ask you to write me a website for organising photos, there's no such thing as 'done' - no matter how good you are, after 4 hours you'll still be able to think of ways to make it faster, more beautiful, more featureful, more scalable, cheaper to operate, etc
Obviously, as a hiring manager I'll notice if you've spent 40 hours on the 4 hour task - but if you've spent 6 hours maybe I just think you're a fast worker with relevant experience and sharp tools. And my sense of how far you can get is calibrated by other prospective hires; if lots of people are spending 6 hours and claiming to have spent 4, my expectations will naturally be high.
This isn't a college application.
The rest of this I'm not interested in. For as long as we've been talking about hiring processes on HN, there have always been staunch defenders of interviews. Lots of people have spent time getting good at them, there are classes on it, there are books, there are drilling exercises. I don't anticipate talking those people out of their investment in interviewing.
People aren't saying that. They are saying that other candidates will put in more time. If I do a professional job in four hours, then an equally talented candidate who puts in eight hours will produce a much more polished effort, and they will get the job.
I guess the fix is to ask the candidate to pick a four hour window, and to ask them to complete the task in that time.
I understand that some people are concerned that they're competing with candidates who will put in 12 hours to do what they should be doing in 4. But that's not their problem. Their problem as a professional is to evaluate whether they can do the challenge in 4 hours; that's the expectation the job is setting.
It is perfectly reasonable for someone to look at the hiring process we're running and say "no, this communicates to me that this job wouldn't be a good fit for me". That's a good outcome! Most jobs aren't a good fit for most people; that's the whole challenge of hiring.
I'm based in the UK, where firing people is harder, so this wouldn't fly. That said, I know interviews are basically useless so I would like something better.
For all its flaws, part of the benefit of an interview is it's time bound and equal for everyone. Similar to a test.
With AI coding this is also largely useless. These “build this thing in 4 hours” assignments come with a literal prepared prompt so that they can be churned out in 10 minutes.
Again, the underlying smuggled premise here is that candidates have to finish the work sample. No they don't. In fact, that's a strong sign it's not an effective work sample. A test that everybody passes isn't a real test; it's just a hazing ritual.
So you saved yourself/the team several possible hours of interviewing, and me quite a few hours - I think it took me about 10 to 15 minutes to see what you wanted in an engineer and that I was not it, and a total of 1 email which felt quite automated (whether it was or not) so there was a very low social cost as well.
> Again, the underlying smuggled premise here is that candidates have to finish the work sample. No they don't. In fact, that's a strong sign it's not an effective work sample. A test that everybody passes isn't a real test; it's just a hazing ritual.
Now this is an interesting take. Usually when people talk about these take home assignments, they talk about assessing the quality of the work. How good is the design? How is the coding? Is it efficient/elegant/whatever?
Here you take a much different approach, saying that the completion itself is the filter. If one person completes your assignment in the allotted 2 hours and another needs 12 but never tells you that, do you not care about that discrepancy?
I'm having a real hard time seeing how this isn't strictly better than an interview, which, as the article (and basically everything written in the last year about interviews) points out, is basically a random function.
The AI thing is an interesting problem, but a solvable one. We continue to hire resume-blind.
Would this company permit an employee taking 3 months unpaid leave to provisional hire somewhere else and have free choice whether they stay or go at the end of it.
Yes, standard interview loops also discriminate, and the more time they take, the more discriminatory they are. Any on-site requirements compound the issues.
Like Yegge says: provisional employment/internships solve all of these issues. You get the best of all worlds: stable employment for the candidate where they get paid a regular wage and aren't under a stressful interview setting, and lots and lots of work samples for you, the employer. It's not perfect. For example, it can be difficult to entrust the provisional employee/intern with anything impactful if you don't know whether they'll be employed at the end. But it is significantly better than the alternatives in most contexts.
It works. I’ve seen it in two different places.
At the second one, the fundamental realization I came to was that it is virtually no different than “regular” employment, where the new employee can get fired for not meeting expectations within an arbitrary time period after being hired. This can be months, or even weeks. From the perspective of the candidate, regular employment and provisional employment have roughly the same level of risk: in both cases they take a job where they might be let go at some point. The benefit of provisional employment is that they know how long they will be evaluated for and against whom. It turns out a lot of candidates do in fact like the all-cards-on-the-table approach and enjoy being given the opportunity to prove themselves on the job.
Let’s say I get someone to do this provisional work. What about my existing job? I can’t just ignore it, and I can’t just causally say at my current company that I’m interviewing anywhere else.
And god forbid you like 2 potential gigs, and they both want you to do the campfire approach. Then my current job is really screwed!
Nobody does this, of course, but then provisional employment is a silly idea to begin with.
FWIW I came up with a provisional employment scheme that I thought would work at a job where I was responsible for hiring pipelines. It involved making an annual bonus (in an industry that was bonus driven) garaunteed and accelerating it so it was 50% sign on bonus and 50% delivered at 6 months. The company balked. Because provisional screening is a very good mechanism. Its must more expensive than hire and fire, at least on paper.
It might be a more salient concern now, in the era of AI agents, and we are much warier today than I was at Matasano, but generally I think this risk is more talked about than experienced.
(For those interested: https://sockpuppet.org/blog/2015/03/06/the-hiring-post/)
There is no perfect interview process, so it’s important companies think about what they’re biasing for or against. A work sample gives a more realistic picture of someone’s ability than a coding interview, but less than provisional employment. But, it’s much easier for people to commit to finding the time for it than to spending 3-6 months in limbo.
Regarding AI and work samples, it seems like this problem is largely solved by having an in-depth conversation with the candidate about the assignment, no? They should be able to explain why they made the decisions they did, pros and cons of their approach, etc. If they can do that, and their solution is good, does it matter if it was LLM-assisted?
To me, "stable" implies I don't even worry about having a full-time job for the next 3-5 years. Anything less isn't stable, unless this would be my second job immediately after the McDonald's burgerflipper one where I was scheduled for random 4 hour shifts seemingly designed to maximize personal inconvenience.
If I knew that my job would only last 6 months, I would have to immediately begin conducting a job search and prepping for it. I put my odds at finding another job before the 6 months is up at less than 60%. Even that number seems naively optimistic now that I've typed it out. Offering me 3-6 months of employment only sounds like a deal if I'm currently unemployed and rent was due yesterday.
If you are going to take a day off to do 3-4 in person interviews at a company then this slots in well.
As an interviewer, I much prefer the signals generated through a work-sample interview. I'm much more confident in the hiring recommendation than I get from a 1 hour zoom session. However, if I look at teams that were built through the work-sample and zoom interviews, I'm not sure the outcomes were that noticeably better.
As an interviewee, I think I understand the frustration being on the other side has. With an in-person interview, I often have a good sense that I bombed the interview or something to improve on or replay in my head, less surprising outcomes. On the work samples it's harder to know whether you're making mistakes, or are being out-competed by someone putting in 4 times the effort to polish the solution beyond what their regular work product would be. Although I had one really good outcome where the work-sample interview really flagged the internal dysfunction of a company.
And then with both interview processes, I still think there is a really big unknown on what the false no-hire rate is, how much effort is getting wasted rejecting candidates that would actually fit the team.
So having to choose a process as an interviewer, I'm with you and would always choose a work-sample interview. On whether it should be considered the "gold standard", I'm much more hesitant, I think there are some limitations that are still hard to control for.
I do wish Starfighter/Stockfighter model had gained more traction, would've been interesting to see a recruiting company specialize in this and then seeding the interview results to multiple companies model work out.
Leetcode style interviews feel so stupid and divorced from the reality of the job, especially the “one weird trick” kind where you’re expected to discern the best possible solution to a problem on the spot and in a pressurized situation.
The reality of the job is usually that when you are under time pressure, a suboptimal solution that does the job is fine (to be fixed later), while if you’re working on something you know is important (hot loop code, core data structures) you have time to think about it and get it right. A leetcode interview doesn’t select for either of those things: it selects for people who have time to grind leetcode problems.
On the other hand, a work sample is realistic: a timeboxed task that you can approach in a familiar environment, without an interviewer breathing down your neck, expecting you to think out loud, railroading you to their preferred solution, etc.
As an interviewer, I always pushed for either work samples, which I quite liked, or coding interviews where we very explicitly said that we just want a working solution, which we could then talk through and look for potential improvements. We also explicitly viewed the coding interviews as being low signal, and tried to make the bar for passing low, so we could get candidates to higher signal conversations.
I do think the work sample route is a little more difficult in the LLM era, in that you are more likely to get a decent performance from a candidate who doesn’t actually know the domain, but a subsequent discussion asking them to explain their approach seems like it would be enough to ferret that out.
If you're in an environment that is open to agents to begin with, one simple thing to do is just tell people to use agents, and ask for the prompts. Prompts are a high-signal artifact, and you can construct rubrics to evaluate them objectively.
I'll get around to writing a piece about this within the next couple months.
Ultimately work samples aren't a technology technique; they're a longstanding concept in management science. Long before the first coding take home was ever given, firms were using work samples to qualify salespeople, support professionals, factory workers, whatever. AI agents are a part of what the work of software development involves, and there's obviously no fundamental reason why you can't work sample them like anything else.
I often found that I learned more about the candidate by the way they phrased their Google searches and how they selected and explored sites for information than from the actual solution they produced.
Those were (and are) absolute rubbish; "in 20 minutes please come up with something that took [Knuth, Dijkstra et al.] years to come up with."
Fortunately AI seems to have killed them (leetcode interviews, that is - Knuth is very much alive and seems to be happily coexisting with AI [1].)
Disclosure: I did enjoy Google Code Jam as a fun thing (even though AI killed it as well), but it's useless as a model or method for hiring.
[1] https://news.ycombinator.com/item?id=47306926
It's not just you. At the end of the day interviewing has been demonstrated to be close to a crapshoot in the best of circumstances, and very few interview schemes are the best of circumstances. Work samples are part of the optimal strategy [1] but even then the signal is quite low.
[1] https://psycnet.apa.org/record/1998-10661-006
Hiring in most of the tech industry is gatekept, and like every protectionist scheme, it isn't organized for productivity.
The conclusion at the end that bringing someone on board is the ideal method is true I'm sure, but even that runs into the issue that employee evaluation is an even worse situation than the interview process.
You can openly see some managers panic when they realize they have no idea what their employees have been doing for the last 6-12 months when they're asked to provide feedback.
1) The worst interview I ever had (BY FAR) was at Google--disrespectful people, no respect for time, I could go on and on. And I went back to try again to get that money showered on me. Worth it in the long run.
2) Their new system for "performance management" is a hoax. Just like at all other places, it "documents" what you should do so they can fire you more easily with unspoken rules and all sorts of arbitrary causes as well. A friend literally hit EVERY pre-agreed target and still got pushed out for "not delivering".
They told me it was all about pseudocode and how I think in advance, then on the actual interview they were being annoying about variable names and spaces after comma, while I was supposed to come up with some clever optimisation that boiled down to: "do you know this obscure theorem already? Cool you pass"
1. Standards got higher. Luckily if you got in early and proved yourself you are OK. But doesn't mean you would pass the current interview.
2. A marathon runner (with rare exceptions) can't run a marathon on a random day. They train for a specific date. Same with interview prep.
Number 1 can still be true, and likely is. But then what's the point of using dated packets to test the hiring committees calibration?
They did, but not with the intention of doing anything about the problem.
This is a question of reliability, the conceptual 'correlation' of a measurement instrument with itself when measuring the same thing.
Reliability is one of two major concepts in psychometrics, the other being validity, the conceptual correlation between a measurement instrument and that part of reality that you're hoping to measure.
The question behind validity is "I want to know X; if I measure Y, how helpful will that be?". And the question behind reliability is "if I measure Z, how accurate will that measurement be?"
https://en.wikipedia.org/wiki/Reliability_(statistics)
https://en.wikipedia.org/wiki/Construct_validity
Yegge calls out both concepts explicitly, though not by name, in this essay:
>> The outcomes from interviewing are statistically terrible. Google did wave upon wave of analysis over the years, and all the results were incredibly depressing.
>> [reliability] To name just a few off the top of my head: interviewers barely agreed with each other. Put the same candidate in front of two of our sharpest people and you’d routinely get a confident “strong hire” from one and a flat “no” from the other.
>> [validity, though the 'problem' here is strongly confounded by a restriction of range issue] And once people were actually on the job, their interview scores told you next to nothing about how they’d do
>> [reliability] Hell, some of our star performers failed their Google interviews four or five times, finally got in after 2+ years...
>> [validity] ...and then outshone everyone else.
The discussion of how interviewing outcomes are statistically terrible would benefit from naming the ways in which they're statistically terrible. Knowing the problem you have is an important step toward solving it.
(And as a side note, the last I heard from Google, you're not allowed to interview more often than once a year. Interviewing five times in two years would seem to violate that policy.)
It is a basic theorem that the validity of any instrument is bounded above by the square root of the reliability. It isn't possible for an unreliable instrument to be tightly correlated to reality, because it is, by definition, not tightly correlated with anything. That's what it means to be unreliable.
Thus, any company that wanted its hiring process to be good would necessarily be extremely concerned with making that process accurate; you need to come to the same decision when you assess the same person. This is something that interviews cannot achieve except at extreme cost. You'd need far more than five interviews to get a reliable assessment from them, despite the claim in this essay that "any more than four interviews and you're just playin' with your food". Of course, the Google interviews aren't supposed to be reliable anyway, so in that sense the claim is probably accurate.
The prescription Yegge offers is valid. Multi-month work assessments will give you a strong, reliable, and valid signal. They're also very expensive.
Another thing the essay completely glosses over is that this problem has been recognized for a long time, and we already know how to do assessments that are reliable, valid, and cheap to perform. They're called standardized tests.
The problem is that companies like Google that have evaluated their own hiring process, by comparing candidates "hiring score" with subsequent on-the-job performance, have found that there is little correlation. So, while the goal (be more concerned about false positives than false negatives) makes sense, their process of trying to achieve this is broken.
Perhaps we could administer this IQ test at age 12, so that the low-scoring individuals can go straight into the fast-food industry, and the rest can pick between the doctor/lawyer/SWE offers that will be showered upon them?
Hiring from a test that measures IQ is a very good idea (and there is a test that's commonly used for hiring purposes, the Wonderlic†); hiring from "an IQ test" is a bit less good. Anyone who wants to subvert the Raven's test will be able to do that. High-stakes tests need more security.
The concept of "IQ" can be toxic in contemporary American politics, so there are many more tests that "happen" to test IQ than there are tests that advertise themselves as testing IQ.
† https://psycnet.apa.org/record/1982-00123-001 : "correlations between Wonderlic IQs and WAIS Full Scale IQs were [0.93] for the main group and [0.91] for the cross-validation group". Note that this test involves only basic math and takes 12 minutes of the candidate's time.
A man walks into a pet store. There's a parrot for $100, it says this parrot speaks perfect English . The one next to it is $1,000, and says, This parrot speaks 12 languages fluently.
Then there's a bedraggled looking, droopy, parrot, and its label simply says One Million Dollars.
Does it sing opera and has successfully run for President? the man asks with a sneer.
This parrot, says the store owner, _thinks_.
That's what this entire post is about - how to evaluate people with a series of attributes, score, correlate, blah blah blah.
Hire them, see if they think. If they don't, fire them. It's cheaper than this credential/signal rigmarole, most of which is about CYA legal b+llsh1t. Yes, it's a simplistic strategy and it doesn't work for Shoogle, Banthropic, Goober, whatever. You know what, boo f*cking hoo. You're a trillion dollar company, suck it up. You have a zombie horde at your doors and you're just upset the "true gems" are hard for you to spot amongst the slavering masses. You're going to heartlessly lay them off anyway in a few years. You SHOULD feel this pain and anguish of having to sort through them, constantly regretting all your choices. That's the only way to have balance in the Universe.
Edit: Didn’t link it initially because I thought it would be hard to find. Turns out it’s not. https://www.pushkin.fm/podcasts/revisionist-history/hamlet-w...
Tangent: people do a lot of stressing out lately about which LLM they should be using for a certain task. I bet this advice applies there, too.
The fact that an organization cannot deal with such a case is a bigger problem in the first place. Eliminating incompetence and malice is among the basic skills of an organization.
Of course the organization should do its best to avoid bad hires. It should do so because of the opportunity cost of not hiring the right person, not because of the damage that they might give to the organization.
In light of this kind of threat, strongly favoring no hire, and even policies like mandatory on-prem work, start making a whole lot of sense.
Why is incompetence so discussed for ICs but rarely for management?
Maybe if more managers were competent and less conflict-averse they would do their jobs better and cycle out incompetence and bad intentions faster.
Ah wait, but middle managers specifically choose conflict-averse, easy to control, domesticated people. That's why you find this particular type of personality in management so often.
So the issue is incentives as well and bigger than initially thought.
The post touches on it (regulation about firing causing legal issues) but the unfortunate truth is that turning away false negatives has little cost to a business as long as you make sure to turn away false positives. Bad hires are extremely expensive, where as the lost revenue of turning down a good hire isn't nearly as bad (it's easy to try to hire them again later).
The stamp idea is good, reputation based hiring is important, as shown by the use of referrals in hiring. I've always wondered why we don't keep track of referrals inside companies in order to figure out who is recommending their peers who legitametly are good fits, vs those who are just trying to help out friends (who might not be the best hires).
Further I think there is also the problem that anti-corruption/anti-nepotism often harms hiring. It's much easier when working in a start up to hire the most talented people you know without other people sitting in, compared to at a big company where I know people who actively know good people they want to hire, but who can't be arsed with the long interview process and high chance of rejection.
If it wasn't for the regulation around firing some sort of staking your job on the line would be effective. You can refer someone and have them skip most of the process but your job is on the line if that was a bad call.
B.A. Political Science, Bangalore University (failed)
Everyone will have some successful stamps, and they accrue over time giving better signal. Hiring managers can make a rational decision about what number of stamps over what period of time is sufficient to evaluate a person, trading against the risk of misreading a profile by accepting someone with fewer stamps.
That's real useful work, if we don't hire them they can show it off to a different hiring company, and it makes the world a better place.
The problem with homework problems now is AI. But I think it would be quite easy to detect if someone just fed it in to Claude when you go through the code with them.
I also think this is pretty bad for candidates. The last thing I want to do is a bunch of unstable gig work for fickle startups in one of the most expensive places to live in the world. Also, I really think this only works if you decouple things like health insurance from employment, otherwise the instability can be too much for a lot of people.
The place was not big - never got much beyond 100 engineers, but produced dozens of founders, VPs, GMs etc at well known companies, as well as engineers with very notable OS projects and lots of high-placed engineering in FAANG companies.
It doesn’t work for software engineers because nobody wants to do free work. Also, free work isn’t consistently, and fairly testable because project requirements can evolve or completely change over a hiring period. Not to mention IP issues.
What has worked in my experience is a synthetic take home exercise that isn’t easily LLM solvable, the same one given to all candidates with the same constraints, and offering to be available to answer their questions by email. Sometimes we can tell a lot about the candidate just from the questions they ask, or how they package the solution, without even looking at the code.
In my experience (FANG hiring) less than 10% of candidates refuse to do this exercise, so it’s worked well for us.
I received one of these when I applied at twitter (pre-musk) and it was so poorly worded and organized that it went badly for me and the interviewer. But I see that as dodging a bullet (google was a much better fit for me).
If I am doing a 6 month contract which is what he is proposing then yeah I want a great day rate and to earn 12 months salary in those 6.
It basically means I invest in a company that doesn't believe in me and probably wont try to help me succeed.
Also mortgages, car loans, life and health insurance etc. are harder to get on a short contract.
Hiring is broken (maybe) but 6 month paid interviews are not the solution.
I think it might actually be a good thing for both employers and employees if health care, housing, and food access was secure enough to give people the freedom to take six month internships, but I have no idea how we get there from here.
Of course, its not that simple. Some companies probably are great at scouting. Yegge mentioned a few ways in the post. Good internship programs, acquihiring, etc.
Most places still interview. Because hiring someone for 3-6 months is still an expense. For large companies, onboarding can take many weeks before you're even thinking about being productive. Interviews don't need to be 100% accurate. They need to be time-efficient and an ok filter to prevent hiring the worst people.
I don't see how a bad job market is going to make skilled people be willing to intern in order to get employment. Employment is a market, if demand goes down, then the reaction will be that price goes down.
The reason students are willing to intern is because the supply is so high and demand so low that you can effectively hire them for nothing (or next to nothing). The interns know that on completion the internship will upgrade them to a new category with new supply/demand. It's the same reason they are willing to pay large amounts for a university degree.
So interviewing will stay the same. If demand collapses, we will see wages drop, and I imagine the supply of software people will react through early retirement, career switching and reduction of people choosing it as a career before we see an upheaval of hiring methods.
In the last few years I was the main tech interviewer for a 300-employee fintech.
For a specific position, one recruiter got around 150 applicants, selected 5 great ones, who did take-home tests and mild-tech interviews. Offers were made to most.
For the same role/salary, but from another queue, a second recruiter got around 900 applicants, cherry-picked about around 70 of them. Out of those, only 40 completed the 1h take-home test. Only 20 delivered it, only 10 implemented the requirements. Of the 10, all were unable to answer even basic questions.
This was concurrently, so it wasn't "affected by AI".
I didn't changed my methods and in fact I didn't even got close to asking hardball questions to the second group.
The second recruiter didn't get their contract renewed and left.
Yeah and you know why? Because like everyone in the industry you make the mistake of trying to hire engineers at the level of expertise and with the knowledge and skills you think they need to do the job. Except, any engineer worth their salary (i.e. their salt, I guess) is going to grow into any job and learn new skills and new technologies that they need to do the job, so the person you really want to hire is someone who is not yet ready to do the job you want them to do but has the potential to learn it really, really well. And the engineers you really hire, if they already have the knowledge and skills that you want them to learn, won't learn them because they already know them, will be bored to hell with the role, and as a result do a sloppy job and move on quickly to another role that is more challenging. That is, if they like to be challenged. If they don't like to be challenged- congratulations: you hired yourself a boring engineer.
That's what's wrong with your hiring practice. And you have no way to fix it because you keep telling yourself you want to hire "talent" but what you really try to hire is "experience". If you hired for talent, you'd get both talent and experience, but if you hire for experience, you often end up getting neither.
Incidentally, my pet theory is that this explains why 90% of modern software is crud.
i've seen a few interview types in my time:
1. technical interviews run by nontechnical or junior engineers who can't judge technical talent. this typically produces sub-par hires. imagine you are made the director of UX design and you have to hire someone but know little about the industry. what type of hires are you going to get?
2. technical interviews that focus on CS skills. can you write a red-black tree in Java? can you write a bubble sort in haskell? The problems here is that this has bias towards new CS grads that dont have much industry experience but just took tests on stuff like this in their degree. google pioneered this style of interview and it's taken off as leetcode. the problem is that experienced devs dont typically write bubblesort algorithms. this type of interview is biased toward younger out of college hires and codecamp hires that have little experience but gring on leetcode sites.
3. technical interviews that have too many opinions for a hire recommendation. this seems to be a new trend where you need 3 or 5 or 7 thumbs up to get hired and any negatives sink the candidate. this does raise the bar, but typically what i see is someone on this panel has a high opinion of his/her abilities and gives everyone they seem a thumbs down.
What i've found is that soft skills and team dynamics are > the technical accumen. I can teach you how to write go(lang) but its much harder to teach you how to influence your peers or to communicate effectively.
Experienced devs probably dont know how to ace your leetcode interview, but they do know how to influence and communicate and when to stay away from bad ideas. You dont get tested on these things. It's analogous to a being combat veteran. You've been there, you've done that you know how to survive. You might not know the new tech, but that can be taught. I have never seen an dev not be able to pickup a new platform/language/skill within 6 months of hire - anywhere.
Here's hoping this will get better one day but i'm not holding by breath.
And that's why the trades require you to perform years of work as an apprentice before you're ever qualified to pass an exam to become a journeyman. Not only is it like a years long internship, but you're paired with someone who has already passed a high bar. You're learning directly from vetted people, and those people can vouch for you.
> University of Waterloo famously sends their Computer Science students through a total of six internships, giving them roughly 2 years of real-world work experience before they graduate.
Still less time than an apprentice is required to work, but it's better than nothing. Most trades require you to be an apprentice for 4 to 5 years.
> The reason is, hiring engineers has historically been so competitive that you couldn’t convince a senior engineer to do an internship [...] So the industry converged on not requiring it.
Which is one reason the trades made it the law that you have to follow the apprenticeship in order to become a journeyman. "The contracting companies don't feel like hiring licensed workers" isn't an option. Companies don't do the right thing unless they're forced to.
This is why we need professional engineer certification for software engineers. We need a rigorous, time-tested, reliable process to ensure engineers have actually done the job in the right way. Otherwise it's a bad guessing game like Steve is explaining.
This is also why we need a software building code. We need to explicitly define exactly how you're supposed to engineer, so that we can create a certification that people can pass. Otherwise designing and building software is completely subjective. Engineering should not be subjective.
This is not some mysterious experimental idea that nobody knows if it'll work. Trades have been doing this for decades. It's not perfect, but it's much better than the alternative.
It doesn't have to be perfect right off the bat. Start by writing down how you do your job. Get some other people who do your job to review that. And get some more people to review the review. The reviewers note down where there's some gap, something you're not doing, some considerations for doing it, etc. All of it gets updated, consolidated. In a few months you have a document that standardizes your profession.
There are millions of people around the world that write software. Get different groups together to document their own specialties. By parallelizing, we can get a basic set of standards for most specialties in ~1 year.
And don't forget there are already thousands of books on how to do each specialty. It's not like all the knowledge is tribal. It's not really that complicated to explain how software and systems work.
I think there’s a lot of ways to do engineering right. It’s very subjective in many ways. Every project has multiple possible paths to success.
I think the real solution is something like the Bar Exam. Apply the hard filter once, publicly, administered by a trusted neutral party. Anyone that gets through is assumed to be good enough, and interviews can assess only firm/team specifics
FIRST - is that before you get to campfire anyone, you have to have done some sort of interview process to boil it down to the one person to do the campfire - so how does that work eh?
SECOND - campfire is deeply invasive to the candidate's life and time.
THIRD - you have to pretty damn sure someone is a hire before doing a campfire. You CANNOT do campfire as an evaluation step, after which there are more interviews.
FOURTH - this is effectively just a really really long version of the take home work test which is absolute bullshit.
FIFTH - there's STILL no science to the campfire. Don't give anyone a fucking test if you don't actually know how to scientifically evaluate the results. And campfire does NOT result in a scientific outcome, it still results in an arbitrary opinion.
SIXTH - any company that wants me to do a campfire - to commit days or even weeks of work as part of them trying to decide to offer me a job - can fuck off. Sorry, the party got spoiled by all the other companies who asked me to do something as part of the interview process and then either ghosted me or gave me some bullshit outcome like "they didn't like your work".
I can tell you how to recruit people and it does not require campfire.
You TALK to people about software development - you engage them in extended conversation about what they have done, what they know, what their interests are, what they have built, what projects they worked on, what went right, what went wrong. You look for people who have BUILT STUFF - this was true before AI and is 100X more true now - anyone who has not built anything today is not worth employing and anyone who has built something must be able to talk about it in depth. This interview processes worked before AI and it works after AI. And finally, you accept the limitations of recruiting which is that people are people and you won't find out how well someone performs until they have been on the job six months - live with it.
Sorry Steve - I love your work but I'd never work for any company that wanted me to do a stupid campfire because they don't know how to actually work out if I can do the job or not.
Talking requires that the interviewer be competent and care. Which seems like a low bar, but it easily degrades. When an engineer is burned out, or is interviewing for a project they have less stake in, or has business requirements forcing them to hire someone fast, the bar lowers. Those engineers now are the gatekeepers, and they let in even weaker engineers.
Even worse, any utility from this is predicated on the founder making good hiring decisions. If they don't understand how to hire a competent engineer and make them care about the company, then it's dead from the start. And a lot of founders or execs in general are awful at this.
Companies that maintain an excellent work culture such that engineers deeply care about the team and have stake in any project they conduct interviews for will do well with this model. I hope we see it used by them.
why not? to expensive? candidates don't have the time? i'd love to do that because it also gives me insights into the company. how they work, etc...
I can tell you how to recruit people
why is interviewing broken then?
You TALK to people about software development
didn't we try that and find out that the sweet-talkers master this and are able to fool everyone? or didn't we find that this leaves out good people who don't do well when questioned like that?
btw: my personal preference for a tech interview is 1 hour of pair programming. (not just live coding)
these days there are so many people being laid off that you still get a large selection of potentially good candidates, also, for someone who has been out of work for some time i would like to test them more intensively. in fact from a social standpoint i should optimize hiring for people who don't already have a job.
There is a way to make the campfire approach a little bit less utopian: California could pass a law that makes it legal to have a second temporary job. So that FAANG engineers for instance, would be allowed to campfire/interview at another company with legal protections. Put a bit of grease in the interview system is good for candidates.
From the candidate's POV what is mostly broken about the application process is all of the ATS gaming, resume tweaking, etc, just to avoid getting filtered out before an interview, as well as the inane leetcode screening that some companies are doing.
A better process might be to replace all initial profile/resume-based screening with task-based evaluation (evaluation could be at least semi-automated - if a computer/AI is going to reject me, I'd prefer it to be based on task evaluation job skills rather than ATS-filter avoidance skills!).
Lengthier on-site interviews/evaluations could also be task-based - for a developer role perhaps a 2-4hr peer-programming or problem solving task. Far less signal that a 3 month provisional hire of course, but maybe a better use of everyone's time than a traditional talk and brief whiteboard challenge process which is clearly failing as a useful filter.
I wouldn't expect companies to forgo the traditional touchy-feely team/culture fit type of screening as well, but better if this came after they'd already determined, as best they can, whether you've got the chops to actually do the job well.
Today I'm 45 with family and have a fancy VP title, but I would have no problem to do this for an interesting role at a cool company.
Just one more thing: the industry should really put a bit more weight on measuring people's potential and the concept of long-term growing and learning on the job. Just saying. You know, like every other industry on this planet.
Of course, the real issue is that a prospective employer doesn’t know what they want. Our “engineering” industry runs on vibes, heroic effort, fashion and hype-cycles. Someone very smart, enthusiastic, quick learning, flexible and affable is the only real job description.
Combine this with the “supply” side entirely overwhelming demand, having to compete with AI, and experience having no value, and for most software developers the future is bleak until a new more equitable equilibrium is found.
Nevertheless in the meantime, at least one can have solace in collecting the authors “failure stamps”.
It's true, but prepping for a typical senior+ onsite loop in big tech still requires weeks of grinding leetcode, re-learning the latest system interview questions and the system interview answer framework, refreshing and rehearsing STAR stories, studying the company and its unique quirks that you're expected to know to pass the culture filter, remembering how to do all of this speedrun-style since you only get 40ish minutes per session, etc.
While that knowledge is more reusable across onsites, it's likely even more work than doing real or pretend-work for the company for a couple of days.
> When candidates get to walk away with something of lasting value that they can keep forever
I'm curious why them getting rejected from the position, even with the work sample they can carry away with them, wouldn't be still interpreted as a negative from future employers. "The other co passed on them, am I the fool for thinking they're good?" type of herd mentality which is often unavoidable.
Won't that "work sample guest book" be treated as the list of all companies that rejected you, a net negative for your personal brand you're projecting?
> (Me paraphrasing what Steve was implying) Take-homes are impacted by AI one-shotting them for candidates
I've been pleasantly surprised by how much you can glean from having the candidate upload their conversation log with the coding agent for whatever take-home you give them.
You need an NDA from a candidate (many would not like to risk - what if someone gets job on the competing company and is sued for revealing some secrets from the campfire interview process).
In many cases interviewee would need to get a laptop from the company, as there are specific requirements about data security (disc encryption, usage of solutions like Fortinet or Zscaler), be added to company SAP to get access to the resources (Office, Teams, etc.), company need to purchase licenses for Office, Github, Jira, etc.
Surely hiring is hard, surely there are false positives and false negatives, but fixing this requires hell a lot of resources and organizational changes and costs.
The problem with provisional employment is that it can take quite awhile for a new hire to be productive. In a complex FAANG environment I'd wager it's about six months before they're not a net drag on the team and maybe 1-2 years until they're close to fully productive. These are complex not just technologically but organizationally and there are tons of hidden rules and micro-decisions that are the difference between a project stalling vs moving forward.
If you want good employees, poach those that others are paying millions already.
[0] https://www.nytimes.com/2016/02/28/magazine/what-google-lear...
1. For juniors, any sort of proof you have passion is enough. 2. Treat mid-levels as seniors. 3. Seniors have to show proof of passion (with longevity and intensity, AI and consultants exist for expertise and menial work) and competence (Take home problems with a long time scale with whatever tools they need. Use AI. I don't care, but be expected to be scrutinized for your design decisions, depth of exploration, and architectural write-ups.)
Now you're amazon/google. You have 200 open positions in a certain site/country. You receive 10000 CVs. Same problem, different scale.
So ok, you need to filter CVs, mmm, which sucks, right, so perhaps we do a screen interview? mmm, low signal, maybe....
He is indirectly responsible or at least part of everything that was terrible in most of the big techs he was part off. By his own admission, each time things were evaluated, the scientific conclusion was that it was "horse shit" and totally randomly hiring results in the end despite having candidates go through an awful process. And now he is full of ideas about what kind of new nightmare we can create to reach the same stupid result...
What I see in his post, and that I find despicable is that it looks like that he has this "superiority" syndrom making him consider candidates and employees as disposable resources.
Instead of considering the process as a "mutual" process, where it costs and the investment on joining the company is also from the candidate side, he is considering that just the employee is a liability for the corp and that the candidate should submit fully to it. Even wasting weeks, month or years without guarantee of stability or not wasting his time.
If you want to know if someone is good at your company in 3-4 interviews, it’s tough, the best they can do is ask these technical questions. Talk to you about your past work, ask you technical what ifs. Most dumb ass companies will ask you to do trick coding leet code crap.
And before you say that this is inefficient consider that despite being terrible for morale and efficiency (proven in un’etica studies) companies still maintain the bottom 10% out or up or out policies.
Companies always love their power on their employee over efficiency.
Or maybe the company changed in the 10 years or so since everyone in that room was hired and the employee needed 10 years ago is not the same as the ones needed now?
Warning, sad boi rant ahead.
I was from a programming bootcamp. I entered the tech because I was the only person in my bigger family members who have the skills/education/chance to earn a big salary. I still remember, I paid $12k back then, and it was me and my mom's only savings. I started tech in my 30s, and mostly worked physical labor work before that.
I was the weakest at that cohort, but I studied really, really hard, until the bootcamp noticed my progress and hired me, albeit at a very low salary. That was my first programming job.
I eventually learned about big tech, and liked the fact that they didn't care about credentials, whether I came from a good university or not, unlike the YC combinator startups who mostly cared about credentials. I learned that they pay really well, and I did, read Steve's blog, "Get that job at Google". It motivated me to study DS&A, and also to get CompSci degree.
Overtime, I lost count, how many interviews I have failed at big tech. I gained more experience, I became older, but I kept studying, and studying, and studying, but I keep failing. I also wasn't sure which area to study, so I ended up studying for everything, from frontend to backend. It was a lot, really, a lot of things to study, from leetcode to JS specific to DOM to backend system design to frontend system design, to behavioral, and frontend interviews back then was still a crapshoot, some companies ask for deep JS/DOM questions, some companies just ask for leetcode questions.
But I kept failing.
I think I'm pretty good, or at least, that's what I thought. I learned quickly, I have no problems for clearing mid level leetcode questions in under 20 mins. Overtime, the bar gets really hard and it became hard level leetcode questions in under 20 mins.
But what made me really bitter over all of these interviews, was that I saw my friends, people I knew, from programming bootcamp, from my CompSci degree, got a job at big tech. I knew, or at least, what's what I thought, that they had less skills than me in programming and in DS&A. But they got into big tech. Some of them were minorities, and during that time, diversity hiring was a thing, and maybe that's why they got in, I thought to myself.
Sometimes I asked them what the interview questions were, and to my surprise, it was easier than my questions. Idk why.
Sometimes I wonder if my luck in interviewing is really bad. I tried everything I could. I bought courses and devour those materials. It's been years and years and years and years. It did wear down on me. I want to cry, which I did sometimes due to keep failing big tech interviews. But it won't do me anything, I can only keep my head down and keep trying.
I ended up making pretty good money in this field, and able to help my family members. I did work with some of those big tech engineers. I realized that those big tech engineers were just average, and I don't think I am less capable than them. That made it sucked even more, because I don't understand why I kept failing, and why the people I know are succeeding.
I do some freelance now, but will finish some of my contract. The job market is scary, and one of my contract will finish soon. I am having hard time getting even recruiters to contact me, maybe because I don't have big tech credentials.
I also did mostly fullstack/frontend leaning lately so I ended up studying frontend interviews. But frontend job market is kinda dying lately, since most companies don't really respect frontend and don't think its worth it anymore. These days, I don't even know what to study anymore, so I need to study everything again, with the addition of all the AI stuffs.
I can't help but to think, and always think even today, especially during this tough economic times, what if I was able to get the big tech job. My family would be proud of me. I would have better companies in my resume, and could stand out more in the job market. I could've done some cool highly scalable projects that I can boast of in my resume. I could've joined some good team, mentored by good engineers. I would have more savings than now.
These days it doesn't matter anymore. The big tech salary era already closed. Things are way way more competitive than before. With my average background, I don't stand even a chance at being asked to interview.
Life is really a mystery. Things that you really need or want, you don't get it no matter how hard you try. But it is wasted away on those who got it.
I usually don't post comments, but seeing this post, I just needed to vent.
Thanks for reading.
Thanks for this article Steve. I need to read this. Maybe it will finally give me closure, after years of failing.
Sorry what.
My record is comparatively humble: I hired around 300 people in tech for small-to-medium size companies, in the small tech backwater of Vancouver, Canada.
My conclusions:
(1) Interviewing is an intractable problem. Start by recognizing that.
- You don't interview the best candidates, but the ones with the best resume.
- You don't hire the best candidates, but the ones that do best at interviews.
- Screening by HR (phone or zoom) is at best useless.
- Timed coding assignments are a waste of time. They are used because they're cheap + provide a [generally wrong] "quantitative signal". Noone's job will consist of solving 8 "leetcode" riddles/day.
(2) "Technology fit" is a dangerous illusion:
- It is very, very, very unlikely that any candidate will be able to pick up right away your tech environment. (ok; exception: you're hiring permanent an existing community contributor for your open source project)
- Your best new hires will be 0% productive in their first month (negative; training will use resources); 15% in their second; 50% in their third.
- Rejecting e.g. "Java" when screening for "C-sharp" is stupid.
(3) The interview process is about building relationships.
- People you don't hire will remember your company from the interview + disseminate.
- Someone you didn't hire today (one of your top rejections) may be super-attractive 2 months later, or next week if your top candidate accepted another position.
- Ownership of the process or at least buy-in from the team (vs. just the hiring manager + opaque "corporate committees") is the first step in a working relationship. Your "superstar" may end up being toxic in the team and you could find that out in the interview.
(4) Five simple rules:
- Treat phone/zoom screening like an advertisement for your company. Ideally do it yourself (hiring manager). Largely ignore feedback from HR :)
- Hire candidates who are: smart + hungry. Programming languages; frameworks; environments are secondary.
- Try to get a sense of the fit with the other humans in the team they'll be working with.
- Take "3-month probation" seriously. Explain it to the candidate + team. Sell it internally. Candidate compensation for a botched probation is reasonable + just money, after all.
- Treat candidates as humans: Send a personalized rejection (from you, the hiring manager _not HR_) to everyone who made it to the interview. Call or zoom everyone who made it to the final round. If you can, provide them actionable feedback on ways to improve their interview process. Leave a "human" door open.
A scoping/guidance suggestion to start: Is the problem of improving hiring more tractable if you don't try to solve it for Google?
What if Google is hopelessly tainted with biases, from having run almost every single person there through a process that you acknowledge is a horrific load of poo. Which not only means the process has become a religion/frathouse that people are proud to have been accepted into, and will defend fiercely, but also greatly determines the strengths and weaknesses you have to work with, if you want to get them to do anything else.
What if you instead start with smaller companies that don't have the baggage of FAANG, and look at their requirements, which may include some mix of (off the top of my head): prior experience, learns whatever is needed, creative/innovative, works as a team member, aligned with company success, professionalism, good for morale, good for catalyst, will have some loyalty during the harder stretches or when tempted by a little more money.
It would be nice if, instead of many 2-person startups making their first hire with the assumption that they should mimic the known-bad processes of megacorps (often based on their own FAANG-oriented interview prep in college), the megacorps were desperately trying to figure out how to hire as smartly and genuinely as SMBs, but at FAANG scale and with all the FAANG legacy workforce biases clawing them down.
what?
Why would this be useful signal?
I'll assert that you're in a professional environment of: dishonest posturing and oversized egos in startups, bureaucracy and butt-covering in established companies, and a culture of thin niceness veneer.
I'd think all of that would be barriers to a useful Permanent Record.
> This system is basically worth exactly what your honesty is worth. If you hand out gold stars to everyone, you’re just reinventing LinkedIn endorsements, which are worthless. Companies whose stamps mean anything will be the ones known for only handing them out for quality work. Being a hard, fair judge is an advantage for everyone.
What is the precedent of that working, in the environment I just characterized?
Given our current environment -- which isn't going to change easily; it works for the people who have most of the money now -- seems like a not very reliable new category of surveillance capitalism. But it gets worse... A lot of techbro founders will immediately see this as the next sketchy, hated tech company, which won't only charge companies a fee to participate, but the real business will be twisting it into off-label uses. For example, with whom and when a person has interviewed, and who which companies are interviewing, is valuable data to sell, no matter how useless the feedback. They could also see how close you could get it to a protection racket, by selling Pro individual memberships on the side.
So we get bad signal for the ostensible service, in exchange for additional techbro dystopia.
The treatment: there's a long failed history of using contractors and hiring the best, which is a recipe for abuse and PTSD.
The wisdom now is that one should prioritize avoiding false positives, but the observed facts are that many people who later do great interviewed poorly.
For false positives, there's the legal risk (hard to fire), but the greater risk is the damage to the organization before they are fired.
The premise is that people are great or not at interview time, and the process just needs to be accurate.
But for both false positive and false negatives, the reality is that how people work out does not vary primarily with their skills (probably because all candidates have enough baseline skills) but is a function of both them and their work+environment - not f(emp) but f(emp, work, others).
For f(*, work, others) there are 2 troubling issues: 1) for underperforming groups, there's a massive incentive to blame someone, and blaming the new hire is the solution that doesn't disturb existing relations; and 2) even effective combinations f(emp, work, others) might not be able to perform as needed, so performance is only a qualified measure of effectiveness.
So the more interesting fact is when low-score hires end up doing well, which is signal for a high-functioning team being able to get the best out of people. And when emp goes on to other contexts successfully, the effect could be persistent (and they could be bringing it transitively to other groups).
Yes, companies need to hire good people. But sometimes it's more important for companies to make good people of those they hire.
I suspect the poster child for selection is Amazon, and for making the best of what you have is Apple. Amazon is great at pursuing new opportunities relentlessly, while Apple is good at managing a very complex supply chain and delivering consistently.
So on the "bundle" theory of companies (from recent discussions about Japanese company diversification), you'd want to optimize hiring to company capabilities and orientation.
If you make "accuracy" at interview time the standard and optimize against false positives, you're almost deliberately hiding from the details that determine what people will grow and contribute the most. (Here's where prior relationships, school networks have a positive effect that pragmatists accept (notwithstanding the social bias).)
Almost any work can be as formative as prior experience and education. I think that depends most on mutual consideration and a growth perspective, and you can interview for that. Like investments, you're looking for growth potential, not past performance.
Ironically, successful teams that make the most of their people don't need to hire so much, so you'll have less signal.
That answers why interviewing can not only be broken but stay broken: you're studying failures and optimizing for anti-failures, but there are a million ways to fail and a billion incentives to do so.
Better to deeply understand and replicate the few successes.