Last summer I had an interview with a large insurance company for a front end role. The take home test was to produce 5 HTML+CSS+JS components to their existing designs (basically to take a PNG and turn it in to a web component). The components were a hero element, an information card, a notification banner, a list and a working calendar with selectable date ranges across different months. They were all to be made without any CSS or JS frameworks and they needed to be responsive. They suggested this would take about 4 hours.
I read through it and spent a couple of hours on it before abandoning the test and sending them an email to withdraw my application. If I was making those components properly I'd spend about week of solid dev time on them - 1/2 a day on each of the first four components and 3 days on the calendar..
Maybe I'm a terrible dev and very slow, but all that test really told me was that the developers there hack things together without spending time thinking about what they're doing, and on reflection I didn't want to do that.
They told me they would mail me a programming assignment, giving me one week but a skilled dev would really only need 2 hours for that. I asked them "what contract"? And they were befuddled I expected to be paid for a week's worth of work. Anyway, I told them, in a non-committal manner, that I would take a look.
The assignment was to write a Django app (module) related to restaurants. Backend, templates, some light pagination, documentation, and at least 70% test case coverage.
I re-adjusted my priorities and did it at my own pace. I didn't bother sending them the finished work, just treated it as a programing exercise. It struck me that with such requirements it's basically production ready and they can use my work with perhaps two or three small adjustments, even if they reject my application.
You are absolutely correct that this would take a magnitude longer to build from scratch.
> a skilled dev would really only need 2 hours for that
Ah. I think that's the problem.
I suspect the "skilled dev" is keyword for "reuse existing project".
I can absolutely see myself reworking a Django app to support a backend, templates, some light pagination, documentation, and at least 70% test case coverage in 2 hours if I had an existing similar Django app with a backend, templates, some light pagination, documentation, and 90% test case coverage and I needed to rework it.
I think your competition cheated in reporting the total time it actually took them
I can respect that position. However, as a hiring manager I can see their perspective.
That perspective being, "How fast are you at good enough?"
Food for thought in the future. As a hiring manager I would have accepted all 5 done for 50% of all screen types, with developer comments on where you got stuck or what compromises you chose to make.
In my opinion that should be made clear in take-home tests. "You are allowed to cut corners, but please comment on WHY you did so"
> That perspective being, "How fast are you at good enough?"
That's a great question to ask.
However, I now ask you if that's really the question you are indeed asking an engineer who already has a full time job with possibly a hour or longer commute and is attacking your problem at the end of the day or week.
If that's the case, what you are really asking the question "How fast are you at good enough when you are extremely tired and nearly exhausted?"
Either that or you're selecting for candidates who are not challenged enough at work and work at a place that does not fill the rest of their employee's time with busy work to make up for the slack so they have time left over to work on your project.
You have thought of the problem from your viewpoint as an employer but have you viewed it from the viewpoint of an employee who's so frustrated with their current employer that they are willing to put up with hours of looking for job posts, doing background research on companies, filling out forms, following up on applications, answering random trivia questions and random take home assignments at the end of their full work day?
Are you really looking for “engineers” who can half-ass a project from start to finish in 4 hours? That’s the quality bar?
If I were in OP’s shoes and feeling frisky I would point out to the hiring manager that for a real world project like that it would take at least 4 hours to just gather the requirements, write a design doc, and provide an estimate. 4 hours to do design, implement, and test something properly is ridiculous. And if you aren’t focused on doing it properly, then——wait, what company are you hiring manager at?
OP’s hiring company was an insurance company, not an Uber-For-Cats startup. The software he or she is eventually expected to write on the job is supposed to work.
> If the constraint is 4 hours, the manager can't expect nothing but 4h quality.
Yes, but how does the manager estimate that it indeed typically should take 4 hours when the numbers he gets are:
1. From his employees who work at that company, intimately familiar with the problem domain, tools and possibly came up with the problem statement in the first place and solved it in 4 hours after thinking about and working on it with no other distractions
2. From candidates self reporting the time it took to complete the problem severely undereporting the time it took them because they want to impress the employer?
Is 4 hours of focused, uninterrrupted time equal to 8 4 hours of unfocused, interrrupted time?
The trouble is that the determination of "fast enough" is provided by a senior developer who has 2 years of experience with that toolchain, and that's a ridiculous expectation.
Given a weekend, I can come up to speed well enough on an unfamiliar toolchain to start being productive. Given a month, my productivity is going to be high enough to be good enough for any shop that isn't looking for rock stars.
I interview candidates on their fundamental skills and ability to solve problems. If you're reasonably intelligent, can communicate well, and have logical and higher-order thinking skills, I don't care what tools you've used in the past. Many times I've recommended someone who has absolutely zero experience with the tools my team uses.
> The trouble is that the determination of "fast enough" is provided by a senior developer who has 2 years of experience with that toolchain, and that's a ridiculous expectation.
I dont think it's a ridiculous expectation.
People rarely apply for Django jobs if all they know and have been working with is Java and have never coded a Django app.
However, I think the OP is severely biased and blinded by their viewpoint as an employer and have failed to view it from the viewpoint of an employee who's so frustrated with their current employer that they are willing to put up with hours of looking for job posts, doing background research on companies, filling out forms, following up on applications, answering random trivia questions and random take home assignments at the end of their full work day
> I interview candidates on their fundamental skills and ability to solve problems. If you're reasonably intelligent, can communicate well, and have logical and higher-order thinking skills, I don't care what tools you've used in the past. Many times I've recommended someone who has absolutely zero experience with the tools my team uses.
This sounds so refreshing and atypical of the standard interview that I would love to give this a go.
My email is my handle at gmail.com
No obligation. I just want to connect with an individual who thinks deeply about real life problems.
I was once given a take home with unlimited time. They wanted me to give them the best possible solution. They would compare all candidates based off of the best work they could produce.
I went all out and wrote a solution that was point free. No variables. Now it is my default paradigm.
A lot of them haven't even heard of the point free style using function composition let alone function composition.
One of the core tenants they wanted me to ascribe to was to not overdo it with abstractions. The fact that I actually used minimal abstraction (I didn't use the abstraction of object orientation or even variables) was lost on them. Instead the foreignness of function composition and of a program that was simply just pipelines of functions appeared as if I overdid it with abstraction.
Less food for thought but more biases and gut reactions and the same typical stuff you see among all people. I learned that people interview for a stereotype based off of their experience, if you introduce something outside of the stereotype then you're a goner.
> Software developers are notioriously bad at esimating.
While that's true, understand that the numbers companies are quoting are not esimates but rather actual time candidates self reported after finishing an assignment.
I expanded a bit more on this in a sibling comment.
Are these tests supposed to reflect real-world programming? If not then the month view "calendar" widget has just three state variables: current view start, selection start, selection end, and the rest is just calculated from that.
4 hours for everything is a bit harsh, but a basic calendar view shouldn't take more than a few hours using just the today's web tech.
It will certainly take more hours if you try to make it accessible or compatible with too many browser versions, or start adding features like DnD range selection, or start optimizing, etc. But if company expected me to make a fully fledged usable component on a home assignment, that they can just use, I'd send them a quote, lol.
> Are these tests supposed to reflect real-world programming?
That absolutely is the expectation.
These are not tests for basic coding proficiency.
These are not tests to see whether you can build a single component or widget.
These are tests to check if you can build a full suite of components or widgets including a README, tests.
... and candidates are apparently delivering.
I can get a GraphQL/Express server up and running from a frequently used template I built, in 90 minutes will full support for Query, Mutation and Subscription.
Create new users, tasks, updates, votes?
Check Update users, tasks, updates, votes? Check
Get realtime notification of changes to users, tasks, updates, votes? Check
Apparently there are others who can not only do this but also implement a react frontend that puts the UI support for these as well in just 30 minutes more.
Either candidates are reusing templates they have built in previous projects and not reporting the time it took them to build the template when they report the time
Or, they are grossly under-reporting the time it took for them to complete the assignment.
We are doing this to ourselves. We should not deflect the blame.
If we feed garbage data to companies and then companies base their decisions off that, it's absolutely unfair to blame companies for not being realistic
The company was still advertising the role a few months ago. There could be more than one, or maybe they hired someone who has since left, or maybe they haven't found anyone yet.
Companies are trying to minimize the time/money they spend on each interview.
Clarifications cost money.
Hoping a company will engage in a "what really are your requirements? are you trying to test my basic coding competency or do you want to see the best I have got?" will elicit a very vague response, if any.
I think if you want to develop solid components, I don't think the time spend is high at all, on the contrary.
Right decision to withdraw the application in my opinion. I like it if I can gather some points with an assignment, but I would have mentioned their ridiculous time estimation and how they determined the required effort. If that wouldn't lead to a significant alteration of the task, the interview would already be concluded.
You pay quite high prices to have such components developed.
I don’t understand these time limits measured in hours. What practical job skill are they testing? If it’s “we just promised a customer a new UI to be delivered working and tested by end-of-day!” then you dodged a bullet by withdrawing.
Maybe I’m just really lucky, but I’ve never had a grownup job where a new, never before seen engineering project was due in a time period of less than a few days.
It's a home assignment, not an engineering job. It shows expectation on how much time at most you're expected to waste on it to get hired there.
Though to be fair, they should make the limit strict, otherwise they're selecting for people willing to do more for free, if they just then compare results without comparing the actual time spent.
An experienced web developer will have template in his head for various typical problems. So that should not be an issue at all. It's just typing it out for the component at hand.
Asking for using web components will of course limit the pool of candidates having a mental template quite a bit, since it's a fairly recent technology, and even then, most people probably never used the raw API, or if they do, they wrap it a bit, so it will not be something they'll remember right away.
> It's just typing it out for the component at hand.
You and I have very different approaches to design.
Before I type out the actual component, I will write the Enzyme via Jest test cases first to ensure I clearly understand what the actual component should be doing in the first place even if I am beyond sure I have a good grip on the requirements.
Real world experience has taught me that the ability to bang out code matters way less than the ability to ensure and verify that the code actually, in reality, behaves as expected by the customer.
That can be verified by clicking through the result, after you bang out the code. There's certainly nothing that prevents verification later, if you bang out the code first.
Automated verification may save some time (in some cases), but is not the only way to verify the component behavior.
You are not accounting for the third option: cheating.
I think there's a major disconnect in candidates wanting to be treated respectfully and companies that offer take homes
I have a suspicion that companies that offer take homes expect the candidate to cheat in some way or the other and begin with assuming the worst.
... or they are simply going off the data that they have.
I think majority of the candidates cheat by simply severely under-reporting the time it took for them to complete the assignment.
99% of the take home offers I receive are "our candidates typically take 2 hours to complete this assignment".
These are not tests for basic coding proficiency.
These are tests to check if you can build a full stack application - backend AND frontend including a README, tests and majority of candidates are typically taking 2 hours to complete.
I can get a GraphQL/Express server up and running from a frequently used template I built, in 90 minutes will full support for Query, Mutation and Subscription.
Create new users, tasks, updates, votes?
Check Update users, tasks, updates, votes? Check
Get realtime notification of changes to users, tasks, updates, votes? Check
Apparently there are others who can not only do this but also implement a react frontend that puts the UI support for these as well in just 30 minutes more.
Either candidates are reusing templates they have built in previous projects and not reporting the time it took them to build the template when they report the time
Or, they are grossly under-reporting the time it took for them to complete the assignment.
We are doing this to ourselves. We should not deflect the blame.
If we feed garbage data to companies and then companies base their decisions off that, it's absolutely unfair to blame companies for not being realistic
Take home coding tests are fine. I really prefer them, honestly. The problem is when you do the test, write great code, they love it and ask you to come onsite, and then you get white-boarded by a group of people who never even looked at your code or resume and get rejected for not being able to cough up a DFS algorithm on the spot.
I've interviewed several hundred people and gave them a take home test of a simple problem that we had encountered in our work.
We asked them to write a simple API that would return the closest pharmacy, given a latitude and longitude supplied to the API, and gave them a CSV with 10 pharmacies. Language didn't matter, and we never even ran it to see if it actually worked.
During the interview we would have them walk us through the code and explain what each part does and why they did whatever (this makes sure they actually wrote it and understand it).
Then, we would throw curveballs like "what would you need to change in your code if there were 100,000 pharmacies", and "how would you need to change it to return the closest 10 pharmacies instead of the closest 1?"
It's normally really difficult to get into someone's head about HOW they solve problems. I really felt like our method shined a light on their programming abilities. Also, how well they might anticipate future usage by building something smart to begin with and not code themselves into a corner.
I agree with you and there is nothing inherently wrong about a take home test. I do prefer taking home a test and working on a feature and talking about it with the hiring panel over whiteboarding. But the expectations they express about the take home work can still be poorly thought out and make me feel like I'm in the wrong for taking a longer time than they budgeted for me. Or that I'm being secretly tested to see if I will ignore the suggestion and throw myself at this exam to show my dedication to their company.
This joke is a suggestion there is still room for improvement.
Oh and I will add, I hate the criticism that a take home test is "free, unpaid work." So is looking for jobs and taking time out of your schedule to interview on site. It is inevitable that in a job search you put in unpaid time to try to find the best possible work experience. But it would be nice to try to find ways to make it less painful for job seekers and more accurate in assessing developers's real ability to be productive team members.
I've repeated this several times now but a great experience I had was with a small company from Ottawa (they may be reading this).
They offered to pay me for time spent working asynchronously on a task with them that they would potentially be able to use. They gave me a retainer up front and had me bill them based on agreed-upon hourly amount.
Ultimately the offer was based on that work. We ended up pressing pause as they were able to find help more local to them so I didn't join up but all around good experience. And it is certainly nice to be paid when it is work they may potentially use!
In this case I think your assumptions were incorrect.
They already had some remote staff and contacted me out of my expressed desire to work remotely. I was caught up in a critical project and needed more time however they needed the extra set of hands earlier than I could commit to. I'm guessing colocation was an added bonus because it is a bit easier.
I think my largest concern was the sustainability of my rate for them in the long term being a small outfit. No regrets one way or the other. And like I said, they paid my retainer up front and it was definitely to my satisfaction.
> I'm guessing colocation was an added bonus because it is a bit easier.
I would love to hear your thoughts on this:
My opinion is that if there's a preference for a co-located candidate, that simply means their processes are not built to put a remote candidate on equal footing.
Hence the preference.
The fact that they have remote workers does not invalidate the fact that those remote workers are at a disadvantage.
> I think my largest concern was the sustainability of my rate for them in the long term being a small outfit
This is one more red flag: companies that have to work remotely because they have to not because they want to.
There is a world of difference between a company that had to settle for remote work just because they could not afford to hire locally from a company that was built remote first because the founders believed in remote work.
The former will always be on the lookout for local candidates, forever deprioritizing remote workers while the latter don't have any such discrimination in mind.
I am glad it turned out well for you in the short term. Maybe you can follow up on how their remote workers are currently doing.
The distinction between the two though is projects and coding tests are work actually potentially useful to the company, things they could take and use where what you're talking about isn't part of the normal work of an employee.
This seems like some sort of myth that happened to one engineer and now everybody is repeating it. Take homes I’ve done are no more useful to the company than any of the 1hr whiteboard problems. The company is spending time and money (a LOT of money at big companies) to interview you, expect to spend some of your own.
> This seems like some sort of myth that happened to one engineer and now everybody is repeating it. Take homes I’ve done are no more useful to the company than any of the 1hr whiteboard problems
You underestimate the insight that comes with looking at different solutions to the same problem.
I have been showing the value of GraphQL to a few startups for a while now to the point where I can get a GraphQL/Express server up and running from a frequently used template I built, in 90 minutes will full support for Query, Mutation and Subscription.
Create new users, tasks, updates, votes? Check
Check Update users, tasks, updates, votes? Check
Get realtime notification of changes to users, tasks, updates, votes? Check
This new insight has tremendous value to a company that's struggling with a RPC like system hidden behind "REST" endpoints.
At a company I know the R&D team routinely interviews PhD candidates of which an hour is kept aside specifically for the R&D team to come up to speed on the latest research that these PhD candidates read up on.
> The company is spending time and money (a LOT of money at big companies) to interview you, expect to spend some of your own.
I dont think you thought this through.
The company is spending time and money to make sure they don't make a hiring mistake. The time and effort sunk in by the candidate is part of the cost subsidized by the candidate.
An argument can be made that it would be cheaper for the employer to hire a candidate on an at-will basis once the candidate passes a basic skill and design test.
I've only done a few take home tests, but never done one (or given one) where the work product was usable for anything other than evaluating candidate skills.
> Take home coding tests are fine. I really prefer them, honestly
Home coding tests are great at gauging real life performance of a candidate, if done right, but in my limited exposure (I don't have a lot of time to work on coding tests just so I can collect data samples to back my opinion), coding tests are handled extremely poorly, with expectations being set extremely vaguely.
Any realistic assignment requires a dialogue between the consumer and producer.
Companies are trying to minimize the time/money they spend on each interview.
Clarifications cost money.
Hoping a company will engage in a "what really are your requirements? are you trying to test my basic coding competency or do you want to see the best I have got?" will elicit a very vague response, if any.
If a company treats a home coding test as a real assignment and provides the candidate with the resources they need, I see no problem.
Unfortunately from my personal experience and anecdotes from others, it's just not the case.
> get rejected for not being able to cough up a DFS algorithm on the spot
DFS is easy. BFS is easy. So is Coin changing, inverting binary trees and doing back flips.
If you invest enough time and effort into it, you can do it.
I am beginning to prefer DFS, BFS, Coin changing, inverting binary trees or even back flips for interviews.
I have to learn it once and can regurgitate until I forget them.
I pay the heavy cost once upfront and amortize it.
No such thing for a home coding test!
I now have to pay the same cost over and over again, unable to amortize it because no two companies are solving the same problem unless you view it as a meta problem and in that case you might very well end up redesigning Django, RoR or a similar meta framework.
Financially it would be cheaper to become a full time contractor and get paid to make POCs and demos unencumbered by NDAs.
As a candidate a home coding test is extremelyexpensive compared to the traditional DFS, BFS, Coin changing, inverting binary trees or even back flips for interviews.
I'm always shocked by the examples that come up when people talk about when talking about whiteboarding. DFS is a simple, understandable, easily modifiable, and useful algorithm with many real world applications.
Sure, DFS is a simple, understandable, easily modifiable, and useful algorithm with many real world applications.
I have coded DFS many times in my life, both at high school and atleast a dozen times each during my Bachelors and Masters and far more times at work because I do a lot of NLP and traversing a tree is a regular day at work.
I can code DFS with you today, when out at a bar over a few beers.
What I struggle with is to code DFS:
1. with people I don't know, am not comfortable with
2. who are looking at me expectantly, expecting me to fail at it (because that's what happens in most of the interviews they have been part of)
3. with time running out
They only time I was able to perform DFS at an interview was at my first job out of college because just the month before I had prepared for my Data Structure final exam and could code DFS (or any of the HackerRank style algorithms) in my sleep no matter what even if all I had was a chisel and a slab of granite.
I was not upset that I never had to actually code the DFS, invert a binary tree and find the minimum coin change (all the 3 questions I was asked for the job) ever after that - it was all C++/Python/Java CRUD, for 3 years because I had 0 professional experience with CRUD at that point so I would have never got the job in the first place.
In fact the official policy at that place was to always use existing commercial libraries (and purchase them if necessary) because it was a CRUD shop and they had high turnover so any custom library would quickly become a liability.
A coding exercise should, ideally, be a starting point for a follow up discussion.
Then discuss things you would do next given unlimited time etc and justify those points. That's plenty insightful into what a developer prioritises and how they approach software development.
That sounds ideal but oftentimes, intentionally or not, that isn't the expectation communicated. And in some cases I have seen take home coding exercises that are literally projects big enough for weeks of work. That particular small startup had copied their take home exam from a bigger company. They told me "work as far as I can," but it is intimidating and accomplishing a reasonable amount in say half a day still would feel like I had barely scratched the surface of the work they "suggested" I do "as much of as I feel like." It made me feel like an inadequate candidate by feeling like I can't possibly put enough of a dent in that task for it to be meaningful.
Oh and by the way the people asking me for that work were significantly more junior than me. I politely passed on the company and even beginning the take home test.
I have also had a company give me an API to build with sparse instructions. I did my best, I probably should have asked them more questions. I got no response. I followed up weeks later to see if there was more I could do with the coding exam or if I misunderstood it somehow. They didn't have anything else to say about the coding exam but they did invite me in for an in person interview. They then ghosted me and my further follow ups about the status of the hiring process.
The thing about the hiring issues in the industry is they aren't as simple as the solution you offered up. They are systemic and it doesn't feel like anyone is even trying to improve them. And that the changes that do happen are equally but differently absurd in their expectations as the mistakes of the past.
Unless the problem is relatively trivial, it's pretty easy to see job candidates not wanting to turn in a quick hack job. But, if this test is early in the interview process--and it probably is--expecting a day of work (plus all the other prep/time) for a given interview is probably not reasonable. Which is a general problem. Assuming a couple of days to interview with a company might not be totally unreasonable in isolation. It starts to become so if someone with a job is interviewing multiple companies.
That said, there are certainly jobs where some sort of "audition" is clearly reasonable and even necessary. The arts are obvious. But I've held a number of positions where writing (and analysis, presentations, etc.) for external audiences was a big part of my job. You can be sure I want to see a writing sample and maybe even a presentation from a candidate for a similar role. If they already have some writing samples, that may be OK. But, if they don't, they're going to have to create one.
Ghosting candidates gives them a chance to come back with an offer in an indefinite time. 3 years after the interview: "hey, with your interest in this position, would you be initested in making a world a better place on a position of L1 support engineer, 250 miles from home, on a 6 month contract?"
Dearth of feedback regarding take-home stuff I've seen described as a liability issue, which I can kind of see. It really sucks, especially when you really thought you hit all the points, but I get it.
The ghosting phenomenon however is a cancer that can 100% be attributed to milliennials. It's wildly unprofessional behavior and didn't exist in any noticeable capacity before they attained positions of power.
Well, I was ghosted plenty enough by “reputable” companies without millennials being in any position of power... Generally it’s poor HR performance or poor HR process which may (or not) tell a bit how serious the company is about HR overall...
I spent a good amount of time on one of these after a phone interview for a company I was mildly interested in. I’m confident it met all requirements, yet I submitted it and got no response one way or another. I could have put more time into it, but it met requirements and I thought, in the real world, would this company need me to make everything perfect or just good enough? However, in these take home tests where the implementation is so simple that submissions are judged on factors other than code correctness, it becomes an arms race between applicants and who has the most time to jump through hoops. I’d rather do a white board test and if I fail, have a chance to redeem myself with a take home test. In my experience it wasn’t worth the time I put into it. At least it helped me make a decision.
At a previous job we assigned folks a take home test, but paid an honorarium for their time. Then part of the interview was a code review session. Different projects for different experience levels, but never more than 15 hours total (usually more like 2-10).
Seems like the best compromise between flexibility for the employee, seeing actual real world work for the employer and allowing the candidate to shine.
If businesses treated take homes with diligence and treated interviewees respectfully, I wouldn't mind reasonable take home assessments. This means the company is required to at the very least to send a "declined appplication" communication and not ghost.
Ideally, a business would give feedback as well but I understand there are people who quibble into lawsuits instead of integrating constructive criticism to improve themselves.
With the current lack of professionalism and respect in the hiring process industry wide, take homes as is are a good way to waste significant amounts of time when job hunting.
These stories are why I'm so glad to have spent my career as a consultant, where this kind of BS doesn't happen (in my experience, anyway). I wouldn't dream of applying at any of these places for this reason alone.
Fair enough but a consultant is an orthogonal role to an employee.
This thread is about employees being frustrated with the current state of affairs.
A consultant is a business owner.
As a consultant, my deliverables are rarely code. They are strategy documents, reports, plans, proposals, market surveys and video conferences.
As a contractor, my deliverables are mostly code.
Neither of them is similar to an employee's role, although, as a contractor, my deliverables match up with those expected of an employee.
Which is why it seems to me that you are conflating consulting with contracting.
Perhaps you meant to imply you love working as a contractor not having to deal with what employees have to but employees have to "deal" with it because their role requires it.
As a consultant, I deliver lots of code in addition to documents. The two are not mutually exclusive.
It's sad that employees have to deal with BS like "whiteboard exams". Their role "requires" it only because their employers have followed that trend like lemmings.
So, yes I'm very happy to get hired (as a consultant) based on merit and proven experience.
I would not call it "orthogonal" (though that does sound smart in those documents), because it's more akin to a parallel than an orthogonal role.
> Their role "requires" it only because their employers have followed that trend like lemmings
You speak as if the employer is a different entity than employees.
it's a collective. a composition of employees.
The employees bring it upon themselves.
Employees have to deal with BS like whiteboard exams because the employees themselves conduct the whiteboard exams!
The reason, why you are not subject to the rules of the employee is because you are playing a completely different game.
As the employees waste each other's time doing pointless pushups to prove how good of a fit they are at a moving company (after all a physically fit person can move more stuff per hour than one less fit!) you help sign up more customers who need to be moved and make it a more pleasant experience so the same customers remember your client.
It might be that you end up doing a few pushups or even move boxes but the real value is in the above.
While every employee at a moving company is focused on maximizing their push up count, you make your client come ahead in the market by retaining more customers and sign more up. then repeat
Here we go again. As this seems to pop up quite often, it's seems like it's an important point for companies and candidates. I think that there are good take at home coding tests, and there are bad ones, there are good in person coding interviews and there are bad ones.
For me personally, I always liked take at home tests, and we are doing this at my current company. We have a small task with two levels, one to complete quite fast and the second level is if you really want to show off and spend more time on it. We use it as the main technical skills evaluation for the in person interview, because it opens up an actual discussion and we can talk about reasoning, see how candidates take feedback and how they can explain what they did. In the same time, there is another department with us that has a take at home task that I wanted just for laughter to try, it took me a whole weekend and it was insane. So I can see where some people are coming from. In the same time, we also had some cases where we paid the people (who requested it) for the time spent on the task, and we hired one, which was one of the most worst decisions that we made, because he turned out to be one of the most selfish and self absorbed people we hired.
In the same time I've been through some ridiculous situations where I was left alone in a room with no phone allowed, no pc, just write on a paper some SQL queries based on some printed tables from a spreadsheet, or, in another case to write on a flip-chart some code to print the first X prime numbers in front of 3 people. This I also find quite ridiculous, but it was 8-9 years ago.
In the end it's also how you feel as a developer about these things, if you are ok with take at home tests, maybe the company is a good fit for you and that is it, if not, you dodged a bullet.
> We have a small task with two levels, one to complete quite fast and the second level is if you really want to show off and spend more time on it. We use it as the main technical skills evaluation for the in person interview, because it opens up an actual discussion and we can talk about reasoning, see how candidates take feedback and how they can explain what they did
This sounds so refreshing and atypical of the standard take home test that I would love to give this a go.
My email is my handle at gmail.com
No obligation. You don't even have to have a current opening. I want to honestly give this a go only to experience what this feels like. That would be value enough for me.
I guess this has to do a lot with hiring culture. Here in the Netherlands, these kind of tests are rare. You get two or three interviews, which are more about your programming experience and your personal strengths and weaknesses. It is as much about whether you fit the company and its culture. Sometimes you are given a tour through the company or are given the opportunity to talk to some of your potential new colleagues, and usually includes meeting your first responsible manager. If everything goes well, you get a job offer and a one or two month trial period, during which both parties can terminate the contract on any moment. In my life, I only once had to do some coding before a white board. I felt like not doing very well, but they thought it was great. The idea that you can select the best person for a job position by testing is not really based science, because there are so many factors involved that you cannot assess with some tests either performed on spot or at home.
It is usually very easy to criticize and not provide any alternative solutions. How should a company process 1000 resumes with a meaningful effort and find candidates to be called for an on-site interview?
If you think a single on-site interview of a candidate, it usually takes a few hours of multiple people from every segment of the company (mgmt, development, HR). Everybody has to prepare beforehand for some time for these interviews as well. So in the end, every on-site interview is costly for a company.
As an engineer, I dislike attending on-site interviews to hire people because I see it as a time that is not used optimally.
Finally, IMHO, the solution is to provide both standard-track and fast-track interviews where exceptional (on paper) candidates can be called directly for an interview and regular candidates would have multi-stage interviews.
If you send out a 1000 4-hour take home tests as a screen and 100 people took upon it, you just wasted 396 man-hours of other people time. Creating a bunch of frustration, disappointment, and unnecessarily accelerating heat death of the Universe.
Outside of Madoka Magica (2011), it's pretty safe to say that no human activities we undertake today will have any impact on the heat death of the universe whatsoever.
Fast tracks for interviews already exist, and they’re not good for the industry. They usually result in a pool of people shuffling from high paid job to job regardless of their competence, all on the basis of who they know. Making it an open industry standard would be disastrous.
Hiring is absolutely one of the best uses of your time in any reasonable environment.
If we're considering hiring for a position with the same expectations as your position, unless that time you've spent nearly doubles long term efficiency over the hours you spend to fill the position, it's easy to see why it's a good use of your time.
limit the number of interviews and only allow the immediate peers and managers to the interview? the people interviewing should be interested in a good candidate. that's enough.
I live in a major metropolitan area not on the west coast. When I’m looking for a job, I change my LinkedIn status to “actively looking”, call a list of recruiters that I have worked with in the past, and usually have two or three offers in hand within two weeks to a month.
For whatever level I’m at at the time, usually companies pay within a narrow band ($10-$20K). Why should I waste time doing an at home coding test when I can get a job that pays just as much just by answering a few behavioral interview questions and describe my past career history?
I’m no special snowflake. I’m just your average everyday CRUD/SAAS software developer/architect/team lead depending on how the wind is blowing.
I think there's a major disconnect in candidates wanting to be treated respectfully and companies that offer take homes
I have a suspicion that companies that offer take homes expect the candidate to cheat in some way or the other and begin with assuming the worst.
... or they are simply going off the data that they have.
I think majority of the candidates cheat by simply severely under-reporting the time it took for them to complete the assignment.
99% of the take home offers I receive are "our candidates typically take 2 hours to complete this assignment".
These are not tests for basic coding proficiency.
These are tests to check if you can build a full stack application - backend AND frontend including a README, tests and majority of candidates are typically taking 2 hours to complete.
I can get a GraphQL/Express server up and running from a frequently used template I built, in 90 minutes will full support for Query, Mutation and Subscription.
Create new users, tasks, updates, votes? Check
Update users, tasks, updates, votes? Check
Get realtime notification of changes to users, tasks, updates, votes? Check
Apparently there are others who can not only do this but also implement a react frontend that puts the UI support for these as well in just 30 minutes more.
Either candidates are reusing templates they have built in previous projects and not reporting the time it took them to build the template when they report the time
Or, they are grossly under-reporting the time it took for them to complete the assignment.
We are doing this to ourselves. We should not deflect the blame.
If we feed garbage data to companies and then companies base their decisions off that, it's absolutely unfair to blame companies for not being realistic
Would you work for a company that spent 0 time vetting you? There has to be a "right" amount of time for evaluating candidates and the amount of time the candidate is required to give up when looking for a new job is only one side of the equation. Unless you want to work somewhere that has no hiring standards, you should not dismiss the time taken for take-home tests as wasted without evaluating their effectiveness vs other methods.
At Job 1: the interview was a white board interview where he wanted me to write a merge sort. He asked a few questions about my background and had to model a database schema and write a few queries to answer done questions.
The 2nd offer was after an interview where I had no language questions, he asked me about how I would design a development process “if I ruled the world”, methodology about dealing with legacy code and integrating it with a mobile app via APIs, my feelings about testing, what trade offs I had to make based on “business realities”, etc.
The first offer was full time, a shorter commute and paid more.
The second offer was contract to perm, if I translated it to an FTE salary it paid less (I ended up making more since I got paid by the hour), and it was a longer commute. They were both supposedly “senior software engineer” jobs.
I took the second offer. I didn’t realize at the time that they were actually looking for a dev lead. When they bought me on perm they matched the offer of the first job.
Are you really not able to tell someone’s experience if they have been in the industry for awhile just by talking to them? The truth is that most developers are “dark matter developers” doing yet another software as a service CRUD app or a bespoke internal app that will never see the outside of the company.
I effectively did this. No whiteboarding, no code stuff, just a phone call and an onsite with coworkers and boss. Small team, range of technical talent, but a much better fit for me compared to the larger (or smaller) standard advertising companies in the area that run 4-6 hour take home tests followed by a day of whiteboarding. You do not need to haze candidates, but you do need to test for basic ability.
As for vetting the company from my side: there's a bunch of mentoring and refactors that needed to happen with prior code, sure, but nothing game-breakingly broken. I've viewed this as a great mentorship setup, not as a "time to post another N+1 select to dailywtf".
To attack the take-home test instead of defending it:
The entire problem with these things: the more complex the problem, the longer the assignment generally takes. Half-an-hour assignments are trivial. 1 day assignments can range from trivial to absolute bogus. Now do these 10 times and congrats, you just worked a week or more for free, on things you're not even sure can be carried over to the future. Most of these companies don't compensate you for taking the tests and they don't assist you either (time is money, friend). Most of these take-home tests aren't universally applied, either: test A at company X is irrelevant to company Y, so they will give you test B.
If anything, this take-home test is a symptom. A symptom that companies have gotten lackadaisical thanks to their power, and a symptom that the industry as a whole needs to evolve. Don't punish capable devs because the guy you hired yesterday couldn't do FizzBuzz, and pay your talent accordingly (both in money and respect) so you don't have to resort to these practices in the first place. Most of all companies are there to take risk, not to give it back to (potential) employees. They get paid accordingly. If they dump the risk on us, then they should also lose a piece of the pie. Unfortunately, the reverse is happening: they have their cake and they are eating it too.
And a personal anecdote: most of the tests I've come across have nothing to do with the job at hand, and frequently the people able to clear these tests are overqualified for the job. Don't give a junior a take-home test made for a senior.
As time goes by, I'm slowly becoming more and more convinced that the only way to accurately and fairly interview programmers is to give them some real world contract work assignments that match the kind of work you want them to do at the job position, and then see how they perform.
This approach has a lot of upsides:
1. Real work gets done
2. The developer gets fairly compensated for their efforts
3. You get to gauge them objectively, based on their actual results
I have been doing software for a while and there has been no realistic situation when a business needed a problem solved from scratch without an collaboration, discussion or feedback within an hour.
I have been in situations where companies needed a solution within an hour and it almost exclusively involved extreme team work and cooperation.
I made a comment on how it would be cheaper for the employer to hire a candidate on an at-will basis once the candidate passes a basic skill and design test: https://news.ycombinator.com/item?id=22368572
A coding exorcise should only be done after the main interview and given only to finalists. Doing well should give one a huge chance of getting the job. The last few hires, we gave the test to three people and chose the best candidate after talking to all of them. Giving a test before even conducting an interview, a real one not a ten minute phone screen, is just an asshole move by shitty companies.
Your approach doesn't save any time or effort for the hiring company though. It's the in depth interviews that take up so much time.
You may not like it but this is their motivator, not being nice
Sure if they don't know how to interview, they're going to waste their own time and still not get good candidates (or if they don't have much to offer as most companies don't). Our interviews were an initial phone screen of I think half an hour followed by an interview with the team for one hour followed by the take home test followed by another half hour to an hour interview to discuss the code and allow for further candidate questions. All interviews are conducted over a conference call with screen sharing for the code review.
You're right. I don't like it because what you're describing is generally how shitty companies operate. Often they won't even look at the coding test. Those companies deserve to get the worst of the worst candidates that they're optimizing for. They fit into the category above: nothing of value to offer the candidate.
The ideal is for the company and the candidate to have an equal investment in the process.
If I spend two hours coding some toy project, I expect the company to spend two hours talking to me. Either interviewing or constructive feedback relating to their rejection.
A company that wastes 1000 people's hours and then ghosts 990 of them should be embargoed by some kind of a computing and technology service workers' union. (You know... the folks who can herd CaTS.)
If the company feels justified in wasting a lot of your time in order to save a little of theirs, before the interviews even start, imagine what they will do after hiring.
Even better when they give you a project with tests in it, where the tests aren't even able to compile, and they then also want you to write tests which does the exact same thing as the non working ones
What if it's paid? We pay people for interview projects where they have full access to our team and resources. This helps us gain an understanding of how they interact as well as their technical chops, produces value from most candidates, and makes sure the candidate is being compensated for using up their free time to work on this.
You're still using their free time even if they're being compensated. In addition, there's a decent chance that, assuming you're in the same field as a current employer, they may be violating their employment contract by taking money from you. I know people will argue this isn't a big deal and "who will know?" but it seems an awkward situation to create.
This is better for sure, but if you're trying to get someone really good to leave their current job (hint: They always have a job) it's as much the time as money so it still doesn't work well
I once took a certification exam sort of like that. The goal was to make the tests pass, except for one random stage where the test itself was bugged. It was noted in the notes for that section, but not in an obvious way.
I worked at a company for a while that had a pretty good take on this, IMHO.
First, the take home test was relevant to the sort of work one might actually do in a reasonable time frame. (Read a csv input file, hit an api per record in the csv, then merge that info and output an updated csv.) Could theoretically be done via shell script; my submission was ~100 lines of python.
Second, submissions were graded mostly for correctness. Style/structure and all the other stuff were nice but usually weren't deal breakers, iirc.
Third, the on-site included a session where we reviewed the submission with the candidate, asked questions, etc... This was pretty instructive, because we could point out a missed edge case or whatnot and see how they'd modify the code to handle it.
Fourth, names & personal identifiers were scrubbed from submissions, to avoid potential bias.
So, it was a better than average process, IMHO. That said, after reviewing a fair number of coding tests myself, I'd note that there are still a number of downsides to take home tests:
a) even if the instructions specify that one should take less than N hours, it's obvious that some applicants spend considerable time on them. So we'd see solutions of wildly differing quality; some with detailed unit tests, documentation, Dockerfiles, vagrant files (!), build scripts, etc... etc... some with just a single file.
b) Even though the primary criteria was correctness, there was some room for assessment of code quality, and subjective bias of the reviewer was undoubtedly a factor. A solution that missed an edge case but which was well structured would generally be fine, but "well structured" is obviously subjective. Or, although applicants could use "the language of their choice," those selecting, say, Java, were at a disadvantage. Most of us were Python/PHP/Ruby folks and the Java submissions looked generally bloated in comparison, and seemed to be graded more harshly.
Not sure what a perfect solution would look like. To resolve issue a) one could impose a strict time limit or use HackerRank or whatnot, but that would create time pressure and up the anxiety. For b) one could rely on purely objective criteria, though that may create other unforseen problems.
My suspicion is that those who do well on HackerRank have invested a lot of time to be familiar with HackerRank.
Is that a valuable and meaningful longterm strategy and worth the investment for a developer?
If you were given an hour of paid time and asked to choose between:
1. learning something valuable and meaningful that would continue to pay dividends for the rest of your life, or
2. learning to use a website that you will probably use a dozen times or less your entire career
... which choice would you, personally, make?
The actual IDE I use at work is nothing like the HackerRank IDE. I have built up muscle memory that just does not work on the HackerRank IDE.
Have you tried to run a Javascript program within HackerRank? Try out how the Map implementation works and how it's different from the Map implementation available when you use Babel.
Timed tests on HackerRank select for candidates who have trained on HackerRank problems inside the HackerRank IDE.
If that's your perfect candidate don't come complaining that your employees solve HackerRank problems all day at work.
I have been doing software for a while and there has been no realistic situation when a business needed a problem solved from scratch without an collaboration, discussion or feedback within an hour.
I have been in situations where companies needed a solution within an hour and it almost exclusively involved extreme team work and cooperation.
Exactly... but even then, how do you run the test cases?
Download them and run them from PyCharm?
I just don't like HackerRank.
Let me rephrase that.
As I grow in my role, I see writing the type of code that can be tested on HackerRank less and less, so investing time in learning how HackerRank works is absolutely not on my radar.
... and getting a HackerRank solution in within that ticking timer absolutely requires me to be comfortable with the HackerRank UI.
This has really annoyed me recently, I've had interviews with companies that expected me to do a test before even seeing them in person (following a phone interview) and their tasks mentioned I should spend around 8 hours on the test. I think this is definitely unreasonable.
I prefer longer interviews where you're working alongside your potential future colleagues. This way you can also find out if you actually want to work with them or not. Interviews should be a two way affair.
The phone and online coding exercises are called pre-screening. Their goal is not to find out if you're a good programmer, but if you can write code at all. This is to save time and see as many candidates as possible.
Yes, i get the screening process; but asking a candidate to spend 8 hours on implementing a client library for their API at the beginning of the application process is definitely unreasonable. It's not a coding exercise, but a full blown deliverable.
I've given up fighting white boarding. If anything, at least it can be gamed, plus if their expectations are bad or you got bad luck on a topic you didn't study for, at least you get to waste their time as well. I'll take that over wasting only my own time on a take-home test.
i would rather take a coding test than suffer an unknown number of office-visits, talking with people who are not interested in hiring, who are stuck talking with me and who are adamant to stop getting interrupted and get back to work.
as bad as coding interviews can go, they are better than spending more than a day to talk to people who evaluate you on how good you are at talking.
>Or: here's a whiteboard, implement some algorithm you last studied 10 years ago and that you will never use in front of 4 people staring at/judging you. You have 5 minutes. :D
I agree with the fact that this style of testing is not directly measuring the skills relevant to the job.
Say what you want, however, a candidate that does solid on the whiteboard despite the irrelevancy of it, is a good candidate because a whiteboard is one hundred times harder than the take home test and most of the problems that will be thrown at the candidate on the job.
I read through it and spent a couple of hours on it before abandoning the test and sending them an email to withdraw my application. If I was making those components properly I'd spend about week of solid dev time on them - 1/2 a day on each of the first four components and 3 days on the calendar..
Maybe I'm a terrible dev and very slow, but all that test really told me was that the developers there hack things together without spending time thinking about what they're doing, and on reflection I didn't want to do that.