I worked at a company for a while that had a pretty good take on this, IMHO.
First, the take home test was relevant to the sort of work one might actually do in a reasonable time frame. (Read a csv input file, hit an api per record in the csv, then merge that info and output an updated csv.) Could theoretically be done via shell script; my submission was ~100 lines of python.
Second, submissions were graded mostly for correctness. Style/structure and all the other stuff were nice but usually weren't deal breakers, iirc.
Third, the on-site included a session where we reviewed the submission with the candidate, asked questions, etc... This was pretty instructive, because we could point out a missed edge case or whatnot and see how they'd modify the code to handle it.
Fourth, names & personal identifiers were scrubbed from submissions, to avoid potential bias.
So, it was a better than average process, IMHO. That said, after reviewing a fair number of coding tests myself, I'd note that there are still a number of downsides to take home tests:
a) even if the instructions specify that one should take less than N hours, it's obvious that some applicants spend considerable time on them. So we'd see solutions of wildly differing quality; some with detailed unit tests, documentation, Dockerfiles, vagrant files (!), build scripts, etc... etc... some with just a single file.
b) Even though the primary criteria was correctness, there was some room for assessment of code quality, and subjective bias of the reviewer was undoubtedly a factor. A solution that missed an edge case but which was well structured would generally be fine, but "well structured" is obviously subjective. Or, although applicants could use "the language of their choice," those selecting, say, Java, were at a disadvantage. Most of us were Python/PHP/Ruby folks and the Java submissions looked generally bloated in comparison, and seemed to be graded more harshly.
Not sure what a perfect solution would look like. To resolve issue a) one could impose a strict time limit or use HackerRank or whatnot, but that would create time pressure and up the anxiety. For b) one could rely on purely objective criteria, though that may create other unforseen problems.
My suspicion is that those who do well on HackerRank have invested a lot of time to be familiar with HackerRank.
Is that a valuable and meaningful longterm strategy and worth the investment for a developer?
If you were given an hour of paid time and asked to choose between:
1. learning something valuable and meaningful that would continue to pay dividends for the rest of your life, or
2. learning to use a website that you will probably use a dozen times or less your entire career
... which choice would you, personally, make?
The actual IDE I use at work is nothing like the HackerRank IDE. I have built up muscle memory that just does not work on the HackerRank IDE.
Have you tried to run a Javascript program within HackerRank? Try out how the Map implementation works and how it's different from the Map implementation available when you use Babel.
Timed tests on HackerRank select for candidates who have trained on HackerRank problems inside the HackerRank IDE.
If that's your perfect candidate don't come complaining that your employees solve HackerRank problems all day at work.
I have been doing software for a while and there has been no realistic situation when a business needed a problem solved from scratch without an collaboration, discussion or feedback within an hour.
I have been in situations where companies needed a solution within an hour and it almost exclusively involved extreme team work and cooperation.
Exactly... but even then, how do you run the test cases?
Download them and run them from PyCharm?
I just don't like HackerRank.
Let me rephrase that.
As I grow in my role, I see writing the type of code that can be tested on HackerRank less and less, so investing time in learning how HackerRank works is absolutely not on my radar.
... and getting a HackerRank solution in within that ticking timer absolutely requires me to be comfortable with the HackerRank UI.
First, the take home test was relevant to the sort of work one might actually do in a reasonable time frame. (Read a csv input file, hit an api per record in the csv, then merge that info and output an updated csv.) Could theoretically be done via shell script; my submission was ~100 lines of python.
Second, submissions were graded mostly for correctness. Style/structure and all the other stuff were nice but usually weren't deal breakers, iirc.
Third, the on-site included a session where we reviewed the submission with the candidate, asked questions, etc... This was pretty instructive, because we could point out a missed edge case or whatnot and see how they'd modify the code to handle it.
Fourth, names & personal identifiers were scrubbed from submissions, to avoid potential bias.
So, it was a better than average process, IMHO. That said, after reviewing a fair number of coding tests myself, I'd note that there are still a number of downsides to take home tests:
a) even if the instructions specify that one should take less than N hours, it's obvious that some applicants spend considerable time on them. So we'd see solutions of wildly differing quality; some with detailed unit tests, documentation, Dockerfiles, vagrant files (!), build scripts, etc... etc... some with just a single file.
b) Even though the primary criteria was correctness, there was some room for assessment of code quality, and subjective bias of the reviewer was undoubtedly a factor. A solution that missed an edge case but which was well structured would generally be fine, but "well structured" is obviously subjective. Or, although applicants could use "the language of their choice," those selecting, say, Java, were at a disadvantage. Most of us were Python/PHP/Ruby folks and the Java submissions looked generally bloated in comparison, and seemed to be graded more harshly.
Not sure what a perfect solution would look like. To resolve issue a) one could impose a strict time limit or use HackerRank or whatnot, but that would create time pressure and up the anxiety. For b) one could rely on purely objective criteria, though that may create other unforseen problems.