Hiring remote software developers: How to spot the cheaters
For the past year, moving to an all-remote workforce has often been positioned as a silver lining to the pandemic. Software engineers, in particular, reported a better work-life balance and a higher level of productivity. With an overwhelming majority of software engineers expressing a preference for remote work, it’s no wonder that more employers are making commitments to expand their remote workforces. Not only will this help companies retain talent, but it also opens up the candidate pool globally, increasing the chances of finding a top-notch engineer. It seems like a win-win situation.
But good intentions need to be paired with knowledge and strategy to have a positive impact. And some employers jumping into remote hiring aren’t aware of the challenges in auditioning their software engineers remotely. Namely, in spotting cheaters.
Huge demand for software developers has yielded sizable salaries – even more so in high-compliance industries like finance. Employers offering generous compensation to attract talented developers are also attracting inadequately qualified developers who are faking ability. And because hiring has gone remote, this is getting harder to spot.
How are software development applicants cheating?
Prior to COVID-19, many companies had engineering applicants take coding skills assessments in person. On-premises testing allowed employers to control the environment and observe the applicant’s process. Now, employers are providing these assessments (and getting observations) remotely, and applicants (almost exclusively at the junior level) are gaming the platforms.
The two most common strategies are plagiarism and identity misrepresentation. In the former, applicants copy and paste code found on sites like Github or they are lifting code from prior assessments administered by the same employer that have been published and/or sold online. (Companies that have only a few variations of a coding challenge will find, with a quick Google search, that prior test-takers have either posted it online or are offering the answers privately. They’ll even sprinkle in some minor differentiations so that it’s harder to catch.) Identity misrepresentation means asking or paying someone else to log in to the test platform and solve the test (or part of it) for the applicant.
Globally, the rate for plagiarism in 2020 was 5.6%, and suspicious connectivity patterns – indicative of session handover to someone else other than the applicant – appear in 6.48% of sessions. We are seeing a slight growth in the percent of sessions with suspicious behaviors, and this growth is visible in both global and financial markets in particular.
Some industries will have higher rates of cheating than others; for example, organizations in the government, education, and non-profit sectors can see up to double the global average for red-flag behavior. The general shortage of HR professionals with deep technical knowledge make practically all employers vulnerable to inefficiencies and the perils of under-qualified tech candidates making it too far into the recruitment funnel. Higher rates of cheating mean that IT professionals need smarter tools to avoid mis-hires.
Addressing this problem needs to be a priority for employers looking to hire remotely on a larger scale or as a permanent practice, because the short- and long-term consequences are always more costly than whatever investments they put into preventative safeguards.
Hiring a person who cheated in the recruitment process is a recipe for disaster, both for the employer and the employee. Job seekers will typically cheat because they lack the qualifications to pass the recruitment process or, sometimes, just lack the confidence that they can succeed. In either case, if the recruitment leads to employment, the nascent working relationship is botched from day one. The lack of qualifications surfaces sooner or later, frequently damaging schedules, reliability, and security of software products and services, not to mention driving business costs up and reputation down.
More alarmingly, common sense and academic research suggest (Peterson et al., 2011; Schneider & Goffin, 2012), says that the lack of integrity has a potential to reoccur on the job, quite possibly leading to security breaches immensely more dangerous than software bugs. Last but not least, it is plainly emotionally difficult for many individuals to grow a healthy relationship towards the employer and the workplace when the relationship started with dishonesty.
What can I do to deal with cheaters when hiring developers?
There is a subtle balancing act in providing an assessment platform that is efficient at sensing fraud, but at the same time provides a good experience for honest test takers. The most successful assessment platforms usually apply a two-pronged approach by mixing and matching fraud mitigation with fraud detection.
Signing the code of honor is an example of graceful and efficient mitigation tactics, rooted in academic research (Ariely, 2007) and confirmed by years of practice. It has been scientifically established that being reminded of moral issues makes an individual less prone to cheat.
It is always wise to protect the platform’s evaluation content. Quality vendors limit the time and number of exposures of the same assessment content, actively monitor scores and pass rates to preempt task depletion and constantly crawl the internet to identify leaked tasks and solutions. Test randomization, a platform feature that enables automated on-the-spot test creation from a set of preconfigured equivalent tasks, is helpful in mitigating cheating, since it’s harder to game a system that is less predictable.
Detection, the second pillar of successful fraud prevention in online recruitment, typically comes in three flavors: plagiarism detection (code similarity checking), detection of identity misrepresentation (ID verification), and detection of suspicious connectivity patterns (IP checking).
High quality source code plagiarism detection is based on comparing a solution with source code accumulated in the platform’s database and scraped from the internet. Application of popular plagiarism detection methods usually does not yield optimal results, because the programming languages have different structures than natural languages and this fact needs to be reflected in the plagiarism detection machinery.
Frequently, cheaters who copy-paste solutions try to confuse the similarity checkers by blindly applying certain code modifications to “apply the makeup” without spoiling the core of the solution (which they usually don’t comprehend). High quality source code similarity checkers are designed to expose such machinations.
It also needs to be noted that in certain cases high degrees of similarity does not necessarily stem from plagiarism. As with each fraud detection, some instances of similarity checking land in the gray area and ideally are flagged for human re-examination. Best-in-class systems reintegrate the outcomes of the manual re-examination and over time learn to minimize the gray-area cases.
ID verification typically involves scanning a passport or ID card with the candidate’s camera in order to verify the face similarity. Providers exist who specialize in identity verification based on these inputs, and it makes sense both for employers and testing platforms to take advantage of such services to limit the cases of identity misrepresentation on the test.
Finally, the monitoring of connectivity patterns for changes of the test taker’s IP address, in-test device switching, switching of IP address to addresses previously associated with fraud, patterns suggesting session handover, and so on, prove to be a good source of signal on fraud and best testing platform flag their occurrences.
As with every fraud detection technology, the platforms are not able to tell cheaters from non-cheaters with certainty, they merely help to assess the probability of fraud being committed. It is important to remember that under unusual circumstances a machine may flag an honest but unlucky test taker as a potential fraudster.
An efficient screening tactic giving a decent benefit of doubt assumes that if a candidate’s session was flagged with a fraud warning, the first follow-up interview should begin with an in-depth discussion of the solution submitted in the flagged session, ideally requesting high-level explanation of the solution structure and what-if exploration of problem variants.
When is it time to advance my remote hiring practices?
There are plenty of reasons for not doing it: companies thinking they are too small, are not experiencing fraud, thinking this will never be an issue them, believing most of the applicants to be honest, or simply trusting people. These ostensible reasons may have merits, but they may also create a false sense of security. Recruitment fraud, just like credit card fraud, is committed by a fraction of the population. This does not mean it can just be ignored, especially seeing that the numbers are not reassuring (your mileage may vary but give or take 10%).
The more your organization chooses to ignore fraud, the more vulnerable it will become, because fraudsters will sniff your leniency and… apply. Letting a fraudster in can be draining, financially and emotionally, and the smaller the organization, the bigger the impact. Lastly, the honest hardworking majority deserves the job market in which fraud is simply not tolerated.