Leveraging machine learning to improve recruitment processes

The challenge

Business Need

RED Commerce is a successful global recruitment agency focusing on SAP and complementary technology roles. To fulfil their customers’ vacancies, they use their CV database and effective shortlisting process. ​They were looking to grow their business and scale up without having to introduce and involve many new recruiters.

Finding a candidate with matching skills and experience is not sufficient to fill a vacancy. A placeable candidate has to be reachable by the recruiter, be interested in job offers, and available when needed. In a standard candidate search process, the recruiter usually focuses on one of the candidates’ characteristics (for example, skills match), sorts the candidates according to that aspect and examines each candidate in detail before deciding to call them. As many of the candidate’s other important features are often ignored, reaching good and placeable candidates can prove time-consuming.

RED decided to address the abovementioned challenges with a recommendation system that looks at the candidates’ multiple relevant features simultaneously and ranks them accordingly – and objectively. This type of unbiased and multifaceted ranking system would aim to allow the recruiters to reach the right candidates quicker, and to reduce the overall costs of the recruitment process.

The solution

Project Details

Scarlet, the recommendation system, consists of two components: “deterministic” and “stochastic”. The deterministic component is responsible for the initial selection of candidates based on vacancy tags. Scarlet uses both manually entered tags as well as tags automatically derived from the vacancy’s title and description.

In order to process vacancy descriptions, the system could not simply rely on finding keywords that correspond to skills found in the skills tree. For example, ‘CO’ could refer to either the ‘Controlling SAP’ module or to ‘Colorado’, depending on the context. A ‘Change Manager’ is a specific role, potentially different from a more generic managerial role which also includes Project Managers. Additionally, job descriptions often contain important information regarding required language skills (e.g. “English and German or Russian”).

To render the system’s functionalities more sophisticated, Objectivity’s team built a module that extracts these logical connections between language names and uses them for building appropriate candidate queries. Thus, for the natural language processing of vacancy descriptions, an augmented keyword-based system was built that addresses the most common issues with the basic keyword-based system.

The initial long list of candidates is passed to Scarlet’s stochastic component, which consists mainly of a hybrid neural network that combines metric learning with a deep neural network into a single model. The metric learning component learned the metric between vacancy and candidate tags on historical data. The inclusion of metric learning increased the performance of the deep neural network compared to a simple dot-product-based similarity measure. Other features, such as candidate availability, are combined with metric learning output. The entire network returns a single number that represents the placement probability for each considered candidate.

During data modelling, Objectivity’s team discovered a typical scenario for real-world datasets: human behaviour deviates significantly from the idealised processes designed by business specialists. Thus, in order to recommend the best candidates, it is not possible to simply rely on a neural network that predicts placements from historical data. Instead, for the final candidate ranking, business logic was combined with neural network-based placement predictions. Thus, the final candidate ranking reflected both the most important aspects of the business process as well as placement predictions derived from historical data.

The results

Business Benefits

Scarlet provides candidate recommendations through the REST API. The recommendation system is queried by Mercury xRM (the system used for recruitment process management) and the returned results are added straight to vacancies.

The following components were built:

• Augmented keyword-based natural language processing module
• Logical tree that facilitates development of correct logic for candidate queries​
• Deep neural network/metric learning hybrid to select candidates most likely to be placed and most likely to be engaged in the recruitment process​
• Fully automated CI/CD triggered by merge/commit to master branch​
• Developed on Azure DevOps, deployed as Azure Function with REST API

Key Achievements

RED’s new recommendation system, Scarlet, has enabled the company’s recruiters to analyse CVs and find the most fitting vacancy candidates more precisely and objectively – and in greater volumes.

Human beings tend to make assessments and decisions in a linear manner – as such, recruiters have a natural tendency to analyse a long list of candidates according to one or two chosen criteria. This type of analysis helps them to generate a shorter list of potential candidates, and the decision-making cycle continues in this manner until a candidate is finally chosen.

Scarlet, on the other hand, is able to assess candidate’s skills and characteristics more holistically – looking at the big picture and generating recommendations in a completely unbiased manner. This also means that the generated list of suitable candidates most closely fits the actual job requirements, contributing to more satisfied clients as the employees they engage are truly up to the task.

With Scarlet, RED’s recruiters can successfully match candidates with available vacancies much quicker than before – this is because the automated system is more time-efficient than manual work by its very design. Additionally, prior to introducing Scarlet, certain more complicated and difficult-to-fill vacancies would remain unfilled, because it was quite challenging to find a truly suitable candidate. Now, thanks to the system’s holistic skills assessment capabilities, even the more demanding vacancies can be filled with suitable candidates, enabling RED to meet more of their clients’ hiring requests.

Another key business benefit that RED was able to manifest as a result of the project concerns future innovation. Having built and implemented Scarlet has brought to light an exciting growth opportunity – the possibility to build a new and inexpensive system on top of Scarlet, which would help to further eliminate the need for manual work. Such a system would be able to conduct the recruitment process without engaging recruiters.

RED’s new system, Scarlet, has enabled the company to leverage the latest automation technologies to render their business more time and cost optimised. It has also allowed them to better support their recruiters’ daily work and meet more of their clients’ needs.