Saville Assessment at the DOP Conference 2019

Great Outcomes: A Modern Perspective on Talent Acquisition

At the British Psychological Society’s Division of Occupational Psychology Annual Conference 2019 we were delighted to present a three-paper symposium discussing our perspective on talent acquisition and assessment.

We challenged our audience of assessment practitioners to reflect on how they add value to their clients by asking them whether they:

  • can robustly evidence the value of their work?
  • know what candidates – the end users of their assessments – want?

We finished by focusing on one specific area of demonstrating value which is receiving ever-increasing attention. We asked our audience to reflect on whether they are looking at candidate engagement in the right way.

Video preview image of Rab's introduction play_circle_outline

Paper 1 - 'Do As We Say, Not As We Do':
Occupational Psychologist and Selection

We began by examining the concept of the return on investment of workplace assessment. Through a literature review we found that hundreds of metrics are currently used to show the value which can be added through psychological assessments at work. We shared the output of this review, focusing on how we analyzed the metrics and clustered them to identify the five areas we believe all assessment practitioners should be using to evidence that they are adding value. They are as follows:

  • Quality – Does the assessment identify the right people, of the right caliber, for the right roles?

  • Engagement – After completing the assessment process, do candidates say positive things about their experience?

  • Efficiency – Do recruiters and other stakeholders (including candidates) spend as little time as was necessary assessing/completing the process?

  • Cost – How cost effective was the assessment process?

  • Diversity and inclusion – Is diversity in the candidate pipeline maintained through each assessment stage?

ROI wheel

We shared a review we conducted of 140 practitioner case studies against this framework and used it to lead a discussion on how, as assessment professionals, we could all do more to demonstrate our value to clients. Whilst many of the case studies reviewed acknowledged the importance of objectivity in an assessment process, we felt there was not the same focus on objectivity when sharing the positive impact these case studies had. We demonstrated how consideration of the five key areas appears to change based on market trends and shared our view that we, as a group of assessment designers, must work harder to provide a more consistently balanced analysis of the impact of our work rather than focus on measurement based on the latest fad.

We went on to explain that there may be ethical concerns about the current practice of assessment practitioners, and explored whether claiming assessments are successful without objective analysis across these five areas could be going against the British Psychological code of ethics and conduct. As an example, if practitioners are not objectively reviewing the impact of their assessments against each of these areas, how can they ensure they are respecting individual and cultural differences, as well as assessing in a scientifically robust and truly effective way?

Video preview image play_circle_outline

Paper 2 - What Do Candidates Really Want?

In the second paper, we moved on to look at online screening processes, often a candidate’s first interaction with an organization. From our experience, online screening processes are designed by recruiters based on what they think their candidates want. But do they really know? We surveyed candidates and recruiters to find out.

We presented the resulting data which included responses from 200 recruiters and 1,029 applicants to get a picture of their respective perceptions of the online screening process.

Our findings showed a disconnect between what candidates say and what recruiters thought they would say. For example, the data suggests that applicants are not looking for a quick fix or a stop gap; 91% of applicants surveyed stated that it is important to them that the online recruitment screening process results in a job that they want to stay in. However, fewer recruiters placed importance on identifying employees who will stay for a long time, with several other factors rated as being important by a greater proportion of the recruiter sample.

We also highlighted the importance to applicants of being selected for a role to which they are well suited. Interestingly, a smaller percentage of recruiters placed importance on identifying the right person for each role. For recruiters, aspects such as minimizing cost per hire and filling all vacancies were more important.

The 17% who placed lower importance on finding those that are best suited to a role could run the risk of increasing their overall costs. Bad hires result in wasted recruitment costs, plus the cost of development / training ultimately needed to re-fill the role with someone more suitable.

Survey table showing order of importance

The role of technology in the online screening process was discussed and data presented on how candidates responded when asked about the device on which they would choose to complete their online assessment. This data was contrasted with what recruiters thought would be the likely choice of candidates. Interestingly, the majority of recruiters surveyed over-estimated how many applicants would want to complete on their mobile phone. The overall results from candidates showed that 57% would choose to complete on a desktop or laptop computer, 32% on a mobile phone and 11% on a tablet. Discussion followed around what the goals should therefore be when considering assessment on different devices.

Data was then presented on the factors that would most likely have a positive effect on whether an applicant chooses to apply to an organization via an online screening process.

We linked these survey findings with the discussion we were sharing in Paper 1. Our takeaway message was that the increased focus on candidate engagement is a positive trend, but it should not come at the expense of measuring an assessment’s effectiveness at delivering quality candidates, through robust validation research.

Paper 3 - Candidates Just Want to Have Fun?
How to Truly Measure Assessee Engagement

In this paper we focused on a specific area of our model which is gaining increasing attention. We went on to question whether, as assessment practitioners, we are looking at candidate engagement in the right way. As had been shown in the previous two papers, engaging the best candidates in the selection process, and in the subsequent job role, is one of the most important success criteria of an assessment process. Therefore, it is critical we understand what is meant by true engagement.

Candidate engagement is being increasingly recognized by assessment developers and assessment users alike as an important consideration. A consistent theme through our symposium is that this is a positive development. However, we argued that there is sometimes an excessive focus on the “fun” aspect of engagement. Whilst certainly a contributing factor, we believe this cannot be the critical meaning of candidate engagement. Indeed, mismatches between supposedly fun assessment processes and the content of a role can be a source of disengagement.

Video preview image play_circle_outline

We highlighted the risk of assessments creating a disconnect between expectations and reality. We pointed to face validity evidence which tells us that it is important for assessment takers to feel they’re doing something relevant to the role – which may not be the impression given when interacting with some assessment content on the market. Assessments may also inadvertently convey the wrong messages about the employer’s workplace culture, or the content of the role, and in these cases there’s an argument that such assessments cannot be truly considered engaging.

Much has been made of the recent phenomenon of “fun” or “enjoyable” assessments. Whilst there is a lot that can be learnt from innovation in this area, we argued that some assessment developers and recruiters incorrectly view how much fun a candidate is having during the process as the key metric of engagement.

We explained that true candidate engagement manifests itself as candidate commitment through the process and, for those who are successful, the subsequent role they enter. It is possible to create – and evidence – engaging assessment processes without sacrificing psychometric rigor or compromising on quality. Using truly engaging assessment processes will mean you identify candidates who will stay, and thrive, in an organization. That is what our research suggests that candidates actually want from a modern talent acquisition process.

Download a full copy of the survey report to find out more

To receive a full copy of our survey report ‘Are You Switching On or Switching Off your Applicants?’, or to find out how Saville Assessment can help you align your assessment process with what applicants really want, please fill in the form below.

    Please send me a copy of the "Are you switching on or switching off your applicants" survey report:

    Email Sent Confirmation

    Thank you for contacting us

    One of our team will get back to you shortly.