Article · 6 minute read
Lessons Learnt from the SIOP Conference:
Don’t be afraid to go ‘back to basics’ with Talent Assessment - define clearly, measure effectively
By Simon Jayne– 15th May 2023
Two colleagues and I recently had the opportunity to attend the prestigious Society of Industrial and Organizational Psychologists (SIOP) annual conference in Boston, Massachusetts. It is the premier conference for psychologists from around the world to gather, network, learn, present research, discuss, debate and socialize.
We were exhibiting our products in the main exhibition hall, trying to catch the eye of conference delegates, mainly I-O (Industrial-Organisational) Psychologists, and when the crowds abated, were able to attend a number of sessions. We were fascinated by the range of subject areas, varying from the technical (concerns about the correction of error in meta-analytic research into the validity of selection methods) to the practical (ways of measuring team effectiveness and performance over extended periods of time), and including Universities considering how to use in-house I-O Psychologists to assess candidates applying for Master and PhD programs… to study I-O Psychology!
The conference provided me with a valuable reminder that I-O Psychology has over 100 years’ worth of research into personnel selection practices and much is known about the effectiveness of different assessment methods at predicting workplace performance. The challenge is to apply this expertise and knowledge to the process of assessing candidates applying for roles in organizations, large and small, as well as for applicants wishing to further their academic studies. It is often the case that recruitment decisions are made by subject matter experts in their chosen field who may not be well versed in the field of human assessment.
The main takeaway from the session on assessment of applicants to Masters and PhD programs for us was that even in the most intellectually high-powered and well-regarded academic institutions, what could be regarded as silo-thinking or a failure to utilize in-house expertise and knowledge can, and does, occur on a regular basis. This is not to judge or blame the institutions because this can happen in any organization.
I reflected on the organizations which I come across who are sometimes tempted (and give into that temptation) to rely on the subjective judgment of hiring managers who ‘know a great candidate when they see them’ and subscribe to the notion that in sales roles ‘a great smile and firm handshake is all you need to succeed’. Because, of course, it saves time and money to rely on gut-feel and intuition, and allows hiring managers to surround themselves with people like them, which can’t be bad, can it?
Perhaps what was highlighted in the session we attended was that a ‘back to basics’ approach could be extremely beneficial for many organizations, be they universities or otherwise, from a talent acquisition perspective, and they can benefit from the knowledge and expertise gained by utilizing sound I-O psychology practice.
Good practice leads to success: Define requirements well, select relevant assessments, measure candidates effectively.
Developing a full understanding of the demands of the role as well as the personal qualities and characteristics needed is the key starting point. This doesn’t have to be highly time-consuming, resource-hungry and expensive. Using off-the-shelf behavioral or competency frameworks can be extremely efficient; you can then utilize card sorts or short online job profiling questionnaires to gather input on the importance of elements of the framework from key stakeholders, including behaviors and relevant skill areas.
Once a clear understanding of requirements has been achieved, the choice of identifying the most suitable assessment methods is much easier, selecting those which best measure the criteria identified. Which assessment methods to consider (and rule out) should be based on a clear understanding of the psychometric properties of those methods, including reliability, validity, utility and fairness. Having talent acquisition professionals attend training in the use of assessments provides a solid foundation on which to select and use the most appropriate methods.
The validity of cognitive measures, including aptitude tests, has long been established as being among the best predictors of work performance and training performance, especially for roles in which there is a high requirement for problem solving and evaluation. Consideration of non-cognitive measures such as scenario-based assessments, e.g Situational Judgment Tests, as well as self-report personality and working styles questionnaires should be made, as well-designed and researched questionnaires possess high levels of predictive power in forecasting job success.
Combine assessments where appropriate to measure more of the whole person
Where high-volume sifting and screening of candidates is needed (e.g. apprentice applicants) the combination of short, efficient cognitive measures, scenario-based assessments and short behavioral questionnaires has proven highly effective at identifying the most suitable candidates for the target roles. Keeping the candidate experience uppermost in mind, using work-relevant assessments, particularly those which give candidates a realistic preview of the role and an opportunity to demonstrate their skills and abilities, as well as short assessment completion times, is to be recommended.
Where appropriate, the use of work samples and multi-method assessment centers may be appropriate at the latter stages of selection process to ‘select in’ candidates, with the caveat that activities such as role plays, group exercises and the like should be used sparingly, as their validity tends not to be high and the scope for subjectivity and bias can creep in.
In essence, the goal is to utilize the most predictive assessment methods as early in the selection process as possible and avoid the use of subjective approaches which, although may be appealing, can lead to ineffective and possibly biased decision making. You can find out more about the approach Teledyne Technologies implemented for their apprenticeship recruitment process in our Case Study here.
Using the knowledge gained by 100 years of I-O psychology research and practice to construct a fair, objective, and effective selection process would seem to be the most sensible approach and avoid relying entirely on the judgment and intuition of hiring managers; however eminent they may be in their field of expertise, they may not be expert in the assessment of humans!
Find Out More
Our expert consultants can work closely with you to design an assessment process that really works for you.
About the Author
Simon Jayne
Simon is a Managing Consultant at Saville Assessment and has supported many organizations in deploying objective assessment for the purposes of selection and development both in the UK and internationally.
You can connect with Simon on LinkedIn here.
The views and opinions expressed in this blog post do not necessarily reflect those of Saville Assessment and its employees.