Why impressions mislead
At senior level, almost every candidate speaks well. They are used to explaining strategy, answering hard questions and appearing convincing in front of a board. As a result, first impressions tend to be uniformly good, and the differences that really matter are hidden behind a manner of speaking.
We also tend to like people who resemble us, to rate confident people highly, and to forgive whatever does not fit our first impression. This is natural, and precisely because it is natural, it needs to be held in check by structure.
What a competency scorecard is
A scorecard is a short list of the competencies that truly determine success in a given role, with a clear definition for each level of assessment. It is drawn up before candidates meet anyone, based on the briefing, not on whoever happens to turn up.
For a leadership position, four to six competencies are usually enough. Examples are strategic leadership, business results, managing change, building teams and influencing stakeholders. Beyond that, the assessment becomes shallow and everyone looks merely average.
Behavioural evidence: probing what the candidate has actually done
A good question asks the candidate to describe a real event, not to give an opinion. Compare the two questions below.
- How do you usually handle major change in an organisation? The answer to this question almost always sounds good, because it describes an intention.
- Tell me about one major change you led. What was the situation, what did you decide, who resisted, what did you do, and what was the result a year later? This answer describes an event, and an event can be checked.
A trained interviewer keeps probing with follow-up questions: what was your own role there, who could describe this from another point of view, and what would you do differently? If a candidate always speaks in terms of "we", the interviewer's task is to find out the "I".
Marking evidence honestly
Each requirement in the scorecard should carry a simple status:
- Evidenced. A real event was described, the details are consistent, and it can be confirmed through work history or references.
- Needs checking. The candidate states it, but the evidence is not yet strong or has not been verified. Mark it clearly so it is checked before the final decision.
- Not yet seen. There is no evidence, and that is information too.
This separation prevents claims from being treated as facts. It also makes the report to the client more honest, because it shows which parts are solid and which still rest on trust.
A score that cannot be explained is only an opinion with a number in front of it.
The role of tools and assessors
Tools, including artificial intelligence, are useful for reading large numbers of CVs and drafting initial summaries. But assessing leadership requires conversation, observation and human judgement. The final assessment should always rest with someone who can be held accountable, and tools should only help prepare the material.
If additional assessments are used, such as a case simulation or a psychometric assessment, treat them as one source of evidence among others. Their results fill one column of the scorecard. They do not replace it.
Reducing bias in practice
- Draw up the scorecard before meeting any candidate, and do not change the weightings after seeing who is ahead.
- Ask each interviewer to complete their own assessment before any discussion, so that the loudest voice does not decide.
- Use the same core questions for all candidates for the same position.
- Record evidence in concrete sentences, not adjectives. Write "led the closure of two branches within six months", not "decisive".
- Keep the assessment of competence separate from matters unrelated to the job.
An example of one scorecard line
To keep this from staying abstract, here is how one competency might look in a scorecard for a head of operations. Note that each level is defined by observable behaviour.
- Competency: improving operations that are already running.
- Strong evidence: has led a process improvement at a facility with measurable results, and can explain the decisions taken, the obstacles faced, and what they changed when the original plan did not work.
- Moderate evidence: has taken part in a similar improvement, but their leading role was limited or the results cannot be explained in detail.
- Weak evidence: speaks fluently about methods but cannot point to a real event.
With a format like this, two different interviewers tend to reach similar assessments. That is the aim: to make the result depend less on who happens to conduct the interview.
The role of references in strengthening evidence
References are often treated as a formality at the end of the process. In fact they are a strong source of evidence when done properly. Ask the candidate to name people who have worked with them from different angles: a superior, a peer and a direct report. Ask the same questions as in the candidate interview, for example what you rely on most in this person and where they need support.
Respect the candidate's confidentiality. References are contacted only with the candidate's consent, and usually at the final stage, when both sides are serious.
Common mistakes in using a scorecard
- Too many competencies, so that none is assessed in depth.
- Giving scores without written evidence, so the numbers cannot be explained to anyone else.
- Adding up scores mechanically without checking whether a must-have requirement has not been met. One failed must-have cannot be offset by a high score elsewhere.
- Changing an assessment after learning the view of the most senior person in the room.
A note for putting this into practice
Choose one position you are currently searching for. Write down its five competencies, and for each one write a question about a real event and an example of an answer you would count as strong evidence. Use that sheet in your next interview, then compare it with your impression after the meeting. The gap between the two shows how much impression influences your decisions.
