Our research for the Careers and Enterprise Company informed the development of the Future Skills Questionnaire – a tool used in schools to measure student progress in the knowledge and skills needed to move on from secondary school and set them up for a positive career.
The challenge
The Careers and Enterprise Company had already consulted widely to develop an updated version of the Future Skills Questionnaire, taking into account feedback from a wide range of stakeholders. The final stage was to carry out cognitive testing with a range of young people to see how well the questionnaire worked in practice.
What is cognitive testing?
Cognitive testing is a technique used to test and improve survey questions. During a cognitive interview, participants are asked about their thought processes when answering the questions. This indicates how participants are engaging with and interpreting the questions, and can identify any questions which are not being understood in the way they were intended.
The approach
Our team recruited a sample of 40 young people aged between 11-18 from a range of genders, ethnicities, socioeconomic groups and with a variety of future aspirations. We also included students with SEND requirements.
Over the course of a week, we carried out intensive cognitive testing via Zoom with these young people, exploring their answers and probing into their understanding of the questions. After each round of interviewing, we made changes to the questions that weren’t performing well to iteratively develop our recommendations and make sure that any changes we suggested had been tested and worked better than the original wording.
The insight
Our research provided recommendations for changing some of the wording of the Future Skills Questionnaire. These recommendations have since been adopted by the Careers and Enterprise Company. They include:
Simplifying some language to be more easily understood, particularly when talking about skills development. For example, when asking about ‘creativity’ many young people immediately think about art or other ‘creative’ subjects, rather than interpreting the skill more broadly.
Changing references to ‘local area’ – our research found that younger students in particular had a very narrow view of their local area, thinking only of their immediate neighbourhood or corner shop. This wasn’t very helpful when trying to capture broad awareness of career opportunities.
Removing references to networking, which was not a concept widely understood by those still at school. We instead asked about ‘making contact with people who work in jobs that interest you’.
You can read the full report produced by the Careers and Enterprise Company here.
Kommentare