The Death of the Multiple-Choice Quiz

We often see learning tech companies commenting about 'skills tags' in their platforms. Random-but-provocative question:

If 99% of the content in your LMS is a class with a quiz at the end, is that truly 'skills data?'

If we pause to think about it, the answer is actually 'No.' It's knowledge data. It's an indicator of the knowledge that a learner has 5-10 minutes after they just watched your content. From a data science perspective, the R-squared of this data has a low predictive value of future employee performance. In our own Bright research, we've found that only 20% of the people who pass a multiple choice assessment can actually demonstrate the skills associated with that assessment. That's a huge gap!

There's a more valuable alternative, and it poses interesting questions for what the 'right' use of a multiple choice quiz should be in the future of learning: NLP-powered simulations.

Through these experiences, learners have to write or speak in wide-opened scenarios. Have a conversation. Answer a question. Make a decision about an insurance claim, or whatever the use case may be. In addition to being a more engaging experience, wide-open NLP-powered simulations have the advantage of being better from a learning science perspective as well.

Multiple-choice based assessments or - even - ‘choose your own adventure’ simulation experiences trigger a different part of the brain than wide-open experiences through a principle called ‘priming.’

When a learner sees 3 multiple choice options and a prompt for ‘Which one would you choose?’, they don’t behave as they would in real life. The skills demonstrated in multiple choice assessments are more akin to probability and rationalization skills. Comparing items in a short list asks the brain ‘which option is most likely to get me a good score?’ instead of ‘what would you actually do?’

There’s still a place for light multiple choice assessments in the skill building process. Bright still offers the option to include these types of experiences in our LXP. But as learning technology increasingly makes wide open interactions more possible, we predict that this mainstay of corporate learning will take a backseat. Want to learn more about the alternative? Reach out!