eLearning Evaluation: Kirkpatrick’s Four-Level Model
I’ve written about how important assessment is for eLearning (or any training, development, or learning effort for that matter), and this is one area where it pays to get back to basics. There’s more to evaluating eLearning than just reflecting on how it went in a general sense. If you want to really do a serious evaluation of an eLearning module or course, then you need a solid framework, and there is none more solid than the classic Kirkpatrick Four-Level Evaluation Model. This article will summarize the four different levels, and then there will be at least an article devoted to each level (maybe more) so you can drill down into the details of what to do.
Donald Kirkpatrick passed away on May 9, 2014 at the ripe old age of 90. He had plenty of time to perfect his life’s work, which is the evaluation model you’re about to explore. He was a past president of the Association for Talent Development (ATD, but back when it was ASTD, the American Society for Training and Development) and a Professor Emeritus at the University of Wisconsin’s Management Institute. He left a powerful legacy in the form of a model for evaluating training and/or eLearning – one with which every learning professional should be deeply familiar.
Download the free whitepaper How You Can Assess The Effectiveness of Your Training – Kirkpatrick Model
He developed this four-level model in his dissertation, then introduced it to a wider audience in a series of four articles appearing in the Journal for the American Society for Training and Development (which is now called TD) from November 1958 through February 1959. In 1975 he collected together all the articles he’d written on the topic and published it as Evaluating Training Programs: A Collection of Articles from the Journal of the American Society for Training and Development. Then in 1998 he tweaked the model a bit more and published his seminal work, Evaluating Training Programs: The Four Levels (Berrett-Koehler Publishers), and the third edition came out in 2006. Here’s a snapshot of the four levels:
Level 1: Reaction
This is where you find out how the participants felt about the eLearning course or module they just experienced. This is a key moment to capture some of the most important feedback you can get from the people you most need to hear it from – your audience. What worked really well? What bombed? What could have made it better? Those are just a few questions. In the article devoted to drilling down into reaction, I’ll share plenty of ideas for how to get the most from a Level 1 eLearning evaluation.
Level 2: Learning
Level two is where you find out if your students actually learned anything. Has their knowledge of the topic increased because of the eLearning course or module? The basis for this evaluation should be whatever specific learning objectives you had in mind for the module or course (you did have specific learning objectives in mind, right?). There are lots of ways to measure learning, and I’ll give you some great ideas for that in an article devoted to Level 2 eLearning evaluation.
Level 3: Behavior
Nearly all worthwhile eLearning in the workplace has the ultimate goal of changing people’s behaviors in some way. This is about how the knowledge they’ve received is being applied (or not) in the their daily work lives. This where some logic to the levels makes sense in terms of sequencing. If you measure behavior and don’t find a change, that doesn’t mean your students didn’t learn anything or that the eLearning was a complete failure. For one thing, you’d want to make sure that the behaviors you want to see are even possible – sometimes conditions simply aren’t favorable, such as a reluctant supervisor.
Level 4: Results
At this fourth and final level is where you try to figure out the real impact of the eLearning in terms of results that matter to the business or organization. This is where you’d want to make sure that your eLearning goals are aligned with organizational goals.
This was just a quick introduction to a classic evaluation model that, although originally designed for traditional instructor-led trainings many decades ago, is the right framework to use when evaluating eLearning nearly 60 years later. There will be four more articles in this mini-series, one for each of Kirkpatrick’s four levels – may he rest in peace.