The Learnability of Your eLearning Courses

There are lots of different ways to evaluate the effectiveness of your eLearning efforts, which is essential to making sure your learning department can document robust return on investment (ROI) and stay in the good graces of your company’s upper management and leadership. While evaluation can run the gamut from simple to complex, one concept that has appeared recently is a dimension being called learnability. I think it’s worth taking a closer look at it to see if it’s something you should work into your eLearning evaluation process.

Content developer and eLearning design company EI Design has come up with a framework that can be used “to predict, measure, and validate the learnability of online courses.” Here’s how the company used its new framework to evaluate the learnability of one of it existing online courses about instructional design. They applied the following categories of learnability metrics to the evaluation process learners engaged in after taking the course:

• Task performance.
• Interface usage.
• Content and cognition.
• Feedback on design elements.
• Interface and navigation help.
• Usability.

The feedback received from learners made a real difference to the course, allowing its creators to make changes that reflected higher learnability for the module. For example, the photos below show the before and after of changing how the module begins. Rather than a lengthy video and welcome screen before getting to the module’s introduction, the elements are combined to more quickly introduce it:

Another before/after example includes changes made to the course’s overview, objectives, and “coming up next” elements. Previously, these were all separate screens users had to go through, which many registered as unnecessary delays. By combining all these elements into one screen and also making it interactive, learnability was enhanced:

In the next example below, there was a screen that was interactive, but the interactivity was dependent on users clicking the people pictured. Unfortunately, many users assumed the people pictured were just part of the background image and missed the interactivity entirely. The improvement was by making the interactivity based on a clearer contextual icon click that was much more intuitive for learners:

The company formed a task force that spent months sifting through a huge array of internal and external sources to identify all the issues, feedback, guidelines, best practices, user acceptance test results, and so forth that could be classified into various factors affecting learnability.

From there the factors were operationalized by creating multiple parameters to measure each factor. These parameters were further refined and organized into the metrics previously mentioned. The metrics are specifically and directly related to the goals and scope of learnability and are aggregated into an overall cumulative learnability index.

The only thing in this process I take issue with is the notion that it is “predictive.” The framework clearly allows for measuring and validating learnability, but you can only do this by having learners take the course and provide their evaluative feedback. The process can do this through real-time surveys, which is great. But to call that predictive is wrong. What it’s doing is allowing you to test learnability, not predict it, although to the extent that over time you get a feel for what kinds of designs result in better learnability, incorporating those into your design of new courses would predict better learnability. However, there is probably going to be enough unique variation from course to course (and learner to learner) to thwart such predictions. It’s better to think of it as a way of testing for learnability and then making the requisite changes to enhance learnability.

I should also note this isn’t a framework you can just pick up and use. It’s a proprietary development of the EI Design, so of course the company wants you to hire them to help you create customized eLearning solutions, which would include their learnability framework. It can also function as an analytics plug-in to their online review and collaboration tool to optimize eLearning course development called eBridge. They’re definitely onto something here, but it will be up to you whether or not it makes sense to engage the company further to benefit from its learnability framework.