Assessing Analytical Capability
If you’ve read my previous article, Laying the Foundation for Leveraging Big Data in Learning, then you already know how important it is to assess the analytical capabilities of everyone on your learning department’s staff. It’s a foundational piece in laying the groundwork for being able to leverage big data for better learning. But how do you go about actually assessing your staff’s analytical capabilities in a meaningful way that gives you insight about how to focus big data training needs?
It begins with an understanding that just because you have access to great data doesn’t mean that good decisions will naturally follow. This is where the intersection of analytics and decision-making becomes the critical factor.
To assist your quest in assessing analytical capability for enhanced decision-making, the Corporate Executive Board (CEB) has developed an important tool called Insight IQ. It assesses a person’s ability to both find and analyze relevant data to make better decisions. This is critical because, as CEB highlights, “Without big judgment, big data magnifies risk, not opportunity.”
A person’s Insight IQ is a combination of three critical elements: Information attainability, information usefulness, and employee capability. People then tend to fall into one of three major decision-making profiles:
Visceral. At one end of the spectrum are the people who tend to distrust analysis in general, going instead almost exclusively with their “gut.” The risks inherent in this approach include an over-reliance on intuition. An inability to adequately account for highly complex environments, greater susceptibility to personal biases, and the lack of any kind of replicable process. This can all get in the way of achieving “buy-in,” especially as visceral types tend to make unilateral decisions.
Unquestioning Empiricist. At the extreme opposite end of the spectrum are those who tend to over-value analysis, placing it over and above judgment. They often fail to adequately account for situational context. Don’t know what to do with wide variation in data, and fail to question whether the data or any assumptions therein might be wrong.
Informed Skeptic. This is the sweet spot of middle ground between the two extremes. They achieve a more optimal balance between analysis and judgment. While they have strong analytical skills, they’re willing to listen to and consider divergent viewpoints.
Here’s the big shocker: When administered to 5,000 employees in 22 global companies, only 38% of employees and 50% of senior managers fall into that key profile of informed skeptics.
Among employees, the visceral decision makers accounted for 19%, with an average Insight IQ score of 45. Unquestioning empiricists made up 43% of the sample, with an average Insight IQ of 48. The Informed Skeptics, by contrast, comprised 38% of the sample, showing an average Insight IQ of 60.
The implication is that while many companies have a handful of people highly skilled in analytics, this has yet to be dispersed widely throughout organizations.
The way out of this dilemma, as described by three CEB writers extolling the virtues of combining judgment and analysis in the Harvard Business Review, is no small feat. It requires “training workers to increase their data literacy and more efficiently incorporate information into decision making, and giving those workers the right tools.
Employees need to recognize that not all numbers are created equal—some are more reliable than others. They must understand the factors and calculations behind the numbers and learn to think critically about the accuracy, sample sizes, biases, and quality of their data. Even people who took statistics in college could probably use a refresher to help them apply what they learned to their current jobs.”
The CEB study showed that the only way to really address this insight deficit is to have training that engages people in conducting actual analyses to gain clearer understandings of information and how to put it to work in decision-making. Utilizing the CEB Insight IQ assessment is the perfect first step in the process.