John Cleave on Overcoming Learning Analytics Hurdles in L&D
Learning expert John Cleave identifies key barriers to effective learning analytics and offers practical strategies for measuring training impact
John Cleave, an expert with a PhD in Computer Science and over 30 years in learning design, explains why many learning and development teams fail to use analytics effectively. He argues that the failure to measure impact represents a significant loss for the industry.
Cleave, in a Q&A with eLearning Industry, cites five primary reasons for the lack of widespread analytics adoption. These include the effort and cost of creating skill assessments, a rigid waterfall development process that stops after deployment, fear of negative results reflecting poorly on L&D leaders, a lack of confidence in evaluation skills, and the inherent difficulty of accurately assessing skills.
Measuring Real-World Learning Impact
Gathering data on behavioral change and job performance is notoriously difficult. Cleave suggests a workaround. He advises identifying key performance indicators within the training's learning experience itself.
For sales training, indicators could include asking good questions or recognizing customer needs. Data on these indicators can then be captured through simulations or case studies embedded in the course. This in-situ data, Cleave contends, often provides a better guess at on-the-job ability than manager reports or self-assessments.
Common Analytics Mistakes and LMS Use
The biggest mistake, according to Cleave, is not doing analytics at all. The second is measuring the wrong thing. He warns that many assessments focus only on information recall because they are easy to create, especially with AI. This data may have no correlation to real-world performance, offering no predictive value.
Despite limitations, learning management systems do provide useful analytics. Cleave outlines several actionable data points.
The Role of AI in Analytics
Artificial intelligence is changing the field. Cleave states that AI feeds on data, making learning analytics invaluable. AI can use this data to personalize training content for individual learners based on their struggles and mastery.
It can also evaluate learner work products, like reports, against a rubric. Also, AI excels at pattern matching. Cleave's team has fed large volumes of xAPI statements into AI to build statistics, uncover behavioral patterns, and identify common mistakes.
Convincing Leadership and Future Trends
To convince a resistant supervisor, Cleave advises probing for the root cause, often fear of negative results. He recommends framing analytics as a formative, ongoing process to improve course design, not as a final judgment. A pilot program's data can be used to make adjustments before full rollout.
Cleave believes the future will see more routine and powerful analytics. AI will make it easier to generate and analyze data from rich sources like simulations. This increased use, he concludes, will ultimately help L&D professionals create better learning experiences.
