Why Does L&D Measurement Remain So Difficult?

Why does L&D measurement remain so difficult? Put a different way, why hasn’t the profession made greater progress over the last ten years? ATD research shows clear progress was made from 2001 -2009 in the percentage using higher-level program evaluations (3, 4, and 5), but little has changed since then. We continue to hear from a majority of colleagues how difficult measurement is.

Many explanations have been offered, however, the most common is that practitioners lack the knowledge. They don’t know what to measure, how to measure, and what to do with the measures once they have them. For program evaluation, though, this is hard to understand because there are so many books written and workshops on the subject.

At a broader level, however, I think the profession has lacked a comprehensive framework that addresses how and what to measure for all the reasons to measure—not just program evaluation. We know that most measurement and reporting activity is focused on informing and monitoring, not on program evaluation. Just think of all the scorecards and dashboards you use; these are not for program evaluation. And there has been no guidance whatsoever on reporting and on how to choose the most appropriate report based on the user and reason to measure.

Talent Development Reporting Principles (TDRp) were created specifically to meet the need for an overarching measurement framework. Peggy Parskey and I wrote the book Measurement Demystified: Creating Your L&D Measurement, Analytics, and Reporting Strategy to share this framework and guidance. Our hope is that practitioners will now have the answers to their questions about what and how to measure, and how to report their results once they have them.

Knowledge and lack of a comprehensive framework, however, probably don’t explain all the discomfort with measurement. For some, measurement implies accountability, and many don’t want to be accountable for results. After all, what if we measure and find that application rates are low, impact is negligible, and ROI is low or negative? What if we find we have poor rates of on-time completion for development and delivery or a high cost per learner? This is a culture issue within L&D and more likely within the entire organization that cannot be solved by reading books or attending workshops. Leaders, starting with the CLO, need to make it clear that they want to measure in order to deliver better results and learn what is working so that opportunities for improvement can be identified.

Even if the issues of knowledge and culture are addressed, there may still be a hesitancy to measure and report due to a lack of confidence. This is typical for analysts and others new to their position who simply don’t have much experience with measurement and reporting. They have read the books and attended the workshops, and they may have a supportive culture, but they just aren’t sure about the right measures to use or how to calculate and report on them. Unfortunately, the only way to gain confidence is to do the work. Mistakes will be made, but we all make mistakes and the goal is that we learn from them. A supportive boss or a good coach or mentor can help, but often that is difficult to find.

Another possible explanation is turnover. Like other professions, it takes years to acquire all the practical knowledge needed to excel at a job. I am speculating, but could it be that those responsible for measurement and reporting don’t stay in the position long enough to truly master it with confidence?

Finally, I’ve heard that measurement and reporting is difficult because they don’t have the resources to do a good job. They have a small budget for measurement and not enough staff. In many cases, this reflects a lack of appreciation for measurement and reporting by the CLO and other senior leaders which in turn may reflect a lack of knowledge or accountability on their part.

By the way, these same issues apply across all of HR regarding measurement and reporting, so it is not just L&D.

If you’re committed to making measurement and reporting a part of your L&D strategy, please join us at our virtual conference November 2-4 to explore this issue in more detail. Kimo Kippen will lead a panel discussion on this very topic on November 3 at 2.45 pm ET, and Michelle Eppler from the Human Capital Lab at Bellevue University will share results from a survey she just completed to probe the reasons. Registration is free.