The Most Important L&D Measures to Capture

The Most Important L&D Measures to CaptureWe have more than 200 measures in L&D which begs the question, “What measures do I really need?” You certainly don’t need 200 and there is no “extra credit” for having more measures. The goal of your measurement strategy is to have the right measures for the coming year or two. After that, your strategy will evolve and you will likely expand or at least modify the list of measures you collect. So, what do you need for the “here and now”?

Since organizations have different needs, the recommended list of measures will be unique for each organization. That said, we can identify a list of starting measures that will apply to most organizations. First, let’s explore the most commonly employed efficiency or quantity measures at the course level. The list includes:

  • Number of participants,
  • Completion rate (percentage of the target audience that completed the course),
  • Completion date (used to determine if the course was delivered on time), and
  • Cost

More practitioners are also beginning to measure activity within online courses like time spent on each task.

At the department level, the learning leader will typically want to see:

  • Aggregate course efficiency measures like total unique participants (no duplications),
  • Total participants (allows for the same person to take more than one course),
  • Average completion rates,
  • Average percentage for on-time completion of development or delivery, and
  • Total cost

At the department level there are additional basic efficiency measures like:

  • Reach (percentage of employees touched by learning),
  • Number of courses offered and utilized,
  • Number of hours utilized, average hours per employee, and
  • Number of courses (and/or hours) by type of learning (instructor-led, virtual, e-learning, blended, clusters)

There are also a number of efficiency metrics for informal learning which measure usage. If you offer knowledge sharing, then you should at least measure the number of communities of practice, the number of active communities and the number of participants. If you offer performance support, then you should measure the number of performance support tools available and used as well as the number of employees using them. If you offer online content through a portal (not e-learning), then measure the number of items available and used as well as the number of users.

A measurement strategy should always contain a balance of efficiency (quantity) and effectiveness (quality) measures to provide a holistic view and to prevent unintended consequences. The most common effectiveness measures at the course level are:

  • Level 1 participant reaction,
  • Level 1 goal owner or sponsor reaction,
  • Level 2 learning,
  • Level 3 application (intended and actual), and
  • Level 5 ROI (Level 4 is addressed below)

All courses should be measured at Level 1 and important courses should be measured at Level 3 (at least intent to apply). If a knowledge check is important, the course should be measured at Level 2. If it has a large audience or requires substantial resources, it should be measured at Level 5.

At a department level, the effectiveness measures are the same but they will typically be reported as averages across all courses for Levels 1-3, but the learning leader should also examine the distribution to see if there are a large number of low scores which are hidden by a high average. Level 5 is best reported as a list of ROIs by project.

For informal learning, Level 1 should be measured periodically (every three to six months) to determine if users are satisfied with their communities of practice, performance support tools, and online content.

In the TDRp framework, we separate Level 4 impact from the effectiveness measures and place it in its own category called outcome measures. We do this because CEOs have told us they most want to see Level 4 from their L&D department but seldom receive it. So, we give it special attention by creating a third category of measures.

Ideally Level 4 is the isolated impact of learning, and Jack and Patti Phillips describe five ways it can be calculated. If it is not possible to isolate impact, then a Kirkpatrick approach can be used to make a convincing case that at least some of the business result was due to learning.  (Note that Level 4 can only be measured using the Phillips approach and that Level 4 is required to calculate Level 5.) Level 4 impact should be measured for any important program (often consisting of multiple courses and informal learning) that is expected to help achieve an organization goal like increasing sales, reducing injuries, or improving leadership or employee engagement.

In conclusion, my recommendation is to start with the basic efficiency and effectiveness measures for formal learning. You should measure to at least Level 3 application for important courses and Levels 4 and 5 for the most important programs and initiatives. Add informal learning measures as you deploy informal learning. You are likely to be more successful if you start small and then grow through time. In other words, don’t start with 50 measures. Instead start with 10-15 and add some each year.

Learn more in Measurement Demystified: Creating Your L&D Measurement, Analytics, and Reporting Strategy (ATD Press, 2021).

Speak Your Mind