Impact and ROI of Learning: Worth Pursuing or Not?

Several articles in the last month have suggested the profession should step back from trying to isolate the impact of learning and the resulting return on investment (ROI). The authors argue that it is difficult or impossible to isolate the impact of learning from other factors and no one will believe the results anyway so why bother. Instead, they call for showing the alignment of learning to business goals, focusing on easier to measure levels 1-3 (participant reaction, amount learned, and application), and finally focusing on employee engagement with learning (consumption of learning and completion rates). Aligning learning to business goals and measuring levels 1-3 are always good ideas, so no disagreement there. And, depending on your goals for learning and the type of learning, measuring average consumption and completion rates may also make sense. However, for certain types of learning there is still a need to measure impact and ROI (levels 4 and 5).

The primary reason is that senior corporate leaders (like the CEO and CFO) want to see it and so should the department head and program director. Research by Jack Phillips in 2010 showed that CEOs most want to see impact and ROI but instead are often provided with only participation data (number of learners, number of courses, etc.), participant reaction (level 1) and cost. While these measures are helpful they don’t answer the CEO’s question of what they are getting for their investment. CEOs and CFOs want to know what difference the training made and whether it was worth the time and effort. The program director and CLO should also be curious about this, not from a defensive point of view (like proving the value of training), but from a continuous improvement perspective where they are always asking what we learned from this project and how we can improve next time.

It is true that level 4, the isolated impact of learning on a goal, is harder to determine than levels 1-3. Sometimes there will be a naturally occurring control group which did receive the training. In this case, any difference in performance between the two groups must be due to the training. In other cases, statistical analysis like regression may be used to estimate the isolated impact from the training. The most common approach, however, is participant and leader estimation which is generally good enough to roughly determine the impact of learning and definitely good enough to learn from the project and to identify opportunities for improvement. In a nutshell, the methodology calls for asking participants to estimate the impact from just the training and to also share their confidence in that estimate. The two are multiplied together to provide a confidence-adjusted estimate of the isolated impact of learning. For example, one participant may say that the training led to a 40% increase in performance (like higher sales) but is only 50% confident in the 40%. The confidence adjusted impact would be 40% x 50% = 20% increase in performance. Repeat for others and average. Best practice would be to ask supervisors what they believe as well. Then share with the initiative’s sponsor and other stakeholders and modify as necessary. Once the level 4 isolated impact is determined, ROI is very straight-forward.

Speak Your Mind