Bringing Standards to the L&D Profession: Three Types of Standard Measure: By Dave Vance

By Dave Vance

Accounting has four types of measures (revenue, expense, assets and liabilities) and three standard statements (income or profit & loss, balance sheet, and cash flow). What do we have in L&D which is similar? Five years ago a group of industry leaders came together to consider a standard framework for L&D. The group included leaders like Kent Barnett then at Knowledge Advisors, Tamar Elkeles then at Qualcomm, Laurie Bassi from McBassi and Company, Jack Phillips from the ROI Institute, Jac Fitz-enz from the Human Capital Source, Josh Bersin from Bersin & Associates, Cedric Coco from Lowes, Karen Kocher from Cigna, Rob Brinkerhof from Western Michigan University, Kevin Oakes from i4cp, Carrie Beckstrom from ADP, Lou Tedrick from Verizon, and a host of others – 30 in all, including myself. After 24 revisions, we all agreed on a framework which we believed would begin to provide the type of standards that accountants have enjoyed for some time and which would help us become more professional.

These leaders recommended a framework consisting of three types of standard measures and three standard reports. So three and three versus the accountant’s four and three. The three types or categories of standard measures are effectiveness, efficiency, and outcomes. The three types of reports are Operations, Program, and Summary. The goal was to keep the framework simple and easy to use. We also wanted to build on the excellent work done in our profession over the last 70 years, especially that done by the Association for Talent Development (ATD, then called ASTD).

Let’s look at the measures first. Effectiveness measures are those that address the quality of the learning program or initiative. In learning we are fortunate to have the four levels popularized by Don Kirkpatrick and the fifth level (ROI) popularized by Jack Phillips. These five levels all speak to the quality of the learning. Level 1 measures the participant’s or sponsor’s satisfaction with or reaction to the learning – certainly an initial measure of quality. Level 2 measures the amount learned or transference of skills or knowledge, and level 3 measures the application of that knowledge or change in behavior. If they didn’t learn anything or if they don’t apply what they did learn, I think we can all agree that we don’t have a quality program. (This may reflect a lack of engagement or reinforcement by the sponsor, but we still have a problem.) Level 4 measures impact or results and, since this was the reason for undertaking the learning to begin with, if there are no results it is hard to argue we had a quality program. Last, ROI provides a return on our investment, a final confirmation of quality assuming we have properly aligned the learning to our organization’s needs and achieved high quality on the previous four levels. Most organizations have level 1 and 2 measures but relatively few measure at the higher levels.

Efficiency measures are about the number of courses, participants, and classes as well about utilization rates, completion rates, costs, and reach – to name only the most common. Typically these measures by themselves do not tell us whether our programs are efficient or not. Rather we need to compare them to something else which may be last year’s numbers, the historical trend, benchmark data, or the plan (target) we put in place at the beginning of the program. Now we have a basis to decide if we are efficient and if there is room to improve, to become more efficient. All organizations have efficiency measures with the most common being number of participants, classes, and hours as well as some cost measures like cost per learner or cost per hour.

That leaves outcome measures. Unlike effectiveness and efficiency measures, most organizations are not measuring outcomes and few even talk about them. That is unfortunate because outcome measures are the most important of the three types, especially to senior leaders who make the funding decisions for L&D. In accounting this would be like reporting expense and liabilities but never talking about revenue or assets. No one would have a complete picture of what we do, and it would be hard for anyone to understand why we have the expenses and why we incur the liabilities. So, what are these all-important outcome measures? Simply put, outcomes represent the impact or results that learning has on your organization’s goals or needs. Suppose a needs analysis indicated that your salesforce would benefit from a consultative selling skills program and product features training. And suppose that you and the head of sales agree that it makes sense, and the two of you further agree on program specifics, including mutual roles and responsibilities, especially how the sponsor will reinforce the learning and hold her own employees accountable. Now, what impact on sales can we expect from this training? How much of a difference can training make? A lot? A little? Enough to be worthwhile? This is the realm of outcome measures which will sometimes be subjective (but not always) but very important nonetheless. Sometimes the level 4 impact or results measure from the list of effectiveness measures will do double duty as an outcome measure. That is okay and the same happens sometimes in accounting. Or other measures will be selected. In any case, with outcome measures we are at last focused on how we align with corporate goals or needs and what type of impact we can have, and this is what your senior leaders have been waiting for.

Next month we will look at the three reports and how the three types of measures populate these three reports. In the meantime, as a profession, let’s start talking about these three types of measures. It would be big step forward if we could just adopt a common language, which by the way, is a precondition to be a true profession.

 

Speak Your Mind