Most organizations aspire to better assess, understand, analyze, and report their L&D measures but are often uncertain about the capabilities needed to move forward. Some even wonder if they need to hire a Ph.D. data scientist. The short answer is no. It is better to start by building capability up from the bottom by ensuring everyone has a basic level of competency and ensuring that those with measurement responsibility have at least an intermediate level of competency.

Your first goal should be for all your L&D staff to understand L&D Measurement and Performance consulting. This is where a common language and framework like Talent Development Reporting Principles (TDRP) can help. It provides a common language for measures and reports and reasons to measure, facilitating communication among all L&D professionals. Everyone should know the essential efficiency and effectiveness measures, including how they are defined and used. Everyone should also have a basic understanding of performance consulting and needs analysis, the importance of reaching an upfront agreement with goals owners and stakeholders on success, and the role learning can play in that success.

Your measurement staff (if you have a dedicated team for this) or others who do Measurement as part of their job need to have at least an intermediate level of expertise in Measurement. This means they should know and be comfortable using most of the L&D measures in our field, at least those for formal learning if your organization doesn’t use informal learning. They should be able to select measures for a program or department-wide use. They should know what type of report is appropriate for each user, depending on their reason to measure. They should be able to write basic survey questions and understand the different options for measuring levels 2 and 3. They should be aware of the differences between the Kirkpatrick and Phillips approaches to level 4 and be able to employ the participant estimation methodology to isolate the impact of learning. They should be able to calculate ROI.

Ideally, you should also have at least one staff member with an advanced level of mastery. This person should be familiar with all the L&D measures, including those for informal learning. This person should also know statistics and survey design and the use of control groups, trendline estimation, and regression to isolate the impact of learning. This person should be able to identify and adjust for seasonality in the data and make year-end forecasts. This person should also be able to formulate hypotheses and test them to confirm or reject their validity.

What does it take to achieve these levels of competency? The good news is that you can meet the basic level with a short introductory workshop or several webinars. A framework like TDRP is easy to understand and remember, as are the 10-15 basic L&D measures. The intermediate level requires more education and experience. You can gain knowledge by attending a workshop on Measurement and reporting (for example, the Measurement Demystified workshop offered by CTR) or a Kirkpatrick or Phillips workshop. You can supplement your education by reading a few of the many books written on the topic of Measurement and Analytics, as well as attending more advanced webinars. Applying the concepts is also essential for an intermediate level of competency.

That leaves the advanced level of competency. This requires more workshops, more reading, and in some cases, courses, for example, to require the requisite knowledge of statistics or survey design. This level also requires considerable experience in the field, applying the concepts. By this stage, the person may be considered a data scientist, but that does not imply a graduate degree and certainly not a Ph.D. Most can achieve this level of competency through continual learning and application. Moreover, a newly minted Ph.D. data scientist may not have the experience to operate at an advanced level. It takes at least five years to reach this level.

In conclusion, your measurement staff should have at least an intermediate competency level which is not hard to obtain. Moreover, all L&D staff should have at least a basic understanding of Measurement and Performance consulting, which is easy to provide. Ideally, at least one person in a large organization possesses an advanced measurement competency, which requires much more work.