Is Data Analytics Your Highest Priority? Maybe It Shouldn’t Be

Data analytics continues to be a very hot topic in our field. Many have identified this as their number one or two priority for the coming year, but should it be?

First, let’s explore some definitions.

Data Analytics

For some, data analytics means any collection or manipulation of numbers. By this definition, all measurement is an exercise in data analytics, including finding the number of participants in a course or the number of courses offered. I think this definition is too broad and not very helpful for our discussion.

In our book Measurement Demystified we define analytics as:

“an in-depth exploration of the data, which may include advanced statistical techniques such as regression, to extract insights from the data or discover relationships among measures.”

Clearly, this definition does not encompass calculating the number of participants or finding the average application rate. In other words, analytics goes beyond arithmetic operations like calculating sums or averages and it goes beyond the simple reporting of a measure.

Analytics could entail looking at a frequency distribution of the data or in-depth analysis of the value of a measure by drilling down to finer levels of detail. It could also be the use of correlation or regression to see if two or more measures are related. (For example, is more learning correlated with higher retention?)

Measurement

So, what some refer to as analytics is just measurement. We define measurement as “the process of measuring or finding values for indicators.” Finding the number of participants or the application rate for learning are examples of measurements. It is true that measurement and analytics are related. Typically, you will measure first to produce measures which may then be analyzed to better understand the value of the measure. So, the measurement comes before the analytics. But this is not always the case. Sometimes, we need to do analysis to identify additional measures we need for further analysis. Bottom line, the two concepts are closely related but synonymous.

Now, back to our question: “Should data analytics be your highest priority?” In my opinion, the answer is no for organizations that have not mastered measurement, which I define as having a comprehensive measurement and reporting strategy. A comprehensive strategy would include the regular measurement of numerous efficiency measures (level 0) for use at both the program and department levels, regular measurement of levels 1-3 for effectiveness measures for programs, and measurement of outcomes (level 4) and ROI (level 5) for selective programs. A comprehensive strategy would also include the aggregation and reporting of these measures across all programs as well as the basic efficiency and effectiveness measurements for informal learning.

If your organization does not have a comprehensive measurement strategy yet, then I believe the greatest benefit will come from building out your measurement strategy. In other words, master the basics before you tackle data analytics which is a much more advanced topic, requiring special expertise. Most organizations today are still not measuring even level 3 application regularly which means they have no idea if their learning is applied, let alone producing measurable results. On the other hand, if an organization already has a comprehensive measurement and reporting strategy, then developing a sound analytics capability is the next logical step. In this situation, a wealth of data will already exist, providing a great foundation for analysis.

Data analytics has tremendous potential for the profession, but let’s be sure we have mastered the basics first.

Speak Your Mind