Latest Resources

Measurement Demystified: Creating Your L&D Measurement, Analytics, and Reporting Strategy

As some of you may be aware, the title of this blog is the name of the book Peggy Parskey and I recently published with ATD. It is the culmination of all our work since 2010 to create a framework and a set of standards and best practices to make it easier for L&D professionals to create a measurement and reporting strategy. It is also the result of input, feedback, and suggestions from the thousands of professionals who attended our webinars, workshops, and conferences. So, in essence—the book is a joint effort by everyone to advance the profession.

For those unaware of the backstory of the book, here’s a little history…

It all began in September 2010 at a CLO Symposium networking event. Kent Barnett, at the time CEO of Knowledge Advisors, and Tamar Elkeles, at the time VP of Organizational Learning for Qualcomm, were discussing the state of our profession. They both agreed that the time had come for measurement and reporting standards— similar to what accountants have in the Generally Accepted Accounting Principles (GAAP). Accountants go to university and learn about the four types of measures (income, expense, assets, and liabilities), the specific measures in each category, how to calculate the value for each measure, and what to do with the measure once they have its value (i.e., in what report it should be included and how it should be used). They wondered if a similar framework would benefit L&D.

Kent and Tamar set forth to form an advisory council to pursue their vision and ideas. They engaged leading practitioners as well as thought leaders in the field, about 28 in all. They also recruited both Peggy Parksey and me to lead the effort. We produced a draft document in early 2011, which would end up going through over 20 iterations as we incorporated feedback from the council and others. The final document was completed later in 2011, and we then began to expand the principles to all areas of HR, which was completed in 2012. The standards were named Talent Development Reporting Principles (TDRp). Similar to GAAP, TDRp provides a framework for measurement and reporting for L&D.

By mid-2012, the foundational work was complete, and TDRp needed a home. We established the Center for Talent Reporting (CTR) as a 501c(6) nonprofit organization in August of 2012 to continue to develop and promote the principles of TDRp. We hosted our first conference and our first workshops the following year. We also started our monthly webinars and blogs. And we never turned down an opportunity to speak and share TDRp with others.

It’s hard to believe that we have been at this for ten years. We’ve learned a lot along the way, so we captured all of our learnings in the book, Measurement Demystified: Creating Your L&D Measurement, Analytics, and Reporting Strategy. It is our intention that the book reinforces the standards of the TDRp framework and will do for the L&D profession that GAPP has done for accounting, which is to provide a common language; categories, names and definitions of measures; standard reports; and guidance on what to measure and how to report.

The TDRP framework begins with four broad reasons to measure: inform, monitor, evaluate, and manage.  There are many more detailed reasons to measure, but we believe a framework should have no more than five elements, or it becomes too difficult to remember and use. These four categories make it easier for us to have discussions about the reasons to measure, which is the starting point for any measurement strategy.

TDRp recommends three categories for measures. Historically, practitioners had divided measures into efficiency and effectiveness buckets which is what I did when I worked at Caterpillar. However, research and practice showed the need for a third category, not because of the number of measures in it but because of its importance. The third type of measure is “outcome,” which means the results or impact from the learning on the organization’s goals. This is the type of measure CEOs want to see the most, and yet it is seldom reported. This is the measure required to make the business case for learning.

Lastly, TDRp identifies five types of reports which contain the measures. Moreover, TDRp ties the type of report to the reason for measuring, which provides much-needed guidance for practitioners. Scorecards and dashboards are great for informing and can also be used for monitoring if thresholds are included. Program evaluation and custom analysis reports are ideal for sharing the results of a program or research. Management reports are the fifth type, and these have a special format to help leaders manage programs and initiatives to deliver planned results. They come in three varieties depending on the user and the need, but all have the same format, just like the basic accounting reports do.

It is our sincere hope that the TDRp framework and the guidance in Measurement Demystified will help the profession advance. Some are just beginning and should benefit significantly from a framework and guidance on how to select measures and reports. Others are further along and can benefit from the definitions of over 120 measures and the advanced report formats. Even experts may benefit from the detailed discussion of how to create plan numbers, use year-to-date results, and create forecasts.

And we are not done yet. We just submitted our draft manuscript for Measurement Demystified: The Field Guide to ATD for publication this December. We provide over 100 exercises to improve your understanding of the concepts and to give you practice in applying them. Hopefully, the Field Guide will significantly increase your skill level and your confidence, enabling you to create a much more robust measurement and reporting strategy.

We look forward to your continued engagement and feedback.

Running Learning Like a Business

The concept of “running learning like a business” continues to gain traction. It means different things to different people but in all cases involves bringing a more business-like perspective to the L&D function. Personally, I like to focus most on its implications for the actual management of learning programs and the L&D department.

So, you ask, what does this mean specifically? What would someone do differently? How can you tell if someone is running their operation like a business? I believe there are two primary elements to running learning like a business. First, you need to set specific, measurable goals or targets for every important measure. This becomes your business plan for a program or the entire department. This plan should be created just before the start of your fiscal year or no later than the first month of the new fiscal year. Second, once you have a business plan and the year is underway, you need to compare progress against plan every month and answer two key questions: 1) Are we on plan year to date? and 2) Are we going to end the year on plan? If you are not on plan or if it appears you may not end the year on plan, then you need to discuss options to get back on plan and decide whether to take corrective action. This is what we mean by disciplined execution.

Let’s look at each in more detail. First, you need good plan. Of course, there is lot that goes into a good plan beginning with an understanding of the organization’s goals and discussions with goal owners about whether learning has a role to play. Think of this as proactive, high-level performance consulting where you seek out the senior leaders and engage in good, open discussion to explore the possibilities for learning to help them achieve their goals. Learning will not always have a role, but at Caterpillar we were able to contribute to each of the CEO’s top seven goals each year.

If learning can make a contribution, then you need to reach agreement with the goal owner (like the SVP of Sales) on what the learning program will look like: target audience, timing, learning objectives, type of learning, design, etc. More than that you and the goal owner need to agree on measures of success (like the impact of learning on sales) and on targets for key efficiency and effectiveness measures to deliver the agreed upon outcome (like impact of learning). This would include agreeing on plans for number of participants, completion dates and rates, learning (level 2), and application (level3). This needs to be bolstered by agreeing on roles and responsibilities – what each of you needs to do to deliver the planned measures. For example, how will the goal owner communicate the importance of the learning before the program and reinforce desired behaviors after the program. (We had a written roles and responsibilities document at Caterpillar signed by both parties.)

Second, once the program is underway, you will begin to receive monthly data on the measures you and the goal owner agreed were important. You will need a monthly management report (called a program report) to compare year-to-date results to plan. If you are not on plan, you need to understand why. Perhaps it is just a matter of timing (for example, it took longer to launch than planned) and left alone the program will get back on track and meet plan expectations. This is why the forecast for how the year is likely to end is so important. If the forecast shows you delivering plan, no further action is required. However, if the forecast shows you falling short of plan by year end, then you need to consider taking corrective action as soon as possible. This may involve actions such as redirecting resources toward this program or coming up new plans for the goal owner to reinforce the learning.

Our focus so far has centered on what we call strategic programs. These are programs directly aligned to the goals of the CEO. However, there are typically other programs of great importance, in some cases even more important, than strategic programs. Examples would be onboarding, basic skills training, leadership, compliance, etc. The same thinking applies. Start by creating a plan for each program with specific, measurable goals. Then execute the plan throughout the year with discipline to deliver the plans.

We have just discussed what it means to run learning like a business from a program perspective. The same discipline, however, can be applied to your CLO’s initiatives to improve results across all programs (like improving the overall application rate) or to improve internal process or system efficiency or effectiveness (like improving customer satisfaction with the LMS or reducing helpdesk wait time). In this case, the CLO needs to set specific, measurable goals for each measure she chooses to manage. These measures are captured in the monthly operations report which she and the directors can use each month to determine if the initiatives are on plan and likely to end the year on plan. If not, they need to decide what action to take.

So, this is what “running learning like a business” means to me. This the hard work of managing L&D with the same type of discipline your colleagues use in sales or manufacturing where they set annual targets and do the best they can to meet those. For many, this is a very different way of managing than they do today. Most have plans of some sort, but many are not specific and measurable. Even some who do have good plans do not execute them with discipline by using the monthly program and operations reports to compare progress against plan.

If you are managing this way today, congratulations! I know you see the value in this and ask you to share your stories with others so they can be inspired by your success. If you are not managing this way, consider giving it a try. You can start small. Try it for one program. And/or try it with the CLO to better manage a short list (3-5) of measures for improvement. I am convinced that running learning like business was key to our success at Caterpillar, including support and funding from our CEO and senior leaders.

If you would like to hear more about this concept, join me for a panel discussion on March 23 at 4.15 ET hosted by Corporate Learning Week.

2020: The Year of Disruption

I think we’ll all be happy to put 2020 behind us. As of December 18, 2020, Covid has claimed more than 300,000 lives in the U.S. and disrupted the economy, our personal lives, and learning along the way. Despite the pandemic’s destruction, some good has come from the disruption—like the SEC rule mandating human capital disclosure.

The impact of the pandemic on learning has been massive. Organizations have been forced to shift from instructor-led learning (ILT) to virtual ILT (vILT) supplemented by more eLearning and portal content. Some learning couldn’t be converted immediately, so learning opportunities and productivity gains were consequently lost. Since companies had to convert learning so quickly, much of vILT didn’t meet quality standards. But despite this, learners were appreciative of the effort and happy to have more learning than none at all.

During this time, the good news is that learning departments have demonstrated their value by offering up valuable strategic advice on how to manage remotely. This has been a huge win for many L&D departments and has dramatically increased their credibility and stature within the organization.

Just as the pandemic has caused us all to reconsider our work-life balance and the possibility of an ongoing work-from-home arrangement, it has also forced us to reconsider the mix of ILT versus vILT, eLearning, and content available in the LMS portal. Actually, this reconsideration is long overdue. Many organizations had little to no vILT before March 2020. Now, they have a lot of vILT, and the quality of the content is slowly improving as it is redesigned for virtual delivery. Live virtual training offers significant benefits, such as travel and facility cost savings. And, if properly designed, it may be able to accomplish the same objectives in less time. I don’t see any chance of organizations reverting to a pre-pandemic model that realized so heavily on ILT.

There is also an opportunity here to think even more broadly about restructuring your learning programs. Many organizations going into the pandemic had long onboarding and basic skills ILT programs. Some went on for a month or more, with a few lasting three, six, or even 12 months. Is this really the best way to learn? And how much of the content will be remembered and applied? Instead, why not use this opportunity to restructure the learning entirely? Reduce ILT or vILT at the beginning to just the basics that learners will need to get started. Supplement this with additional vILT or eLearning combined with performance support in the workstream at the time of need. For example, following the initial (now shortened) course, participants might take a short vILT course or eLearning module every month over the rest of the year. And this is combined with performance support, which is easy to access and provides them just the tools they require at the time of need.

Last, let’s not forget the SEC’s new rule mandating human capital disclosure. Over the coming years, this transparency will spread to all types of organizations and radically alter the expectation of both investors and employees for information on important human capital metrics like diversity, pay equity, leadership trust, and employee engagement. By 2030 people will wonder why this type of information was not always available. Disruption is coming, and there will be no going back.

So, take this opportunity as you plan for 2021 to continue to think outside the box, just as you have been forced to do this year. Don’t go back to the old ways even when offices reopen.  Double down on new ways of learning and embrace the opportunity to disclose your important L&D and HR metrics publicly.

Best wishes for the holidays and may everyone be healthy and safe.

Covid Has Changed Our L&D Word—Where Do We Go from Here?

by June Sonsalla,
Vice President, Head of Learning & Development,
Thompson Reuters

I 2014 I attended an event focused on Resilience at a boutique hotel near the harbor in San Diego. The day of the conference, it was a temperate 70?F, the palm trees were bending in the breeze, and the air smelled of saltwater and wildflowers. In a pre-pandemic world, resilience lacked urgency. At best, it was a new way to describe change readiness. At worst, it put the onus on employees to manage increasing work volume with the same resources. That day back in 2014 seems like a lifetime ago. I’ve since moved to the midwest, there’s a different president in the White House, and I must wear a mask while in the grocery store.

Fast-forward to 2020. March of this year I started a new position at Thompson Reuters and only spent nine days in the office before moving to my home office indefinitely. A global pandemic, increased focus on social justice, and inclusion as a critical input for all talent processes require a completely different way of working to get the same results as in prior years. In addition, adults are reporting more adverse mental health conditions as they navigate uncertainty. All of these variables mean new demands on human resources function and on each of us individually. If I had a crystal ball, I would have taken better notes during that Resilency workshop I attended back in 2014.

So, how do we overcome new obstacles, move forward, and contribute as much or more value to the organization than before? What matters most to our organizations now? What measures do we use to ensure continued progress and long-term success? And, how do we move ourselves and human resources from merely “getting by” to excelling in the new normal?

A Special Invitation…

I invite you to explore the answers to all of these questions with me at a very special event… the 2020 CTR Conference. During the session, Covid-19: Challenges, Change, and Continued Innovation, a few of my L&D colleagues and I will discuss how the pandemic has accelerated long-term trends creating permanent transformations in what and how L&D delivers strategic value. Hope to see you there for a lively discussion.

Measure and Appreciate Talent Outcomes

by Jeffrey Berk, COO, Performitiv

Learning’s ‘holy grail’ is to measure business outcomes. These include measuring how common business results are connected to learning. Results such as revenue, expenses, quality, productivity, customer satisfaction, risk, cycle time, and employee retention are common examples of business outcomes where L&D desires to show its connection.

However, let’s not forget about important talent outcomes. Talent outcomes are not the business results mentioned above, yet they play a very important role in measuring learning’s impact, especially for strategic, visible, and costly programs. Over the past year, Performitiv has partnered with L&D experts to identify the main sources of talent outcomes, such as culture, engagement, leadership, knowledge and skills, and quality of hire. These outcomes are extremely important to executives for programs such as leadership and onboarding, as well as programs that are part of material change or reorganization.

The next challenge is to figure out how to measure these talent outcomes. Unlike operational business outcomes, talent outcomes are not as clear or obvious. They require evidence of impact but neither perfect nor precise statistical measures of impact. As a result, we can ask questions on evaluations to understand if the programs were connected to the talent outcomes.

Here are some sample questions of talent outcome indicators on evaluations:

  • Culture: As a direct result of this learning, I feel more connected to our organization’s values and beliefs
  • Engagement: As a direct result of this learning, I am more motivated and committed to this organization
  • Leadership: As a direct result of this learning, I feel I am significantly better leader for this organization
  • Knowledge and Skills: As a direct result of this learning, I am confident I will significantly improve my job performance
  • Onboarding/Orientation: As a direct result of this learning, I am confident and comfortable to perform my job requirements at a high level

While not perfect, the above guidance may be used to build your own talent outcome indicator questions, as they are reasonable evidence of impact. For example, the Engagement indicator question may be used to help an executive understand whether participants in a program felt significantly more motivated and committed to be with the firm as a result of participating in a program. The program manager can then look to learning objectives, exercises, and support materials to show the connection of the program toward engagement.

2020 CTR Annual Conference

Learn More about Learning Measurement to Track Impact

Jeffery Berk will be discussion Learning Measurement in depth at the 2020 Virtual CTR Annual Conference, October 27-29, 2020. Join him for his session: Learning Measurement Using Impact Process Mapping, October 28 at 2:00 pm (ET). Registration is completely free.

REGISTER>>

A Continuous Improvement Approach Designed by Learning for Learning

by Kent Barnett, CEO, Performitiv

Ten years ago, a group of professionals came together to create an industry standard for Talent Development Reporting. Under Dave Vance’s leadership, that work turned into Talent Development Reporting principles (TDRp) and the Center for Talent Reporting. This work has withstood the test of time and has become the industry standard we all desired.

We now have the opportunity to leverage TDRp and create a much needed continuous improvement process that helps our industry optimize its impact. Other industries created their own continuous improvement process with a dramatic impact on their contributions, such as Six Sigma and Net Promoter System.

Many of us who have contributed to the development of TDRp have created a similar continuous improvement process, the Learning Optimization Model (LOM). The LOM is being deployed by several leading organizations and experts. It has six core components that can be implemented over time.

One of the key components is to introduce a new way of measuring effectiveness and impact. It leverages the simplicity and power of Net Promoter Score but is designed specifically for Talent Development. It is a great way to compare formal and informal learning while identifying way to improve impact.

Another component, the impact process map, helps us understand how all the measures we have fit together. More importantly, it helps us understand what to do with the data.

A third component of the LOM is hard data. Many of us believe that learning needs to take ownership of its financial contributions similar to manufacturing and sales. A vice president of manufacturing doesn’t control all aspects of gross profit, but it is a hard measure critical to managing manufacturing’s contribution. A vice president of ales doesn’t control all aspects of sales, but sales growth is hard data critical to managing impact. All business units with a P&L have workforce productivity goals measured by revenue divided by labor cost. Shouldn’t a CLO know the workforce productivity goals for his/her customers and take ownership to help achieve them?

Another component of the LOM is financial modeling. Jack and Patti Phillips of the ROI Institute have been an integral part of developing the LMO. Financial modeling is designed to incorporate the ROI process by helping to do analysis on the best mix of resources prior to rolling out certain strategic programs.

If you’re interested in learning more about the Learning Optimization Model, Join me at the CTR Conference, October 27-29, where I will go more in-depth.

2020 CTR Annual Conference

LEARN MORE ABOUT THE LEARNING OPTIMIZATION MODEL


Join us at the 2020 Virtual CTR Annual Conference, October 27, for the session: A Continuous Improvement Approach Designed by Learning for Learning presented by Kent Barnett, Founder and CEO of Performitiv.

LEARN MORE >>

SEC Publishes Final Rule on Human Capital Reporting

by David Vance, Executive Director, Center for Talent Reporting

The U.S. Securities and Exchange Commission (SEC) just published its final rule on human capital reporting on August 26, 2020. This follows the proposed rule issued one year ago. The final rule makes very few changes to the proposed rule and mandates for the first time, public reporting of human capital metrics by companies subject to SEC reporting requirements, which includes all US companies issuing stock, bonds, or derivatives. The rule becomes effective 30 days after publication in the Federal Register which should happen in September, 2020.

Today, companies have to report only one human capital metric—number of employees. The new rule will still require the reporting of full-time employees, but additionally, encourage companies to report part-time and temporary employees if they are important to a company’s financial results.

More importantly, the new rule mandates for the first time that companies provide “to the extent such disclosure is material to an understanding of the registrant’s business taken as a whole, a description of a registrant’s human capital resources, including any human capital measures or objectives that the registrant focuses on in managing the business.”

The qualifying word: “material” means anything that an investor would want to know before buying or selling a stock, bond, or derivative. The SEC goes on to specifically call out the areas of “attraction, development, and retention of personnel as non-exclusive examples of subjects that may be material, depending on the registrant’s business and workforce.”

SEC Chair Clayton commented in the public release, “I cannot remember engaging with a high quality, lasting company that did not focus on attracting, developing, and enhancing its people. To the extend those efforts have a material impact on their performance, I believe investors benefit from understanding the drivers of that performance.”

In other words, the SEC expects to see these three areas discussed and reported if they are material, and it is hard to imagine companies where they are not. Consequently, public companies will need to start disclosing and commenting on them as early as  October , 2020. And this is just the starting point. The rule calls for all material matters to be disclosed; therefore each company will need to decide what other human capital matters might be considered material by an investor. Depending on a company’s  situation, this might include total workforce cost or productivity, diversity (especially at the leadership level), and culture (revealed by employee engagement and leadership surveys). Discussion may also be required about the implementation of a new performance management system or a significant change in compensation and benefit philosophy.

Where can companies get guidance on specific metrics they might use to meet the new rule? The 2018 recommendations by the International Organization for Standardization (ISO) includes 10 metrics for public reporting by all organizations and an additional 13 for reporting by large organizations. ISO also recommends 36 other metrics for internal reporting, which are organized by area or cluster. The recommended metrics for the three SEC focus areas with metrics recommended for all organizations include:

  • Attraction: Time to fill vacant position, time to fill critical vacant positions, percentage of positions filled internally, percentage of critical positions filled internally
  • Development: Development and training cost, percentage of employees who have completed training on compliance and ethics
  • Retention: Turnover rate

These metrics provide a starting point for a company’s human capital reporting strategy to meet SEC requirements. Additional metrics are available from ISO for the area of development, including percentage of employees who participate in training, average hours of formal training per employee, percentage of leaders who participate in training, and percentage of leaders who participate in leadership development.

In addition to the metrics for the three focus areas, ISO recommends a number of others that could address areas of material concern such as workforce cost,  productivity, diversity, and leadership trust. Companies will need to identify their own metrics not addressed by the ISO and may wish to supplement with other data (such as employee engagement). They will also need to report on material initiatives which may not have metrics.

Publication of the final rule by the SEC, combined with the comprehensive recommendations by ISO, ushers in a new era of transparency in human capital which will fundamentally change the way organizations operate. And these changes will go far beyond US publicly traded companies. In 5-10 years, privately held companies, nonprofits, and other types of organizations will be compelled (or shamed) into adopting the same level of transparency.

In the future of human capital transparency, what investor will buy stock in a company that refuses to disclose material human capital information, which is already the primary driver of value in many companies? What employee will work for an organization that refuses to share its key human capital metrics because they are embarrassed to share? Why would anyone work for an organization where the culture is terrible, they don’t invest in their people, turnover is high, and there are not enough employees to do the work? Today, you can say you didn’t know. Tomorrow, you will know.

Change is coming. The new world of human capital transparency has arrived.

Learn More

The final rule on human capital reporting is part of a much larger revision to what is called Regulation S-K which governs what companies must disclose to investors initially (form S-1), quarterly (form 10-Q) and annually (form 10-K). The name of this effort, which has been underway by the SEC for four years, is Modernization of Regulation S-K items 101, 103, and 105. Item 101 governs description of the business and this is where new rules for human capital reporting are found. (Item 103 covers legal proceedings and item 105 risk factors.) Read the rule at here (pages 45-54) for human capital.

The ISO 30414:2018 Human Resources Management—Guidelines for Internal and External Human Capital Reporting is available for purchase. ISO is just beginning to release technical specifications on how the metrics are defined and used. By mid 2021, specifications should be available on all 59 metrics. The specifications for five of the eight training and development metrics are now complete and in publication. They should be available soon.

Attend Our Free Webinar On Public Reporting

Join us September 16 for our Public Reporting webinar, where cover both the SEC rule and ISO recommendations in greater detail.

Attend our Free Virtual Conference

We will be covering the subject of Human Capital Reporting at length at our Virtual Conference, October 27-29. Jeff Higgins, Adjunct Professor – Data Science(DSO), Human Capital Analytics at University of Southern California – Marshall School of Business, has been actively involved in both the SEC and ISO efforts and will be keynoting on the subject. We will also have a panel on the SEC and ISO as well as a follow-up session on the recommended metrics.

Optimizing Investments in Learning

by Peggy Parskey, Associate Director, Center for Talent Reporting & John Mattox, II, PhD, Managing Consultant, Metrics That Matter

You might wonder why L&D leaders need guidance for optimizing investments in learning. Recent research from The Learning Report 2020 (Mattox, J.R., II & Gray, D. Action-Learning Associates) indicates that L&D leaders struggle to convey the impact of L&D. When asked what they would do better or differently to communicate value to business leaders, 36% of L&D leaders indicated they would improve processes related to communication. More telling though is, 50% indicated they would improve measurement. To convey value, leaders need a story to tell. Without measurement there is not much of a story.

This is where Talent Development Reporting principles (TDRp) are so relevant. It begins with a measurement framework that recommends gathering three types of data: efficiency, effectiveness, and outcomes. Efficiency data tells the story of what happened and at what cost. How many courses did L&D offer? How many people completed training? How may hours of training did learners consume? What costs did we incur? Effectiveness data provides feedback about the quality of the learning event (e.g., instructor, courseware, and content) as well as leading indicators of the success of the event (e.g., likelihood to apply, estimates of performance improvement, estimates of business outcomes, and estimates of ROI). Outcomes are the business metrics that learning influences such as sales, revenue, customer satisfaction, and employee satisfaction. TDRp recommends gathering data for all three types of measures so L&D leaders can describe what happened and to what effect.

The table below (Table 11.1 in Learning Analytics) shows a variety of measures for each category.

Key Performance Measures for L&D

Efficiency MeasuresEffectiveness MeasuresOutcome Measures

  • Number of people trained

  • Number of people trained by learning methodology (instructor led, e-learning, virtual)

  • Reach (percentage of people trained in the target population

  • Cost of training per program

  • Decrease in costs

  • Cost of training per learner

  • Cost of training per hour


  • Satisfaction with training

  • Knowledge and skills gained due to training

  • Intent to apply learning on the job

  • Expectation that training will improve individual performance on the job

  • Expectation, that individual performance improvement to lead to organization performance improvement

  • Return on expectations


  • Increase in customer satisfaction

  • Increase in employee performance

  • Decrease in risk

  • Increase in sales

  • Increase in revenue

Source: Learning Analytics ©2020, John R. Mattox II, Peggy Parskey, and Cristina Hall. Published with permission, Kogan Page Ltd.

In addition to this guidance about what to measure, TDRp provides guidance on how to report results using methods familiar to business leaders. Using these measurement and reporting approaches, L&D leaders can tell a robust story to business leaders that they can connect with.

In April 2020, Learning Analytics, Second Edition (by John Mattox, Peggy Parskey, and Cristina Hall). The book helps L&D leaders improve the way they measure the impact of talent development programs. The second edition includes a chapter on Optimizing Investments in Learning, where TDRp is discussed in detail. TDRp is featured heavily in the book because it helps L&D leaders connect their efforts to business outcomes by measuring the right things and reporting them in a way that business leaders understand.

In Chapter 11, the authors connect TDRp to the Portfolio Evaluation methodology. This approach implies that business leaders are interested in how learning and development programs impact four business areas: growth, operational efficiency, foundational skills, and risk. By aligning courses with one of these business areas and using TDRp, L&D leaders can demonstrate the effectiveness of the courses in preparing the workforce to improve each business area (portfolio).

The book also provides guidance on how to use TDRp to spur L&D leaders to act on the data they report. The book indicates L&D leaders need to shift reporting in three ways to make it more actionable (see graphic below). Reports should provide:

  • Analytics that improve business decisions
  • Insights that link to evolving business challenges
  • Information that connects talent data to business impacts

Using Data to Spur Action

Using Data to Spur Action

Source: Learning Analytics ©2020. Reproduced with permission, Kogan Page, Ltd.

Another critical aspect of conveying a meaningful message that drives action is to tell a compelling story. Chapter 11 of the book includes three critical elements of a data-driven story:

  • Scene Setting—Connect all data and results back to the desired business outcomes and goals.
  • Plot Development—Emphasize logical and clear connections between learning results and outcomes; focus on the central message of the results, not peripheral findings; and note any areas where L&D is creating value for the organization.
  • Resolution—Clearly and succinctly outline the justification for key findings; suggest improvements, recommendations, and nest steps. Continue the conversation on how to help L&D improve.
2020 CTR Annual Conference

LEARN MORE ABOUT THE USE OF DATA AND LEARNING ANALYTICS


Join us at the 2020 Virtual CTR Annual Conference, October 27, for the session, Everything You Wanted to Know about Learning Analytics but Were Afraid to Ask.

LEARN MORE >>

Why We Need Human Capital Metrics and Reporting in Company Disclosures

by Jeff Higgins
Founder and CEO, Human Capital Management Institute

Can we agree that the coronavirus pandemic is above all a human crisis requiring people-centric measurement and solutions?

The last great crisis in the U.S. was the financial crises of 2008-2009. If the pandemic is a uniquely human crisis, the most powerful driver in our economic recovery is what we do next with our human capital and people practices.

Given the importance of people and human capital practices in organizations, does it not make sense that such a critical resource and source of value creation be included in public disclosures?

Without standard human capital metrics, everyone is flying blind (risk planning, investors, policy makers, and corporate executives).

Human capital risk planning is a new focus for many companies. It’s a topic frequently mentioned but has seen little action beyond companies adding boilerplate legal risk language in public reporting.

Reliable, validated human capital reporting has long been a black hole for investors, including institutional investors representing employee pension funds. This may change, as investors and policy makers press for insightful evidence of risk mitigation and future sustainability including global standards like ISO30414 Human Capital Reporting Guidelines (December 2018) and US SEC proposed rule on human capital disclosure (August 2019). Recently the SEC has reaffirmed its want for enhanced disclosure and more focus on workforce health and welfare.

“We recognize that producing forward-looking disclosure can be challenging and believe that taking on that challenge is appropriate,”  explained the SEC in an April 2020 letter.

While critical for investors, human capital reporting is just as important for employees. Think about a person deciding where to work. Wouldn’t they want to know key human capital metrics to make their decision, such as the diversity of employees and leaders, the amount invested in developing employees, the percentage of employees and leaders who take advantage of training, the retention or turnover rate, and culture measures, such as leadership trust index or employee engagement score?

It is critical to note that organizations disclosing more about their human capital have historically performed better in the market.

EPIC research shows:

Reliable human capital data is often lacking yet raw data is readily available, along with proposed ISO HC reporting guidelines.

The enhanced usage of people data and analytics has been a positive business trend long before the COVID-19 crisis. But stakeholders, including executives and investors, have not always had information that was relevant, valuable, and useful for decision making.

Turning people data into decisions requires skill, effort, discipline, and money, which is lacking in many HR departments. Priorities have often been elsewhere in building employee engagement and experience metrics, along with other “hot” technology driven vendor offerings.

Even experts acknowledge that the value of human capital is not always easy to measure. Nevertheless, ISO guidelines do exist for internal and external human capital reporting. Until now, relatively little has been offered publicly in most U.S. companies.

“We may not always want to know what the data tells us,” said one member. As we have seen with the COVID-19 crisis, not wanting to know what the data tells us, can be a recipe for disaster and  not one that leads to economic recovery.

The Need for Transparency Has Never Been Greater

Many of the critical metrics and practices needed most are not transparent to the very stakeholders (such as employees, investors and government policy makers) who must start and sustain the economy recovery. 

Moving forward, enhanced transparency around workforce management and human capital measures will be needed. Without such data, organizations will have a difficult time adapting to changing markets and workforce needs and building critical partnerships. Such data is vital to building trust within the workforce and attracting, engaging, and retaining talent.

Increasingly, as organizations collect more data about their workforce, employees will want to benefit from that data.

“We saw less concern about privacy when we saw the workforce data was shared and being used to protect the workforce,” said one member.

Investing In and Developing the Workforce Has Never Been More Important

The power of learning, development, and people investment has never been greater nor more directly tied to economic results. A greatly changed world requires a new level of learning investment, adoption and innovation.  

Will organizations see the need for learning investment and people development as a continuing need? Will they measure it to better manage it? Will they begin to see human capital measures as a means for learning about what is working and what is not? Will they begin to see human capital reporting as a leading indicator of recovery? 

2020 CTR Annual Conference

IN-DEPTH LOOK AT ISO'S L&D AND HR METRICS FOR HUMAN CAPITAL REPORTING

Join us at the 2020 Virtual CTR Annual Conference for this session where industry experts share the definitions of the 23 ISO disclosure metrics and how to use these to in your human capital reporting strategy.

LEARN MORE >>

Three Measures You Should Capture Now During COVID-19

I hope everyone is healthy and managing through these very trying times. And, I hope the same for your family and friends. I know this pandemic has not been easy.

All of us by now are well into our Covid-induced new world where we have shifted from instructor-led training (ILT) to virtual instructor-led training (vILT) and eLearning, supplemented by content available through our employee portals. On top of all of these changes, you have probably had to create new courses on safety and return-to-work procedures. Perhaps you have provided guidance to your leaders about how to manage remotely in stressful times. Some have even gone into cost reduction mode to help offset the loss of income.

While you’ve been really busy, I hope are ready to ‘come up for air’ now. If so, here are a few measurements to think about. If you have not already implemented them, there is still time to collect the data before it gets lost.

Make Time to Collect and Analyze Data

First, be sure you are collecting data on your vILT and any new courses you have put online. You may be using a new platform like ZOOM, that were not set up to connect with your learning management system (LMS), and the number of participants may not be recorded and not be receiving typical follow-up survey. In this case, be sure to set up a system to capture the number of participants manually—perhaps by asking instructors to send in counts. And, consider sending a survey out to at least a sample of participants—even if you have to do it manually. If you cannot easily generate the email list of participants, send to a sample of all employees and make it more general. For example, ask whether you have taken a vILT course, online course, or downloaded content in the last three months. Then, get feedback on each.

Feedback, especially for vILT, is important so you can compare to ILT and understand what participants like better about it as well as what they dislike. Surprisingly, some organizations are reporting higher or equal scores for vILT. And these results are from vILT which was probably rushed into production (i.e., presentations from ILT that were used and repurposes for vILT). Imagine how much better vILT could be if it were actually designed for virtual delivery. This is the data you and your leaders will need to decide if you want to permanently change your mix to less ILT in favor of more vILT when the pandemic is over.

Capture Your Efficiency Savings

Second, be sure to capture the efficiency savings as a result of switching to vILT and online courses. Typically, vILT and eLearning will be less expensive because there is no instructor or participant travel and the course may be shortened. To be fair you should really only count the savings where an alternative to ILT was provided. Of course, there will be savings from simply cancelling ILT with no replacement, but that is not really a savings. It is just a reduction in offerings. You can estimate the cost of ILT and the cost of vILT so you can calculate the difference. Don’t be afraid of estimating—it’s a common business practice.  If you have too many offerings to make a calculation for each course, come up with the  average ILT and vILT cost, find the difference, and multiply it by the number of participants.

Capturing the dollar savings from switching to vILT is important because there is real value there and it should factor into your decision about continuing with a mix of learning modalities after the pandemic ends. The savings will be especially large for organizations where participants or instructors incur travel and lodging costs.

Calculate Your Opportunity Costs

Third, calculate the reduction in opportunity costs by switching to vILT and e-learning. Opportunity cost is the value of participants’ time while in class and traveling to and from class. This adds up—especially for half-day or full-day courses and can easily exceed the accounting costs (i.e., room rental, supplies, instructor, etc.) for some courses.

Calculating opportunity cost is simple. Take the average hourly employee labor and related cost (including benefits, employer paid taxes, etc.) from HR and multiply it by the reduction in hours from switching to vILT and e-learning. For example, suppose you replaced an 8-hour ILT with a 5-hour vILT, which also saved participants one hour of travel time.  The reduction in hours would be 3+1=4 hours. Multiply the 4 hours by the number of participants and by the average hourly compensation rate. You can use the average hours for an ILT course and compare it to the average for vILT or e-learning to make the calculation at an aggregate level.

It’s important to account for savings in travel time—because employee’s time is valuable. By eliminating travel, you are giving time back to employees when you move to vILT or e-learning. Share the opportunity cost savings with your senior leadership, CFO, and CEO along with the savings in course costs. The opportunity cost savings, in particular, is likely to be an impressive number. Think about this savings when you decide whether to go back to ILT or keep using vILT.

I hope you find these measures helpful. You really need them to make an informed decision about what to do when the pandemic ends. I think the profession has a great opportunity to use this experience to become much more efficient and effective in the long run.

L&D Leadership in a Time of Great Change

[FREE VIRTUAL CONFERENCE] July 14 & 15, 2020. 12:00 – 4:15 pm EDT. This two-day virtual conference details the specific tasks L&D organizations have to carry out to achieve significant and continuous workforce productivity. Presentations will detail how to organize these tasks so that training executives can perform them systematically, purposefully, with understanding, and with a high probability of accomplishment.

Hear case studies, winning strategies, evidence-based results from strategic management leaders and learning executives charged with the awesome responsibility of managing L&D in today’s new business climate. Register Today!

Talent Leaders Virtual Exchange Americas

Virtual Event | June 17, 2020
REGISTER NOW

The Talent Leaders Virtual Exchange will give talent executives the much needed opportunity to hear how their peers are dealing with unprecedented disruption during this global pandemic. In a time when the situation changes almost daily, it’s important that talent executives are making quick decisions in the moment while also planning for a post pandemic world.

The Talent Leaders Virtual Exchange will offer insights from many perspectives on what is working now and what strategies are being put into place for the future. Topics covered include:

  • Communicating with a concerned, remote workforce
  • Lessons learned in the face of a global pandemic
  • Capturing the voice of the employee in a crisis
  • How talent leaders are delivering value during the time of COVID-19
  • Managing your employer brand in uncertain times
  • Preparing for a post-pandemic workforce

>> REGISTER

 

Forward to the Past—Reflections on How Our Profession Evolves (Or Not)

If you follow this blog, you know that I’ve been struggling this past year with the profession’s new focus on upskilling employees for future needs. I have been trying to understand two things:

  1. How future needs can be reliably identified now, and
  2. Assuming needs can be identified, why would anyone want to provide training now, when the skills and knowledge will not be needed or applied for years to come—resulting in scrap rates of 100%.

To me, this seems to go against everything we have learned over the last 20 years in terms of good performance consulting and designing with application and impact in mind.

I was excited to see multiple topics on upskilling and reskilling at a recent CLO Symposium. I looked forward to learning more about this topic, given its incredible popularity.

During the sessions, I had expected to learn that the identifiable future skills would be tangible skills—like programming or new manufacturing techniques (which would be difficult to predict with enough specificity to train for today). Instead, the future skills identified by the presenters were all soft skills like communication, teamwork, innovation, creative thinking, and problem solving. This reminded me of current research that cited soft skills are very important to future success.

That same research also indicated that employers are looking for those same soft skills in their employees today. If these skills are needed today, we can use our performance consulting tools to identify the gaps and design learning to address them. This all means that these soft skills can be applied today, which means we are really not talking about future skills—we are talking about skills to meet current needs, which are considered important for the future.

The CLO Symposium presenters shared these skills in the context of  being ‘new’—as in organizations should start to train employees on them. This is where I felt I was fast forwarding to the past. When Caterpillar U was founded in 2000, we had a brand new LMS containing hundreds of courses. In the ‘old days’ training organizations used to brag about how many courses they had in their ‘paper’ catalog. Guess what was in that catalog…soft skill courses like communications, team building, writing, and problem solving. The catalog also included a lot of job function skills too, but the point is, soft skill courses are not new. They have always been in demand and always will be. So, can we please apply a little historical context and be more careful about how we define  ‘future skills’? What we are really talking about is ‘forever skills.’

As I reflect on the past, it’s interesting to note what happened to learning leaders bragging about how many courses they offered. Company executives began asking for results. They wanted to know how these hundreds of courses specifically aligned to business goals and needs. Executives demanded measures beyond how many courses were offered. This led to ‘strategic alignment’ where the emphasis is how courses proactively and specifically align to meet the goals of the enterprise.

For many years, strategic alignment had been a hot topic at industry conferences and many books have been written on the topic. How many times did I hear strategic alignment mentioned at the CLO Symposium conference? Zero. I understand that focus areas change and evolve, but we don’t seem to be building on our past. Instead, we’re re-inventing it as with identifying soft skills as future skills. We will advance more quickly and soundly as a profession if we have a sense of our past and can skip the re-learning phases.

Let’s remember that soft skills will always be important as will strategic alignment. There is no need to cycle back and forth and rediscover the value of each. A holistic future can include all that we have learned which will provide the strongest foundation upon which to build, and then we can focus more of our energy on inventing what is truly new and special.

Thoughts on the Coronavirus and What It Means for L&D

The situation surrounding COVID-19 is evolving on a daily basis. Uncertainty is high and it may be weeks or even several months before we know how all this plays out. Several things are clear, though.

First, the US and world are likely headed into an economic “timeout” and quite possibly a recession. Hopefully, this will be brief but even a brief downturn will cause significant harm. Many in the service industries are going to lose their job or have their hours severely reduced. Other companies will experience falling sales as people cut back or delay their spending. Most organizations have already restricted travel. Even if the worst of the virus is over by summer and travel restrictions are lifted, companies may implement cost reduction measures for the remainder of the year to partially offset a decline in revenue for the first half of the year.

So, if your organization is one which is negatively impacted, now is the time to implement the recession planning that we have talked about for years. Be prepared to start reducing variable expenses, including the use of vendors and consultants. Where appropriate, shift from instructor-led learning to virtual ILT, e-learning, and performance support. Prioritize your work and be prepared to focus on only the most important. Get advice from your governing body or senior leaders on where they would like you to focus your efforts. If you have not already done so, clearly align your programs to the top goals and needs of your CEO.

Second, COVID-19 presents an excellent opportunity to make significant progress in moving away from instructor-led learning and towards virtual ILT and e-learning. Most organizations have been moving in this direction for years but say they are not yet at their desired mix of ILT to vILT and e-learning. Well, now is the perfect opportunity to make this transition. Travel restrictions will prevent employees from traveling to a class and will also prevent instructors from flying to class locations. And social distancing discourages bringing employees together for training which is why so many universities have announced that they are shifting entirely to remote learning for at least the next month or so. The private sector should be equally responsive. Use the existing platforms to conduct classes virtually and ramp up the marketing of your e-learning and portal content.

Third, the virus also presents a once-in-a-lifetime opportunity to highlight performance support. What can we in L&D do to help our colleagues adapt to this new world and still do their jobs? People will be working remotely from home, perhaps for the first time. What performance support do they need? Most will not need a 30-minute e-learning course on working remotely but they will need steps to take to get connected. They will need help setting up virtual meetings. What can you provide them to make their life easier and to prevent a flood of calls to IT or HR for support? And, building on this, moving forward what opportunities do you have to replace some of your existing ILT or e-learning with performance support? Take this opportunity to truly integrate performance support into all that you do.

On a personal note, I wish each of you the best and hope you stay healthy. The situation will stabilize and eventually return to normal, but in the meantime let’s see what we can do to help each other through these challenging times.

Top Five New Year’s Resolutions for the Learning Profession

January is always a good time to reflect on the past and ponder the future. It’s also a good time to make some resolutions to improve in the coming year and decade. Here are my top five suggestions for the profession in the areas of measurement, analytics, reporting, and running learning like a business.

1. Resolve to measure more at levels 3 (application) and 4 (impact).

In November, ATD released its latest survey on evaluation that showed 60% of respondents evaluated at least one program at level 3 which is up from 54% in the 2015 study. Likewise, 38% measured at least one program at level 4 which is a slight increase from 35% in 2015. The good news is that the numbers are moving in the right direction. The bad news, is that they should be much higher. Over 80% measure levels 1 and 2. This should be the target for level 3 as well. And, we should measure level 3 for more than just a few programs. It should be measured and managed for all important programs. Once we start measuring application at a higher rate, we can turn our attention to measuring more programs at level 4. Phillips has shown us that this measure is what CEOs most want to see from us and we’re not giving it to them. We need to fix this.

2. Continue our shift from order takers to strategic business partners.

This will require us to have a discussion with the CEO before the year starts, to discover the goals fro the coming year and the goal owners. We need to talk with the goal owner to determine if learning has a role to play, and if it does, work with their staff to recommend an appropriate program. The goal owner then needs to approve the program and we need to reach agreement on the planned impact, other key measures of success, timing, and roles and responsibilities. One of the most important responsibilities of the goal owner will be to ensure that supervisors reinforce the learning with their employees in order to achieve a high application rate. All of this should be completed before the new year starts.

3. Be more disciplined and honest about how well aligned courses are with your organization’s goals.

Most learning departments would say they are well aligned but really, they’re not. What do I mean by “alignment”? It means the courses were designed (or purchased) in response to an identified need in support of a goal. If you say you are “aligned” to your organization’s top five goals, you should have a strategic alignment table which lists the top five goals—and under each one, you show the programs designed (or purchased) specifically to help achieve that goal. Instead, most use backward mapping, which starts with the course catalog and then seeks to map a course to all relevant goals. Since most courses indirectly support most goals, many conclude they are well aligned. For example, you might find that courses to improve communications skills are “aligned” to each of the top five goals (like increasing sales). The two approaches are very different—which explains why in surveys such a high percentage report they are aligned.

4. Be more sophisticated in your analysis of data.

Historically, data has been aggregated and reported as a total or average. Say for example, we report that the average participant reaction (level 1) is 76% favorable or that the average application rate is 51%. These are excellent summary measures but often do not tell the entire story. Perhaps the overall 76% favorable rating is due to the majority of participants being very satisfied but a sizable minority being very dissatisfied. We would want to know this information so we can explore the reasons (i.e., wrong target audience, bad instructor, etc.). Look at the distribution, not just the average or total and then use the information at a “by-name” level to take action (i.e., letting the supervisor know they are going to have to work harder with an individual to get application). This is the potential of microdata, which is really just a fancy term for data at the individual employee level. Ken Phillips is doing interesting work in this area.

5. Create and use more management reports.

Today, the industry primarily generates scorecards and dashboards. Scorecards show historical data in tabular form (like number of participants or courses for the last six months) while dashboards include some visual and/or interactive displays. Both however are designed to inform by sharing historical results. In other words they are “backwards” looking. Of course, it is important to know where we have been so we need scorecards and dashboards—however we also need to look forward. For this, we have management reports that show the plan (or targets) for the year, comparison of year-to-date results with plan, forecast (how the year is likely to end), and forecast compared to plan. They are designed to help leaders manage their initiatives to deliver promised results by answering two questions: “Are we on plan year to date?” and “Are we going to end the year on plan?” Management reports complement scorecards and dashboards by providing that forward look which can be used to actively manage initiatives.

 

Corporate Learning Week 2020

 

 

 

Corporate Learning Week | March 23-26 | Austin TX | www.clweek.com

This year’s Corporate Learning Week (CLW) conference celebrates 25 years as an essential gathering for senior corporate learning leaders and innovators. If you’re a learning, training and development professional who wants to increase performance, maximize profits, make more strategic and tactical decisions, and offer more effective training and development to future-proof your organization’s workforce knowledge shortages—then, this conference is for you.

This year’s event will focus on helping L&D leaders to utilize innovative technology to deliver measurable, quantifiable results—results that empower diverse talent and ultimately drive business outcomes. In short, CLW 2020 will help address the fundamental questions and challenges that remain in today’s environment:

  • How do you make your L&D more effective in face of disruption?
  • How do you find and realize new training potential in a rapidly digital environment?
  • How do you align your training and development outcomes to corporate goals?

CLW 2020 will equip L&D practitioners with practical skills to go from strategy-setting to practical execution to communicate the business outcomes and results back to senior executives.

Reserve your spot today!

Upskilling Revisited: What Strategy Makes the Most Sense?

In last month’s blog I shared my skepticism about some of the current upskilling initiatives. I have been thinking more about it since then. In fact, it is hard not to given the near daily references to the need for upskilling. Just this morning I saw a reference to a World Economic Forum study on the future of jobs which indicated that 54% of employees will require significant reskilling or upskilling by 2022.

Several things come to mind. First, I wonder about the terminology. How should we define these two terms? The term “upskilling” seems to suggest providing a higher level of skill while “reskilling” suggests a different skill, which may be higher, lower, or the same in terms of competency. Is that what we mean? And how does this differ from what we have been doing in the past? Haven’t you been reskilling and upskilling your employees for years? Isn’t that what the term training means? (Unless you have been “downskilling” your employees!) Is “reskilling” just the new term for “training”? Don’t get me wrong. If you can get a larger budget by retiring the term “training” and emphasizing “reskilling or upskilling”, then go for it. But let’s be careful about how we use the terms.

Second, I wonder about the methodology being used to conclude that upskilling is needed. As I mentioned last month, one of my worries is that we are not doing a needs analysis to confirm this need or identify precisely what the need is. I know some organizations are hiring management consultants to tell them what skills will be needed in the future but how do the consultants know what your needs will be? I know other L&D departments are asking their own organization leaders the same question, but how good is the knowledge of your own leaders? My concern is that this methodology will generate the type of common, somewhat vague, needs we now hear about all the time like digital literacy or fluency.

Third, what exactly do we do once we have identified these vague needs? A traditional needs analysis would identify the performance that needs to be improved and then, assuming learning does have a role to play in improving that performance, recommend training specifically designed to close the gap. If we were to provide training to improve digital fluency, exactly what will be improved? Are we back to increasing competency simply to increase competency? Training for training’s sake? So, my concern here is that even if we increase competency in these areas of “need”, nothing of measurable value will improve.

Fourth, what is the time frame? Put another way, how exactly is the question being asked of leaders about future skills? Is it about “future skills” with no time period specified, or is it about skills that will be needed in one year, five years, or ten years? When does the “future” start? It seems to me that the further out we go, the more likely we are to mis-identify the actual needs, especially given the rapid pace of change. Even if we could identify a need one or two years in the future, do you really want to develop a course today to address it, given that the need may evolve? Moreover, if you begin upskilling employees today, they are not going to have an opportunity to apply the new skill for one or two years, by which time they will have forgotten what they learned.

Here is my suggestion for discussion. Doesn’t it make more sense to focus on the skills needed today and in the very near-future and to deploy the learning at the time of need rather than months or years ahead of time? Ask leaders what skills their employees need today or will need in the very near-future, and then follow up with a proper needs analysis to confirm or reject their suggestions. If we can do a better job meeting the current and very near-future needs of the workforce, won’t our employees always be well positioned for the future? And isn’t a reskilling/upskilling initiative tailored to specific employees and delivered at the time of need likely to be far more impactful than an approach based on vague needs for a large population delivered months or years before the new skills will actually be used?

Reskilling—Are We Thinking About This Clearly?

It seems that “reskilling” is on everyone’s mind these days. How often do we hear about an organization’s need to reskill its workforce for the future? And, this isn’t just coming from L&D—it is also coming from senior leaders. So, reskilling is the shiny new object that is attracting a lot of attention; and as my regular readers know, the brighter that new object, the more skeptical I become that we are on the right path.

Before sharing my doubts about reskilling, let me first acknowledge that there are situations where it makes perfect sense. For example, suppose that a manufacturing plant is going to be modernized in six months. The current employees do not have the skills to run the new automated plant but some would be interested in learning. I think this would be a perfect case for reskilling so the company can keep its existing employees who are loyal and committed to the success of the organization. By all means, provide the training for the new skills they will need to succeed. And, if there are not enough positions in the newly automated factory, provide new skills for the remaining employees to fill other positions in the company. So, I don’t have any issue with this use of reskilling where specific, existing employees need new skills to do a different job because their current job is about to be eliminated.

Let’s explore the other end of the reskilling spectrum, where I do have serious doubts about our current path. Some organizations are looking ahead and trying to define what skills will be needed in the future, often five to 10 years in the future.

In my experience, it is sometimes difficult to get good direction and consensus on skills needed in the short term (like for the next year), yet alone for the future. I have a hard time imagining how the process will work to define skills needed in five to 10 years. Who will L&D rely on to define these needs? Department heads, the CFO, CEO, governing body, a consulting company? How good is their forecast likely to be? Do these people have a track record for forecasting skills or will this be their first time? I can say from my experience at Caterpillar that making accurate long-term economic sand sales forecasts was virtually impossible. So, in a fast-changing world, I am skeptical of our ability to accurately forecast skills five to 10 years out.

Even if we could accurately forecast skills, does it make any sense to start training for those skills today? The assumption here is that the skills are not needed today but will be needed in the future. Given what we know about how quickly knowledge dissipates and about the importance of immediately applying what was learned, why would we start training now for a future need? Wouldn’t it be better to wait until just before the need actually exists and then offer the training, combined with the opportunity to immediately apply it and receive reinforcement from management?

Now, at this point, some may say that the time frame is not five to 10 years but instead just a few years—maybe even just one year ahead. Furthermore, some may say that leaders can readily identify the skills their current employees have and that these skills will be even more in demand in the future. Examples of this may be processes like design thinking, adaptability, communication and teaming. In this case, we can use our standard needs analysis and performance consulting process to identify the skills gaps and address them. Now however, I would contend that we are no longer talking about reskilling, but instead traditional training.

I believe reskilling connotes new skills that are not needed in one’s job today, but will be needed for a different job either today or in the future. And, I think it makes perfect sense to provide these new skills to existing employees whenever there is a good fit and when the need is immediate. I don’t believe we should spend much time trying to identify skills that will not be needed for five to 10 years down the line; and I certainly don’t believe we should start training for those skills today. Instead, let’s focus our reskilling efforts on those employees who need (or want) to find a new job in the next 12 months and who require new skills. For the vast majority of employees who simply want to keep learning to improve in their present position, let’s just continue to call this “training”.

Human Capital Disclosure May Soon Be Mandated by the SEC

In an earlier blog I talked about the ISO standards for human capital reporting which were published in December. These called for the voluntary public disclosure of measures for both large organizations and small/medium-sized organizations. While some European and Asian governments are likely to adopt the ISO recommendations as law, the United States Congress was never likely to follow suit so it appeared that adoption in the U.S. would be voluntary and slow.

The U.S. Securities and Exchange Commission just published proposed rule making which, if implemented, will bring human capital disclosure to U.S publicly traded companies much sooner than anyone imagined. This is a game changer for our profession and a VERY BIG DEAL!

On August 7 the SEC proposed to fundamentally change the way publicly traded companies report. Under the current rules, the SEC specifies 12 items that must be included in the narrative description which accompanies the financial statements. Companies have to address all that apply to them. For example, these items include a discussion of the principle products and services offered, new products and segments, sources and availability of raw materials, whether the business is seasonal, competitive conditions, and the current backlog of firm orders. The last required item is the number of employees, which is the only human capital measure currently required for disclosure.

The SEC proposes to move away from an explicit list of required items and instead adopt a principles-based approach where each company must discuss whatever is material with the understanding that the list of topics will differ by industry and company. The SEC does, however, share a non-exclusive list of items that it believes will apply to most companies. This list includes five items from the current list of 12 and two new items: HUMAN CAPITAL and compliance with government regulations. (The other five items are revenue-generating activities, development efforts for new products, resources material to the business, any business subject to re-negotiation or cancellation, and seasonality.) Furthermore, simply disclosing the number of employees is no longer sufficient. In their own words:

“Item 101(c) (1) (xii) [the current rules] dates back to a time when companies relied significantly on plant, property, and equipment to drive value. At that time, a prescriptive requirement to disclose the number of employees may have been an effective means to elicit information material to an investment decision. Today, intangible assets represent an essential resource for many companies. Because human capital may represent an important resource and driver of performance for certain companies, and as part of our efforts to modernize disclosure, we propose to amend Item 101 (c) to refocus registrant’s human capital resources disclosures. Specifically, we propose replacing the current requirement to disclose the number of employees with a requirement to disclose a description of the registrant’s human capital resources, including in such description any human capital measures or objectives that management focuses on in managing the business.” (page 48 of the proposed rule Modernization of Regulation S-K Items 101, 103, and 105)

Did you just feel the earth shifting below your feet? You should have. The importance of this proposed rule for our profession simply cannot be overstated. Many in the profession have worked years to increase the visibility and use of human capital measures. The time may finally have come for the U.S.

While it is true that the SEC will not prescribe specific human capital measures, it does provide some examples, including measures for attraction, development and retention of personnel. The test for disclosure, however, is clear: materiality. Can you imagine any CEO or CFO telling analysts, the public and their own employees that people are not a material contributor to the company’s success, especially after saying for years that people are the company’s greatest asset? I don’t think so. So, if this rule making is finalized, human capital disclosure is coming and coming soon.

Most companies will rely on their heads of HR as well as accounting for guidance on what to include in their narrative on human capital. If for no other reason than risk mitigation, these leaders in turn will look to the human capital profession for guidance. And they will find ISO30414:2018 which are the human capital reporting standards published in December 2018. These standards recommend the external reporting of 23 measures for large companies and 10 for small medium. These measures will be a natural starting point as companies decide what to discuss so if you don’t yet have a copy, get one, and be prepared to proactively help your organization be a leader in human capital reporting. The ISO document is available for purchase at https://www.iso.org/standard/69338.html.

The proposed SEC rule is available at https://www.sec.gov/rules/proposed/2019/33-10668.pdf. The rule is 116 pages, but the section on human capital is under section IIB7 pages 44 -54.

Can you Justify your Learning and Development Projects?

By Jack J. Phillips Ph.D., Chairman, ROI Institute

Daily headlines in the business press focus on the economy. While it is booming in some areas, other areas are slowing, and economic uncertainty exists everywhere. During uncertainty, executives must take steps to make sure the organization can weather the storm—whenever or wherever it occurs.

One way executives meet this
challenge is to ensure that expenditures represent investments and not just
costs. If an expenditure is considered a cost, it will be frozen, reduced, or
in some cases eliminated. If it is considered an investment that is producing a
return, it will be protected and possibly even enhanced during difficult times.
For example, many learning and development budgets are now being frozen or
reduced, even with record profits. While this seems illogical, it happens. Now
is the time to reflect on your budget and your programs. Can you withstand top
executive scrutiny? Are you ready for ROI?

The most used evaluation system
in the world is the ROI Methodology® to measure impact and ROI for a few major
programs. The ROI Certification® is the process to develop this valuable
capability. This valuable certification provides participants with the skills
needed to analyze the return-on-investment in practical financial terms. The
results are CEO and CFO friendly. Over 15,000 managers and professionals have
participated in this certification since it was launched in 1995, underscoring
the user-friendly nature of this system.

Don’t have the time or budget?
Several approaches are available to reduce the amount of time and cost needed
to develop this capability. For more information on ROI Certification, contact
Brady Nelson at brady@roiinstitute.net.

ROI Institute is the global
leader in measurement and evaluation including the use of return on investment
(ROI) in non-traditional applications. This methodology has been used
successfully by over 5,000 organizations and in over 70 countries.

Bridge the Gap from Training to Application with Predictive Learning Analytics

by Ken Phillips, CEO, Phillips Associates

In my previous blog post, I discussed the concept of scrap learning and how it is arguably the number one issue confronting the L&D profession today. I also provided a formula you could use to estimate the cost of scrap learning associated with your training programs.

In this post, I’ll share with you a revolutionary methodology I’ve been working on for the past several years called Predictive Learning Analyticts™ (PLA). The method enables you to pinpoint the underlying causes of scrap learning associated with a training program. It consists of three phases and nine steps to provide you with the data you need to take targeted corrective actions to maximize training transfer (see figure below). 

While the specific questions and formulae for the scores are proprietary, I hope you can apply the concepts in your organization using your own survey questions and your own weighting for the indexes. Even if you adopt a simpler process, the concepts will guide you and the article will give you an idea of what is possible.

Phase 1: Data Collection and Analysis

Unlike other training transfer approaches which focus mostly on the design and delivery of training, PLA offers a holistic approach to increasing training transfer. Built on a foundation of three research-based training transfer components and 12 research-based training transfer factors (see chart below), PLA targets the critical connection among all these elements. In short, PLA provides L&D professionals with a systematic, credible and repeatable process for optimizing the value of corporate learning and development investments by measuring, monitoring, and managing the amount of scrap learning associated with those investments.

Training Transfer Components & Training Transfer Factors

Phase 1: Data Collection & Analysis

The objective of phase one, Data Collection & Analysis, is to pinpoint the underlying causes of scrap learning associated with a training program using predictive analytics and data. Five metrics are produced and provide L&D professionals with both direction and insight as to where corrective actions should be targeted to maximize training transfer. The five measures are:

  • Learner Application Index™ (LAI) scores
  • Manager Training Support Index™ (MTSI) scores
  • Training Transfer Component Index™ (TTCI) scores
  • A scrap learning percentage score
  • Obstacles preventing training transfer

Data for calculating the first three measures: LAI, MTSI, and TTCI scores, is collected from program participants immediately following a learning program using a survey. The survey consists of 12 questions based on the 12 training transfer factors mentioned earlier. Data for calculating the final two measures are collected from participants 30 days post program using either a survey or focus groups and consists of the following three questions:

  1. What percent of the program material are you applying back on the job?
  2. How confident are you that your estimate is accurate?
  3. What obstacles prevented you from utilizing all that you learned if you’re not applying 100%?

Waiting 30 days post program is critical because it allows for the “forgetting curve” effect—the decline of memory retention over time—to take place and provides more accurate data.

LAI Scores

LAI scores predict which participants attending a training program are most likely to apply, at risk of not applying and least likely to apply what they learned in the program back on the job. Participants who fall into the at-risk and least likely to apply categories are prime candidates for follow-up and reinforcement activities. Examples include email reminders, micro-learning or review modules, and coaching or mentoring to try and move them into the most likely to apply category.

MTSI Scores

MTSI scores predict which managers of the program participants are likely to do a good or poor job of supporting the training they directed their employees to attend. Managers identified as likely to do a poor job of supporting the training are prime candidates for help and support in improving their approach. This help might take the form of one-on-one coaching; a job aid explaining what a manager should do before, during, and after sending an employee to training; or creating a training program which teaches managers how to conduct pre- and post-training discussions with employees.

TTCI Scores

TTCI scores identify which of the three training transfer components and the 12 training transfer factors affiliated with them are contributing the most and least to training transfer. Any components or factors identified as impeding or not contributing to training transfer are prime candidates for corrective action.

Scrap Learning Percentage

The scrap learning percentage score identifies the amount of scrap learning associated with a training program. It provides a baseline score against when follow-up scrap learning scores can be compared to determine the effect targeted corrective actions had on increasing training transfer.

The obstacles data identifies barriers participants encountered in the 30 days since attending the training program that prevented them from applying what they learned back on the job. Waiting 30 days to collect the data allows for the full range of training transfer obstacles to emerge. For example, some are likely to occur almost immediately—I forgot the things I learned—while others are likely to occur laters—I never had an opportunity to apply what I learned. Frequently mentioned obstacles are prime candidates for corrective actions to mitigate or eliminate them.

Phase 2: Solution Implementation

The objective of phase two: Solution Implementation, is to identify, implement, and monitor the effectiveness of corrective actions taken to mitigate or eliminate the underlying causes of scrap learning identified during phase one. Here is where the “rubber meets the road,” and you have an opportunity to demonstrate your creative problem-solving skills and ability to manage a critical business issue to a successful conclusion. Following the implementation of the corrective actions, it is now time to recalculate the amount of scrap learning associated with the training program. You can then compare the results to the baseline scrap learning percentage calculated during phase one.

Phase 3: Report Your Results

The objective of the third phase: Report Your Results, is to share your results with senior executives. Using the data you collected during phases one and two, it is time to show that you know how to manage the scrap learning problem to a successful conclusion.

In Sum

Scrap learning has been around forever, however what is different today is that there are now ways to measure, monitor, and manage it. One of those ways is through Predictive Learning Analytics™. Alternatively, you might employ the concepts to build your own simpler model. Either way, we an an opportunity to reduce scrap learning.

If you would like more information about the Predictive Learning Analytics™ methodology, email me at: ken@phillipsassociates.com. I have an ebook that covers the method and a case study illustrating how a client used the process to improve the training transfer of a leadership development program.

Business Alignment: A Critical Success Factor for L&D Organizations

by Peggy Parskey, Associate Executive Director, Center for Talent Reporting

Business Alignment
Signpost with Alignment wording

If you have attended a Learning and Development (L&D) industry conference within the past three years or listened to a panel of senior L&D leaders, it’s likely that someone has raised the topic of alignment. Numerous blogs from ATD, SHRM or even technology vendors confirm that alignment is clearly top of mind.

Given the attention paid to the topic of alignment, you might think that L&D has nailed down the principles, process, and outputs of business alignment. Unfortunately, you would be wrong. We haven’t nailed down this process by a long shot. And therein lies the problem: The L&D industry does not have a consistent definition of what alignment is, let alone how to achieve it.

Do a quick Google search on L&D business alignment and you will get thousands of articles on the topic. Click some of the links and you will find as many suggested approaches to alignment as search results. Some authors suggest that business alignment is about getting the right KPIs to support business goals. Other suggest that alignment is about doing a gap analysis between the current and future state to identify the needed training programs. Some bloggers recommend conducting interviews with business leaders as input to an L&D strategy. And a few organizations with whom we have worked view alignment as a simple mapping exercise: “We need to grow sales this year, we have a bunch of sales programs. Voila, Alignment!”

With all of these possible approaches, how should an L&D leader navigate the alignment process?

Principles of Effective Alignment

At least four principles should guide leaders who are grappling with achieving business alignment:

  1. Alignment isn’t a nice to have, it’s a must have
  2. Alignment requires engagement and commitment from both L&D and business leaders
  3. Alignment has both strategic and tactical components
  4. Alignment isn’t simply about what L&D offers but also how it offers them

1. Alignment Isn’t a Nice to Have

Strategic and tactical alignment is critical to ensure that L&D is investing its scarce resources in the right place at the right time on the right programs. When L&D doesn’t execute this process or doesn’t manage it effectively, the consequences can be significant. If L&D delivers the wrong programs to the wrong audiences, the organization will lack the needed capability to achieve its goals. Furthermore, other organizations now have to pick up the slack created by L&D. Non-L&D functions may develop shadow learning organizations, or hire external resources to fill the gap left by L&D. If L&D wants to fulfill its purpose to develop the needed capability for the current and future workfoce and improve L&D performance, then effective alignment is the critical success factor.

2. Alignment Requires Mutual Engagement

Discussions about L&D alignment often imply that the business has a peripheral role to play. L&D leaders get the strategic goals from above or they interview a few business leaders to understand priorities. At that point, the business seems to disappear from view.

If L&D leaders want to achieve effective alignment, the business can’t be a bystander. Senior business leaders must engage with L&D not only to communicate their priorities but also to ensure L&D has a role to play in achieving those priorities. Moreover, the business is an influential voice to ensure that L&D has the appropriate resources and support to deliver on its promise. Furthermore, the business will have the primary responsibility for reinforcement, without which there is likely to be little application to the job. If the business doesn’t understand its role, then it is incumbent on L&D leadership to spell it out and create shared accountability for success.

3. Alignment Has Both Strategic and Tactical Components

The alignment process must start at the strategic level. Business leaders establish strategic priorities, set goals and then allocate resources to achieve them. Based on these priorities. L&D then determines if and where it plays a role. If the organization plans to launch a new product line, L&D and the business must agree that L&D should own the process to train employees on these new products. If the organization wants to capture new demographic for its products, the business and L&D may agree that L&D has a minimal role to play. Regardless of the objective, business and L&D leaders need to clarify if L&D has a role as well as the importance and urgency of that role.

Having reached agreement on the strategic priorities, L&D must also ensure it aligns tactically. As an example, imagine that $500K is allocated to develop customer experience competencies. L&D practitioners then dive into the details to assess specific organizational needs. During this process, performance consultants discover that a primary root cause of underdeveloped customer experience capability is the lack of quality resources for call center personnel. At this point, L&D discovers that its requirements have changed. Training, while necessary, is not at the heart of the under performance. At this point, L&D may simply turn over the requirement to the business to develop the needed content for call center employees and invest its scarce resources elsewhere.

The point of this example is that what appears to be development need at the strategic level, may not translate into a development requirement when practitioners study the requirements more fully. It is incumbent on L&D practitioners to adjust their approach to ensure they stay aligned.

4. Alignment Isn’t Just About What L&D Offers

Alignment isn’t just about what programs L&D offers, but increasingly, how it offers them. In a fast-paced world, instructor-led training (ILT) has a long development cycle, is expensive to produce, and requires a large time investment for learners. Yet, according to ATD’s 2018 State of the Industry Report, ILT methods still comprise 67% of all training hours with self-paced at 29% and mobile learning a mere 2%.

Learners need content at their fingertips. They need a rich reservoir of material in different forms (white papers, how-to guides, videos, checklists, case studies) that are easy to find and easy to consume. Content must be high quality and useful on the job.

Alignment requires not simply that L&D builds capability and improves performance, but also does it in the most efficient and effective manner appropriate to the need.

Conclusion

Alignment is critical to ensure the L&D function meets the needs of the business by building the necessary skills to run the business and achieve its strategic objectives. This post focused on the key principles at the heart of alignment. In subsequent posts we will explore the importance of treating alignment as a continuous process and building skills to manage the end-to-end process effectively.

I recommend you learn more about alignment from respected industry leaders who can provide guidance on how to achieve meaningful alignment with the business. Below are four resources you should check out:

The Business of Learning by Dave Vance. See Chapter 4 which discusses strategic alignment in depth.

Attend the CTR Measurement and Reporting Workshop, which addresses the importance of alignment and how to engage business partners in the process.

Read the recently published IDC PlanScape document on the importance of L&D alignment to maximize the impact of training investment.

Read the upcoming 2nd Edition of “Learning Analytics,” which will be published in February 2020. (The current edition may be found here. Look for the second edition at www.explorance.com in February 2020. The second edition has a chapter devoted to the topic of strategic and tactical alignment.)

We’d like to hear from you. If you’re having business alignment challenges, don’t hesitate to email Dave Vance or Peggy Parskey.

The Greatest Issue Facing L&D Today

Scrap Learning

by Ken Phillips

What is arguably the top issue facing the L&D profession today?

The answer is scrap learning or the gap or difference between training that is delivered but not applied back on the job. It’s the flip side of training transfer and is a critical issue for both the L&D profession and the organizations L&D supports because it wastes money and time—two precious organization resources.

Now, you might be wondering, “How big is the problem?”

Two empirical studies, one by KnowledgeAdvisors in 2014 and one by Rob Brinkerhoff and Timothy Mooney in 2008, found scrap learning to be 45 percent and 85 percent respectively in the average organization. To add further credibility to these percentages, I’ve conducted three scrap learning studies over the past few years and found the scrap learning percentages associated with three different training programs in three separate organizations to be 64 percent, 48 percent, and 54 percent respectively. Added together, these five research studies results in an average scrap learning figure of approximately 60 percent in the average organization.

To further highlight the magnitude of the scrap learning problem, consider its effect on the amount of wasted organizational dollars and time. According to the 2018 ATD State of the industry research report, the average per employee organization training expenditure in 2017 was $1,296 and the average number of training hours consumed per employee was 34.1. Using the KnowledgeAdvisors and Brinkerhoff scrap learning percentages mentioned above, you can see in the table below just how much scrap learning costs the average organization in wasted dollars and time.

Cost of Scrap Learning in Wasted Dollars & Time

Average per employee training expenditure (=) $1,296(X) 45% scrap learning(=) 583 wasted $
The average per employee training expenditure (=) $1,296(X) 85% scrap learning(=) 1,102 wasted $
The average number of training hours consumed per employee (=) 34.1(X) 45% scrap learning(=) 15 wasted hours
Average per employee training expenditure (=) $1,296(X) 85% scrap learning(=) 29 wasted hours

Taking all of this data into account reminds me of James Lovell’s famous quote during his Apollo 13 mission to the moon when an oxygen tank aboard the space capsule exploded putting both the flight and crew in great peril: “Houston, we have a problem!”

If you would like to take a crack at estimating the cost of scrap learning associated with one of your training programs, you can use the Estimating the Cost of Scrap Learning Formula below. To gain the most useful insight, you should make every effort to collect the most accurate data possible for each of the input variables. Also, when selecting an estimated percentage of scrap learning associated with the program, variable 4 in the formula, you should get input from several people familiar with the program such as other L&D colleagues, participants who previously attended the program or perhaps even the managers of program participants and then compute an average of their estimates. Gaining the input of others will increase both the accuracy and credibility of the estimate and remove any concerns that the scrap learning percentage is merely your opinion.

Estimating the Cost of Scrap Learning Formula

Wasted Participant Dollars

The length of a learning program in hours _____
X the number of programs delivered over 12 months _____
X the average number of participants attending one program _____
= the cost of scrap learning in wasted time _____
X the average hourly participant salary + benefits _____
= the cost of wasted participant dollars _____ (A)

Wasted L&D Department Dollars

Administrative expenditures (e.g., materials, travel, facility, facilitator, delivery platform, food, etc.) for one program _____
X the number of programs delivered over a 12-month period _____
X the estimated percent of scrap learning (45-85%) associated with the program _____
= the cost of wasted L&D department dollars _____ (B)

Total Cost of Scrap Learning

Cost of wasted participant dollars (A) _____
+ cost of wasted L&D department dollars (B) _____
= total cost of scrap learning _____

With this information in hand, you are now ready to pinpoint the underlying causes of scrap learning and take targeted corrective actions to mitigate or eliminate these causes and maximize training transfer. How to do this will be part two of this blog article.

Change the Conversation about Funding Measurement

The L&D profession continues to struggle to get budget and attention for more robust measurement strategies. Learning professionals, particularly those with measurement or analytics responsibilities, appreciate the value of measurement and know that further budget would be well spent but often cannot convince senior leaders in learning to make the investment, let alone senior leaders outside learning. So, how do we make the case for more resources?

My advice is that we take a stealth approach. While some senior leaders will readily see the reason for more robust measurement, in my experience most will not. Senior leaders always have many more requests for budget and staff than they can grant, and typically measurement by itself doesn’t rise to the top of the priority list. And, since you’re already providing learning, measurement seems like an add-on or a ‘nice to have,’ but not something that is essential. This is especially true if most of the measurement currently being done is simply to report activity or historical results.

The alternative is to change the conversation. Instead of talking about measurement, talk about what will be required to deliver the planned results from the learning initiative. In other words—talk about management rather than measurement. Of course, you cannot manage without measures, so management will be the trojan horse that gets in measurement. This approach makes measurement the means to the end. It also focuses on the most important purpose or use of measurement which is to manage.

How would this work? Start with programs aligned to your organization’s key goals or needs. For these initiatives you need to partner closely with the goal owner, like the head of sales or manufacturing. Both parties need to agree on program specifics like learning objectives, target audience, and completion date. Most importantly, they need to agree on mutual expectations for the impact of the learning, which may be the isolated impact of the Phillip’s approach or the more subjective expectation of the Kirkpatrick approach.

In either case, both parties will need to agree on all the relevant measures and their targets that are required to deliver the planned impact. These measures would include efficiency measures like number of participants, completion rate, completion date, and cost as well as effectiveness measures like participant reaction, learning, and application rate. These measures will have to be managed to plan the development, delivery, reinforcement, and impact stages in order to deliver the ultimate measure, which is impact. (ex. The program will have to be completed by all participants by April 30 with a 90 percent initial pass rate a 90-day application rate of 85%.)

Notice that you now have a robust measurement strategy which is an integral part of the management for this initiative. In fact, you will not be able to meet expectations without it and consequently you should refuse to do the learning if a senior leader suggests stripping out the measurement. However, since measurement was not presented separately (no budget or staff identified for it, simply part of the plan), the whole issue of stripping it out is not likely to come up. You can follow the same approach for other learning initiatives including those not directly aligned to the goals of the organization. In every case, you should identify some measure of success and should identify the relevant measures and the targets required to deliver the success.

This is the stealth approach—treat measurement as a means to the end—embed measurement in all key initiatives by getting agreement with goal owners and senior leaders upfront on the planned outcome and on targets for all the relevant efficiency and effectiveness measures. In addition to measures to ensure expectations are met, you may need to employ measurement and analytics upfront to better understand the issue, optimum learning solution, or modality. Again, measurement will be integrated into the management of the initiative. It will not be a stand-alone item and will not be presented as such.

You may already have a dedicated measurement group to serve as an integral support staff; however, every program manager needs to work with goal owners and leaders to agree on a measure of success and the relevant efficiency and effectiveness measures and their targets. They need to be comfortable with measurement. If you want to expand your measurement efforts, look for ways to do this as part of key programs and initiatives (the end) rather than just as expanding your measurement group (the means).

In conclusion, try changing the conversation from measurement (which senior leaders generally do not understand or value) to measurement or to help you with your measurement strategy. They really don’t care. Instead, engage them to manage their program to deliver planned results which they do care about and which, by necessity, will include measures. Try this approach and see how it works for you.

 

2019 CTR Conference a Great Success

We are just weeks back from our Sixth Annual CTR Conference
in Dallas. We weren’t sure it could top last year’s but I think it did. We had
117 participants and they were very enthusiastic and engaged. Lots of great
sessions and discussions. And of course some delicious food in Grapevine!

Patti Phillips kicked it off with a talk on ROI, emphasizing
how important impact and ROI are and how they can be calculated. She included
some very interesting history and the story of how she met Jack. When we came
back together later in the morning after breakouts, Peggy facilitated a panel
on scrap reporting and measurement literacy, and then Brenda Sugrue, 2018 CLO
of the Year, shared some thoughts from her career. Six more breakout sessions
followed in the afternoon leading up to the awards ceremony at the end of the
day.

CTR recognized Jack and Patti Phillips as the first
recipients of the Distinguished Contributor Award, acknowledging their truly
significant contributions to our field including the 1983 Handbook of Training
Evaluation, the concepts of isolated impact and ROI, and the successful
implementation of ROI in over 70 countries. Then we recognized the winners of
the TDRp Excellence Awards: SunTrust Bank for L&D Governance, and Texas
Health Resources and Heartland Dental for L&D Measurement. Jack and Patti
recognized three winners for ROI studies as well.

Day two began with Steven Maxwell of the Human Capital Management Institute sharing his perspective of what CEOs and CFOs want from learning as well as his thoughts on the newly released Human Capital Reporting standards, both of which got people thinking. After six more breakout sessions, Justin Taylor, General Manager and EVP of MTM, led a panel on the impact of learning to wrap up the conference. With a focus on just the measurement, reporting and management of L&D, everyone could focus on the issues of most concern to them and share ideas and questions with each other in detail. If you didn’t make this event, you missed hearing from industry thought leaders and leading practitioners.

Next year we are moving the conference to the fall, so mark your calendars for October 28-29, 2020. We will continue to begin CTR Week with a pre-conference workshop on measurement and reporting (October 26-27) and end the week with post-conference workshops..  Next year will be the 10th anniversary of TDRp and we hope you can join us for the celebration.