Latest Resources

Are You Spending At Least 20% of Your Time Reinforcing Learning?

By Dave Vance

Training may not be rocket science, but it is a lot more complicated than it first appears. This is especially true when it comes to reinforcing the learning so that it will be applied and have the planned impact. After all, if the learning is not actually applied, there will be no impact and it will be “scrap learning”, regardless of how well it is designed or brilliantly delivered. So, your learning can be well received (high level 1 participant reaction) and the target audience can demonstrate mastery of the content (high level 2), but if it is not applied (level 3) there will be no impact (level 4) and the ROI will be negative (level 5). Unfortunately, all too often we don’t even measure level 3 to find out if the learning was applied. We deliver the training, hope for the best, and move on to the next need. We need to do better.

Our greatest opportunity here is to help the sponsors (who requested the learning for their employees) understand their very important role in managing and reinforcing the learning. Learning professionals can provide advice but cannot do this for them. Sponsors need to understand their role in communicating the need and benefits of the training to their employees before the course is delivered. (This will require clarity upfront on exactly what new behaviors or results the sponsor expects to see.) Ideally, the employee’s supervisor will also communicate expectations immediately preceding the course. The sponsor needs to ensure that the intended target audience takes the course and follow up with those who did not. Once employees have taken the course, the sponsor and direct supervisor need to reinforce the training to make sure the new knowledge, skills or behaviors are being used. They need to again clearly state what they expect as a result of the training and let the employees know what they are looking for. The employees (individually or as a group) should meet with supervisor to discuss the training and confirm application plans. The sponsor will need to be ready with both positive and negative consequences to elicit the desired results. All of this constitutes a robust, proactive reinforcement plan which should ideally be in place before the training is provided.

As you can tell, this takes time and effort. How much effort? My belief is that it will take at least 20% of the time you dedicate to planning, developing and delivering to do this well. So, if it takes five weeks to plan, develop and deliver a course, are you planning to spend at least 40 hours on reinforcement? Many sponsors and supervisors have no clue about the importance of reinforcement and will not do it unless you can make them understand the importance. Many think that once they have engaged L&D, their job is done and training will take care of everything else. However, this cannot possibly be the case. The target audience employees don’t work for L&D, and L&D professionals cannot make employees in sales or quality of manufacturing do anything. L&D certainly cannot compel these employees to apply their learning – only the leaders who asked for the training can do this. So, you need to convince the sponsor that they have a very important role in the training. L&D can do the needs analysis, design and develop the training (with help from the SMEs), and deliver it, but only the sponsor and leaders in the sponsor’s organization can make their employees apply it. Impact and results depend on both L&D and on the sponsor – neither one can do it alone. On this we must stand firm as learning professionals. In fact, if a sponsor tells you that they don’t have time for these reinforcement tasks, you need to respectfully decline to provide the learning because it is likely to be a complete waste of time and money. And don’t let them shift the burden of reinforcement to you. It is your responsibility to assist them but it is their responsibility to do it.

With this understanding, are you dedicating enough time to reinforcement? At Caterpillar we found that we had to redirect resources from design and delivery to reinforcement. In other words, you will likely have to make a trade off. We decided it was better to deliver less learning but do it well (meaning reinforced so it was applied) rather than deliver more learning which would be scrap. What is your strategy?



The Promising State of Human Capital(Report)

CTR is happy to provide the Promising State of Human Capital Report sponsored by CTR, i4cp, and ROI Institute.

This Valuable document  is available for downloa thanks to CTR and our partnerships with i4cp and ROI Institute.  This Report ties in with the ROI Institute webinar with the same name posted on our site for download.

Partnership with the Corporate Learning Network

The Center for Talent Reporting, is pleased to announce a partnership with the Corporate Learning Network. CLN, is an online resource for corporate learning leaders and academic professionals. The Corporate Learning Network believes the Future of Learning will be created through multi-disciplinary approaches and peer-led exchange. With live conferences, community webinars and virtual forums, they bring together stakeholders across the L&D spectrum to help you realize your plans for improved learning outcomes and organizational success.

Learn more about  CLN at

CLN goals are similar to CTR, and we believe this partnership will further the change and growth in HR and L&D.

Management: The Missing Link in L&D Today by Dave Vance

Despite great progress in so many areas of L&D, there is one area which has not seen much progress. This is the business-like management of the L&D department and L&D programs. Yes, department heads work to implement new LMS’s on time, and program managers and directors work to roll out new programs on time but there is still an opportunity to dramatically improve our management. Let’s look at programs first and then the department as a whole.

At the program level a really good manager would work with the goal owner or sponsor to reach upfront agreement on measures of success for the program like the planned impact on the goal. A really good manager would also work with the goal owner and stakeholders to identify plans or targets for all the key efficiency and effectiveness measures that must be achieved to have the desired impact on the goal. Examples of efficiency measures include number of participants to be put through the program, completions dates for the development or purchase of the content and completion dates for the delivery, and costs. Examples of effectiveness measures include levels 1 (participant reaction, 2 (knowledge check if appropriate), and 3 (application of learned behaviors or knowledge). A really good program manager would also work with the goal owner upfront to identify and reach agreement on roles and responsibilities for both parties, including a robust reinforcement plan to ensure the goal owner’s employees actually apply what they have learned. Today, many program managers do set plans for the number of participants and completion dates. Few, however, set plans for any effectiveness measures and few work with the goal owner to reach agreement on roles and responsibilities, including a good reinforcement plan. Virtually none use monthly reports which show the plan and year-to-date results for each measure and thus are not actively managing their program for success in the same way as their colleagues in sales or manufacturing.

At the department level, a really good department head or Chief Learning Officer would work with the senior leaders in the department to agree on a handful of key efficiency and effectiveness measures to improve for the coming year. Then the team would agree on a plan or target for each as well as an implementation plan for each, including the name of the person responsible (or at least the lead person) and the key action items. Examples of efficiency measures to manage at the department level include number of employees reached by L&D, percentage of courses completed and delivered on time, mix of learning modalities (like reducing the percentage of instructor-led in favor of online, virtual, and performance support), utilization rate (of e-learning suites, instructors or classrooms), and cost. Examples of effectiveness measures to be managed at the department level include level 1 participant and sponsor reaction, level 2 tests, and level 3 applications rates. Both the efficiency and effectiveness measures would be managed at an aggregated level with efficiency measures summed up across the enterprise and effectiveness measures averaged across the enterprise. A really good CLO would use monthly reports to compare year-to-date progress with plan to see where the department is on plan and where it is falling short of plan. A really good department head would use these reports in regularly scheduled monthly meetings to actively manage the department to ensure successful accomplishment of the plans set by the senior department leadership team. Today, very few department heads manage in this disciplined fashion with plans for their key measures, monthly reports which compare progress against plan, and monthly meetings dedicated to just the management of the department where the reports are used to identify where management action must be taken to get back on plan.

In conclusion, there is a great opportunity to improve our management which in turn would enable us to deliver even greater results. This requires getting upfront agreement on the key measures, on the plan for each one, and the actions items required to achieve each plan. For the outcome measures it also requires reaching agreement with the goal owner on mutual roles and responsibilities. Once the year is underway, good management also requires using reports in a regular meeting to identify problem areas and take corrective actions. Our colleagues in other departments have been doing this for a long time and with good reason. Let’s get onboard and manage L&D like we mean

The Promising State of Human Capital Anayltics (Webinar by ROI Co-Sponsored by CTR)

Talk with any human capital executive about the field of human capital analytics and they will generally agree: the best is yet to come. That doesn’t mean that many companies aren’t already performing incredible feats with people data—a few are profiled in this report—the statement is a testament to the opportunity that most can see in this burgeoning field. And it’s a testament to the constant new and innovative ways professionals are using people-related data to impact their organizations. This study surveyed analytics professionals in organizations of all sizes worldwide, and asked very pointed questions on how those organizations are using human capital analytics today, if at all. The results were more affirming than they were surprising:

  • Budgets for human capital analytics are increasing along with executive commitment.
  • Relatively few companies are using predictive analytics now, but expect to in the future.
  • Most are using analytics to support strategic planning and organizational effectiveness.

Successful companies tend to be those that purposefully use data to anticipate and prepare rather than to react to daily problems.

It’s clear in both the data from the survey and follow-up interviews that were conducted, that the future focus of
professionals in the human capital analytics field will increasingly be on using analytics to guide strategic decisions and affect
organizational performance. To sum up the state of human capital analytics in one word: Promising.


This webinar introduces the new research study that demonstrates the progress that
continues to be made with human capital analytics. Participants will learn:

  • Four key findings on the state of
    human capital analytics
  • How high-performance
    organizations are building
    leading human capital analytics
  • What Google, HSBC, LinkedIn
    and Intel are doing to drive
    business performance through
  • What you can do to move your
    analytics practice forward

We want to thank Patti Phillips, and Amy Armitage for the opportunity to co-sponsor this webinar. The recording and PPT have been made available to anyone who missed the webinar.


The Promising State of HCA

3.60 MB 25 downloads


Webinar: Innovation in L&D: Building a Modern Learning Culture

Tuesday, July 19 @ 11:00–12:00 p.m. CST




Join Caveo Learning CEO Jeff Carpenter and a panel of forward-thinking learning leaders from Microsoft, McDonald’s, and Ford as they explore innovation in L&D.

More and more, stakeholders throughout the business are bypassing the learning function to create learning outside the learning & development organization. To win back the hearts of these stakeholders (and win a bigger share of the organizational budget), learning leaders must deliver solutions that are exciting, cutting-edge, efficient—in a word, innovative.










By pushing beyond their comfort zone to embrace new ideas, concepts, and technologies, L&D organizations ensure their continued relevance and enhanced ability to deliver tangible business results.

In this webinar, you’ll learn how to:

  • Foster a culture of innovation and creativity in your learning organization
  • Reexamine and reconfigure outdated training through a lens of strategic innovation
  • Develop innovative training and eLearning programs within the confines of business processes and templates

Register today!

Webinar Presenters

Rich Burton, Group Project Manager at Microsoft

Jeff Carpenter, CEO of Caveo Learning (CTR Advisory Council Member)

Gale Halsey, Chief Learning Officer of Ford Motor Company

Rob Lauber, Chief Learning Officer of McDonald’s Corporation

Who Should Attend

Chief Learning Officers (CLOs), VPs of Training, Training Directors and Managers, Human Resources VPs and Directors, CEOs, and COOs.

Webinar: ROI from a TDRp Perspective: Plan, Deliver, and Demonstrate Business Value

August 17, 11am CT


Join Patti Phillips, CEO of ROI Institute, for an introduction to ROI and to learn how ROI and TDRp work together to help you plan, deliver and demonstrate business value. She will share the steps for a successful program implementation starting with the upfront planning and continuing through the calculation of actual ROI at the end. You will come away with a better appreciation of both ROI and TDRp as well as how they can both help you deliver better business value,

Register Today:-

Dr. Patti Phillips is president and CEO of the ROI Institute, Inc., the leading source of ROI competency building, implementation support, networking, and research. A renowned expert in measurement and evaluation, she helps organizations implement the ROI Methodology in 50 countries around the world.

Since 1997, following a 13-year career in the electric utility industry, Phillips has embraced the ROI Methodology by committing herself to ongoing research and practice. To this end, she has implemented ROI in private sector and public sector organizations. She has conducted ROI impact studies on programs such as leadership development, sales, new-hire orientation, human performance improvement, K-12 educator development, and educators’ National Board Certification mentoring.

Phillips teaches others to implement the ROI Methodology through the ROI Certification process, as a facilitator for ASTD’s ROI and Measuring and Evaluating Learning Workshops, and as professor of practice for The University of Southern Mississippi Gulf Coast Campus Ph.D. in Human Capital Development program. She also serves as adjunct faculty for the UN System Staff College in Turin, Italy, where she teaches the ROI Methodology through their Evaluation and Impact Assessment Workshop and Measurement for Results-Based Management. She serves on numerous doctoral dissertation committees, assisting students as they develop their own research on measurement, evaluation, and ROI.

Phillips’s academic accomplishments include a Ph.D. in International Development and a master’s degree in Public and Private Management. She is a certified in ROI evaluation and has been awarded the designations of Certified Professional in Learning and Performance and Certified Performance Technologist.



New Corporate Memberships Benefits

CTR is happy to announce additional benefits for Corporate Members.

  • A Free Q&A with Dave Vance to answer your TDRp, Measurement, or Reporting Questions
  • A Free Review of your List of Measures or Reports
  • A Free Private Intro to TDRp Webinar for Your Team
  • A $100 Discount on the CTR Confernece
  • A $200 discount for the Basics Workshop
  • A $500 discount for a Custom Workshop

Please contact Andy Vance at for more details

Our Newest CTR Advisory Council Member Todd Harrison

Todd Harrison, Ed.D                                                                             Todd

[Director, Talent Solutions]

CTR is pleased to announce Todd Harrison as our newest Advisory Council Member. Todd will be taking over for Kendell Kerekes. We are thrilled to have such an experienced and renowned individual added to our ranks.


Dr. Harrison joined CEB Metrics that Matter (MTM) in 2012, after nearly 15 years of corporate experience in various learning and development leadership roles, where he was an active practitioner of the MTM system.  At MTM, he is accountable for the MTM Professional Services team of approximately 40 people, and provides strategic leadership to the Metrics that Matter (MTM) Client Advisory and Consulting teams within this group.  These two teams have primary responsibility for delivering ongoing support services to MTM technology clients, as well accountability for the development and delivery of learning and human capital consulting measurement solutions, respectively.  Specifically, the Professional Services team helps measure the impact and effectiveness of learning and development programs within an organization intended to unlock the potential of organizations and leaders by advancing the science and practice of talent management.  Dr. Harrison’s business responsibilities include oversight of a portfolio of more than 600 clients and an annual global P&L goal of nearly $5M.

Dr. Harrison has extensive knowledge and expertise in several areas of talent management to include:

·        Succession Planning ·        Leadership Development ·        Talent Analytics
·        Learning Strategies ·        Employee Engagement ·        Competency Design
·        New Hire Onboarding ·        Performance Management ·        Organization Development



  • Doctorate in Organizational Leadership (2016) – Indiana Wesleyan University
  • Master of Arts in Human Resource Development (1995) – Webster University
  • Bachelor of Science in Journalism (1986) – Murray State University

Professional Experience

  • Director, Talent Solutions, Metrics that Matter: CEB, Chicago, IL (2015 – present)
  • Director, Consulting Services, Metrics that Matter, Chicago, IL (2014 – 2015)
  • Senior Consultant, Metrics that Matter, Chicago, IL (2012 – 2014)
  • Director, Global Leadership & Organizational Development, Stryker, Kalamazoo, MI (2010 – 2012)
  • Director, Leadership & Associate Development, Anthem Blue Cross/Blue Shield, Indianapolis, IN (2002 – 2010)
  • Vice President, Human Resources, Total eMed, Franklin, TN (1999 – 2001)
  • Director, Leadership & Organizational Development, American Rehability Services, Brentwood, TN (1997 – 1999)
  • Lieutenant Colonel (Retired), United States Army (1984 – 2005)

Professional Affiliations

  • Association for Training Development (1993 – Present)

CEB Platinum Sponsor

CEB is a best practice insight and technology company. In partnership with leading organizations around the globe, we develop innovative solutions to drive corporate performance. CEB equips leaders at more than 10,000 companies with the intelligence to effectively manage talent, customers, and operations. CEB is a trusted partner to nearly 90% of the Fortune 500 and FTSE 100, and more than 70% of the Dow Jones Asian Titans. More at

CEB is a Platinum sponsor of CTR. With their continued support, CTR has continued its mission to bring standards and measures to the HR community. CEB was a big participant in the February 2016 CTR Conference. Helping to draw many participants making this conference, our largest yet. We look forward to continuing our relationship with CEB.

For information on sponsorship CTR please visit 

Join CEB For a selection of webinars in June.


All time are in CT


Addressing Pay Equity
June 7th 11am-12pm
Join CEB on June 7 for their complimentary webinar as they examine pay equity concerns for organizations with U.S. workforces. Register now at
Keep Quality through Onboarding
Jun 9th 11am-12pm
Best practice onboarding can maintain and actually improve quality of hire by up to 12%. Hear how the best companies prioritize employee integration in onboarding during CEB’s complimentary webinar June 9. Register now at
Organizing HR to Lead Enterprise Change
Jun 9th 9am-13:30am
Only 21% of organizations are able to initiate change as soon as the need arises. Learn how to better influence change implementation where it happens during CEB’s complimentary webinar June 9. Register now at
The Talent Within
Jun 23rd 11am-12pm
74% of organizations want to increase their internal hiring this year. Hear how the best companies improve internal mobility, and yield greater quality of hire during CEB’s complimentary webinar June 23. Register now at
Four Imperatives to Increase the Representation of Women in Leadership Positions
June 28th 12pm-1pm
Increasing representation of women in leadership positions has a real, tangible impact on an organization’s financial and talent outcomes. Join CEB on June 28 as they discuss the common myths surrounding women in leadership, and how to engage and retain a more gender-balanced workforce. Register now at


About CEB

CEB is a best practice insight and technology company. In partnership with leading organizations around the globe, we develop innovative solutions to drive corporate performance. CEB equips leaders at more than 10,000 companies with the intelligence to effectively manage talent, customers, and operations. CEB is a trusted partner to nearly 90% of the Fortune 500 and FTSE 100, and more than 70% of the Dow Jones Asian Titans. More at




Big Data, Analytics, and L&D By Dave Vance

Big Data, Analytics, and L&D By Dave Vance


Big data and analytics continue to generate a lot of excitement in the human capital field. It seems like a new conference is announced almost monthly on the topic. So, what does it all mean for L&D?

Short answer: Not much yet.

Don’t get me wrong. I like data and I like analytics, and the new applications in HR are very exciting, but the buzz is primarily around using statistical tools like regression and correlation to discover relationships among measures (variables) outside of the learning field. Some of the most often cited analytical work is the determination of the factors that explain retention which will allow organizations to take proactive steps to retain desirable employees. There is also work to better explain the factors behind employee engagement, although even this is often in service of increasing retention. I have not seen any conference descriptions or magazine articles where big data and analytics have been applied to learning.

There are certainly applications of analytics to L&D but most of these have been around for a while and are not receiving much current attention. Let’s begin, though, with some definitions.   “Big data” generally refers to the ability to capture more measures at an individual employee level than even before and, once captured, store them in a data base(s) which is easy to manipulate for analysis.  Think of data on employee engagement scores, ratings, individual development plans, past and current assignments, projects, supervisors, referrals, past employment, participation in company clubs and activities, use of benefits, number of sick days, number of vacation days taken, etc., by name for each employee. The “analytics” part is the application of statistical techniques to tease out relationships between these measures which may have been expected or, in some cases, totally unexpected (no one was looking for it). You can see the potential of putting big data together with analytics. For example, you might discover that a leading indicator of regrettable turnover is high use of sick days or lack of involvement in company activities or lack of opportunities to participate in projects. With this type of knowledge you could identify those at risk and intervene early to keep them from leaving.

How might this be applied to L&D? You can imagine using big data to show that application and impact are lower for employees with poor supervisors who don’t reinforce the learning. But we already know this. You can imagine using big data to show the importance of identifying the right target audience, but we already know this, too. I am sure there are opportunities to discover some new relationships, but I believe L&D is more mature than other HR disciplines and thus we already have discovered many important relationships.

While big data is not being used much in L&D today, we have been using “little data” quite effectively for some time. We routinely measure number of participants, courses, hours, costs, completion dates, participant reaction and amount learned. We measure application rates but not nearly as often as we should. Typically we do not put by-name employee measures in a data base which also has information about the supervisor, employee engagement scores and other potentially interesting measures so we end up with “little data” instead of “big data”.

And we have been using analytics for some time. We have been using the intended application rate as a leading indicator of actual application for years. We also use the other “smart” questions from the post event survey as leading indicators for application and impact. Patti and Jack Phillips have helped many organizations use regression analysis to isolate the impact of learning from other factors. This effort has really matured lately with the advent of organizations like Vestrics which use general equilibrium modeling (sets of regression equations estimated simultaneously) to isolate the impact of ALL the factors with a high degree of statistical certainty. So, analytics, including high-level analytics, does have a place in L&D.

In summary, in L&D we have not made much use of big data but we have been using little data for a long time. We have not made extensive use of analytics but we have used leading indicators for years and some are using regression to isolate the impact of learning. In part, the lack of application to L&D currently reflects our relative maturity. We have a better understanding of the basic relationships among measures than our colleagues in talent acquisition or total rewards because we have been measuring longer, and thus there is not as a compelling case to be made for trying to discover new relationships. That said, there are opportunities going forward especially in terms of isolating the impact of learning and demonstrating to nonbelievers the power of strong sponsorship and reinforcement.

CTR Accepts the Community Partnership Award

Dave Vance was present and honored to accept the Community Partnership Award, awarded to CTR by the University Of Southern Mississippi Department Of Human Capital Development, one of the few departments in the country to offer a PhD in Human Capital Development. CTR was selected in recognition of our inviting USM PhD students to our conference in Dallas and for our leadership in the profession though TDRp. It was awarded Friday April 29 at the Third Annual Awards Ceremony held at the Gulf Park Campus. Six other awards were given, and graduating Masters and PhD students were recognized.

Dave Vance, (Executive Director of the Center for Talent Reporting).

Dave is a frequent speaker at learning conferences and association meetings. He also conducts workshops and simulations on managing the learning function. Dave is the former President of Caterpillar University, which he founded in 2001. Dave received his Bachelor of Science degree in political science from M.I.T. in 1974, a Master of Science degree in business administration from Indiana University (South Bend) in 1983, and a Ph.D. in economics from the University of Notre Dame in 1988. Dave was named 2006 CLO of the Year by Chief Learning Officer magazine. He was also named 2004 Corporate University Leader of the Year by the International Quality and Productivity Center in its annual CUBIC (Corporate University Best in Class) Awards. Dave’s current research focuses on bringing economic and business rigor to the learning field as well as the development of computer-based simulations to help learning professionals increase their effectiveness and efficiency. His first book, The Business of Learning: How to Manage Corporate Training to Improve Your Bottom Line, was published in October 2010.

Setting Standards for our Profession: Three Standard Reports By Dave Vance

Last month we talked about the need for standards and introduced three types of measures which would serve to organize our measures just as accountants use four types of measures in their profession. This month we will introduce three standard reports for L&D which contain the three types of measures. These three reports, used together, provide a holistic and comprehensive view of learning activity just as the three standard accounting reports (income statement, balance sheet, and cash flow) do for financial activity.

The three reports are Operations, Program, and Summary. While a version of these reports (called detailed reports) contains just historical information (like actual results by month or quarter), we will focus on reports for executive reporting which in addition to last year’s results include the plan or target for the current year, year-to-date (YTD) progress against plan, and, ideally, a forecast of how the year will end if no special actions are taken (like extra staffing, budget or effort beyond what is currently planned). The reports are meant to be used by the department head and program managers in monthly meetings to actively manage the function to deliver results as close to plan as possible. Of course, before the year starts, L&D leaders will need to decide what measures are important for the year and what reasonable plans or targets would be for those measures.

We will start with the Operations Report which contains effectiveness and efficiency measures. Recall from our discussion last month that effectiveness measures are about the quality of the program (like levels 1-5) and efficiency measures are about how many, at what cost, etc. (like number of participants, hours, utilization rate, and cost). The Operations Report is meant to capture the 5-10 most important effectiveness and efficiency measures at an aggregated level which may be for the enterprise, region, business unit or product group depending on the L&D department’s scope. So, for example, the Operations Report might show last year’s actual, current year plan, YTD results, and forecast for unique and total participants for all the courses being offered. It might also show the average level 1, 2 and 3 for all courses being offered. The department head would use this report monthly to see if the department is on track to meet plan for the year. If it appears the department is not on track, then the senior L&D leaders need to understand why and agree on what actions to take to get back on plan.

The Program Report also contains effectiveness and efficiency measures but its focus is at the program or initiative level rather than the enterprise level. It is designed to show the key programs in support of a single company goal like increasing sales by 10% or reducing operating cost by 5%. Under each identified program would be the most important effectiveness and efficiency measures. The Program Report also would contain an outcome measure showing the planned impact or contribution from learning on the goal. So, the report pulls together the key programs and measures required to achieve the desired impact on the goal. The owner of the goal (like the SVP of sales) and the L&D leaders would agree on the programs; outcome, effectiveness and efficiency measures; and plans or targets for the measures before the programs begin. The goal owner and L&D also need to agree on their mutual roles and responsibilities, including how the owner plans to reinforce the learning. Program Reports would be used by the department head and program managers each month to ensure everything is on plan to deliver the agreed-upon results.

While the Operations and Program Reports are used to manage the function each month, the Summary Report is designed to show the alignment and impact of learning as well as its effectiveness and efficiency. The Summary Report starts by listing the company’s top 5-7 goals in the CEO’s priority order and then shows the key learning programs aligned to each. (In some cases there will be no planned learning.) Where learning does have a role to play in helping achieve the company goal, the report ideally will include an outcome measure or some measure of success. Next, the report will share three-four key effectiveness measures and three-four key efficiency measures. The target audience for this report is the CEO, CFO, SVP of HR, other senior leaders as well as employees of the L&D department. The report lets the senior company leaders know four important things: 1) L&D knows what the company’s goals and priorities are, 2) L&D leaders have talked with goal owners and the two parties have agreed on whether learning has a role to play and if it does, what kind of contribution may be expected from learning, 3) L&D leaders are focused on improving the effectiveness and efficiency of learning, and 4) L&D leaders know how to run learning like a business by setting plans and then managing each month in a disciplined way to deliver those plans.

Use of these three standard reports and the processes associated with creating them will dramatically improve the impact, effectiveness, efficiency, stature, and credibility of L&D. Along with adopting the three standard measures, it will also be a major step towards L&D becoming a true profession with our own standards and reports.

Dave to Talk at the ATD International Conference and Exposition.

Dave be talking at the 2016 International Conference and Exposition in Denver on Wednesday May 25 from 10-11. Topic is Talent Development Reporting Principles: Your Guide to measurement and Reporting for L&D.

Hope to see you there.

2016 CTR Conference Slides

Please visit for access to the CTR conference slides from 2016 and 2014.


Bringing Standards to the L&D Profession: Three Types of Standard Measure: By Dave Vance

By Dave Vance

Accounting has four types of measures (revenue, expense, assets and liabilities) and three standard statements (income or profit & loss, balance sheet, and cash flow). What do we have in L&D which is similar? Five years ago a group of industry leaders came together to consider a standard framework for L&D. The group included leaders like Kent Barnett then at Knowledge Advisors, Tamar Elkeles then at Qualcomm, Laurie Bassi from McBassi and Company, Jack Phillips from the ROI Institute, Jac Fitz-enz from the Human Capital Source, Josh Bersin from Bersin & Associates, Cedric Coco from Lowes, Karen Kocher from Cigna, Rob Brinkerhof from Western Michigan University, Kevin Oakes from i4cp, Carrie Beckstrom from ADP, Lou Tedrick from Verizon, and a host of others – 30 in all, including myself. After 24 revisions, we all agreed on a framework which we believed would begin to provide the type of standards that accountants have enjoyed for some time and which would help us become more professional.

These leaders recommended a framework consisting of three types of standard measures and three standard reports. So three and three versus the accountant’s four and three. The three types or categories of standard measures are effectiveness, efficiency, and outcomes. The three types of reports are Operations, Program, and Summary. The goal was to keep the framework simple and easy to use. We also wanted to build on the excellent work done in our profession over the last 70 years, especially that done by the Association for Talent Development (ATD, then called ASTD).

Let’s look at the measures first. Effectiveness measures are those that address the quality of the learning program or initiative. In learning we are fortunate to have the four levels popularized by Don Kirkpatrick and the fifth level (ROI) popularized by Jack Phillips. These five levels all speak to the quality of the learning. Level 1 measures the participant’s or sponsor’s satisfaction with or reaction to the learning – certainly an initial measure of quality. Level 2 measures the amount learned or transference of skills or knowledge, and level 3 measures the application of that knowledge or change in behavior. If they didn’t learn anything or if they don’t apply what they did learn, I think we can all agree that we don’t have a quality program. (This may reflect a lack of engagement or reinforcement by the sponsor, but we still have a problem.) Level 4 measures impact or results and, since this was the reason for undertaking the learning to begin with, if there are no results it is hard to argue we had a quality program. Last, ROI provides a return on our investment, a final confirmation of quality assuming we have properly aligned the learning to our organization’s needs and achieved high quality on the previous four levels. Most organizations have level 1 and 2 measures but relatively few measure at the higher levels.

Efficiency measures are about the number of courses, participants, and classes as well about utilization rates, completion rates, costs, and reach – to name only the most common. Typically these measures by themselves do not tell us whether our programs are efficient or not. Rather we need to compare them to something else which may be last year’s numbers, the historical trend, benchmark data, or the plan (target) we put in place at the beginning of the program. Now we have a basis to decide if we are efficient and if there is room to improve, to become more efficient. All organizations have efficiency measures with the most common being number of participants, classes, and hours as well as some cost measures like cost per learner or cost per hour.

That leaves outcome measures. Unlike effectiveness and efficiency measures, most organizations are not measuring outcomes and few even talk about them. That is unfortunate because outcome measures are the most important of the three types, especially to senior leaders who make the funding decisions for L&D. In accounting this would be like reporting expense and liabilities but never talking about revenue or assets. No one would have a complete picture of what we do, and it would be hard for anyone to understand why we have the expenses and why we incur the liabilities. So, what are these all-important outcome measures? Simply put, outcomes represent the impact or results that learning has on your organization’s goals or needs. Suppose a needs analysis indicated that your salesforce would benefit from a consultative selling skills program and product features training. And suppose that you and the head of sales agree that it makes sense, and the two of you further agree on program specifics, including mutual roles and responsibilities, especially how the sponsor will reinforce the learning and hold her own employees accountable. Now, what impact on sales can we expect from this training? How much of a difference can training make? A lot? A little? Enough to be worthwhile? This is the realm of outcome measures which will sometimes be subjective (but not always) but very important nonetheless. Sometimes the level 4 impact or results measure from the list of effectiveness measures will do double duty as an outcome measure. That is okay and the same happens sometimes in accounting. Or other measures will be selected. In any case, with outcome measures we are at last focused on how we align with corporate goals or needs and what type of impact we can have, and this is what your senior leaders have been waiting for.

Next month we will look at the three reports and how the three types of measures populate these three reports. In the meantime, as a profession, let’s start talking about these three types of measures. It would be big step forward if we could just adopt a common language, which by the way, is a precondition to be a true profession.


CTR Conference A Great Success


Our 2016 CTR Conference held in Dallas last month was a great success! We had 134 participants, almost double the number at our last conference and far more than we had hoped for.  If you were not there, you really missed a great gathering. Everyone there was interested in measurement, reporting and management so there was a tremendous amount of focused energy, sharing and interaction. Cushing Anderson from IDC and Ed Trolley from NIIT provided the keynotes to start off each day, and they were not shy about sharing their thoughts! We had a great panel on Disruptive Ideas and a very nice tribute to Jac Fitz-enz, the father of human capital analytics. There were 16 breakout sessions hosted by industry thought leaders over the two days and our participants often struggled to decide which session to attend. Unlike many conferences, most of our speakers came for more than just their session and even asked questions of their colleagues during other sessions which livened things up considerably. Finally, we followed the 1.5-day conference with two half-day workshops: CLO for a Day and Strategic Alignment. Attendees really had fun with CLO for a Day which is the only computer-based simulation for our profession.  We are starting to plan next year’s conference so watch for the Save the Date message coming soon.

Conference survey results  

What were the most beneficial aspects of the CTR Conference?

What topics or speakers should we consider for the 2017 CTR Conference?


All Roads Lead to ROI

Have you ever thought about ways to prove the value of your work as well as show proof that it was worth the investment?

If you have, you’re not alone.

As being part of the business world, we all want to know we’re spending money for a good reason.

This one-hour webinar will present real-life case studies detailing what is being accomplished and how it is being used to make human capital analytics work within organizations.

Five types of projects will be discussed:

  • Converting data to money
  • Showing relationships and causation
  • Applying predictive models
  • Conducting impact and ROI analysis
  • Forecasting ROI

It’s time to make human capital analytics work for you! 

Register Now

Develop Your L&D Team’s Competencies to Deliver on Business Goals Webinar Recording

In this webinar, Caveo’s Strategic Learning Partner Alwyn Klein uses real-world case studies and proven concepts to show how to develop your L&D team to become proactive and high performing, balanced with the appropriate mix of roles and competencies. You’ll learn to…

      • Leverage the L&D Compass for your team’s journey
      • Plan team development strategies and tactics
      • Determine future competencies and roles
      • Maintain a focus on current and future business alignment
      • Deliver consistent and measurable business value

Please Visit  for access to the recording.

Develop Your L&D Team’s Competencies to Deliver on Business Goals (Webinar)

Tuesday, March 15 @ 11:00–12:00 p.m. CST

Develop Your L and D Team's Competencies to Deliver on Business Goals

More than half of training spend is done outside of learning & development organizations, and a big reason is the limitations of L&D team members’ competencies.

Strategic thinking, a performance improvement mindset, ROI and financial acumen, effective instructional design, executable program management, and proficient technology skills are just a few competencies that are required for your L&D organization to gain credibility and deliver on business goals.

By taking an inward focus on the development of their teams, learning leaders can solve these common pain points:

  • Team members that lack key L&D competencies
  • Lack of L&D “readiness” to support the next key business strategic initiative
  • Ineffective execution of learning initiatives to meet business goals
  • L&D team members that have “business” experience, but have underdeveloped performance improvement and instructional design experience, or vice versa

In this webinar, Caveo Strategic Learning Partner Alwyn Klein will show you how to develop your L&D team to become proactive and high performing, balanced with the appropriate mix of roles and competencies.

We’ll share real-world case studies and proven concepts to help develop your L&D team. You’ll learn how to:

  • Leverage the L&D Compass for your team’s journey of development
  • Plan team development strategies and tactics
  • Determine future competencies and roles
  • Leverage expert and industry resources
  • Maintain a focus on current and future business alignment
  • Deliver consistent and measurable business value

Register today and take the first step toward building an L&D team that drives business value!

Who Should Attend

Chief Learning Officers (CLOs), VPs of Training, Training Directors and Managers, Human Resources VPs and Directors, CEOs, and COOs.

Webinar Presenter: Alwyn Klein

Alwyn KleinAlwyn Klein is a popular conference presenter and facilitator with over 20 years’ experience in the L&D field, having previously headed up a team of over 60 learning professionals. He is a Certified Performance Technologist with the International Society for Performance and Improvement and a Certified Professional in Learning and Performance with the Association for Talent Development. In 2014, Klein won South Africa’s prestigious Chief Learning and Development Officer of the Year Award. He is constantly working to implement innovative, people-focused training initiatives that are strongly aligned to organizational goals.

We Don’t Exist to Increase Employee Engagement: By Dave Vance

We Don’t Exist to Increase Employee Engagement

By Dave Vance


A surprising number of L&D professionals seem to believe that our primary mission, our main purpose, is to increase employee engagement. They are wrong and their mistaken belief will lead them to misallocate resources, choose inappropriate learning, and deliver suboptimal outcomes.

It is always good to start at the beginning, and the beginning for any department is to know why it exists. What is its mission or purpose? Ideally, learning leaders have thought carefully about this and have created a written mission statement or at the very least can articulate the mission informally. In most organizations, the learning department has the capability and potential to do much more than provide or facilitate learning to increase employee engagement. L&D has the potential to help the organization achieve many, if not all, of its goals. This means helping to achieve business goals like increasing revenue by 10%, reducing costs by 5%, improving patient satisfaction by 3 points, or reducing injuries by 20%. This also means helping to achieve HR goals like improving the leadership score on the annual survey by 5 points, meeting all compliance-related goals, and yes, increasing employee engagement. It  means providing the basic skills needed by employees new to a position and the more advanced skills needed by experienced employees especially in knowledge-based companies like those in consulting and accounting. Last, learning can help address many needs and challenges that fall below these high-level goals but which nonetheless must be addressed for the organization’s overall success.

So, learning has a very broad reach and can help an organization achieve many of its goals and address numerous challenges. A good mission statement should reflect this broad reach. For example, “Help our organization achieve its goals” or “Help our organization be successful”. If L&D’s mission is simply to increase employee engagement, then whose mission is it in your organization to help achieve all the other goals and meet the numerous needs and challenges that can be addressed though learning? It is true that your sales department could do its own learning, quality its own, manufacturing its own, customer care its own, and so forth. Basically, any department that needs learning does its own. That leaves L&D to address just HR goals and maybe just employee engagement if a separate department takes care of leadership development. This is a very sad state for L&D and for your organization as a whole. Think about what this state implies. Most often, other departments do not have learning professionals so the quality of the needs analysis and the learning is low. Someone is assigned to take care of the training needs on a part-time basis and they may not be happy about it. And training people in these departments are all isolated from one another with no opportunity to pool resources, share knowledge, and specialize.

For most organizations, the learning department can have a much more powerful impact if it has a broad mission which includes helping the organization achieve its business goals and if it is organized to support more than a single department. Let’s nor limit ourselves to simply addressing HR goals like increasing employee engagement. Instead, let’s address business and HR goals as well as the basic and advanced skills our employees need for success. What is your mission?

A New Year’s Resolution to Better Manage Learning

A New Year’s Resolution to Better Manage Learning

By Dave Vance

In the spirit of the New Year, here are my suggestions for ten steps to better manage learning to have greater impact on your company and to run your department more effectively and efficiently.

  1. Be clear about your mission. Why was the learning function created and what is it expected to do? Is your primary purpose to help the business achieve its goals, increase employee engagement, improve retention, or what? The answer is hugely important.
  2. If your mission is to help the company achieve its business goals, know what those goals are each year. Meet with the owners of those goals and reach agreement on whether learning has a role to play.
  3. If your mission is to increase employee engagement or retention, then meet with the head of HR and other senior leaders to plan how learning can contribute.
  4. In either case, set specific, measurable outcome-related goals for your learning initiatives. How much of an impact can learning have on achieving the business or HR goals? A lot? A little? You and the owner of each goal need to be crystal clear on this.
  5. Once you have agreement on the planned impact of learning, you and the goal owner need to reach agreement on your respective roles and responsibilities. What, exactly, must each of you do for learning to have the agreed-upon planned impact? The planned impact will only happen if both of you work together to make it happen.
  6. Set specific, measurable goals for the L&D department for the coming year. These generally will involve targeted improvements in effectiveness and efficiency measures across all the learning you offer. Examples of effectiveness measures would be improvement in participant and sponsor satisfaction, first-time pass rates, and application rates. Examples of efficiency measures would be number of participants, percentage of courses or hours that are web-based or blended, percentage of employees reached by learning, and completion dates.
  7. Create a convincing business case for the coming year incorporating the planned impacts and actions and the costs to achieve them. Include this in a written business plan for the year which would include an executive summary, last year’s accomplishments, a discussion of current and proposed staffing and spending, the business case, work plans, and a measurement & evaluation strategy. Have the plan approved by your CEO and governing board.
  8. Create reports containing plan and year-to-date results to use each month to manage the department to deliver the planned results. Update monthly. You should have a report for each major goal supported (program report) and a report for the key effectiveness and efficiency measures (operations report). You should also create a one-two page, high-level report with no learning jargon to share with the CEO, governing body, senior leaders and employees to highlight your alignment to and impact on key company goals and your efforts to improve effectiveness and efficiency (summary report).
  9. Use these reports in regularly scheduled monthly meetings to review progress against plan and, when results are not on plan (which will be often in the real world), to discuss and decide on appropriate management actions to get back on plan.
  10. Look for ways to celebrate your success and to continuously improve. It is a journey and you will never be perfect.

Note that the ten steps above did not include a request for more staff or budget. You can begin implementing these steps with your current resources if you are willing to make some trade-offs. For example, you may need to re-assign some staff or reduce the number of projects. At the end of the day, you may have to decide if you want to do better or if you want to do more. My suggestion is to consider doing better rather than doing more, which in the longer run is actually the best way to also do more. Show your company’s senior leaders you are an outstanding manager of the resources they have given you and they are likely to give you more. So, manage learning with business-like discipline by knowing your mission, aligning your learning priorities to that mission, setting specific & measurable goals, and using monthly reports to ensure that planned results are delivered.

Best wishes to all in this great profession in the New Year!



The Move to Learner-Centric or Self-Directed Learning: Don’t Be Fooled by the Current Hype!

The Move to Learner-Centric or Self-Directed Learning: Don’t Be Fooled by the Current Hype!

By Dave Vance

In just the last month I have seen more and more about the wonderful new world of learner-centric or self-directed learning and it is really beginning to worry me. Article after article talks about the need for companies to stop trying to direct employee learning and instead shift to a role of curator and enabler to make it easier for employees to find whatever learning interests them. Here is what concerns me.

First, the history is wrong. Each article begins by stating that in the “old days” corporate L&D departments directed all the learning for all the employees. This is patently false. At Caterpillar we certainly never dreamed of directing, prescribing or controlling ALL of the learning for ALL of the employees. And I have never met a CLO or L&D department head who believed it was their mission to control all the learning in their company. So, what are these authors talking about? Yes, working with senior company leaders we did develop learning to address particular needs and help accomplish company goals. Yes, we did have orientation programs where we provided helpful learning to new employees who otherwise would not have a clue about what they needed to know. Yes, we had compliance-related learning. And, yes, we provided a number of general education courses and suites of online learning for employees to access to further their own development.  And we also facilitated the creation of individual development plans using required, suggested and discretionary formal and informal learning. But we never even entertained the notion that we, as a corporate university, would prescribe all the formal learning for all the employees or that we would control all the informal learning. The concept itself is ridiculous. So, let’s begin by getting our history right.

Second, the notion that employees always know best and should be left to choose all their own learning is equally ridiculous. Many employees new to a company or position simply don’t know what they need to know. Why wouldn’t the business want to provide direction? A needs analysis may even indicate that experienced employees lack some important skills, knowledge or capability to perform at the desired level. Why wouldn’t the business want to provide that learning to the appropriate employees? Now, it should go without saying that L&D professionals should of course use all the tools available and appropriate to meet the need. So, the “old days” of reliance solely on instructor-led learning should be behind us as should use of only formal learning. Even 10 years ago at Caterpillar we had 35,000 employees actively engaged in 2000 communities of practice and we always considered performance support tools. So, by all means, make full use of all the new learning openly available on the internet both in your business -directed learning and in your employee -directed learning. But don’t stop providing business-directed learning to employees where it makes sense.

Third, the “new role” for L&D is a move in the wrong direction and it is clearly a demotion. Some now suggest that L&D should play a more passive role and simply respond to the desires of employees.  Basically, L&D leaders should find out what employees want and give them more of it. This is fine if the mission of L&D is simply to help boost employee engagement. But our mission should be much, much more. Yes, we can help increase employee engagement by providing or facilitating more learning opportunities, but we are also in the best position to help our companies achieve their business goals. This requires strategic partnerships with the businesses resulting in business-centric or employer-directed learning. More than any other department in a company, L&D is in a unique position to help the company achieve most of its goals by becoming a valued, trusted, strategic business partner. Advocates of self-directed learning appear to be going in the opposite direction and focus only on supporting the goal of engagement or other HR-related goals like retention. HR goals are not the same as business goals like increasing sales, reducing costs, or improving quality. L&D leaders should strive to support BOTH business goals and HR goals – not simply HR goals. So, it would be a huge mistake for L&D to move away from a very active role supporting the company’s business goals to a more passive role supporting only HR goals. I would also suspect that the budget and staffing for an L&D department that moves away from actively supporting business goals will eventually be cut 50%-75% in line with their new nonstrategic, passive role focused only on one HR goal.

Let’s get the discussion going on this topic. Let’s identify where self-directed learning makes sense and where business-directed learning makes sense. Let’s not get carried away by the current hype.

Two Surprises at CLO Symposium By Dave Vance

Two Surprises at CLO Symposium

By Dave Vance


I was surprised by two things at the Fall CLO Symposium in Austin. First, I was surprised by the L&D mission statements of many speakers. Second, I was surprised by what is apparently an industry-wide and unchallenged swing to a learner-centric approach. The two surprises may be related.

My first surprise occurred as I listened to three speakers in a row start their talk by stating the purpose of their L&D group. I applaud them for starting with a mission because I believe a good, clear mission statement is vital. My surprise came as each one stated their mission in a way that was not directly tied to their business. The first and third said the purpose of their function was to increase employee engagement. Don’t get me wrong, I firmly believe that providing employees with learning opportunities will generally increase their engagement. Furthermore, highly engaged employees are likely to work harder and provide more discretionary effort, both of which should translate into better corporate results. And the more engaged they are the less likely they are to leave which avoid the considerable cost of replacing them.

So, engagement is a great thing, but should that really be their primary purpose? In industries with very high turnover, perhaps it will be a top business goal. For most organizations, however, there are a number of business goals which the L&D department should be supporting like increasing revenue and productivity, decreasing operating costs, improving quality and safety, enhancing customer or patient satisfaction, and sparking innovation. Not a single mention of any of these. It seems to me that the primary purpose of L&D should be to help the organization achieve its goals. This should be done by aligning programs to the business goals so you can impact those directly AND by finding ways to increase engagement which will contribute indirectly (but importantly) to the business goals. In other words, I think your CEO would like to hear about more than engagement.

The second speaker said their mission was to “leverage technology”. Period.  This reflects an absolute confusion between the means and the end.  Leveraging technology is a means to an end – it will never be an end in itself. Why do you want to leverage technology? To accomplish what? The “what” would give us insight into the mission. Of course you should leverage technology, but you need to be clear upfront on why. So, three out three speakers did not provide a compelling reason for the L&D function to exist. And we wonder why L&D gets cut first?

My second surprise may be related to the first. For the first day and a half, it seemed every speaker I heard on learning (I did miss some so consider this an imperfect sample) talked about the wonderful move to a learner-centric or self-directed approach. Some companies have already reorganized their departments around this approach. Common elements in support of this approach are trying to find out what learners want and giving them more of it. Much talk about the need to move away from creating content or teaching classes. L&D’s new role will be that of curator. Some at the conference even told me that company goals and learning needs are changing so rapidly now that there is no role for any business-centric or company-directed learning. Worse, I didn’t hear anybody challenge these assertions. If there was a debate over the last couple of years I missed it, and it seems this is now the accepted wisdom in the field. Pardon me, but I object!

We need to have a good discussion around this topic before our profession does some serious damage to itself. Like so much else, it seems to me there needs to be a middle ground here. It is wonderful that employees want to learn and that there are more ways than ever before for them to do just that. The potential to connect with and learn from others has never been greater, especially when you think about social media and all the online learning available. L&D departments should certainly find ways to help employees find the learning they need and do what they can to encourage informal learning. But that does not mean L&D departments should stop providing business-centric learning which we will define as learning planned to help achieve business goals. Senior leaders and experienced employees have wisdom. Why wouldn’t you want to take advantage of that wisdom and let new employees know what they will need to be successful or what experienced employees need to take their performance to the next level? True, a good employee with enough time will probably figure out what they need to know and may be able to secure that knowledge informally, but why not speed the process up? Plus, I think many employees would appreciate the guidance. So, it seems to me there will always be a role for business-directed learning, not just for compliance but to help employees more quickly be as productive as possible in pursuing the business goals.

I think the learner-centric approach does make perfect sense when the learning is not aligned to business goals but in support of an effort to increase employee engagement. Many employees want opportunities to grow and develop outside the requirements to be successful at their current job. Self-directed learning is perfect in this case since each employee will have unique needs, and L&D departments should help facilitate this process by making as many learning opportunities available as possible.

So, my two surprises may be related after all. If the mission of L&D is simply to increase employee engagement, a learner-centric approach makes perfect sense. However, if the mission of L&D is to help the organization achieve its business goals, then there is most definitely still an important role for business-centric learning. Blind acceptance of the current industry trend away from business-centric learning will put an end to our quest to be valued, strategic business partners. If we continue in this direction we will not deserve a seat at the table and L&D budgets should be dramatically reduced.