Investing in employee development has become a good tone for companies, and for some time it was enough to persuade the suits to launch a training program. Now, they’re asking many more questions. The most common: what is the payoff? In 2018, in the US only, total training expenditures amounted to $87.6 billion. It’s only fair that in return, company leaders want to know what value eLearning initiatives bring in terms of numbers.
In this blog post, we’ll introduce you to the ROI methodology and provide you with a detailed plan on how you can show training value to stakeholders. But first, take our short quiz to find out if your company is ready to get started with an ROI evaluation.
Return on investment, or ROI, is an ultimate measure of success that shows the ratio of what you put in and get back as a result. The idea of calculating ROI is to convert both training costs and benefits into a single value, e.g. dollars. This is actually great because you can make a direct comparison and come to a clear conclusion on the effectiveness of training.
The superpower of ROI is that the metric is familiar to any accountant or financial expert, and has credibility among executives. And that’s a big deal. If they can easily see the relation between training efforts and business results, the training project draws much more attention and thus gets more respect and resources.
The formula for calculating eLearning ROI looks like this:
A positive ROI means that the training has paid off, and a negative ROI indicates losses.
Example 1: The Beta company has invested $65,000 in employee training. As a result, they managed to increase sales and earned $175,000.
ROI = (175,000 – 65,000)/65,000*100 = 169%
So, for every dollar invested in the program, the company got $2.69.
Example 2: The Gamma company spent $25,000 on training. Trained employees brought the company an additional $7,500.
ROI = (7,500 – 25,000)/25,000*100 = -70%
In this example, the ROI is negative, which means that obtaining an additional $7,500 wasn’t worth the efforts and resources put into the training.
It’s easy to multiply and divide numbers when it comes to abstract examples. In practice, finding out how much you managed to earn by assigning employees a certain course can be a challenge, especially if the course is not directly related to sales or something that has a direct monetary value.
It is worth remembering that the ROI formula itself isn’t a magic ball that answers any question. Let’s say a group of analysts told you that the ROI of your last course was 7%. So what?
Without additional data gathered, the ROI metric doesn’t tell the whole story. For a correct interpretation of the results, we need an established methodology that includes other performance metrics as well. Let’s look at this methodology in more detail.
The most renowned model for measuring training effectiveness is Donald Kirkpatrick’s 4-level model. The stumbling block in Kirkpatrick’s methodology was that he supposed that trying to measure training results in money was often impossible or didn’t make sense. That’s why Jack Phillips upgraded Kirkpatrick’s model and presented his 5-level ROI Methodology.
A comparison of the Kirkpatrick and Phillips models
Phillips refined the model and added Level 0 (Inputs) and Level 5 (ROI) — the hard numbers stakeholders are craving. He also developed approaches for isolation of learning effects from the influence of other factors, such as marketing. This seriously increases the credibility of the whole evaluation.
“Yes, all the solutions are working in harmony, but you need to know the relative amount of each contribution. If you do not know what caused the results, you can’t manage it. You can rest assured that marketing will show the effects of advertising, promotions, and even pricing strategies. Marketing is ahead of us on this issue and they work hard at it. If we do nothing, as the Kirkpatricks suggest, we lose credibility and support.”
According to Phillips’s methodology, ROI evaluation consists of four stages: planning, data collection, analysis, and reporting.
The ROI Methodology Process Model. Source: ROI Institute
Clear goals put down on paper guarantee that you’ll be asking the right questions and won’t miss anything while collecting information. Patti Phillips recommends elaborating objectives from the top and then go down the list, starting from ROI metrics and finishing with learners’ reactions to the training.
| Learning objective example | |
| Level 5: ROI | ROI 150% |
| Level 4: Results | Reduce the number of defective items by 20% in 12 months. |
| Level 3: Behavior |
|
| Level 2: Learning |
|
| Level 1: Reaction | The course rating is 9 points or higher (out of 10 possible). |
At this step, you develop and work through three documents: a data collection plan, a data analysis plan, and an ROI calculation plan. This will be your project scope. It’ll allow you to monitor the progress and let you evaluate your own success in the evaluation process.
The project scope should contain information about the objectives of the training, how you’re going to achieve them, a list of tools, and the deadlines. You also need to determine who is responsible for the implementation of each of the plan items.
You can download a free project plan template and other super-useful resources on the ROI Institute website.
If you invest some effort in this task, you’ll get a transparent step-by-step guide for implementing the project. Sign it off with the stakeholders; this will help you make sure that your expectations coincide, and agree on the criteria for marking each stage of the plan as completed.
At this step, we apply the traditional Kirkpatrick approaches: we evaluate the learners’ reaction to the studied content and how well it was digested. After the training is completed, we monitor how the acquired knowledge is applied in practice and what result it brings.
Here are some tools that can be used for gathering information:
| Strongly disagree | Absolutely agree | ||||
| 1. The course topic was relevant to me. | 1 | 2 | 3 | 4 | 5 |
| 2. The instructor has a good knowledge of the subject and is able to provide learners with a comprehensible explanation. | 1 | 2 | 3 | 4 | 5 |
| 3. I will be able to use the information from the course in my daily work. | 1 | 2 | 3 | 4 | 5 |
An example of evaluating changes in employees’ behavior (Level Four).
The business outcome is affected by a large number of different factors, from pricing strategies to the personal charm of a certain sales rep. Isolation is an essential step that helps to determine the amount of improvement directly related to the program. It ensures the credibility of the whole evaluation methodology.
Here are the most common isolation techniques:
To implement this step, you need to put a dollar tag on each result connected with the training program, so that you can compare it with the program costs. As a rule, hard data such as performance, time, and quality can be easily converted:
There is a Step 0: Inputs at the bottom of the Phillips pyramid; now it’s time to revise and fix the final costs of the program. This may include both direct and indirect expenses: instructors’ salaries, the cost of an LMS, the cost of a PR campaign for the program, the purchase of equipment, and so on. And don’t forget to include the cost of the very evaluation of the program performance.
Now, it’s time to get back to the formula we already mentioned:
As a rule, ROI is calculated per year. If there are no desired results in the first year, there’s probably no chance there will be in the future. But in some cases, when you launch a long-term large-scale program, such as a corporate university, it may take several years to achieve the maximum effect. You need to consider that as well.
In most cases, training programs bring intangible benefits along with monetary results. For example, they can improve the public image of the company and increase the team spirit and loyalty of the employees.
The challenge is that soft data can be subjective, so attempts to convert it into monetary values and use in an ROI calculation can seriously affect the accuracy of the final results. Therefore, it’s better to show a lower ROI metric and provide a strong report on intangible benefits than to demonstrate higher ROI while risking the credibility of the entire program evaluation.
If you have doubts on whether to convert or not to convert, use the guidance from the ROI Institute:
To convert or not to convert. Source: ROI Institute
The last but not least step in evaluating training performance is reporting. This step may require some additional planning, as the future of the project depends on it, and the stakes are high.
At this point, you have two major tasks:
a) Prepare reports for different audiences
Think about what kind of reports you need to prepare. Different audiences need different information, but most likely you’ll need two versions of each report: a detailed one and a basic one.
| Audience | Detailed report | Brief report |
| Stakeholders, executives | Complete report on the evaluation | Executive summary |
| Management | Streamlined report, case study | One-page summary |
| PR department | Case study (allowed for external use) | One-page summary |
| Employees | Articles | Brief article or newsletter |
b) Communicate results
How are you going to present the report to a certain audience? Make a list of channels you can use, and adapt reports for them. For example:
| Audience | Live | Online |
| Stakeholders, executives | Face-to-face presentation Printed bulletin | Whitepaper |
| Management | Webinar | Whitepaper Sequence of emails |
| Employees | Brochure Information stand | Video Blog posts |
The classic book on using Kirkpatrick’s model and measuring learning outcomes at four levels.
The absolute must-read book if you’re going to measure ROI for training programs. The author generously shares use cases, ready-made tables, and forms that will save you plenty of time. She even gives an example of a speech to defend the project at the stakeholders’ review.
Everything you need to know about the methodology in less than an hour.
To calculate ROI, you will need to collect and analyze a huge amount of information. The good news is that this process can be automated at least partially. In this article, you’ll learn what kind of statistics can be gathered by an LMS, and what conclusions can be made with its help.