Many companies implement training to improve their employee productivity and work performance. But how can you actually tell that it works? And how do you know what you should fix in your courses to make them more effective? The best way is to start evaluating training programs.
In this article, you’ll learn what training evaluation is, why it is important, and how it can benefit your company. You’ll also learn about the best evaluation models and tools you can use to assess the quality of your programs.
Training program evaluation is the regular analysis of how efficient and effective training programs are. It involves feedback from learners, comparing their job performance before and after the program, accomplishing the company’s strategic priorities, and much more.
It helps companies find out whether training programs meet their goals and objectives. The results of training evaluation are used to improve training programs, or cancel them if they prove unsuccessful.
Evaluating whether your training is sufficiently effective obviously requires additional resources. Still, the benefits of training evaluation outweigh the challenges. In short, it helps companies:
To evaluate training programs more systematically, you can use specific training evaluation models. There are quite a lot of them, and each offers its own approach to assessing courses. We’ve collected 5 models that have shown to be the most popular and trusted over the years.
One of the oldest and, without a doubt, the most famous one is Kirkpatrick’s model. It is still used more than sixty years after its development.
According to this model, training programs should be evaluated on 4 levels: Reaction, Learning, Impact, and Results. Let’s take a closer look at each level.
When learners complete your course, assess their reactions. Ask them to complete a survey with questions like:
If you want to get more detailed results, use a Likert scale survey. Unlike binary questions, a Likert scale survey shows semitones between good and bad, which can be helpful in training evaluation.
Measure how much was learned in the course. For example, create online quizzes to see what knowledge and skills they have and haven’t acquired during the training. In order to gain the most precise results, some companies also choose to create pre-quizzes. In this way, you will know for sure what your learners knew before the course, and what they learned from taking it.
See if an employee’s behavior has changed after taking the training program. And if it has, how much? The most effective way to do this is to compare 360-degree reviews (feedback from the employee’s colleagues, supervisors, subordinates, and others) on the employee both before and after they took the course.
This is the most important level of training evaluation. After all, better results are the primary goal of corporate training. Evaluate what impact your course has had on business by analyzing quality, efficiency, productivity, and customer satisfaction ratings.
We need to mention that Kirkpatrick’s model has some limitations:
Don Kirkpatrick admitted these limitations and suggested a better way of using his method: try working backward through his four levels during the design phase of program development. In this way, you’ll be able to determine what outcome you’d like to get, and then develop your training program accordingly.
The Phillips ROI model can be thought of as an extended version of Kirkpatrick’s taxonomy. They both have the same levels, except for an extra one in the end – Return on investment (ROI). While Kirkpatrick’s model evaluates only a return on expectations (ROE), the ROI model can actually tell you if it was the right decision to invest in a training program.
How to measure ROI with the Phillips model:
Gather business data from before, during, and after the course, and measure the impact that a training had on your company’s profit, productivity, performance, etc. – depending on what the program was aiming for. Then, compare the training cost to the value it provided. If the value exceeds the training cost, you’re doing great. If not, you need to find which level or levels turned out to be ineffective and improve your training.
Another method building off of Kirkpatrick’s model is Kaufman’s five levels of evaluation. He divided the first level into two, grouped Kirkpatrick’s levels 2 and 3 as ‘micro’ levels, and added a fifth level to evaluate results for both customers and society.
How to use Kaufman’s five levels of evaluation:
Measure the amount of resources, such as time and money, that were invested in your training program.
Assess learners’ reactions to the course.
Evaluate individual or micro benefits of your training by stating whether training objectives for individual learners or small groups were met. This means you should find out if your students acquired new knowledge and skills and applied them in their work tasks.
Gauge how well learners utilize new knowledge and skills in their on-the-job performance.
Measure payoffs for your company as a whole. A payoff can be an improvement in performance, a reduction of costs, or increased profitability.
At the final level, you are to evaluate the impact that your course had on what Kaufman termed ‘mega-level clients.’ By these, he means business clients or society as a whole.
Kaufman’s model isn’t entirely applicable to real life. Measuring how much impact your training had on society is far too expensive, complicated, and impractical. Still, Kaufman offered some useful ideas, such as splitting the first level into two and evaluating the content you provide and the resources you invest separately. You can use some of his levels as an addition to your basic model.
CIRO stands for Context, Input, Reaction, and Output. Unlike other models, it is designed specifically for management training evaluation. So, if you need to assess management courses, this model might be your best option.
Evaluate your company’s current situation. Try to list all the factors that may affect the results of a training. Meanwhile, during this stage, find out what your organization is lacking in terms of performance. As a result, you’ll have a list of needs that should be sorted according to the following three levels:
The ultimate objective
The ultimate objective is the elimination of organizational deficiency, which could be poor customer service, low productivity, or low profit.
Intermediate objectives
Intermediate objectives are those that help achieve the ultimate objective. They may require a change in an employee’s work behavior. Let’s say your ultimate objective is to enhance productivity. To determine intermediate objectives, you need to understand the reason behind the current low productivity. After investigating, you might discover that your employees lack motivation, they can’t manage their time, or there are some communication problems between different departments that result in prolonging the time needed to complete tasks. So, your intermediate objectives will solve these issues.
Immediate objectives
As immediate objectives can help with changing employees’ behavior, they usually involve the acquisition of new skills and knowledge as a result of training, or changing employees’ attitudes.
Your goal at this stage is to identify the best training intervention option. Gather information about all potential methods and training techniques. Also, think about how you will design, manage, and deliver your course to your learners. Analyze your company’s resources and determine the best way to use them to achieve your objectives.
At this stage, ask your learners’ opinions of the course. Focus on three aspects:
What you’re interested in at this point is not only whether learners liked or disliked the course, but also what changes they would like to see in this training program. Collect their suggestions and consider them when improving your program.
At this stage, you present the results of the training. There are four levels of measurement:
Choose one of them in accordance with the purpose of the evaluation and available resources.
Imagine there’s a company that produces wooden furniture. In order to increase sales, they invested in salesperson training programs. The training turned out to be highly effective – sales increased. But here’s the problem: the factory workers didn’t receive the training, so they couldn’t build furniture fast enough to finish new orders. As a result, clients’ waiting time increased, and it affected the company’s business negatively. Anderson’s model helps to prevent such situations.
Unlike other models, Anderson’s model of learning evaluation, also known as Anderson’s Value of Learning Model, focuses on a company’s entire learning strategy, rather than a particular training program. It consists of three stages that help to determine the best learning strategy for an organization’s needs.
Determine whether the current training programs target the strategic priorities of your company. Let’s return to our example. Let’s say the company’s strategic priority is to increase the number of sales and expand its position on the market. Does salesperson training aim for these goals? Yes.
Measure the contribution of learning to strategic results. At this point, our company uses different measures to assess how much effect the training has had on accomplishing the primary goals. Collecting the research results, we can see that the program helped increase the number of sales, but it didn’t lead to an increase in the company’s market share. Moreover, over time, the number of customers started decreasing because the waiting time was too long.
Set the most relevant approaches for your company.
The choice of an approach depends on stakeholders’ goals and values. Anderson offered 4 categories of measure:
Here’s a table that might help you find the best approach for your organization.
You should choose a category that’s relevant to your situation and establish an approach that will help fulfil your organizational needs.
In our example, the furniture producer decided the organization required learning value metrics, and it expected short-term benefits. Using the table above, the company determined that it should use Return on Investment Measures to find the most relevant training approach. They calculated the cost of training programs for factory workers, compared it to the possible sales profit, and concluded that it was worth investing in.
Training evaluation tools are the means you use to evaluate training programs. They are usually divided into 4 categories: questionnaires, interviews, focus groups, and observations. We’ll add one more – LMS Reporting. To obtain the deepest and most accurate assessment, these methods are often used collectively. Now let’s look at the specifics of each of them.
Questionnaires are the training evaluation method most often used. They consist of a set of questions that aim to obtain useful information from the participants. This tool is great for assessing the reaction of learners after a program.
There is a wide choice of software for creating quizzes and surveys. If you want your questionnaires to be interactive, customizable, and engaging, try iSpring Suite. With this software, you’ll be able to create 14 question types, add images to both questions and answers, and use many other features that will help you put together questionnaires that really work.
By the way, with this tool, you can not only evaluate your training programs but also create them. The toolkit allows you to create online courses right in PowerPoint and enhance them with quizzes, dialogue simulations, screen recordings, and interactions. Check out this course to see what training programs and questionnaires you can create with iSpring Suite.
The aim of interviews is to gather both opinions and facts. Unlike questionnaires, interviews can reveal some insights into employees’ attitudes, behaviors, and mindsets. Apart from a classical face-to-face session, you can also conduct an interview by telephone or via the internet.
One can think of this method as a mixture of a questionnaire and an interview, because it enables you to both cover a large group of people and get insights. If you’re looking for qualitative data, in an endeavor to get a clear picture of employees’ perspectives, but don’t have enough resources to conduct interviews with all workers, focus groups may be what you need. Simply unite people according to a certain characteristic of your target audience, such as a common working area, common mistakes in job performance, and age, and then conduct an interview to collect their reactions, insights, reviews, and suggestions.
An observation is perhaps the best way to see changes in behavior and attitude after training. Unlike all other methods, observation doesn’t rely on what employees say about themselves or each other. By just watching a person working, you can see with your own eyes whether they apply new skills and knowledge in their job. Nevertheless, it has its limitations.
A learning management system (LMS) is software for delivering online programs to your learners. Within this system, there is another system, LMS Reporting, that collects and analyzes data from your online programs. With its help, you can reveal the weak points of your courses. For example, you’ve designed a training program and assigned it to your employees. Time goes by, but you can’t see any results. This is when you might want to take a look at LMS reports.
There, you may see that the majority of your employees dropped the course at the same point. You find this place in the program and analyze a possible loophole or an issue and then you fix it. In time you see results – employees continue the course and manage to complete it.
You can check out the benefits of an LMS and its reporting system yourself by getting an iSpring Learn 30-day free trial. Besides providing reports, iSpring Learn enables you to create courses, implement gamification in your eLearning, automate repetitive tasks, and much more.
Training programs can be evaluated either while they are being developed or after they’ve been delivered. These two types of evaluation are called formative and summative. Let’s take a closer look at them.
Formative evaluation aims to discover issues and loopholes in your course and fix them before it’s too late, meaning while the course hasn’t been delivered to learners. You can run a user acceptance test to check whether the eLearning platform works as intended, or you can have a focus group or a subject matter expert go through the training to reveal potential weaknesses or errors in your course.
Summative evaluation takes place after learners have completed the course. It includes surveys, interviews, tests, etc. – any technique that will provide you with feedback from course participants. This type of evaluation allows you to make improvements in the program for future learners.
Both types of evaluation are important for improving training programs. The best approach for any company would be to evaluate their training programs both before and after those were delivered. But a company doesn’t always have enough resources to do this. But even if you choose only one of the types, it may still enhance your training program system.
As you can see, there are many evaluation methods and tools that help to assess training programs. Each of them has its own benefits and limitations, and that’s why it’s unnecessary to select a single best approach. But we can say with certainty that the most effective method of evaluation would be using several models and tools, depending on your company’s goals and objectives.
We hope you found this article helpful. Now that you know how to evaluate training programs, be sure to read up on how to develop training programs. Good luck!
We know this is a long article, and you may have forgotten half of it by now. So, we’ve collected the key points here for you to refresh your knowledge.
To evaluate training programs, you need to choose and use an evaluation method that will help you find out whether a training meets the company’s goals and objectives.
Training evaluation is extremely important because it helps companies understand what programs they should invest in, why some programs are unsuccessful, what they should do to fix them, etc.
Training evaluation models are methods used to evaluate training programs. The 5 most popular are Kirkpatrick’s four-level training evaluation model, The Phillips ROI model, Kaufman’s five levels of evaluation, The CIRCO model, and Anderson’s model of learning evaluation.
Training evaluation tools are the means you use to evaluate training programs. They are usually divided into 4 categories: questionnaires, interviews, focus groups, and observations.
The quantitative tools of evaluation are the ones that give you quantitative data, which you later analyze. The tools that refer to this category are various types of questionnaires and LMS Reporting.
In short, to analyze training evaluation data, you need to ask yourself three questions:
Everyone connected to the training. A company doesn’t waste money on ineffective training, instructional designers learn to improve their courses, and employees study engaging content that helps them acquire new skills and knowledge that make their work more productive.