Training Program Evaluation: How To Achieve Perfection

Many companies implement training to improve their employee productivity and work performance. But how can you actually tell that it works? And how do you know what you should fix in your courses to make them more effective? The best way is to start evaluating training programs.

In this article, you’ll learn what training evaluation is, why it is important, and how it can benefit your company. You’ll also learn about the best evaluation models and tools you can use to assess the quality of your programs. 

What Is Training Program Evaluation?

Training program evaluation is the regular analysis of how efficient and effective training programs are. It involves feedback from learners, comparing their job performance before and after the program, accomplishing the company’s strategic priorities, and much more.

Why Evaluate Training

It helps companies find out whether training programs meet their goals and objectives. The results of training evaluation are used to improve training programs, or cancel them if they prove unsuccessful.

Benefits of Training Evaluation

Evaluating whether your training is sufficiently effective obviously requires additional resources. Still, the benefits of training evaluation outweigh the challenges. In short, it helps companies:

• Discover loopholes in courses. A part of training evaluation is to assess the quality of employees’ work, which might reveal some loopholes in training programs. Let’s say the quiz results show that almost all of your workers failed the question on data security. Then, upon checking the course, you discover that this issue is not addressed adequately. As a result, you might add a module on data security to the training program.
• Save money. By comparing training costs to results achieved by training, companies can see how profitable courses are, which training programs are worth investing in, and which are a waste of money and time.
• Improve materials and tools. Without training evaluation, it’s impossible to gauge the quality of the learning content and the effectiveness of the tools they use to build courses. Do learners drop courses after a few slides? Your introduction might not be engaging enough. Try shooting a motivational video to trigger their interest and explain why the course is important. By analyzing the evaluation results, you can see both weak and strong points, and adjust training programs accordingly.

Training Evaluation Models: A Guide Through The Best

To evaluate training programs more systematically, you can use specific training evaluation models. There are quite a lot of them, and each offers its own approach to assessing courses. We’ve collected 5 models that have shown to be the most popular and trusted over the years.

Kirkpatrick’s Four-Level Training Evaluation Model

One of the oldest and, without a doubt, the most famous one is Kirkpatrick’s model. It is still used more than sixty years after its development.

According to this model, training programs should be evaluated on 4 levels: Reaction, Learning, Impact, and Results. Let’s take a closer look at each level.

Level 1: Reaction

When learners complete your course, assess their reactions. Ask them to complete a survey with questions like:

• How satisfied are you with the learning experience?
• Did the training content meet your expectations?
• Did you learn anything new? 
• How would you rate the quality of the training?
• Do you find this training useful?

If you want to get more detailed results, use a Likert scale survey. Unlike binary questions, a Likert scale survey shows semitones between good and bad, which can be helpful in training evaluation.

Level 2: Learning

Measure how much was learned in the course. For example, create online quizzes to see what knowledge and skills they have and haven’t acquired during the training. In order to gain the most precise results, some companies also choose to create pre-quizzes. In this way, you will know for sure what your learners knew before the course, and what they learned from taking it.

Level 3: Behavior

See if an employee’s behavior has changed after taking the training program. And if it has, how much? The most effective way to do this is to compare 360-degree reviews (feedback from the employee’s colleagues, supervisors, subordinates, and others) on the employee both before and after they took the course.

Level 4: Results

This is the most important level of training evaluation. After all, better results are the primary goal of corporate training. Evaluate what impact your course has had on business by analyzing quality, efficiency, productivity, and customer satisfaction ratings.

We need to mention that Kirkpatrick’s model has some limitations:

• Limited use. It can tell you only whether your training works or not. That means if you use this model, you won’t get data that actually helps improve the course. 
• Arguable structure. The idea of linear causality suggested by Don Kirkpatrick is supported by fairly little evidence. This means you can’t be sure that positive results at level 1 will give positive results at level 2, and so on.

Don Kirkpatrick admitted these limitations and suggested a better way of using his method: try working backward through his four levels during the design phase of program development. In this way, you’ll be able to determine what outcome you’d like to get, and then develop your training program accordingly.

The Phillips ROI Model

The Phillips ROI model can be thought of as an extended version of Kirkpatrick’s taxonomy. They both have the same levels, except for an extra one in the end – Return on investment (ROI). While Kirkpatrick’s model evaluates only a return on expectations (ROE), the ROI model can actually tell you if it was the right decision to invest in a training program.

How to measure ROI with the Phillips model:

Gather business data from before, during, and after the course, and measure the impact that a training had on your company’s profit, productivity, performance, etc. – depending on what the program was aiming for. Then, compare the training cost to the value it provided. If the value exceeds the training cost, you’re doing great. If not, you need to find which level or levels turned out to be ineffective and improve your training.

Kaufman’s Five Levels of Evaluation

Another method building off of Kirkpatrick’s model is Kaufman’s five levels of evaluation. He  divided the first level into two, grouped Kirkpatrick’s levels 2 and 3 as ‘micro’ levels, and added a fifth level to evaluate results for both customers and society.

How to use Kaufman’s five levels of evaluation:

Level 1a: Input

Measure the amount of resources, such as time and money, that were invested in your training program.

Level 1b: Process

Assess learners’ reactions to the course.

Level 2: Acquisition

Evaluate individual or micro benefits of your training by stating whether training objectives for individual learners or small groups were met. This means you should find out if your students acquired new knowledge and skills and applied them in their work tasks.

Level 3: Application

Gauge how well learners utilize new knowledge and skills in their on-the-job performance.  

Level 4: Organizational payoffs

Measure payoffs for your company as a whole. A payoff can be an improvement in performance, a reduction of costs, or increased profitability.

Level 5: Societal Outcomes

At the final level, you are to evaluate the impact that your course had on what Kaufman termed ‘mega-level clients.’ By these, he means business clients or society as a whole.

Kaufman’s model isn’t entirely applicable to real life. Measuring how much impact your training had on society is far too expensive, complicated, and impractical. Still, Kaufman offered some useful ideas, such as splitting the first level into two and evaluating the content you provide and the resources you invest separately. You can use some of his levels as an addition to your basic model.

The CIRO Model

CIRO stands for Context, Input, Reaction, and Output. Unlike other models, it is designed specifically for management training evaluation. So, if you need to assess management courses, this model might be your best option.

Stage 1: Context

Evaluate your company’s current situation. Try to list all the factors that may affect the results of a training. Meanwhile, during this stage, find out what your organization is lacking in terms of performance. As a result, you’ll have a list of needs that should be sorted according to the following three levels:

The ultimate objective
The ultimate objective is the elimination of organizational deficiency, which could be poor customer service, low productivity, or low profit.

Intermediate objectives
Intermediate objectives are those that help achieve the ultimate objective. They may require a change in an employee’s work behavior. Let’s say your ultimate objective is to enhance productivity. To determine intermediate objectives, you need to understand the reason behind the current low productivity. After investigating, you might discover that your employees lack motivation, they can’t manage their time, or there are some communication problems between different departments that result in prolonging the time needed to complete tasks. So, your intermediate objectives will solve these issues.

Immediate objectives
As immediate objectives can help with changing employees’ behavior, they usually involve the acquisition of new skills and knowledge as a result of training, or changing employees’ attitudes.

Stage 2: Input

Your goal at this stage is to identify the best training intervention option. Gather information about all potential methods and training techniques. Also, think about how you will design, manage, and deliver your course to your learners. Analyze your company’s resources and determine the best way to use them to achieve your objectives.

Stage 3: Reaction

At this stage, ask your learners’ opinions of the course. Focus on three aspects:

• Program content
• Approach
• Value addition

What you’re interested in at this point is not only whether learners liked or disliked the course, but also what changes they would like to see in this training program. Collect their suggestions and consider them when improving your program.

Stage 4: Output

At this stage, you present the results of the training. There are four levels of measurement:

• Learner 
• Workplace 
• Team or department 
• Business 

Choose one of them in accordance with the purpose of the evaluation and available resources.

Anderson’s Model of Learning Evaluation

Imagine there’s a company that produces wooden furniture. In order to increase sales, they invested in salesperson training programs. The training turned out to be highly effective – sales increased. But here’s the problem: the factory workers didn’t receive the training, so they couldn’t build furniture fast enough to finish new orders. As a result, clients’ waiting time increased, and it affected the company’s business negatively. Anderson’s model helps to prevent such situations.

Unlike other models, Anderson’s model of learning evaluation, also known as Anderson’s Value of Learning Model, focuses on a company’s entire learning strategy, rather than a particular training program. It consists of three stages that help to determine the best learning strategy for an organization’s needs.

Stage 1

Determine whether the current training programs target the strategic priorities of your company. Let’s return to our example. Let’s say the company’s strategic priority is to increase the number of sales and expand its position on the market. Does salesperson training aim for these goals? Yes.

Stage 2

Measure the contribution of learning to strategic results. At this point, our company uses different measures to assess how much effect the training has had on accomplishing the primary goals. Collecting the research results, we can see that the program helped increase the number of sales, but it didn’t lead to an increase in the company’s market share. Moreover, over time, the number of customers started decreasing because the waiting time was too long.

Stage 3

Set the most relevant approaches for your company.

The choice of an approach depends on stakeholders’ goals and values. Anderson offered 4 categories of measure:

• Emphasis on short-term benefits 
• Emphasis on long-term benefits
• Senior management trust in learning contribution 

• The organization requires learning value metrics

Here’s a table that might help you find the best approach for your organization.

You should choose a category that’s relevant to your situation and establish an approach that will help fulfil your organizational needs.

In our example, the furniture producer decided the organization required learning value metrics, and it expected short-term benefits. Using the table above, the company determined that it should use Return on Investment Measures to find the most relevant training approach. They calculated the cost of training programs for factory workers, compared it to the possible sales profit, and concluded that it was worth investing in.

Training Evaluation Tools

Training evaluation tools are the means you use to evaluate training programs. They are usually divided into 4 categories: questionnaires, interviews, focus groups, and observations. We’ll add one more – LMS Reporting. To obtain the deepest and most accurate assessment, these methods are often used collectively. Now let’s look at the specifics of each of them.

Questionnaires

Questionnaires are the training evaluation method most often used. They consist of a set of questions that aim to obtain useful information from the participants. This tool is great for assessing the reaction of learners after a program.

Pros:

• Allows accumulation of a large amount of information
• Cost-efficient
• Time-efficient
• Covers a large group of participants

Cons:

• Low response rate
• Some unreliable responses
• Cannot clarify vague answers
• Subjective interpretation of questions by employees

There is a wide choice of software for creating quizzes and surveys. If you want your questionnaires to be interactive, customizable, and engaging, try iSpring Suite. With this software, you’ll be able to create 14 question types, add images to both questions and answers, and use many other features that will help you put together questionnaires that really work.

By the way, with this tool, you can not only evaluate your training programs but also create them. The toolkit allows you to create online courses right in PowerPoint and enhance them with quizzes, dialogue simulations, screen recordings, and interactions. Check out this course to see what training programs and questionnaires you can create with iSpring Suite.

Interviews

The aim of interviews is to gather both opinions and facts. Unlike questionnaires, interviews can reveal some insights into employees’ attitudes, behaviors, and mindsets. Apart from a classical face-to-face session, you can also conduct an interview by telephone or via the internet.

Pros:

• A better understanding of employees’ perspectives
• Allows asking clarifying questions
• Flexible

Cons:

• Time-consuming
• Doesn’t cover a lot of learners – one at a time

Focus groups

One can think of this method as a mixture of a questionnaire and an interview, because it enables you to both cover a large group of people and get insights. If you’re looking for qualitative data, in an endeavor to get a clear picture of employees’ perspectives, but don’t have enough resources to conduct interviews with all workers, focus groups may be what you need. Simply unite people according to a certain characteristic of your target audience, such as a common working area, common mistakes in job performance, and age, and then conduct an interview to collect their reactions, insights, reviews, and suggestions.

Pros:

• Receive detailed feedback from many people at the same time
• Ability to ask determining questions

Cons:

• Time-consuming
• Several people are required to manage (moderator and assistant)
• Comfortable environment required for people to speak freely and honestly

Observations

An observation is perhaps the best way to see changes in behavior and attitude after training. Unlike all other methods, observation doesn’t rely on what employees say about themselves or each other. By just watching a person working, you can see with your own eyes whether they apply new skills and knowledge in their job. Nevertheless, it has its limitations.

Pros:

• Cost-saving
• Gives a more realistic view, not being based on opinions
• Gathers valuable non-verbal information
• Can be conducted while a person is completing the course

Cons:

• Time-consuming (one person at a time)
• May provide unreliable information, as people tend to behave better when someone’s watching
• An observer may misinterpret what they see
• Doesn’t lead to better understanding reasons for an employee’s attitude or behavior

LMS reporting

A learning management system (LMS) is software for delivering online programs to your learners. Within this system, there is another system, LMS Reporting, that collects and analyzes data from your online programs. With its help, you can reveal the weak points of your courses. For example, you’ve designed a training program and assigned it to your employees. Time goes by, but you can’t see any results. This is when you might want to take a look at LMS reports.

There, you may see that the majority of your employees dropped the course at the same point. You find this place in the program and analyze a possible loophole or an issue and then you fix it. In time you see results – employees continue the course and manage to complete it.

Pros:

• Time-saving
• Automatic process
• Objective feedback provided by the system
• Helps to find weaknesses in training programs easily
• Works 24/7

Cons:

• Can only be applied to online programs
• Doesn’t lead to better understanding reasons behind an employee’s attitude or behavior

You can check out the benefits of an LMS and its reporting system yourself by getting an iSpring Learn 30-day free trial. Besides providing reports, iSpring Learn enables you to create courses, implement gamification in your eLearning, automate repetitive tasks, and much more.

When Is the Best Time to Evaluate Training?

Training programs can be evaluated either while they are being developed or after they’ve been delivered. These two types of evaluation are called formative and summative. Let’s take a closer look at them.

Before training is launched (Formative)

Formative evaluation aims to discover issues and loopholes in your course and fix them before it’s too late, meaning while the course hasn’t been delivered to learners. You can run a user acceptance test to check whether the eLearning platform works as intended, or you can have a focus group or a subject matter expert go through the training to reveal potential weaknesses or errors in your course.

After training is completed (Summative)

Summative evaluation takes place after learners have completed the course. It includes surveys, interviews, tests, etc. – any technique that will provide you with feedback from course participants. This type of evaluation allows you to make improvements in the program for future learners.

Both types of evaluation are important for improving training programs. The best approach for any company would be to evaluate their training programs both before and after those were delivered. But a company doesn’t always have enough resources to do this. But even if you choose only one of the types, it may still enhance your training program system.

To Sum Up

As you can see, there are many evaluation methods and tools that help to assess training programs. Each of them has its own benefits and limitations, and that’s why it’s unnecessary to select a single best approach. But we can say with certainty that the most effective method of evaluation would be using several models and tools, depending on your company’s goals and objectives.

We hope you found this article helpful. Now that you know how to evaluate training programs, be sure to read up on how to develop training programs. Good luck!

Takeaways

We know this is a long article, and you may have forgotten half of it by now. So, we’ve collected the key points here for you to refresh your knowledge.

How can you evaluate training programs?

To evaluate training programs, you need to choose and use an evaluation method that will help you find out whether a training meets the company’s goals and objectives.

Why is training evaluation important?

Training evaluation is extremely important because it helps companies understand what programs they should invest in, why some programs are unsuccessful, what they should do to fix them, etc.

What are training evaluation models?

Training evaluation models are methods used to evaluate training programs. The 5 most popular are Kirkpatrick’s four-level training evaluation model, The Phillips ROI model, Kaufman’s five levels of evaluation, The CIRCO model, and Anderson’s model of learning evaluation.

What are training evaluation tools?

Training evaluation tools are the means you use to evaluate training programs. They are usually divided into 4 categories: questionnaires, interviews, focus groups, and observations.

What are the quantitative tools of evaluation?

The quantitative tools of evaluation are the ones that give you quantitative data, which you later analyze. The tools that refer to this category are various types of questionnaires and LMS Reporting.

How can you analyze training evaluation data?

In short, to analyze training evaluation data, you need to ask yourself three questions:

• Does it meet my expectations?
• If it doesn’t, why not?
• If it does, why?

Who benefits from training evaluation?

Everyone connected to the training. A company doesn’t waste money on ineffective training, instructional designers learn to improve their courses, and employees study engaging content that helps them acquire new skills and knowledge that make their work more productive.