You’ve invested a great deal of resources into employee training and development. And with that comes an expectation to measure its impact. After all, if you can’t measure it, you can’t improve it.
You’re all about improvement, so you’re looking for a guide that’ll tell you everything you need to know about how to evaluate a training program. This comprehensive training evaluation guide will help you identify what’s working, what’s not, and why. Together we’ll explore the following practical topics:
- Select the appropriate training evaluation technique
- Determine what you’ll measure
- Choose the right training evaluation tools
Select the appropriate training evaluation techniques
When it comes to the evaluation of training programs, it’s best to start at the beginning. So before you decide what to measure, or how to measure it, choose the evaluation technique that’s most helpful for your needs.
Not sure which training evaluation techniques are on the menu? Here are some of the most popular methods used today.
What are the methods of training evaluation?
There’s a long (and we mean long!) list of training evaluation techniques to choose from, and this can be overwhelming. But there are five techniques that are most often trusted by companies today. Some of these techniques are referred to as models, or training evaluation methods, and we’ll use these terms interchangeably.
- Kirkpatrick’s Four-level Training Evaluation Model
- The Phillips ROI Model
- Kaufman’s Five Levels of Evaluation
- Anderson’s Model of Learning Evaluation
- Summative vs Formative Evaluation
Kirkpatrick’s Four-level Training Evaluation Model
This method of evaluating training programs might be one of the oldest, but it’s still one of the most well-loved. Why? Because it breaks the evaluation process down into 4 simple levels – or rather, steps. Here’s how it works:
Step 1: Evaluate learners’ reactions to training. This is commonly measured after training. Ask learners to complete a survey about their overall satisfaction with the learning experience.
Step 2: Measure what was learned during training. Use assessments to measure how much knowledge and skills have changed from before to after training.
Step 3: Assess whether or not (and how much) behavior has changed as a result of training. The best way to measure behavior change is through workplace observations and comparing 360-degree reviews from pre- and post-training.
Step 4: The final and most important step is to evaluate the impact of your employee training program on business results. Here, it’s common to measure results like productivity, quality, efficiency, and customer satisfaction ratings.
In modern times, professionals have suggested that this process should actually be reversed. After all, step 4 is the most important one. If you agree with this approach, start by identifying the results you want to achieve, and work backward from there.
Whichever direction you choose to apply the steps toward, the eLearning industry has come to rely on Kirkpatrick’s model for good reason. Its logical, staged approach is easy to apply, and once the evaluation is complete, you’ll have a deep and wide understanding of employee learning during training.
The Phillips ROI Model
This model is the same as Kirkpatrick’s (see technique above), but with an extra step. The fifth step of the Phillips ROI model is to evaluate the program’s return on investment (ROI). To do this, you need to measure the difference between your training cost and training results.
When the results of training are so great that they exceed the cost, then you’ve achieved a positive training ROI. You can pat yourself on the back and continue the great work.
When the cost of training is larger than the results, something needs to change. But what?
Well, the awesome thing about using methods like the Phillips ROI model is that it’s easy to spot the areas that need improvement. Let’s look at an example.
Imagine that you measure positive results at steps 1 and 2 of the evaluation process, but not at steps 3 and up. This tells you that learners enjoyed the training experience (step 1), and that they demonstrated new knowledge and skills when they were tested after training (step 2). However, when it came to changing their behavior in the workplace (step 3), something went wrong.
You might do some investigation and discover that managers aren’t encouraging employees to practice their new skills on the job. Maybe they’re even discouraging it. Once you fix that broken link in the chain by getting managers to support training, your ROI improves. Useful, right?
Kaufman’s Five Levels of Evaluation
Kaufman’s model is another one of the training evaluation methods that takes Kirkpatrick’s approach a step further. You can think of this model as Kirkpatrick’s with a twist. This is what it looks like in practice:
Step 1a: Measure the resources that were invested into your training program, like time and costs in developing materials.
Step 1b: Evaluate learners’ reaction to the training process. (This step is similar to the first step in Kirkpatrick’s model.)
Step 2: Assess whether or not the training objectives for individual learners or small teams were met. For example, did they learn new skills? The focus here is on individual (or micro) benefits of training.
Step 3: Measure the practical impact of the benefits in Step 2. For example, are employees applying their new skills on their job? This is similar to Kirkpatrick’s third step.
Step 4: Measure the greater (or macro) benefits for the business, like increased profitability or reduced costs. Think of this as step 4 of Kirkpatrick’s model.
Step 5: Evaluate the effectiveness of your employee training program in relation to societal benefits. For example, how did training improve your company’s ability to add value to its clients or society as a whole?
The main advantage of using Kaufman’s Five Levels, rather than Kirkpatrick’s Four Levels, is Step 1a. Evaluating the benefits of training against the resources invested into training gives you ROI. And the great thing about ROI is that it can be a very persuasive tool when requesting more training resources from company leaders.
We know what you’re thinking: this model could be difficult to apply in reality, particularly when it comes to step 5. If you’re wondering how to evaluate a training program in a way that’s more focused on your business strategy than society as a whole, this next one’s for you.
Anderson’s Model of Learning Evaluation
This is one of the training managers’ favorite evaluation techniques for training, because it helps them keep their business strategy a priority. And what happens when your training directly supports your strategic priorities? Success!
The easiest way to explain this technique is with an example. So here it goes.
Suppose that a private healthcare facility only has enough staff and equipment to treat 100 patients with the level of care they promise. Now, suppose that their training manager develops a program to help the marketing team win new patients.
If the training is effective, and many new patients are admitted to the facility, the business is at risk of taking on too many patients. The increased volume might have a negative impact on the level of care patients receive, which could damage the facility’s reputation.
On the other hand, a training program that gives nurses the knowledge and skills to avoid waste, and thus reduce costs, would benefit the business. So, quite simply, this model ensures that training is delivered (and evaluated) where it’s needed the most.
If you’re interested in this technique, then follow the three stages of Anderson’s Model:
Stage 1: Evaluate your current training programs against the business’ strategic priorities. If we return to the healthcare facility example above, we’d realize that there is a misalignment between the training program that aims to increase patients, and the strategic priority to deliver high-quality care for patients.
Stage 2: Measure the contribution of training to strategic results. For example, a training program that helps nurses reduce waste could be measured by the percentage of decrease in material costs at the healthcare facility.
Stage 3: Find the most relevant approaches for your company. Here’s where you decide whether the ROI is worthwhile. This final step will depend on your company’s approach. For example, you might compare the contribution you measured in stage 2 to the resources that were invested into training. Or, you might ask whether the percentage of decrease in costs was big enough: did it meet your expectations?
If you’re not satisfied with the ROI measured in stage 3, then it’s time to make some improvements to your training programs.
Summative vs Formative Evaluation
A thorough evaluation will give you the best insight into the drawbacks of your training. So, it’s important to know how to assess a training program both while it’s being developed (formative evaluation), and after it’s been delivered (summative evaluation).
Let’s dive a little deeper.
Formative techniques of training evaluation aim to catch problems (and fix them) early on, before they negatively impact learning. For example, before a new course is delivered, you might run a user-acceptance test to ensure that the platform is user-friendly. Or, you could ask a subject matter expert to evaluate the course content against the difficulty level of assessments.
Summative techniques are also known as post-training evaluation techniques, because they happen after training is completed. Typical examples include Kirkpatrick’s four levels of training evaluation and Anderson’s Model of Learning Evaluation.
Determine what you’ll measure when evaluating your employee training program
Before you evaluate the effectiveness of your employee training program, you need to decide what the indicators of “effectiveness” are. Is training a success when employees become better at their jobs? Or is a happier, healthier company culture a sign that training is working? Is it, maybe, both?
The point is, you’ll probably want to include more than one measure of training effectiveness. The more measures you include, the more information you’ll have to help you improve your program.
Wondering where to start? Pick any or all of the popular training effectiveness measures below, and you’ll be on the right track.
New skills and knowledge
When it comes to learning, training is the pillar, right? For example, if you were training sales staff in persuasion techniques, you’d want them to be more persuasive when the training was over. This makes the acquisition of new skills and knowledge one of the top measures of training effectiveness.
The measure of knowledge and skills development is sometimes referred to as ‘learning performance’, because it relates to an employee’s performance as a learner, rather than their performance on the job. There are lots of easy ways to evaluate learner performance using a learning management system (LMS), but we’ll discuss those in the next section of this article.
One measure of training effectiveness that’s often overlooked, even when using the best types of training evaluation methods, is the learning experience. Why does this matter? Because when the learning experience is poor, employees are less likely to engage with training content, which means that they’re less likely to learn the skills that will make them better at their jobs.
This, of course, is a big problem. In fact, it could result in loads of time and resources being wasted on a training program that never achieved its objectives. So, be sure to measure employees’ perceptions of training delivery and content. Their feedback could be one of the best ways to measure training effectiveness, offering the best tips for improving your training.
Did you know that for many employees, learning is the number one reason they feel happy at work? This is because learning helps employees to grow and develop, and often opens up new career opportunities, too. Wouldn’t that make you happy?
And the great thing about happy employees is that they tend to work harder, stay committed for longer, and produce better results. So, while employee happiness might sound like a strange indicator at first, it’s actually one of the best results you can hope to see for your business.
If you’ve never considered measuring the impact of training on your company’s culture, it’s time to start. Culture is the special ingredient that makes your business unique in a highly competitive world. So, you need to protect it with training that fosters workplace norms and values that are good for business.
When you deliver employee onboarding training, sensitivity training, or anything else that might impact culture, make sure to evaluate success based on culture. You can do this by looking for changes in the number of HR complaints (for example, harassment) after training, or assessing peer review scores for teamwork and positive attitudes.
So far we’ve mentioned four measures to use when you evaluate the impact of your employee training program, but none of them are business results. So, for this next measure we’ll look at the impact of training on the efficiency of employees or teams.
Efficiency can be measured in different ways depending on your industry and the specific department you’re training. For example, a manufacturing company might train their assembly line staff on new equipment, and then measure how many more units can be completed per day. On the other hand, an online tech business could measure how many tickets their customer support team closes after completing a training program.
Finally, it’s crucial to evaluate the real impact of a company’s employee training program on its financial position. The real economic impact of your training can be measured by changes in revenue and profit.
When training is successful, and all the measures of training effectiveness you use show positive results, then you should see an increase in sales and income, or a reduction in costs – or both! When it’s both, you’ll certainly also benefit from a rise in profits.
Choose the right training evaluation tools
You wouldn’t measure length with a thermometer, right? So, before you start collecting information about the results of your training, make sure that you have the right tools for the job. Here are some of the most common training evaluation tools to choose from. Feel free to use more than just one to measure training effectiveness and track employee training.
This is the process of observing employees as they complete a task or process, or engage in a team activity. Often, the observer will use a journal to record what they see (it’s true, even the best evaluators can’t remember everything!).
There are many advantages to using observation as a training evaluation tool. You get to observe learning and behavior changes in a real workplace setting, and it costs nothing more than the observer’s time. It also tends to be more accurate than self-report questionnaires which can be biased, or influenced by poor memory.
Still, this tool has some downsides. First, you need to find someone objective and knowledgeable with enough time on their hands to watch each employee for an hour or more. Then, even if you find an observer, there’s the risk that employee behavior will change simply because they know they’re being watched. This can skew the results.
Sometimes, though, observation is the best tool. This is especially true when behavior changes aren’t easy to measure quantitatively. For example, sales skills are easy to measure by the number of sales an employee makes. Creativity, on the other hand, is tough to measure on the job. So, observing an employee’s creative ideas and input during meetings is a good solution.
Tests are a great way to measure changes in knowledge and skills, and they come in all shapes and sizes. Written assignments can be time-consuming to grade, but luckily the right LMS will give you the tools to create automatically-graded quizzes that are fun and interactive, too.
Perhaps the best part about tests is that you can measure a specific skill or knowledge area without the distraction of being observed. For example, you could measure a medical sales rep’s understanding of a new product with a few multiple-choice questions completed in a private and quiet environment. Plus, once you’ve set up a quiz on your LMS, you don’t have to invest any more time into this tool.
But there’s a catch. Tests usually don’t measure knowledge and skills in the same environment in which they’ll be used – the workplace, that is. So you won’t know whether an employee is able to apply what they’ve learned when there are other distractions and pressures at play.
It’s also worth mentioning that tests aren’t the best measure for skills like persuasion, which are better assessed in practice (think role-plays). And when it comes to skills for high-risk jobs, like pilots and surgeons, tests aren’t enough on their own. More realistic assessments, like simulations, are necessary, too.
Perhaps one of the most common training evaluation tools and techniques used today is the survey. A survey, or training evaluation questionnaire, collects data through a series of questions, usually in the form of multiple choice.
Why are surveys so popular? Probably because they’re highly efficient! You can design one survey, and send it out to millions of employees at the click of a button. If your survey is delivered via your employee training software, it gets even better, because you can access the results as an easy-to-interpret and downloadable report.
There’s just one important limitation that you should know about: not many people like questionnaires! Sadly, surveys get an average response rate of 30% to 40%. So it’s important to explain to employees that surveys help you improve training, and that you really do want to hear their feedback.
Because surveys ask for people’s perceptions and opinions, rather than hard data, this tool is best suited to measuring how successful the learning experience was. You can ask employees what they liked about training, whether the platform was easy to use, and if the content was useful to improving their work.
Interviews can be conducted face-to-face or online. But either way, they’re as effective as questionnaires – and even more so. Why? Because not only can you ask employees a set of questions, but you can answer their questions and delve deeper into their responses, too. This flexibility often means that you get more valuable and detailed information from employees about their training.
Unfortunately, the same flexibility can result in a few problems for this evaluation tool. Each interview has to be conducted separately, which means that you lose valuable time that both the employee and the interviewer could be using to get work done. Plus, if each interview includes slightly different questions, it can become tricky to compare or summarize results.
Still, if you’re exploring the reasons behind other results, this is the tool to do it. For example, if most employees rate the learning experience poorly on a questionnaire, then interviews could help you find out why. Or, if they rate the learning experience favorably, but don’t improve on-the-job performance, you could use interviews to identify the reason for this gap.
Focus groups are carefully facilitated discussions among a small group of employees who all completed the same training. These are great tools for exploring what employees think and feel about training, and to get suggestions for future improvements.
Of course, focus groups are a little less time-consuming than interviews, because you can question a number of people at the same time. A group dialogue can also lead to deeper conversations about topics that might not have been explored in a one-to-one setting.
This makes interviews a particularly effective way to unpack obstacles to training success, and to explore ideas for improvement. Just watch out for group conflict or any other dynamics that could damage your ability to gather constructive information about training.
If training doesn’t improve job performance, it isn’t working. So, performance records are surely an important measure to include in any training evaluation. The performance records you choose to use will depend on your training. But some common examples are deals closed, support tickets solved, units made and customer satisfaction ratings.
The biggest advantage of performance records is that they’re based on numbers, not opinions. This makes them free from bias, and a trusted source of information to judge your training success by. Plus, if your LMS software integrates with your HR tool (like TalentLMS integrates with BambooHR), you can compare training and performance records more easily.
The only downside when it comes to performance records is that they sometimes create more questions than they answer. Yup, performance data shows you where a problem exists, but not why it exists. So to get to the bottom of “why”, you’ll need to leverage more qualitative tools, like interviews or focus groups.
If excellent training results are a top priority for you, then you need to find ways to continuously improve your training program. Just follow the three steps in this article. Start by deciding on a method, then select your measurements, and choose the right training evaluation tools. Once you’ve set up the right method for you, it will be much easier to evaluate and improve your employee training program. And your colleagues will be celebrating your training success in no time!
The post How to evaluate a training program: The definitive guide to methods, techniques & tools appeared first on TalentLMS Blog.