Measuring Compliance Training Effectiveness
In his latest article, Thomas Fox, Compliance Evangelist based in Houston, TX, addresses the question, “What if you want to take your post-training analysis to a higher level and begin a more robust consideration of the effectiveness of compliance training?”
Since at least 2017, the Department of Justice (DOJ) has emphasized the need for a determination of compliance training effectiveness. In the 2020 Update, it stated under the section entitled “Form/Content/Effectiveness of Training” the following questions, How has the company measured the effectiveness of the training? Have employees been tested on what they have learned? How has the company addressed employees who fail all or a portion of the testing? Has the company evaluated the extent to which the training has an impact on employee behavior or operations?
Shawn Rogers analogized compliance training to one of the many subsystems that make up an automobile; specifically, the windshield wiper system. He stated, “This might seem odd, but it provides an interesting analogy. The windshield wiper system is critical to vehicle safety. Nobody would want a car that didn’t have windshield wipers. The wipers are a subsystem of the car’s overall safety system. The car’s entire safety system is designed to (1) prevent crashes and (2) protect the occupants when a crash occurs.”
Equally, a compliance training program is very important. No company executive would ever say a company does not need to conduct compliance training. However, compliance training is just one part of the overall compliance program. It is not the entire program. The compliance program is designed to (1) prevent compliance violations and (2) protect the company when a compliance issue occurs. Rogers believes “a good compliance training program provides clarity to the employees on how they are to behave in their daily jobs. When properly implemented, compliance training helps employees stay within the guardrails. The compliance training program fulfils a very specific purpose, but there are other parts of the compliance program that have to be in place as well to prevent compliance lapses to keep the company safe.”
But there are ways to measure training effectiveness. You can show that the training was aligned to the company’s risk profile. Through user surveys and focus groups you can measure whether the learners feel that the training is applicable to their role and you can measure user satisfaction. You can ask learners to give examples of how they have changed the way they do their jobs. One approach is to start at the beginning with an evaluation and move outward. This approach fulfills the DOJ requirements to answer the following questions: Have employees been tested on what they have learned? How has the company addressed employees who fail all or a portion of the testing? Has the company evaluated the extent to which the training has an impact on employee behavior or operations?
What if you want to take your post-training analysis to a higher level and begin a more robust consideration of the effectiveness of compliance training? Joel Smith, the founder of InHouseOwl, considered an analysis of return on investment (ROI) for compliance training. He advocated performing an assessment to determine ethics and compliance training ROI to demonstrate that by putting money and resources into training, a compliance professional can not only show the benefits of ethics and compliance training but also understand more about what employees are getting out of training (i.e., effectiveness). The goal is to create a measurable system that will identify the benefits of training, such as avoiding a non-compliance event such as a violation of the Foreign Corrupt Practices Act (FCPA).
Smith’s model used four factors to help determine ROI for your ethics and compliance training:
1) Engagement
2) Learning
3) Application and Implementation
4) Business Impact
These same four factors can be used to determine compliance training effectiveness.
Figure out what you want to measure. Before you ever train an employee, you should have a goal in mind. What actions do you want employees to take? What risks do you want them to avoid? In compliance training, you want them to avoid non-ethical and non-compliant actions that would lead to potential violations. Your goal is to train employees to follow your Code of Conduct and your compliance program policies and procedures, so you avoid liability related to actions.
Were employees satisfied with the training? What is their engagement? The next step is to get a sense of whether employees feel that the training you provided is relevant and targeted to their job. If it’s not targeted, employees will likely not be committed to changing risky behavior. One way to obtain such data is through a post-training survey. This should provide insight into determining if employees thought the training was beneficial and effective in answering their questions and concerns.
Did employees actually learn anything? A critical part of any employee training is the assessment. You must know whether they actually learned anything during training. You can collect this data in a number of ways, but for compliance training, the best way is to measure pre- and post-training understanding over time. Basically, each time you train an employee, measure comprehension both before and after training.
Are employees applying your training? A survey should be used to determine employee application and their implementation of the training topics. To do so, you must conduct surveys to understand whether they ceased engaging in certain risky behaviors or better yet understand how to conduct themselves in certain risky situations. These surveys can provide a good sense of whether the training has been effective.
The beauty of using surveys is that it provides feedback on not simply the compliance training to determine effectiveness but a much wider variety of areas for your compliance program. These surveys can provide critical information on the state of your compliance program and provide substantive feedback for further inclusion back into your compliance program. Testing your program and using that information in a feedback loop is another key component of a best practices compliance program.
The importance of determining effectiveness of your compliance program was enshrined by the DOJ in its 2020 Evaluation. The 2020 Evaluation demonstrates the DOJ wants to see evidence of the effectiveness of your compliance program. This is something that many Chief Compliance Officers (CCOs) and compliance professionals still struggle to determine. Both the simple guidelines suggested herein, the more robust assessment and results provide you with a start to fulfill the precepts set out in the 2020 Evaluation, but you will eventually need to demonstrate the effectiveness of your compliance training going forward.
Author:
Tom Fox
The Compliance Evangelist
Founder, The Compliance Podcast Network
The Salt® learning platforms can give you an impressive level of reporting granularity
- How many times users have attempted each question
- How long it has taken them to complete the training
- What areas they are finding challenging
And much more!
