I deal with the business effectiveness of training every day, and the Kirkpatrick model is the methodical basis to design our e-learning training. We also use the simplified Kirkpatrick model to measure the effectiveness of training. I must admit, that after getting to know the new Kirkpatrick model, also by being part of the Kirkpatrick Partners certificate training, I am disappointed with the directions in which the changes of this model are heading. I am under the impression that without a specialized consultative firm, using the model in practice is impossible, and it definitely is not a model for trainers, even HR specialists. Here are 4 reasons for my disappointment.
To find out more about the new Kirkpatrick model go here.
It cannot be used commonly
The level of complexity makes it difficult to understand the model, and the costs of usage are higher than the benefits. Kirkpatrick Partners state point-blank that the model is useful only for projects with strategic meaning, those where we can allow ourselves to raise the high costs of effectiveness measurement. The number of business measurements and strengthening actions during the full implementation of the model is so high, that registering these results and taking correctional actions requires great effort from the entire organization, and thus generates high costs. This way, most training unfortunately stay outside the model. In practice it could mean that most of the training costs in organizations remains without effectiveness control.
It goes beyond the impact of HR professionals
It is not clear to me who should own the process of testing the effectiveness of training in the company. The scope of tools goes way beyond the traditional understanding of HR functions. Supposedly there is nothing wrong with that. I am convinced that HR should look at the company and its business goals with a wide perspective and it should be the goals that set the directions of HR’s actions. The problem lies in the fact that application of the model is better suited to the competence of the financial controller than the HR professional. These two functions require, however, a completely different basis and competences, which go together like oil and water. For this very reason these two functions are separate in companies. This way, the process remains without an owner. Long story short, the model went too far into controlling, and it does not deal enough with training. The method of measuring the effectiveness corresponds more to the concept of the Balanced Scorecard, with particular emphasis on cascading goals. In practice it became a model for managing strategic change in the organization, and not ensuring the effectiveness of the training model. If I, as a manager, was faced with the challenge of large changes in the organization, I would probably reach, in the first place, for the Norton Kaplan model, and not the Kirkpatrick model.
It lacks coherence and clarity
The model takes into account both the participants satisfaction survey and the effectiveness of training and this is another source of the problem. The level of satisfaction and knowledge are aligned with each other and both are expected to lead to changes in behavior. If satisfaction is no longer a condition for the acquisition of knowledge (and I agree), then why should it be a condition for behavior change? The second issue is adding to the level of knowledge elements such as attitude, motivation and confidence. I do not understand why they are part of the second level. The simplest solution would be to move elements related to attitude (attitude, motivation, self-confidence) to the first level and name this level the “attitude”. Then we could return to the chain of levels, in which the condition for acquisition of knowledge is the right attitude of the participants. In this situation the satisfaction assessment would land outside the model, as bearing no effects on business training. It does not mean that it cannot be separately researched, but it is not the participant’s smile that is the condition for change, but his attitude to change, and only this should stay in the training effectiveness model. I am all for this approach.
It does not use the ROE potential
ROE is great at the concept level. The “return on expectation” indicator gives more flexibility and avoids expressing the performance measurements in monetary value, which is the main drawback of ROI. Unfortunately ROE is only a concept. The Kirkpatrick model does not define any formula for ROE. I understand that it is assumed that in any situation expected indicators may be different and otherwise counted. Agree. This, however, increases the complexity of the system and makes it difficult to implement. It gets even worse. ROE was supposed to be an answer to the key weakness of ROI, which is the need to extract the estimated impact of training on the achieved results. In the Kirkpatrick model we deal with the whole chain of leading indicators, and the higher in the hierarchy these indicators are, the more we have a problem of isolating the impact of training on the value of the indicator. In fact ROE became more like ROI, but only in their defects.
The level of complexity, high costs, and non-universality mean that the application of the model requires a lot of determination. In practice this may discourage trainers and HR professionals from its implementation. That’s too bad, because it is a model with great potential and it’s a shame that it can be used only in those rare cases in which we have a high enough expertise and enough time and money for implementation.