Measuring Effectiveness

What does effectiveness or its measurement really mean? When discussing learning or any kind of change, effectiveness refers to the degree of success in achieving specific goals. Learning and changes, in general, aim to bring about some form of transformation, and measuring effectiveness involves identifying and concretizing the success in reaching these transformational goals.

To measure effectiveness in any way, two things are needed:

• A need or goal
• Information about the starting point

The need or goal is crucial because the learning content or the reform project should aim to fulfill this need or goal. To define the need or goal, it's essential to have a clear understanding of the target audience and/or the subject matter that the learning or change aims to impact. And finally, to find out if something has changed, you need to know the starting point.

Measuring effectiveness can, in the best case, reveal how efficiently reform projects or, for example, training acquisitions have been, even in monetary terms.

Prerequisites for measuring effectiveness

Prerequisites for measuring effectiveness

Start with defining the need or setting the goal

The need or goal indicates what one wants to achieve. What is the desired outcome? In our example, it could be the "reduction of workplace safety violations." Once the need or goal is clear, we can examine what needs to change. In this context, it might involve the workforce adhering more closely to workplace safety regulations. Already, it becomes apparent that achieving the goal might involve implementing some form of additional training. In this context, supplementary training is a reform project undertaken to achieve the goal.

Define the target audience or subject matter

Once the goal or need is clear, the next step is to consider what group or issue needs to change. In our example, the behavior of the staff should change for the number of workplace safety violations to decrease. The target audience is, therefore, the organization's staff. The target audience or subject matter is the focus of the reform project or learning content, and the change in their behavior is what will be monitored.

Assess the current situation

To confidently say that something has changed, or to talk about effectiveness, the initial situation must be known. The starting point refers to the situation that exists before any change-oriented actions are initiated. Various methods can be used to determine the current situation, depending on the subject matter. In our example, the starting point could be the number of workplace safety violations on a yearly, quarterly, and monthly basis, possibly broken down by units.

Additionally, information on how many staff members have completed safety-related training and the time elapsed since completing the training could be included. By doing this, a new metric, already in use in the organization, is introduced.

Implement the reform project

Once the need is identified, the target group defined, and the current situation mapped out, it's time to implement the reform project. In our example, this means providing occupational safety training to the entire organization's staff. Once the reform project is implemented, or during its course, the impact can be monitored.

Different methods for measuring effectiveness

There are countless ways to measure effectiveness, depending on how effectiveness is defined and whose perspective is considered. In this text, I have chosen to examine effectiveness at the individual, team, and organizational levels and perspectives.

Individual Level: Effectiveness at the individual level can be subjectively validated. For our example of learning content related to workplace safety, measurable learning objectives could be formulated, such as those related to the content of the material. The learner can then self-assess how well these objectives are met.

Practically, this can be achieved by presenting the learning objectives to the learner at the beginning and end of the learning content and asking them to evaluate how well they believe they have achieved these objectives at both points. Allowing the learner to assess their own competence supports the self-regulation process of learning and makes learning more manageable and verifiable for the learner. Reflection on one's own learning can also be facilitated through a simple assessment of the fulfillment of learning objectives.

Team Level: At the team level, assessments of the fulfillment of these learning objectives can be utilized collectively by examining the self-assessed starting level of the learners and comparing it with the final level. This provides information on the perceived effectiveness of the learning content across the entire target group and the accumulation of knowledge.

Organizational Level: At the organizational level, a review of these learning objectives, along with their initial and final levels, can be combined with existing organizational metrics, such as the number of workplace safety violations over a specific period and the number of completed trainings.

At the organizational level, tracking effectiveness in this way enhances transparency, both for management (learning objectives reveal how perceived learning occurs in different teams, and other metrics show broader changes) and for decision-making based on data. It is also crucial for the organization to know which learning content produces the desired results: that is, which training is effective or which parts of the training yield the best outcomes. This way, training content can be adjusted if necessary to better achieve the set goals.

And finally, when effectiveness monitoring is carried out as described, the return on investment for training or other change initiatives can be seen in monetary terms, specifically whether workplace safety violations (and any associated costs) have changed as a result of the implementation of the reform project.

Measuring effectiveness helps verify the profitability of investments made in change initiatives

Reform projects always incur costs—whether in terms of money or time invested in change initiatives. To make the best possible decisions for the organization, data-driven decision-making is crucial. Data is obtained through measurement, and the effectiveness of changes and training can be surprisingly easily measured.