8 elements for successful performance measurement

Many people and many organisations are struggling to develop performance measures that effectively drive performance improvement.  They are often wading under the ad-hoc morass of inherited, legacy Key Performance Indicator (KPI) and performance reports.  The bureaucracy involved in producing, analysing and reporting on these performance measures has taken over from effective, action-oriented Maintenance Improvement, and the purpose behind the production of the KPIs has been lost.

This paper discusses eight essential elements necessary for a successful performance measurement system, and in doing so it outlines a possible method that could be used to redefine, and refocus, performance reporting in order to motivate the organisation towards higher levels of maintenance and asset management performance.

1. Create an inspiring vision

Why are we measuring performance at all? The only valid reason for measuring performance is to see whether we are making progress towards a specific long-term goal. 

At a personal level, that goal could be the achievement of some financial security – to obtain enough money to be able to afford that dream home, or to be able to afford to send your children to a good private school, or to be able to enjoy a comfortable retirement. Or it could be less tangible – the achievement of some personal satisfaction, or recognition for work well done. However these goals are highly personal, and also inspirational – they are something we really want to achieve.

At an organisational level, organisational vision and mission statements are, almost without exception, completely uninspiring. They do little to generate a sense of purpose, or a sense of excitement amongst members of the organisation. Typically, most organisational mission or vision statements look something similar to the following:

If you worked for this organisation, would this inspire you to great things?  On the other hand, look at some of the other vision statements that some of the world’s leading organisations have used[1].

• To experience the joy of advancing and applying technology for the benefit of the public (Sony)
• To make people happy (Walt Disney)
• To solve unsolved problems innovatively (3M)
• To experience the emotion of competition, winning and crushing competitors (Nike)

What do you notice about these Vision Statements? A number of things – they are:

• Short
• Simple
• Inspirational, and
• Focused less on what will provide benefits to the organisation, and more on what will provide benefits to the individuals within the organisation (although, obviously, it is hoped that the two are closely linked)

If you wish to create a high-performance organisation, then it is essential that all members of the organisation are focused on achieving the same goal – and the simpler, more focused, and more inspirational the goal, then the greater the chance that it will be achieved.

And, more importantly, this vision will form the basis for deciding on what is important to your business, and, therefore, which measures you should be reporting.

2. Use a small number of easily-understood performance measures

When it comes to Performance Measures, less is definitely more! Many organisations are buried in performance measures, for legacy and historical reasons, but the purpose behind these performance measures and reports is often forgotten.

The only valid, productive use for Performance Measures in any organisation is to motivate action which improves organisational performance. Performance Measures are a means to an end – they are the means by which organisational performance, through effective action, can be improved. Yet, in many organisations, the production and analysis of performance measures acts as an end in its own right.

I suggest that you try the following, simple exercise at your work place. 

First, collect all of the daily, weekly, monthly, quarterly and annual reports that are produced on a regular basis. Include shift logs and reports, shift production reports, cost reports, schedule performance reports – any report that records or explains performance that is produced by, issued by, or received by anyone in the Maintenance function – from tradesperson all the way up to manager. If these are electronic in format, print them out. I bet you will be surprised by the number and size of reports that are produced. I am prepared to bet that the resulting pile of paper will be substantial!

Finally, consider how much time and effort has gone into collecting and calculating the data for these reports. Think about how much time and effort has gone into the considered analysis and explanation of the results contained in these reports. And then think about how much effective action has resulted from the issue of these reports. Simply performing this exercise frequently identifies significant opportunities to either:

• Eliminate reports completely, or
• Significantly reduce the size of reports

Further, the more performance reports and performance indicators you have, the greater is the risk that two or more indicators reported in different reports will contradict each other. This can come about either because:

• The same performance indicator is reported in two or more reports, but the definitions of these indicators are different, or they use different data sources, or
• Similar performance indicators are used which present the same concept in a slightly different manner.

An example of the first situation is often the case when Equipment Reliability data (Mean Time Between Failures – MTBF) is reported from both the Maintenance Management System, and from a Process Control (Production) System. Typically, the Maintenance Management system will only record a failure event for each unplanned event for which a work order has been raised within the CMMS. However, the Process Control system will record a failure event for ALL unplanned stoppages due to maintenance, regardless of whether a Work Order was raised or not. The result is different reported figures for MTBF. 

As a direct consequence of this, often a significant amount of time and effort is spent in trying to reconcile the two different figures, and in attempting to “solve” the problem of different data. All of this time or effort is distracting decision makers from the main task of actually taking appropriate corrective action, if required, to bring performance back “on track”.

An example of the second case, that often applies in capital intensive industries, is where Maintenance costs are reported against budget both on a simple cost basis, and on a unit cost ($ per unit of output). Frequently, in capital intensive industries, a large portion of costs are fixed on a monthly basis, regardless of the total production output of the plant. As a result, when production is above budget for any particular month (usually not as a direct result of any maintenance action), then Unit Costs are below budget. However because some costs typically DO vary directly with production output, then often Costs (in total $ terms) will be above budget for that particular month. So, in this case, depending on which measure is used (Total Costs, or Unit Costs), the monthly cost performance in a high production month can be interpreted as being either better than, or poorer than, budget. And of course the reverse is true in a low production month.

Once again, in this case, considerable time and effort can be wasted debating whether performance really is good or bad – diverting attention from other, more important matters.

3. A balanced set of measures is essential

Many of you have probably heard of the balanced scorecard, and some of you may well have a balanced scorecard in place at your organisation – but specifically how does this apply to maintenance?

The concept of the Balanced Scorecard was introduced by Kaplan and Norton in the mid 1990’s. Specifically, they consider that an effective performance reporting system should focus on performance from four key perspectives:

• The Owner/Shareholder’s view of your business
• The Customer’s view of your business
• Internal Processes
• The Learning Organisation

This all makes sense from a total business point of view, but requires some further consideration when translating this concept into something practical at lower levels within the organisation – particularly when we consider, as we will in the next Element, that different performance indicators may be appropriate for different positions, job roles, and even individuals within the organisation.

In general terms, when considering these four elements with respect to maintenance, I have found it useful to conceptualise them as follows:

• Owner/Shareholder’s view – consider performance indicators that relate to financial performance, or exposure to risk (safety performance, for example). These are the areas of performance that those with a financial stake in the business are most likely to be interested in.
• Customer’s view – if we consider “Production” to be “Maintenance’s” customer (a dangerous and one-sided view, in my opinion – but that is a topic for a future paper, not one that there is enough time to consider now), then we may consider measures that “Production” may be interested, such as equipment performance, on-time delivery of service, quality of workmanship etc.
• Internal Processes – these are measures of the performance of the internal management processes within the “Maintenance” function. They could include measures relating to the quality of the planning process, or the success in achieving the plan, or whatever internal maintenance management process is important to you.
• Learning Organisation – Here I would consider that the focus is on measuring how well the organisation is improving. From experience, the most effective improvements are achieved when the business is focusing on no more than a handful of fairly specific activities – and so the measures that you would put in place here would be specific to whatever improvement activities you are attempting to progress. For example, if you were aiming to complete a large-scale RCM implementation program, then you may measure progress towards having completed and implemented RCM analyses on all the equipment items that have been selected for analysis. If you were in the process of implementing TPM, then you may have some measures in place that determine the extent of Operator “TLC” – Tightening, Lubricating, Cleaning, or of the extent to which a Visual Workplace has been successfully implemented.

Clearly, there are a large number of performance measures that you could select from among these categories, but, with Element 2 in mind, it is important to select only a few performance measures. I would suggest aiming for no more than 2-3 measures under each of the four categories above.

How should you select the most appropriate measures? First, I would recommend starting by brainstorming a selection of measures under each of the categories above, for the position or department being considered. 

Then, second, assess the measures in terms of their:

• Relevance – is there a key link between this measure and the organisation’s overall mission/vision/goals
• Reliability – does the suggested performance measure accurately reflect performance in the selected area?
• Understanding – how well is the performance measure understood by those whose performance is going to be measured?
• Availability of Data – is the data required to calculate the measure readily available, or easily obtainable?
• Timeliness – how quickly does the measure respond to changes or improvements that may have been made? Is the measure a “leading” or a “lagging” measure of performance?
• Controllability – to what extent can the person or group, to whom the measure is being reported, influence performance, as reported by the measure?

Of all of these factors, this last one is the most important – and so often ignored, in practice, that it is worthy of discussion separately, as our fourth key element of an effective performance management system.

4. Performance measures are most powerful in the hands of those who can influence the outcomes

Let’s take our substantial pile of reports that we talked about earlier, and perform one further analysis step. For each of the reports in the pile, identify who receives the report, and for each individual receiving the report, consider the extent to which the actions and decisions of the individual receiving the report will influence the results of the reported data. Frequently, you will find that the individual has very little capability to influence the outcomes. In this situation, providing this data cannot motivate the individual to action.

Frequently, I visit operations where a range of data is reported to shopfloor personnel. This frequently includes high level Performance Indicators, such as costs/tonne etc. The question that I ask is, in any large organisation, such as an alumina refinery, or a large minesite, what is the capability of an individual tradesperson to significantly influence the cost/tonne of the entire operation. The answer is, almost none. So reporting this information to shopfloor personnel does not influence the decisions and actions of that individual. It does not motivate him to reduce costs, simply because there are too many other things, which are outside his span of influence, which will have a far greater effect on the outcome of this measure. If we wish to use performance measures to motivate people to make effective decisions, then we must, instead, focus on selecting performance measures for which they can influence the outcome.

Another example is the reporting of Overall Equipment Effectiveness (OEE). Kodak makes a difference between Overall Equipment Effectiveness (which is a measure of the “effective” utilisation of equipment within its scheduled runtime) and Total Effectiveness Equipment Performance (TEEP), which is a measure of the “effective” utilisation of equipment assuming continuous 24 hour/day, 365 day/year operation. Drawing this distinction can be very useful, as Maintenance and Operational personnel often have little control over the scheduled runtime of equipment (these often being determined by such factors as overall market demand, and senior management capital allocation decisions), and so OEE is a more effective measure at this level. On the other hand, senior management should be concerned more with total return on assets employed, and so TEEP is a more effective measure at this level.

This leads us to the realisation that the performance measures that should be reported will be different at different levels within the organisation, and will also, more than likely, be different for different job roles within the organisation. It would not be hard to imagine a pyramid of performance measures that may apply.

5. What you measure is less important than how you decide what to measure

The final, vital aspect in selecting appropriate performance measures is to realise that the objective of measuring performance is to motivate those who have the capability to influence performance to make decisions, and take actions which will result in improved performance. They are unlikely to do so of they do not “believe” the measures, or if the measures are perceived to be externally imposed, and therefore not relevant. The strongest performance measures are those that are “owned” by those who can influence performance, and effectively used by these people to drive performance improvement. So while Elements 2 to 4 have outlined some of the principles, tools and techniques which can be used to select appropriate performance measures, it is most important that these principles, tools and techniques be used in a highly participative process, involving those whose performance is to be measured, in order to ensure a high level of ownership of, and therefore commitment to, the selected measures.

This implies that the process of selecting performance measures is best done through a series of facilitated workshops. And it also implies that compromises may need to be made in the selection of measures, whereby a performance measure that is perhaps technically less accurate, or perhaps less relevant, may be preferred to another measure, which has a high level of understanding and ownership by those whose performance is being measured.

Only if a high level of ownership of the selected measures is obtained will meaningful performance improvement take place. Almost without exception, all performance measures can be “doctored” so that reported performance does not reflect reality. For example, a measure of percentage of planned work completed can generally be forced to 100% by closing all planned work orders within the scheduled work week, regardless of whether the work has actually been done. So gaining ownership, and ensuring that people use the performance measures as an active tool for performance improvement is vital.

6. A measure without a target is meaningless

Look at the performance reports, charts, tables and graphs that are produced at your organisation. Do they all contain clearly understood target levels of performance? You would be amazed at how few do. For some reason, many organisations assume that everyone knows what the targets are, and, therefore, what constitutes good or bad performance. In reality, what then happens is that current average performance comes to represent the unofficial target level of performance, and, as a result, no performance improvement is ever seen. 

An effective control system, as all good engineers are aware, requires four key elements:

• A target or reference point against which measured output can be assessed
• A measurement system for measuring actual output
• A means of comparing actual with the target, and
• A method for adjusting inputs so that the desired output is achieved

This can be illustrated diagrammatically, as shown in Figure 2 below.

Without a target there can be no control, or incentive for improvement, so make a point of ensuring that every performance report shows a target level of performance. With this in place, everyone can understand the gap between current and desired performance. Note that, in some cases, for the first time you may actually have to decide what target level of performance should be!

7. Be aware of the “natural” variability of any process

Of course, most engineers understand that any process has a certain degree of natural variability. But it is absolutely essential that this variability is understood when analysing performance data, and before taking appropriate action. Imagine, for example, the following performance data as shown in Figure 3.

Clearly, performance is dropping, and a manager may decide to take some action, usually by making his displeasure with the situation known. Following this action, performance data continues for the next two weeks as shown in Figure 4.

Clearly (in his view), his action has had the desired results, as the plant throughput has increased significantly. Obviously, “kicking butt” when performance is low is an effective strategy to turn performance around. Having had the desired effect, the manager may then choose (in a temporary fit of remorse) to reward his people, and let them know that he is happy with the improved performance. After this act of generosity, performance trends as shown in Figure 5.

Despite the reward that he has given his people, performance has dropped. Clearly, this reinforces in the manager’s mind that rewarding people for good performance is a counter-productive strategy, as all that it leads to is lower performance. But what if we overlay on this chart the long run confidence limits for this process, based on statistical analysis of the variability of past performance – the limits within which we can expect that 95% of results will lie. This represents the natural variability of the process – any variation within these limits is due to the process itself, rather than any external intervention. This is shown in Figure 6.

From this chart, we can see that all the data points lie within the Upper and Lower Control Limits for the process. In other words, despite the impression that the manager had that his actions were influencing the results, the reality is that all changes were as a result of the natural variability of the process, rather than his actions. 

Furthermore, none of the results on the chart indicate that the process is “out of control”, and therefore requires some action or intervention. So the manager has actually been wasting his time focusing on results that do not require his attention.

So you can see that understanding the natural variability of performance is important in allowing us to focus on the things that deserve our attention.

8. Make sure you “close the loop”

The final vital step in establishing an effective performance management system is to ensure that the control loop is effectively closed.

As we saw earlier, an effective control system requires four key elements, as illustrated below:

• A target or reference point against which measured output can be assessed
• A measurement system for measuring actual output
• A means of comparing actual with the target, and
• A method for adjusting inputs so that the desired output is achieved

In this element, we focus on the last of these points – closing the loop so that effective improvement action takes place. This is one area where, in comparison with other countries and cultures that I have worked in, Australians are not particularly good. There is a tendency to assume that, just because performance is measured, reported, and explained, then improvement action will automatically take place. But this will only happen if an effective decision making, and improvement action process is in place. There are a few requirements for this to happen, among them:

• Those capable of influencing performance actually have the performance reported to them.
• Those capable of influencing performance think that it is important that improvement takes place
• Those capable of influencing performance have an incentive to improve performance
• Those capable of influencing performance are part of the action planning process, and commit to taking defined actions to make improvements, and
• There is follow up to ensure that the improvement actions which have been committed to, actually happen.

You would be amazed at how rarely there is any formal mechanism for reporting and discussion of performance results amongst those that can affect performance within Australian industry. This is particularly true among shop floor level personnel. Frequently, in performing my consulting work, I ask tradespeople how, at the end of the day or the week or the month, they know whether they have done a good or a bad job. In almost all cases, their response is that they either do not know at all, or that they are satisfied when they have met their own personal standards. Rarely do they get any feedback, either formal or informal, on their individual performance, or the performance of their work team in comparison with targets that have been set in conjunction with their supervisor, or the management team. 

On the occasions that they do get some feedback, this is generally by means of posting a few charts on a noticeboard, which they can either choose to look at, or ignore as they wish. These charts are generally produced by somebody else, in a distant office, usually using a computer, and so there is little ownership of the results by these people – the computer produced them. In any case, simply posting a few charts on a noticeboard clearly indicates to them that these charts are for information purposes only – they are not required to take any action based on this information (other than, perhaps, looking for a new job elsewhere if it looks like the company is about to go out of business!)

Even amongst higher level personnel, performance reporting meetings are generally treated as an opportunity to explain poor performance, rather than to take action to resolve poor performance, or to take advantage of performance improvement opportunities that may arise. The focus is on writing a list of excuses on a report, or a whiteboard, rather than on removing those excuses once, and for ever.

Clearly, if we are going to generate improvement, we need to create an environment where we get out of the “blame game”, and start effectively using reported performance as an opportunity to identify improvement opportunities. This can only be achieved if those being held responsible for measured performance have a high level of ownership of the performance measures (as we discussed earlier), and also have a high level of ownership of the recommended solutions. There can be no better means of improving the ownership of performance measures than by getting those who are expected to act on the measures to also produce these measures – that way they cannot ignore the real results, and pass them off as being somehow “inaccurate” or “irrelevant”. In addition, they must be part of the discussion and action planning process to improve performance – this includes shopfloor personnel. If these discussions are led by a manager or supervisor, then this sends a clear message that this activity is important to the organisation, and to them.

And once the improvement actions have been agreed upon, there must be an effective follow up process in place to ensure that the agreed actions have actually taken place, and that they are resulting in the desired improvements. This, once again, requires active management by supervisors and managers.

Finally, the incentive for improvement is far greater if those responsible for making the improvements have some “skin” in the outcome – either financially or non-financially. There should be some form of reward for those that generate performance improvement. This could be through some form of financial bonus scheme, or could be simply through inclusion in the regular performance appraisal process, or both. Equally, informal opportunities should be taken to celebrate success through such activities as providing certificates, gifts or awards for reaching significant targets, or simply through organising a celebratory social function. And never underestimate the impact that a simple, public “pat on the back” can have in motivating individuals and teams.

So the message here is to focus strongly on closing the loop in order to generate real improvement.

Conclusion

So in summarising, there are a few key points to take note of if you wish to develop a truly effective performance management system, and use Performance Measures to drive improvement. These are:

• Ensure that there is a high level of ownership of the performance measures used by those that can actually influence performance.
• Make sure that the measures that you select are:
  • Few in number
  • Balanced, and focused on the key areas that are important to your business
  • Congruent with your business goals
  • Controllable by those whom you are holding accountable for performance
• Ensure that targets (and more importantly target ranges) are established for all performance measures used.
• Make sure that you effectively close the control loop – ensure that the performance management process is action-oriented, and that improvement actions are actually put in place.