The Impulse To Compare

What is the real Value of Benchmarking your IT Services Contract?


Realistically, benchmarking involves two, maybe three compulsions:

1. Comparing yourself against competitors who may be better than you

2. Comparing yourself against competitors over whom you seek to maintain a competitive advantage

3. Simply not knowing but wanting to find out where you stand in relation to your competition/the market/industry

The question of timing (i.e. why now?) is usually based around a critical point in a purchase or decision-making process. That's because this is the time when a greater understanding of your environment will have the greatest impact - a change in vendor, a change in technology, a renewal or renegotiation with a supplier... this is when a lot of money will be spent.

At this point, a decision could have long-term impacts for both savings and added expenditure, and lock you into a decision, the ramifications of which will be felt for some time. It's at this point that a proper look at and understanding of, your environment is most critical.

It’s just about the money, right?

In the end, yes, but on the face of it, merely comparing two prices to deliver a service is overly simple. One of the biggest flaws in comparing yourself to somebody is to look at a limited set of criteria (for example, 'price' alone) rather than taking a holistic view that encompasses much broader criteria.

In the UK, Sainsbury’s Supermarkets recently paid for a full-page advertisement showing two bread rolls with ham, salad and cheese, one from them and one from their competitor Tesco. Sainsbury’s mused that with today's price matching guarantees in place between supermarkets, it would seem that there was little in the way of competition in the mind of the consumer. Hence, two bread rolls, both with the same ingredients and the same price.

However, they noted that whereas all of THEIR ham and cheese was locally sourced in the UK, the Tesco ingredients were not.

Their price was the same but used cheaper ingredients in an identical-looking offering. Which is to say, just because two things are called the same thing, and priced the same...doesn't mean they ARE the same.

The price of something is merely the most visible feature of an offering, especially when what's provided seems to be very commoditized. In order to understand a price, you need to investigate what is received for that price such as:

- Service - Quality - Timeliness - Reliability - Guarantees

So the question is (a) why benchmark, and (b) how to benchmark?


Because it's better to know than not to know

At its most basic, it's because you care. Benchmarking (despite the above supermarket example) is most frequently performed on background, back-end processes that are not sexy, visible, or intrinsically interesting. But they do make a big difference to the outcome of a business' performance. Companies are built on foundations such as this. If those foundations are not solid then the company will suffer. Additionally, publicly visible things like market pricing are easier to measure using internal resources, than back-end items where the price is not necessarily published. That's why you formally hire external benchmarking organisations.

Because while context is important, so is the market in which you operate

At best, most companies are limited to knowledge of their own environments. They can't say whether their performance is comparatively good or bad, and without context, they may not know whether gains can be made in improving performance, or whether those improvements could be made but only in a timeframe and at a cost that is impractical.

Because you might have outsourced the delivery, but you're still responsible

More than ever companies have outsourced many of their back-end processes, whether it be for assumed cost savings, simplicity, or to concentrate on core business. While these benefits could simply be assumed to be in play, it also makes sense to verify them and confirm that the decision to outsource has had the expected benefits. Just because a company has outsourced some of its tasks and processes doesn't mean it should ever absolve itself of the responsibility for that.

To summarise, benchmarking is a way of ensuring that what are frequently considered the boring, tricky, and obscured tasks of the business, are being done appropriately.

Yes, it's political!

Politically speaking, benchmarks can be difficult. It is easier to avoid finding out that things you're responsible for aren't going well than risk embarrassing yourself by revealing that things AREN'T going well. That said, if you work for an even halfway enlightened company, merely sticking your head in the sand and avoiding the possibility of bad news isn't really going to keep people feeling you're probably doing a great job, particularly in the absence of any supportive evidence.

Most organizations would agree that ‘every Benchmark is political’ - and it's true! Whenever you move from something nebulous and unknown to something documented and reported, the data will potentially raise questions that point to issues that could more easily have been obscured had those questions never been asked. But that's a good thing if you have any interest in working for a company that is working well.

Benchmark results can be a pretext for change. The point is that when you know where you stand, you can move with more confidence. If you work for the kind of company where finding out something is wrong is considered WORSE than actively hiding that something is wrong.... then perhaps a benchmark is not for you.


If you're choosing between healthy fruits, comparing apples and oranges is actually quite valid

The idea of comparing one thing to another invariably brings up the notion of 'apples to apples' versus 'apples to oranges' comparisons. The problem is that we live in a world where almost all vendors, outsourcers, off-shoring companies and marketers are trying to convince you of how unique THEY are... and how unique YOU are, as a way of ensuring that comparisons are not easy. The key comes in understanding the framework for comparisons/benchmarking – the selection of peers, the normalization of data, the extraction of unique aspects of your environment and analysis of common aspects.

It is not an easy process, but neither is it shrouded in mystery. Benchmarks should not be taken on faith and their methodology should not arise from a black box.

Select Valid Peers

Any comparison between two items raises the question of whether they can be validly compared. In a market filled with competing fruits, a keen comparison of apples versus oranges can be quite useful, and not necessarily as problematic as assuming that two ham sandwiches are the same merely because they look the same and are made of bread, cheese and ham. Depending on your aims, a comparison against a market average, a top quartile or nearest equivalent organization is valid.

Select Valid Measurements

If you do something unique or operate in a unique environment, then comparisons become trickier. The temptation might be to simply toss aside the value of a comparison based on this and live in blissful ignorance. But there are certain measurements that are valid regardless of environment, and additionally, there are companies out there who DO operate in the same environment - and you're probably competing against them.

Question the Outcomes

A benchmark is not merely a number and a set of graphs. It's an opportunity to improve the way you operate, get a better understanding of the pressures and from that make meaningful decisions about how your business is performing and being run. Given its potential importance, it would be madness not to question the outcomes. Establishing that a valid methodology and peer group selection criteria exists is important, so that when outcomes based on those peers and that methodology are provided, the next steps can be taken with confidence. A good benchmarking organization will provide a conclusion that is backed up with data, analysis and reasoning.

A benchmark is especially useful when considering a change in vendor, a change in technology, a change in how an IT process is managed (e.g. going from an internally managed environment to an outsourced one or vice-versa), a contract renewal or renegotiation with a supplier.

The three key criteria for selecting an appropriate benchmarking organization should be - • a clear and transparent methodology; • the number of contracts to which they have access which translates to the depth and breadth of data points; • their level of experience.

Benchmarks should be viewed in a positive light and as an opportunity to improve a relationship with an existing customer/vendor and identify areas that can be improved with the right kind of governance or process improvement.



comments powered byDisqus
Led lighting small

Read next:

How To Strategically Manage The Dual Game Of Re-Defining And Defending A Business