Are we focusing on the wrong BPM Metrics? Interview with Forrester's Craig Le Clair

Business is in the habit of collecting all kinds of data on everything from quality through to effiiency. But in today's ultra-competitive environment how much of that data is really telling companies the one question they really need to know: how am I actually doing?

Unfortunately, the answer is that many companies are measuring what's easy to measure rather than what actually counts, according to a new Forrester Research report.

In this PEX Network interview, Craig Le Clair, an analyst with Forrester Research who has been doing research into this area, gives us insight into what his research has found, why current BPM metrics are inadequate to deal with the new realities of business in the digital era, and offers some suggestions on what to do about it.

Editor's note: This is a transcript of a recent Process Perspectives podcast. Listen to the original podcast here: BPM needs to go Moneyball

Download a highlight of the research findings: Using Metrics to Drive BPM Excellence

PEX Network: Tell me about your research. What got you interested in the question of which metrics companies are actually using?

Craig Le Clair: What really got me started reading the Michael Lewis book Moneyball - about baseball -and I apologise to those from the UK and Europe who don’t study that game as much as we do over here! One of the interesting aspects of that book was that here’s a game that has the most exposed statistics than anything that’s followed. You grow up watching this little 3 x 12 matrix and it summarises the entire game.

So that would mean we should be looking at four metric categories: agility, customer experience, quality, and productivity. So the genesis of the research really was "okay, that’s an interesting theory, Craig. You sit around and you think about these things; let’s go out and get some data and see if that’s true". So that was really why we hooked up with IQPC and trying to get a better understanding of whether that metrics framework, whether that notion of "Moneyball for business" made sense.

PEX Network: What would you say are some of the key problems, then, or some of the key challenges confronting how companies are actually using metrics today?

Craig Le Clair: The major challenge is that the business process management metrics in the categories that I talk about are really operational metrics. They’re metrics that exist in the middle of the organisation, and they’re really important because they are leading indicators of performance. In the end, they affect the ultimate metrics which are the balance sheet metrics - net income, revenue, and so forth.

But one of the big challenges is that companies do not link the operational metrics - the BPM metrics, if you will, in our BPM framework of metrics - in any systematic way to the balance sheet numbers and the C-Suite that is making decisions about the company do not have adequate transparency to the metrics which are leading indicators. By the time your quarterly report or your annual report is done, your balance sheet is fixed; it’s too late to affect any change. It’s too late to look at those numbers and make decisions to affect those numbers - they’re lagging indicators. So, that’s one set of challenge.

Then, there’s a whole set of challenges around metrics in general. How do you design metrics to generate the right behaviour, both culturally and organisationally, in companies? And that’s why there’s not a very strong use of metrics, because they have to be dealt with very carefully because they affect all of these human organisational and other issues in companies.

The third area is just the technology problem. There’s a big gap in understanding between IT and the business with regard to what can be measured and what’s important to be measured. IT does not understand the business well enough, and the business doesn’t understand what IT can actually measure and do well. Exacerbating the problem is that the underlying rigid legacy systems - packaged applications - really do not provide flexible information access and the kind of analytics and metrics. So companies are now building layers on top of that, BPM being one of them. Things like business intelligence and analytics to try to really rectify that. So there are a number of problems. It’s not an easy problem, but it’s a very important area.

[inlinead]

PEX Network: In your report, you say that the data shows that tomorrow’s processes will derive from the voice of the customer and other approaches that start with assessment of the customer experience. What do you mean or envisage by that?

Craig Le Clair: It’s pretty clear. You look around today at work or at leisure and you just see consumer technology everywhere. We forecast there’ll be 250 million smart devices, in the US alone, by 2016 – those devices being phones, smart phones; another 150 million tablets. So the consumer - the customer - has a tremendous amount of computing power and, right now, businesses are really still tied to the systems that were architected and designed in the 70s, 80s, and 90s.

There’s a big gap there, and the companies that can close that gap fastest will close the gap fastest. That means interacting with customers in the new customer experience manner that they expect to be interacted with through multichannels - more in the moment, more on the run, more contextual, more personalised.

That’s going to become a major battleground for companies. It’ll see some companies that will just cease to exist as viable companies. The level of innovators out there coming up with new business models based on consumer technology and social and big data vastly exceed any number of innovators that have existed in the past. Think of it, every year from some great university in the UK or in Europe you’re graduating 500 or 1,000 entrepreneurs who just want to tear down your existing business.

There’s the reason that customer experience and voice of the customer - and voice of the customer implies getting data from the outside world - has been increasing in importance. If you think about big data for instance, it’s getting data from social, being able to understand that this group of teenagers at this age will be at the mall on Friday night. That type of analytics is going to become more important to deal with that very important battleground that’s emerging.

PEX Network: What actually needs to happen, then, to win on this "battleground" to enable companies to focus better on the customer experience?

Craig Le Clair: The research that we did was clear on two points; agility measurement is important. Business agility is very important, being able to respond to these changes and the events. Companies need to think about their systems and their culture and their organisation to become more flexible in architecture to allow rapid changes to the systems and processes they depend on. That’s the first thing.

You have to think in terms of these other metric categories that we really focused on in this research - agility metrics and customer experience metrics. Take a look at analytics and BPM as very important technologies to really support the development of these metrics. So enterprise architects become a key role here to create that flexible architecture to allow the evolution of these key metrics.

So when you’re doing a BPM or a transformation project, internally, you need to have these metric categories and a metric perspective up front and build that into the BPM project so it’s not an afterthought. Oftentimes the measurement of both the programme and the measurement that will, ultimately, be delivered to the business are afterthoughts. But they need to be engrained right up front and, as I was saying earlier, try to link those BPM - those operational metrics to business goals. There have been a number of strategic approaches that have tried to do that, Balanced Scorecard being one. But we haven’t really operationalized or institutionalised it. We haven’t made those connections in a systematic way. That needs to be a continual focus.

PEX Network: My final question, then, is - for the listeners out there who might now be thinking oh, no, I really need to go back to the drawing board and redesign my company’s metrics - what should they do? What advice would you give them on getting started?

Craig Le Clair: I would look at Forrester’s metric framework and I would look at others as well. But try to have broader discussions between IT and the business; specifically, around metrics and accountability. That is the way that IT can understand the business. They’re not going to understand all the nuances of the business but, understanding the metrics that are important to the business, and doing that trade off of what the business needs and what can be extracted from the IT environment, those conversations are going to give IT the level of understanding that’s really workable of the business.

Getting that dialogue going is very important, organisationally. Another thing is to take a look at what we call untamed business processes, which are really these end-to-end processes that cross departmental silos and we looked at a lot... in the research we looked a lot at value streams. We didn’t look at specific functional areas. We looked at how an end-to-end process operates, cross department, cross information silos and tried to measure the productivity and how BPM projects were successful or not in attacking that value stream. What were the most important metrics for those value streams? So think of it in terms of this value stream end-to-end process approach which really gets to the untamed process that transcends the departments and introduces a lot of non-value added activity. So I think communication’s really important and really looking at metrics in a new light is really critical.