Wil van der Aalst, known as the ‘godfather of process mining’, spoke with PEX Network ahead of his session at PEX Live: Process Mining 2021, to offer an expert insight on the history and current state of process mining, as well as his best practices for implementing the technology in 2021.
In this interview, Van der Aalst reveals how he started his process mining journey and offers advice on process mining-related topics including automation, artificial intelligence (AI) and machine learning (ML).
PEX Network: You are known within the world of process excellence as the ‘founding father’ or ‘godfather’ of process mining. What inspired you to start working on this area in the late nineties?
Wil van der Aalst: I had been working on process modeling, discrete event simulation and workflow management for many years, and while I enjoyed working with process models, I was also disappointed by the impact of these approaches in reality.
Simulation models rarely described reality and did not help to uncover the real problems in business processes. Often, the process of making the simulation model was more important than the results themselves. Similarly, most workflow projects failed and in many organizations the workflow product that was purchased never became operational. For that reason, I wanted to analyze the real processes based on data.
I started to work on this in the late 1990s and developed some baseline algorithms to discover Petri nets from event logs. My group’s research at Eindhoven University of Technology led to the open-source process mining framework ProM.
For a long time, our group was the only one working on process mining in a systematic manner and we worked on dozens of process mining algorithms covering the whole spectrum of topics including conformance checking, predictive analytics and decision mining.
Despite little interest from industry at first, the process mining market gradually grew to the 35 process mining vendors that exist today, which came from our research and explains why people refer to me as the ‘founding father’ or ‘godfather’ of process mining. The large-scale adoption in industry happened only in the last five years, so while many organizations are using process mining today, we are still only at the start of its development.
PEX Network: How would you define ’state-of-the-art’ process mining and what does it look like in practice?
WvdA: Today’s commercial process mining tools work well and have proven their value. These tools still resort to producing directly-follows graphs (DFGs) based on event data, however, rather than using more sophisticated notations that are also able to capture concurrency. Moreover, to tackle complexity, DFGs are seamlessly simplified by removing nodes and edges based on frequency thresholds. Process mining practitioners tend to use such simplified DFGs actively. Despite their simplicity, these DFGs may be misleading and users need to know how these process models are generated before interpreting them.
Activities that have a flexible ordering – for example, due to concurrency – lead to spaghetti-like DFGs with loops, even when activities are executed once at most. DFGs can be simplified using frequency-based thresholds. This may lead to all kinds of interpretation problems, however, due to ‘invisible gaps’ in the model.
Performance information mapped onto DFGs can be misleading and it is unacceptable to only use them in the long run; for example, the average time reported between two activities is conditional and the only situations considered are where they directly follow each other. However, there are plenty of process mining techniques that use higher-level models, such as business process management and notation (BPMN) or Petri nets. Moreover, there are more robust ways of filtering, but these are typically not used for performance reasons.
PEX Network: Which limitations do you see with existing process mining tools?
WvdA: Commercial process mining tools tend to focus on the discovery of the DFGs that I mentioned before. Although valuable, there remains a gap between process modeling tools using BPMN and process mining tools, for example.
Organizations can get confused when seeing the real process models with all deviations and details while having oversimplified process models that only exist on paper. Here, conformance checking plays a pivotal role as it can bridge the gap between the high-level normative process models and the recorded event data. Therefore, organizations need to ensure process mining tools cover the basics, that are better process discovery and conformance checking. After this, they can focus on other elements including predictive analytics and comparative process mining.
PEX Network: Who within the business should implement and own process mining?
WvdA: The beauty of process mining is that, not only does it combines processes and data, but it connects business and IT. The multifaced nature of process mining complicates matters, however.
In my view, process mining should be supported strongly by upper management. When starting with process mining, 80 per cent of the efforts and time are spent on locating, selecting, extracting and transforming data, and only 20 per cent is spent on the actual analysis. This, combined with privacy concerns, is often used to kill process mining initiatives before they can show value.
In addition, large groups of consultants, auditors, quality managers and process owners are unaware of the capabilities of today’s process mining tools. One can still become a certified Six Sigma Black Belt or internal auditor without being able to apply process mining.
Decision-makers are often unable to see that process mining is different from machine learning and artificial intelligence, and managers may be afraid of process mining results due to the increased transparency of processes that could reveal mismanagement, inefficiencies and compliance problems.
Since process mining is generic, it makes sense to create process mining competence centers on a corporate level. This way, experiences can be reused and process mining rolled out to many operational processes in an organization.
Process mining competence centers should not be created next to BPM and Six Sigma centers, however. It is important to merge all process-related initiatives and ensure that the technical competencies are well represented to be able to extract the data from existing systems, and integrate process mining results in existing management dashboards and decision support systems.
PEX Network: Where do you see most companies making the wrong decision when it comes to automating processes?
WvdA: The focus of traditional workflow management is on automating structured processes that need to be performed for many cases. Only in these settings is it cost-effective to automate workflows. There will always be manual processes, because it is either not cost-effective to automate them or human creativity is required.
Robotic process automation (RPA) is an interesting intermediate technology using ‘bots’ to replace people interacting with information systems. RPA can be cost-effective for less frequent processes, because the back-end systems do not need to be changed.
Having RPA in place makes it important to determine what to automate and in what way. This is why process mining is an enabler for RPA. More broad, process mining helps to distribute the work and monitor the interplay between parts of the process that are automated, bots and people. Process mining can observe processes in a unifying manner.
After introducing RPA, the process still needs to be monitored by process mining. Conformance checking can be used to check for deviations, predict problems, and signal problematic handovers between software agents and humans. All of this will help to realize more incremental and cost-effective digital transformations. I strongly believe in hybrid intelligence, with most tasks being best carried out through a flexible combination of software agents and humans.
PEX Network: Why should companies invest in process mining in 2021?
WvdA: In my view, process mining is unavoidable in the long run. For organizations that are “in control”, it is relatively easy to get started with process mining and quickly reap the benefits. Organizations that have major data quality problems and spaghetti-like processes that are out-of-control will need to address these problems anyway. For sure, the Covid-19 pandemic is reducing budgets for innovation and will make it difficult to invest in technologies like process mining.
The Covid-19 pandemic has demonstrated the importance of processes, reliable data and the ability to respond to entirely new situations. When processes change dramatically, process mining still works and decision-makers can use process mining results to make informed decisions.
I expect that in the longer term, the pandemic will fuel new digitalization initiatives, because it made people realize that reliable data and robust processes are critical for the survival of an organization. Many problems showed the gap between the boardroom and political ambitions, and everyday reality with paper forms and excel spreadsheets.
PEX Network: What is your process mining vision?
WvdA: I hope and expect that process mining will become the ‘new normal’. Bridging the gap between process modeling and process mining using conformance checking and better discovery techniques is the first step.
The next step will be to allow for more complex data. Traditional process mining techniques assume that each event refers to one case and that each case refers to one process. In reality, this is often much more complex. There may be different, intertwined processes and one event may be related to different cases (convergence) and, for a given case, there may be multiple instances of the same activity within a case (divergence).
Object-centric process mining relaxes the traditional assumption that each event refers to one case. An event may refer to any number of business objects and using novel process mining techniques one can discover one integrated process model showing multiple perspectives. Moreover, processes change over time and the same process may be performed at different locations. It is, therefore, valuable to compare the corresponding process variants using comparative process mining.
Following these steps will lead to more mature forms of process mining.