Keith Swenson’s Go Flow blog continues to produce thought-provoking discussions of BPM issues. Check it out if you are not a subscriber. His latest concerns simulation, one of my hot buttons. A couple years ago I wrote that simulation was a “fake feature” – one of those things vendors put in the tool to tick off the Gartner checklist but which don’t do anything useful. Since then the situation has not improved to any great degree. This is too bad, because, as Keith suggests, simulation can be of great value in projecting the expected performance improvement from a process change before committing the resources needed to make that change.
But it would be better to say it could be of great value, if the tools were any good. I recently did a small consulting project for a BPMS vendor on what was good and not so good about their product. They really hyped their simulation tool, but I had to tell them it was, in my opinion, mostly useless, because it did not distinguish between the active time of a process activity, which consumes the assigned resource, and wait time (sometimes called lag time), which does not. It considered the total time to be active time.
The reason I call it useless is that in most process improvement projects, the problem is not too few widget-tweakers assigned to the widget-tweaking step, causing a backlog when there is a spike in widget orders. That simulation use case, which I call optimizing resouce allocation, is real in certain heads-down scenarios, such as call centers and backend clerical processes, but it’s not the main one BPM project teams are dealing with.
Far more common is the process improvement use case, which aims for improvement in some metric, usually cycle time but occasionally cost or quality, based on changing flow of process activities. The resource assigned to an activity – approval by a manager, for instance – is not fully dedicated to that activity, nor even other activities described in concurrent simulation models. The active time to perform the task bears no relationship to the actual time to complete it.
Separation of active time and lag time parameters for an activity is just one of many real-world requirements ignored by many simulation tools. Here are some others:
- Event probability and time of occurrence. In BPMN, events provide an expressive visual language for describing the exceptions that occur in real-world processes. In fact, these exceptions are usually at the root of performance problems in the as-is process. To project the expected to-be improvement, you need to be able to assign a probability and mean time of occurrence for events in the process model. Most simulation tools ignore events.
- Repeating activities. BPMN has two types of repeating activities, called looping (DoWhile) and multi-instance (ForEach). You need a simulation parameter to model the number of iterations.
- Instance properties. In most simulation models, the probabilities at each node are uncorrelated. In real-world processes they are highly correlated. For instance, the duration of a particular task, the probability of taking a particular gateway output, and the probability of some event occurring usually track together. In other words, certain classes of instances tend to take longer, tend to take path 1 rather than path 2 out of the gateway, and have a higher than usual probability of some in-flight event. Some BPM Suites have optimization routines in their BAM/Analytics component that actually figures these correlations out for you! But try to find a simulation tool that lets you correlate them in the simulation model. One way is to define the simulation parameters not as a simple number (mean and standard deviation) but as an expression of one or more instance properties, such as orderValue, which could take values high, medium, and low. This makes configuring instance generation more complex, as you need to define the rate of each type, but it could provide much better output.
- Contingent resource assignment. Most simulation tools let you assign tasks to roles with some defined cost-per-hour or cost-per-use parameter, or groups of same. But not many let you say assign to role A as the primary resource, but if no member of role A is available then assign to role B.
- Prepopulation of backlogs. Simulation models generally start empty, meaning no instances in the system. Besides the obvious distortion at the beginning of the simulation period, this does not allow the resource allocation optimization use case to apply when it is really needed, i.e. when an actual running process is jammed up with backlogs and you need to try various alternatives for working out of it.
- Access to raw output. The prebuilt metrics and charts provided by simulation tools are convenient but they rarely provide the detail you need for real analysis. For that you need the raw output, one record for each process instance, and also one record for each activity or event instance, dumped into Excel or a database. From that you can create histograms, provide activity-based costing, and perform other useful analysis. Without it you basically have eye candy.
A simulation tool that provides these 7 features would be really useful in BPM, but nothing I have seen comes close. The tool I use in my BPMessentials training, Process Modeler for Visio from itp-commerce provides 3 of them, and I don’t know of other tools that provide more than that.
At the risk of creating another execute v transform argument, I suggest that an effective simulation strategy would be take an existing simulation tool and transform the BPMN model into it. Of course, that might be easier when there is a BPMN schema.
Apart from the analysis available from simulation, complex processes can be animated in the simulation tool which can save a lot of hand waving in from of an abstract BPMN diagram.
Bruce,
another great summation of a major shortcoming of existing simulation tools on the market. In general, they all suffer from varying degrees of these problems – and the traditional problem of “bad data in, bad data out” – if you are simulating a process and putting in the data and probabilities but can’t be assured that those are based on “real world” values, then how valuable can the simulation really be?
I especially like your point about event probability/occurrence – this typically isn’t handled at all by simulation tools and for SOME models that renders the simulation completely useless.
For instance properties – I have seen simulations take into account real-world probabilities/correlations in the simulation – but without starting from real-world measured data, the data entry becomes simply overwhelming except on toy models. I would label that point really “correlation of properties to outcomes” rather than just instance properties…
I have worked with a tool that provides pre-loading the data -either data that is generated from the simulation (essentially “simulating” a process load that might be 3-6 months after starting the process from zero), or by using a real point in time, making a change, and simulating from that point as the starting point (with in-flight instances, etc).
I still find that, in general, real-world BPM models are a little too complicated for the real-world simulation software- though I have been impressed by the progress of just one or two of the vendors in terms of their progress on this score, and I think the simulations that take into account REAL data are a big step up to those that depend on manually inputted data.
Bruce,
I fully agree with your 7 points. However, there are tools that support most of them if not all. I personally have experience with ARIS Business Simulator and it supports these 7 and many more. (I do not represent the vendor in any capacity, I’m just a user.) The information posted on the web – http://www.ids-scheer.com/en/ARIS/ARIS_Software/ARIS_Business_Simulator/5289.html – is quite a bad marketing compared to other vendors but is some starting point anyway.
Regards
Ivo
Hi Bruce –
I certainly agree with your basic premise, but am compelled to point out that Global 360’s simulator (part of our Insight360 product) offers all the features you describe. The possible exception is contingent resource assignment, which doesn’t work exactly as you described it, but I think we can achieve the same effect. I’m happy to arrange a demo sometime if you like.
I’d also argue that prepopulation of backlogs isn’t really necessary if you have the raw output described in your last bullet – you can simply perform your analysis on any given window of output, including one that starts after the simulation has been running for a while.
The problem with simulation, in my opinion, lies in most businesses’ lack of understanding of their processes. This makes coming up with parameters for the baseline simulation scenario very difficult. We can use production data for this, but you run into the classic problem: differences between implementation models and the business model that simulation-running analysts care about.
I think the root cause of this issue is simply an overall immaturity of BPM efforts. Only now are we starting to see the more advanced BPM practitioners really use simulation, and as you suggest, it can be a very powerful tool for change impact analysis.
Regards
Colin Teubner
Colin,
Good to hear from you. I’ll take you up on the demo. (I should have added another thing… don’t require a BAM runtime to get model simulation.) I also agree with you (and others) that users don’t understand enough about either their own processes or how to use simulation to do simulation effectively. But it’s a chicken and egg problem. Until the tools are good, users won’t know how to use them. So fixing the tools, unfortunately, comes first. (Welcome to the hard realities of a BPMS vendor!)
I’m not sure I understand your prepopulation scenario. The issue is not starting after simulation has run for a while. It’s starting from the current actual condition, and simulating from there. Can G360 do that? If so, that’s great.
–Bruce
Lanner supplies L-SIM a JAVA component specific for simulation of BPMN, we have been involved in simulation for years and so fully endorse the power and usefulness of the technique.
L_SIM can support all the points raised by Bruce….we also advise partners on how to implement it properly. We feel the key is for BPMS vendors is to use a proper simulation engine and provide users with the necessary capability.
L-SIM is used inside ARIS by IDS Scheer, so we appreciate IVO’s comments!
Thanks Bruce, I agree with many of your observations. People talking about simulation features obviously have little hands-on simulation experience in real-life settings where people are involved. For example, the Yerkes-Dodson Law of Arousal can be observed in any office situation but is never supported by tools. Moreover, people are involved in so many (informal) processes that it is very difficult to simulate one process in isolation, etc.
For a study into these effects, see the following paper:
W.M.P. van der Aalst, J. Nakatumba, A. Rozinat, and N. Russell. Business Process Simulation: How to get it right? BPM Center Report BPM-08-07, BPMcenter.org, 2008.
http://is.tm.tue.nl/staff/wvdaalst/BPMcenter/reports/2008/BPM-08-07.pdf
It lists these problems, and identifies the effect of some of these phenomena.
Cheers, Wil.