Today I read a terrific post from John Reynolds, IBM BPM Product Manager and author of the Thoughtful Programmer blog, entitled Tools for Subject Matter Programmers - Learning from Turbo Tax.
Subject Matter Programmers are Subject Matter Experts who know just enough about Programming to craft their own solutions.... I'd like to see a world where Subject Matter Programmers can tackle building something like Turbo Tax.I just about fell out of my chair because this is exactly what I have been lately trying to do in the BPMN space. I fit John's definition of Subject Matter Programmer to a T. I know a lot about BPMN but almost nothing about programming. To write an app like TurboTax, you ordinarily would need something like database design and SQL skills, Java skills for the program logic, and web app skills, say JSP or PHP, for the UI. I have none of those. But for the past few weeks, I have been trying to build something like TurboTax for BPMN, i.e. a tool that creates "good BPMN" - meeting all the best practices of my Method and Style approach - not by drawing the diagram but by filling out a simple questionnaire.
I am still working on it, but the only reason this is remotely possible is I am using a tool called WebRatio. It is a tool explicitly designed to allow Subject Matter Programmers to build database-driven web applications. I met Marco Brambilla, one of the founders of the company, at a BPMN conference in Lucerne last November. To demonstrate the tool, the WebRatio guys quickly developed a more robust implementation of my online BPMN validation tools (for Method and Style validation as well as BPMN-I). In the past few weeks I have been trying to build the TurboTax for BPMN tool myself using WebRatio.
Why would I have an interest in such a tool? In my BPMN Method and Style training, most of the details of "good BPMN" can already be validated with one click in the itp commerce Process Modeler for Visio tool that we use. Of all the features we have added to the course, this built-in style rule validation has made the biggest improvement in the speed of learning and quality of student work. It's basically like spelling and grammar checking in MS Word. But there are a number of things that it does not check:
- Should a merge gateway be AND, OR, or XOR?
- Should an end event be a Terminate?
- Should the multi-instance marker be on the subprocess as a whole or on individual tasks at the child level?
Also, I have come to believe that constructing a process model by drawing a flow diagram from start to end encourages, to a degree, bottom-up thinking - constructing the whole by assembling parts - rather than top-down - starting with the process as a whole and then dividing it into parts based on basic principles. By structuring the questionnaire in my TurboTax for BPMN app, I can guide the thinking to be top-down, and generate the diagram from the answers. This is the theory, anyway.
One of the key concepts of my "method" is the idea of end state. That phrase is not mentioned once in the 508 pages of the BPMN 2.0 spec. It is a business concept, meaning how did an instance of an activity (or process) end, successfully or in some particular exception end state? It is an adjective or noun-adjective phrase, like Order complete or Out of stock. In the initial implementation of my tool, the end states of an activity (except for multi-instance) are mutually exclusive, meaning each instance of the activity must end in exactly one of the enumerated end states. If the downstream flow depends in any way on information received or developed in an activity, some decision made in the activity, or some exception that occurs in the activity, each outgoing transition from the activity must correspond to a distinct end state. If the downstream flow is always the same, then the activity has just a single default end state.
End states are helpful for understanding the exceptions that could occur in the process and capturing in the model all the possible flow paths. In a BPMN diagram, each end state is depicted as an end event (of the subprocess expansion) labeled with the name of the end state. If an activity has more than one end state, a gateway following the activity tests the end state, with one gate per end state.
The questionnaire works like this. (If you've read my book BPMN Method and Style, you will instantly recognize this as "the method.") Step 1 is describing and naming the process, how it starts, what each instance represents, and enumerating the end states of the process as a whole. Step 2 is creating the high level map, a subdivision of the process into 10 or fewer activities, according to certain principles (including alignment of activity instance with process instance), and enumerating the end states of each activity. Optionally you can assign actors to each activity. The high level map is just a list, not a diagram. In Step 3, for each end state of each high level map activity, the user specifies the transition, i.e. to a next activity or process end state.
That's it (at least for the top-level diagram). Note the user is not asked about gateways at all. Those constructs are derived.
When the user is done with defining the flow in this way, the program generates the BPMN, first in the internal object model, then in BPMN 2.0 XML, and finally in a diagram. I am still working on the first piece of that. First, the program needs to generate the splitting gateways (AND and XOR). In some cases there are subtleties whether what could be an OR gateway is modeled AND-then-XOR or XOR-then-AND. Splitting gateways ensure that each start event or activity has only one outgoing sequence flow. At this point, activities and end events may have multiple incoming sequence flows. For each of those, the program inserts a merge gateway. The hard part is determining whether that is a simple merge (XOR) or a join, and if the latter, whether AND or OR. That requires generating constructs called tokens and following them from start event to end event. Splitting gateways generate child tokens, and merging gateways sometimes consume them. An algorithm determines at the end whether a merge gateway is XOR (which can usually be removed), AND, or OR, as well as whether an end event is Terminate.
I have most of this algorithm modeled in WebRatio. I have not yet tackled the scenario of loopbacks, i.e. where a branch of the flow loops back to a preceding merge. I plan to begin by assuming it is XOR, following the tokens in that case, and checking consistency when the token loops back to the merge. Some loopbacks are illegal in BPMN (in the sense that they create multiple simultaneous instances on the same path in the model), and I am hoping this technique will catch those. If you have experience on this, please contact me.
At this point I will have, for the top process level, all the information required to generate BPMN XML. In WebRatio, that requires a bit of scripting using Groovy (a Java variant). I think there's no escaping this bit of programming. What about the diagram? The itp tool can import BPMN XML, without layout information, and generate a diagram. That's Visio and my tool is a web app, so possibly I could call my tool as a web service from itp. Better would be a tool that provided its own web service that generates a (read-only) BPMN diagram from submitted XML. If you have that, call me!
So... I am not yet John Reynolds' poster child for Subject Matter Programmer, but I'm making progress.
If you want to understand my Method and Style approach to BPMN - the best way to ensure your process modeling efforts can be shared across the organization, I suggest my class on March 5-7 from 11am-4pm ET, 8am-1pm PT, 5pm-10pm CET. Click here for more information, or here to register.