I’m in the process of updating my 2006 BPMS Report series on BPMInstitute.org to the new and improved 2007 version. A major change from last year is a beefed-up evaluation scoring. I’ve discovered that many users (and most vendors) are happy to skip the 25-page walkthrough of the product and go straight to the scorecard at the end. Which product “won”? I haven’t figured out the presentation – it will probably be some 2-dimensional thing like the Forrester Wave or Gartner MQ – but I’m close to having a finished scoring methodology. It’s probably asking for trouble, but I’m publishing it right here so that you can comment upon it.
The basic plan is this. I define 4 process types: Task Routing (basic workflow), Production Workflow, Case Management (emphasizes content, collaboration, and unstructured processes), and Integration-Centric. The characteristics of each type are explained in the report overview, but most of you can imagine what they are. Each BPMS is scored against all 4 process types using 12 sets of criteria, but the weightings of each set may differ from one process type to the next. Also, the capabilities affecting the individual criteria may be process type-specific.
Here are the 12 sets of criteria, things I’m looking for in each (some are process type-specific), and the percentage weighting of the set for each process type:
1. Architecture and Environment (Weightings: task routing 10%, production workflow 10%, case management 8%, integration-centric 10%)
· Unified environment for workflow, integration, rules, BAM
· Minimal programming required
· Scalable
· Clustering, hi availability
· Leverage J2EE or equiv platform services
· Web-hostable/SaaS-compatible runtime
· Support for standards
· RAD/iterative support
· Component discovery and reuse
2. Modeling (Weightings: task routing 10%, production workflow 10%, case management 4%, integration-centric 5%)
· Usable by business process analysts (not developers)
· Support for BPMN (full support)
· Integration of BPA (ARIS, ProForma, MEGA, etc)
· Stay in sync w/design (shared or roundtripping)
· KPI modeling
· Model repository, publishing
· Support for industry models (SCOR, ITIL, etc)
· Team collaboration
· Simulation
3. Human Workflow (design) (Weightings: task routing 10%, production workflow 10%, case management 8%, integration-centric 5%)
· Dynamic task assignment
· Flexible routing
· Form design
· Screenflow design
· Worklist design (columns for business data)
· Internationalization support
4. User Experience (runtime) (Weightings: task routing 15%, production workflow 7%, case management 13%, integration-centric 5%)
· Ajax web forms
· Offline task participation
· Guided task performance
· Portal design and integration
· Reassign/delegate task
· Production workflow support (getNext item)
· User sort/filter/query worklist
· Instance status tracking
5. Content/Collaboration/Case Management (Weightings: task routing 10%, production workflow 7%, case management 13%, integration-centric 5%)
· OOTB attachment support in task UI
· Doc upload, annotate, edit
· Viewers
· Checkin checkout, versioning
· Integ with 3d party ECM
· Archiving
· Retention mgmt
· Team room support
· Discussion, chat, presence detect
· Process knowledge repository
· Case mgmt support
6. Business Rule Management (Weightings: task routing 5%, production workflow 7%, case management 8%, integration-centric 10%)
· Rule repository
· Rule design
· Rule maintenance app
· Rule change impact analysis
· Rule engine
· Rule-process integration
7. Integration (Weightings: task routing 5%, production workflow 7%, case management 8%, integration-centric 10%)
· Adapters ? Introspectng/self-generating; packaged – Mainframe, pkg app, b2b, .Net etc
· Async integration, callback, reliable messaging, WS-Addressing, security etc
· Data transformation mapping, XSLT/XQuery engine
· ESB, mediation
· Registry/repository
· Metadata library
· B2B support ? trading partner gateway, EDI
8. Events and Exceptions (Weightings: task routing 5%, production workflow 6%, case management 13%, integration-centric 10%)
· Event listeners and adapters
· Full bpmn event support
· Wait for event
· Interrupt by event + exception flow
· Manual suspend/resume instance
· Error propagation, handling without programming
· BAM events and actions
· Transaction rollback and compensation
9. Performance Mgmt/BAM (Weightings: task routing 5%, production workflow 10%, case management 4%, integration-centric 10%)
· Metrics and kpis ? OOTB and User-defined
· Dashboard design – Charts, reports, alerts and notifications
· OLAP-style breakdowns
· Bam rules and actions
· Drilldown to root cause
· Instance monitoring
· Optimization actions
· Predictive analytics, e.g. expected finish
10. Governance (Weightings: task routing 5%, production workflow 10%, case management 4%, integration-centric 10%)
· Enterprise repository of models and components
· Link processes to goals and KPIs
· Role based access control and authorization
· Version control of model and implementation components
· Change request, approval, and implementation workflows
· Audit trail on component changes
· Change impact analysis
11. Solutions and Services (Weightings: task routing 5%, production workflow 6%, case management 4%, integration-centric 10%)
· Industry solutions ? documented, QA?ed, supported
· Partner industry solutions
· Professional services
12. Installed/reference customers (Weightings: task routing 15%, production workflow 10%, case management 13%, integration-centric 10%)
· Task routing implementations
· Production workflow implementations (volume, industry)
· Case management implementations
· Integration-centric implementations (volume, industry)
Products will be scored in each of the 12 categories from 0-5, as in the Forrester Wave, based on the bullets listed here (as amended); in some cases the scoring will be process type-specific. Then these scores will be weighted as shown here (or as amended) for each category. One or more categories may be split off to form the second axis of the final result, as Gartner and Forrester both do in theirs.
So there you have it. If you see something missing or improperly weighted here, please let me know, either by comment or by private email.
Hi Bruce,
Your list seems to be almost completed or over-completed even.
If you look to Gartner definition of BPM or some other industry definitions, I will be very surprised to see a single BPM-platform which has BPM, EDI and Business Rules Engine out-of-the-box.
These 3 elements are so different, that probably nobody will be able to deliver them in a single solution offering.
On the other hand you could add the following elements in your matrix definition:
1. Support of workflow patterns, like: Split, Or, Xor, And, Branch, Rollback, Skip, Milestone, Synchronous Subprocess, Asynchronous Subprocess, Dynamic Sequental/Parallel/Hierarchiery
2. Support of XPDL or BPEL
3. Dynamic process engine, so meta-driven BPM Engine versus code-driven BPM Engine.
4. On the-fly-process upgrade and manipulation.
Best,
Hans, I think that Bruce has covered some of the things that you think are missing. BPMN support in the modeling section would cover #1, and presumably XPDL/BPEL would be included in the standards support part of the architecture section, covering your #2 point, although I agree that it would be good to state that explicitly.
My concern is more around how the information is gathered: do the vendors self-report on this, do you get a demo from the vendors to prove the existence of these features, or are you doing hands-on evaluations?
Thanks Sandy for the chance to elaborate. The purpose of the reports is to create a short list of BPMS products to bring in for the POC, customer reference due diligence, and cost discussion. The information is mostly based on product documentation, supplemented by briefings and direct questioning of the vendors, a few demos (usually at vendor’s insistence).
I’ve done these before using a POC-style creation of my process using the tool. With today’s system architectures, that’s difficult and expensive, and I think actually puts more emphasis on toy applications than what you can glean from the documentation… but maybe that’s self-serving.
Thanks also for your response to Hans. The one thing I would add is that B2B support (EDI, trading partner gateways etc) is commonly provided by integration-centric BPMS vendors (IBM, webMethods, Vitria, etc) but I agree often not marketed as part of the suite or integrated with the BPMS tooling. I’m a bit on the fence whether to include that or not.
On the aspect of WS-* standards support, this Web Services Standards Overview might come in handy:
http://www.innoq.com/resources/ws-standards-poster/
Looking at that overview, I’m asking myself what we actually mean when we talk about supporting WS-* standards…
From looking at your chart, I’m really glad I just said “support for standards” and not “support for WS-* standards”. Does anybody take that chart seriously? For BPM, I sure don’t.
Quite right…I knew there were a lot of these WS-* Standards, but looking at this chart made me realize that it has already become virtually impossible to keep track of them all.
Question remains – although not exactly in the context of this posting – how we should approach the Standards issue. How best to apply a ‘selective pragmatism’ that will help us single out the essential ones and neglect others.
One a general note, Thanks for this most interesting blog of yours.
Regards
Paul
Bruce,
This is an extensive list, which really shows how broad the BPM space is, spanning the spectrum from fully automated Web Services process orchestration and Enterprise Service Buses through modeling, simulation, rules, human centric through to content based systems.
I think that in soliciting comments you are going to highlight every individual vendor’s bias, and the single differentiating feature they feel they have over a threatening competitor. Which will risk pulling your list down to the checklist level that we all expect and dislike.
And that’s why your list is good as it stands. Its not too granular – so it enables you to position systems in an appropriate space so that organizations can really shortlist the systems they should look at for their requirements. And in doing so, the BPM buyer can decide themselves whether this standard is more important that that standard, but only once they have decided the type of BPMS best fits their business objectives.
I’m looking forward to spending a little time looking through this list to see how the Global 360 products fit, and how I should suggest we manipulate the list to fit my interests better (just joking!!!).
Nice work.
Phil
Phil: “how I should suggest we manipulate the list to fit my interests better” — LOL!
a good Market driven list 😉
but when i have Pattern 15 with sub order parallel process with various asynchron and synchron paths i use jbpm 😉 , they have a good dynamic dsl flow diagram tool lool
where is semantic concerns in your list ?
bpm joke : when two BPM specialists have a talk they generaly do not agree about their diagrams arguing, redefinning terms they use and finaly use pen and paper 😉
when Two mathematicians talk they exchange foumulas and mathml and have time to laught about poor bpm semantic papers making jokes about bpm specialists 😉
pushing you to your best
Bruce, thanks for sharing your knowledge and experience with us. I’ve been looking at BPM as a modeling approach to help bridge the communications gap between business and IT, but it seems that there is a strong inclination by the object oriented development community to co-opt it as an SOA modeling notation. Did UML never provide the rigor that was needed? I hope that BPM will retain its “business friendly” characteristics – or at least that the tools will NOT require programmer level expertise just to design the models.
I’m still catching up – trying to get my hands around the business value – so your contributions both here and at the BMN Institute have been invaluable.
Keep on educating!
Bruce, you address “Role based access control and authorization” in Section 10. I don’t kniow how far you intend to look at it, but I’d like to see discussions of a plugable Access Management interface, or the ability to Customize Access Management through a WebService or something like that. Access Management usually implies access to resources and entitlements, but that can leave questions about data access in limbo. What if two distinct companies or organization are using the system. How does the BPMS prevent data from one company from being viewed from another, even though they may have similar roles. (Eagely awaiting your 2007 report)
Bruce, good list.
Some area’s I miss…
An area on the professional services sides would be training. Training capabilities for a BPM-S vendor are critical, I think.
Another area is BPM Methodology. Does the vendor have some approach/model/best practices for BPM projects.
Some loose thoughts:
– Support for Work time Calender (e.g. when delaying tasks for X workdays, skip weekends and official holidays). Also very handy for Simulations!
– Integration with LDAP/User Profile Database
Regards,
Roeland Loggen