Launching Activiti v6 (Activiti Community Day Paris 2015)
- 1. Launching Activiti 6 June 10, Paris
- 2. Joram Barrez & Tijs Rademakers Activiti Leads, Alfresco
- 3. Defining moments of mankind (*) 3 (*) Random pick and highly subjec9ve
- 5. How did we get there?
- 6. Five years ago … • 1 March 2010 : Joram & Tom join Alfresco • 17 May 2010 : Alfresco launches Activiti 5.0.alpha1 • Huge interest from day 1 6
- 7. The past five years 7 Ac9vi9.org stats: • Displayed here: unique users Jan-‐May 2015 • Over 5 years: 900 000 unique visitors • Huge uptake beyond anything we had imagined • > 100 contributors • Across the whole globe, all kinds of scale in all sorts of industries
- 10. The past five years • The API still looks very much the same as five years ago • The concepts have been kept stable • The database schema evolved, but still looks very familiar • The coverage of BPMN 2.0 has expanded • The number of features/integrations has only grown • Our core goal has always been stability • Without comprimising in use cases • +30 releases on this foundation 10
- 11. In the past five years … • The codebase has grown seriously – By different people, with different focus/use cases – (of course, all code still has to pass the gatekeepers, i.e. us) • We have learnt a lot on how Activiti is used (and sometimes how it is misused/abused) – Feedback through various open source channels – Feedback from running it at Alfresco’s customers • Software architectures are changing – Lightweight, REST, container-less, reactive, … – With more and more data produced/processed 11
- 12. Five years ago … • We chose the right things: – Clean and easy to use API – Lightweight, embeddable engine – Standardized on BPMN 2.0 – Stateless scaling – Many pluggability points – Clear docs (and a lot) – Liberal Apache license – Have a UI that demonstrates the features • Today, all the above still applies (very much so!) 12
- 13. Five years is a long time in IT … 13
- 14. So are you saying Activiti 5 is … bad? • Activiti v6 is not a rewrite • Activiti v6 is not a new engine • Activiti v6 is an evolution of the core engine, taking in account all we have learned in the past five years and what we (*) believe that is needed to cope with the use cases of 2020 in the simplest and cleanest way. • After five years of we were really tired of that ‘5’ 14 (*) the Ac9vi9 community
- 15. “Making sure we are ready for the next five years” 15
- 16. Did Activiti v6 just drop out of the sky? 16 hOps://github.com/jbarrez/ac9vi9-‐neo4j July 2013 Nov 2013 Jan 2014 (commercial offering) Dec 2014 V6 works starts ‘officially’ On hOps://github.com/Ac9vi9/Ac9vi9/commits/experimental May 2015
- 17. Just because we’re really proud of this graph 17
- 18. Activiti v6 Core engine evolution design goals • Backwards compatibility with Activiti v5 – Crucial and non-negotiable – API, DB Schema, concepts, integrations, … • Simple(r) runtime structure and execution • Removal of bloated parts of PVM within the core engine – ActivityImpl, ExecutionImpl,. … – Direct execution of Bpmn Model – Straight algorithm for defining runtime structure • Dynamic process instance/definition modification support • Performance, performance, performance 18
- 20. V5 Problem 1 – Model Transformation 20 • BPMN 2.0 XML -> Internal model (PVM datastructures) -> PVM operations – ActivityImpl, ExecutionImpl, … – not 1-1 with BPMN concepts • Leakage between layers – Example (from the userguide):
- 21. V5 Problem 1 – Model Transformation 21 • V6 version: • (Sequence flow lookup can be simpler, just wanted to show the nuts and bolts)
- 22. V5 Problem 1 – Model Transformation 22 • Hard to reason about patterns in structure for runtime execution • The problem is not clear for simple to even complex process definitions • But once you hit the _really_ complex process definitions … – And yeah, that tends to happen – Think something like a nested use of multi instance parallel nested embedded subprocesses with multiple compensation events and handlers with a couple of boundary events. – Then you are happy if you can reason in BPMN to get out of that
- 23. 23
- 25. Execution tree • Stored relationally
- 26. V5 Problem 2 : Execution Tree • Cause – (over)optimization and reuse entities – focus => getting the number of DB inserts as low as possible • Consequence – Lots of if-elses for specific patterns – Hard to look at execution tree and deduce process definition • Which is crucial for dynamic process definition/instance (without hacking) 26
- 27. API Call (refresher) 27 Command
- 28. V5 Problem 3 • Every API call boils down to – DB changes through one or more EntityManagers – Manipulation of the runtime execution structure • Manipulation of internal model (see problem 2) • Feeding that model into AtomicOperations (PVM core engine) • Problem: atomic operations are started from ExecutionEntity object – New atomic operations are created to continue process – New atomic operations are put on a stack, bound to the ExecutionEntity – Multiple stacks when having multiple ExecutionEntities • Some patterns can’t be executed on v5 due to that • We tried to fix that, multiple times. Brave men have wept. 28
- 29. V5 Problem 4 • Logic is tightly coupled to DB layer • DB code is spread between EntityManagers and the Entity object itself – Kinda like we couldn’t decide between using DAO or Active Record pattern • Really (impossible) to change persistence layer in v5 29
- 30. Why a v6? • We could have tried to solve these problems on the v5 foundation • But it would be a hack. • We would get away with it for a while. But it would bite us back on the longer term • So we decided to solve these problems at once (they are all interconnected, really), let the foundation evolve. Which is a big enough change to call it v6. 30
- 32. Direct execution of BPMN Model V5 • BPMN 2.0 XML parsed to PVM representation • Step in process == ActivityBehavior • ActivityBehavior – interpretes execution tree through PVM model – manipulates execution tree structure – creates new AtomicOperations V6 • BPMN 2.0 XML parsed to BpmnModel, 1-1 java representation of XML • Step in process == ActivityBehavior • ActivityBehavior – interpretes execution tree through BPMN model – manipulates execution tree structure – puts new engine operation on agenda 32
- 33. Execution tree V5 • Optimize and reuse existing execution entities as much as possible • No clear guidelines when execution entity is created or reused – Many types and flags • Scoped • Concurrent • Concurrent root • Event scoped • … V6 • Execution entities are never reused • Have clear rules when a new execution is created – Process instance = execution. Is not used for activity execution – Scope execution = parent for other executions within subprocess. Is not used for activity execution 33
- 34. Centralized Agenda V5 • Multiple stacks of operations during API call execution • PVM representation is continued through Atomic Operations, executed in the context of an ExecutionEntity. V6 • One centralized agena per API call • Behaviours interprete BPMN structure and put new operations on the centralized agena. 34
- 35. Persistence Code Everywhere V5 • Persistence code is in EntityManagers, Entity and probably some other places V6 • All persistence code is moved to its EntityManager. • All EntityManager are interfaces (will be) 35
- 36. High Level Flow 36 Ac9vi9 v6 Engine API call Service Service Service Service Service CommandContext I n t e r c e p t o r s t a c k Agenda Opera9on Opera9on Opera9on Opera9on Opera9on Run loop En9tyManager & other internal classes Con9nueMul9InstanceOpera9on Con9nueProcessOpera9on DestroyScopeOpera9on EndExecu9onOpera9on ExecuteInac9veBehaviorsOpera9on TakeOutgoingSequenceFlowsOpera9on TriggerExecu9onOpera9on Add opera9ons to agenda Command
- 37. Benefits • In the ActivityBehaviors, we reason about BPMN structure and patterns – Easier to to understand – More powerful, as nothing is lost in translation • The execution tree structure is predictable and simple – Allows for more advanced behaviors – Hardest part in v5 was often the cleanup. This becomes child’s play now. – Leads to support for dynamic instances/definitions • Note: these are only the biggest changes in the v6 engine. • Source is at https://github.com/Activiti/Activiti/tree/activiti6 37
- 38. 38 Demo
- 39. A few examples of why the evolved engine is better
- 40. Loops and Tail Recursion • V5 à StackOverFlowException • V6 à All good 40
- 41. 41 Demo
- 42. Tricky Inclusive Gateway • A and B completed. C completes going to E. • V5 à process instance never completes • V6 à All good 42
- 43. Concurrency after boundary event • Timer fires – V5 : 1 task – V6: 2 tasks (good) 43
- 45. Backwards Compatibility • API • DB Schema • Process Definitions • Java Delegates • Integration with other frameworks (Spring, Camel, …) • Embeddability • Concepts 45
- 46. What is not backwards compatible? • Minor stuff that is quickly migrated: – Method signature changed for JavaDelegate/ActivityBehaviour • ‘throws Exception’ is gone – ActivityBehaviours that use PVM datastructures/methods – BpmnParseHandlers that act upon PVM ActivityImpl – Execution queries that depend on a certain number of executions – Assuming that an execution == process instance for simple processes • Expect a Migration Guide soon 46
- 47. Backwards Compatibility – How? • The upgrade of 5.x.x à 6.0.0 is like a regular update • Drop jar, auto upgrade db (or execute DDL changes) • BUT: all existing process definitions / deployments are tagged as ‘v5’ 47
- 48. Backwards Compatibility – How? 48 DB Ac9vi9 v6 Engine run9meService.startProcessInstanceByKey(“”) v5 Mini v5 engine V5 process instance Same applies for every point where A process instance can be con9nued: • Task complete • Signal • Event triggered • Job executed • … Shared resources!
- 49. Migration of process definition to v6 • Backwards compatibility is enabled by having ‘activiti5-compatibility’ dependency on classpath • Old process instances keep running against v5 ‘mini-engine’ – Runtime execution is different from v6 • Running process definition in v6 is – Migrating your custom logic – Deploying a new version of the process definition 49
- 50. 50 Demo
- 51. Roadmap and wild ideas
- 52. • All of the next slides contain – Real roadmap items – Well worked out ideas – Half baked ideas – Dreams • We have a roadmap discussion after this session! 52
- 53. Bulk Insert/Delete • Lower traffic between engine and database • Work already good under way – Bulk insert PR – Execution bulk delete in v6 • Includes reordering for correct order • But needs performance tests to validate • Avoid delete queries if no data exists – Eg execution à related tasks, jobs, variables, … 53
- 54. BPMN Support • Ad-hoc subprocess • Event subprocess – More event types – Non-interrupting • Java-based gateway • Message intermediate throw event + better pool support • Event-based gateway parallel semantics • Escalation 54
- 55. Dynamic process definitions/instances • Support for dynamic features – Tasks, processes related to process instance – Re-route a process instance in a custom way – Store the result as a new process definition • Algorithms are most likely applicable to process definition migration logic 55
- 56. Variables • New variable scopes – Global / Local (v5 semantics) – Scoped (within subprocess) • Variable mapper – Multi instance finished – Call activity finished • System variables • Transient variables 56
- 57. Execution tree fetching • Fetch the execution tree when only one execution is needed • Subsequent execution queries don’t hit the DB • Cache the trees beyond the API call context? – Solve cluster cache problem? • Always or configurable? – Need performance metrics 57
- 58. History • Async history – Performance ++ – What about transactions? 58
- 59. Runtime Swapping • Swap out behaviours, listeners, expressions at runtime • With specific conditions – Allow to patch in flight process instance by overriding behaviour 59
- 60. More hook points • Pre/post transaction commit • Pre/post execution listener/behaviour/delegate • Pre/post process first parse/ subsequent cache load • … 60
- 61. Delegates, ActivityBehaviours and Listener scope • Allow for – singletons (process engine scope - v5 semantics) – Stateful instances process definition scope – Stateful instances process instance scope 61
- 62. Debug Mode • Collecting data during runtime execution – Agenda operations – Execution tree changes – % of time spent in DB vs Logic 62
- 63. Simplify Unit Testing • Currently, writing a unit test – Create process definition xml – Put it on classpath, link it to unit test (@Deployment or RepositoryService) – Write unit tests moving process instance • Ideas – Meta notation (annotation/methods/…) putting process in certain state – Generating a unit test skeleton from the Modeler 63
- 64. Real Concurrency • The new foundation should allow for real parallelism • Haven’t tried it yet • Will lead to a lot of fun with transactions 64
- 65. Docs • Now: tech guide • Future – Work more from use cases and examples – Tijs will write a new book? – Living book? 65
- 66. Other • Community Driven • Bring it on 66