Monitoring & Orchestrating Your Microservices Landscape using Workflow Automation (Parts 6/7 of 7)

On Wednesday, March 11, 2020, I conducted the webinar titled “Monitoring & Orchestrating Your Microservices Landscape using Workflow Automation”. Not only was I overwhelmed by the number of attendees, but we also got a huge list of interesting questions before and especially during the webinar. Some of them were answered, but a lot of them were not. I want to answer all open questions in this series of seven blog posts. Today I am posting the final two in the series. Note that we also started to experiment with the and discuss to make this more frequent, so keep an eye to our community for more opportunities to ask anything (especially as in-person events are canceled for some time). Camunda’s question corner Part 1: (6 answers) BPMN & modeling related questions Part 2: (12) Architecture related questions Part 3: (6) Stack & technology questions Part 4: (5) Camunda product-related questions Part 5: (3) Camunda Optimize specific questions Part 6: Questions about best practices (5) Part 7: Questions around project layout, journey and value proposition (3) Questions about best practices Q: Business data versus workflow data: if you cannot tear them apart, how can you keep them consistent? Are the eventual/transactional consistency problems simpler or more complex with Camunda BPM in the equation? This is quite a complex question, as it depends on the exact architecture and technology you want to use. Example 1: You use Camunda embedded as a library, probably using the Spring Boot starter. In this case, your business data could live in the same database as the workflow context. In this case you can join one ACID transaction and everything will be strongly consistent. Example 2: You leverage Camunda Cloud and code your service in Node.JS, storing data in some database. Now you have no shared transaction. No you start living in the eventual consistent world, and need to rely on “at-least-once” semantics. This is not a problem per se, but at least requires some thinking about the situations that can arise. I should probably write an own piece about that, but I had used this picture in the past to explain the problem (and this might help also): very basic blog post So you can end up with money charged on the credit card, but the workflow not knowing about it. But in this case you leverage the retry capabilities and will be fine soon (=eventually). Q: Is it a good idea to save the process data into a single complex object with JSON notation? As always: It depends. Our . general recommendation is to have as little data as possible in a workflow, and if the data lives somewhere else, only store references But if the data cannot live anywhere else, serializing the data into the workflow is at least an option. And if you serialize it, JSON might also be a good idea. Please note, that you cannot query for data in that process variable any more (or do a text query at max). So it depends. The best practice: might help to judge that. Handling Data in Processes Q: How to handle the filtering of information allowed or not to be seen by a user of the process? I guess this refers to task lists. There are two layers to look at it. Is it important that the user is not able to get the data at all — not even by looking at the data transferred to his browser via JSON in the background? If yes, you need to work with data mappings to make sure only the variables are available in the task that should be readable and configure permissions accordingly. If no, the easiest option is to simply not show certain data in the forms for users. Q: How to do microservice versioning and workflow versioning and manage both in harmony and congruent? I love this question, but it depends. might give you a good starting point for the workflow angle of it. Versioning Process Definitions For the microservices angle of it there are tons of other material available discussing and overall versioning approaches. This is a rabbit hole I don’t want to follow here. tolerant readers Q: What are the best practices for handling version changes when dealing with a BPMN that changes key pieces of its workflow microservices? This seems to be similar to the last two questions. Questions around project layout, journey and value proposition Q: Where is the best place to start when moving from old legacy monolith workflow systems to Camunda so there is minimum disruption. It depends on so many things. The How to migrate to Camunda whitepaper Can give you some first guidance. Q: How to convince monolith customers with legacy to move to microservices? You need to find the current pain points and show how microservices can provide a cure. Companies need to understand the real benefits. You should not apply microservices just because it is hip. The major benefit is around business agility. Microservices are small, autonomous services that do one small thing very well — and then need to work together. The less they need to communicate to anyone else the more efficient they can work and the more agile they can be when any change is needed. My favorite : quote around that is from Jeff Bezos And the metaphor I always use is a three-legged race. If you tie together different teams, so that they need each other to change or deploy anything, you make them all slower. If you cut the bonds, every team can run faster. Of course, this leaves you with the challenge of how these services collaborate — that’s why you inevitably also will stumble over workflows :-) I searched a bit but did not find a recording where I go exactly over this storyline, probably The Role of Workflows in Microservices gives you at least a glimpse of how I think about that. Q: What are the advantages of using Camunda for microservices orchestration? Following up on the last answer I am convinced, that you need orchestration capabilities in your microservices architecture. I discussed this in this webinar, but also for example in . Complex event flows in distributed systems The second part of this question then is: Why to use Camunda instead of any other workflow tool. While I might be a bit biased on this, I am convinced that Camunda has the strongest offering in that space because of the following characteristics: Developer-friendly: No fluff & unrealistic low-code promises, but a great integration into the developers’ world, including tooling and procedures. Highly-scalable: Camunda can back small workflow applications as well as global, mission-critical core business processes. And we h :-) elp the Nasa to get to Mars BPMN/DMN standards-based: We are completely based on well-known and widely adopted standards, that are not only directly executable but also provide visual diagrams, that non-it folk can understand, which brings us to: Business-IT-collaborative. Wrap-Up Thanks for following along in this series of posts. We looked at a lot of questions. You are brave if you made it till here — kudos!