Mule 4 migration + Common Integration Challenges : MuleSoft Virtual Muleys Meetups

All contents © MuleSoft, LLC
Mule 4 Migration
Planning
Angel Alberici
Anu Vijayamohan
Integration
Challenges

MuleSoft Meetup:
Virtual Muleys (Online Group - English )
Moderator: Angel Alberici
September Edition, 2021
Recording

MuleSoft Meetups: Virtual Muleys
8

Find all Virtual Muleys recordings:
https://mule.is/MQRdF
This is not an Ad!

Announcements & Updates

Show the value of Integration to Business, track usage of Anypoint through Time, learn where you are to
decide where you want to go! (Open source)
10 Minutes:
https://www.youtube.com/watch?v=sG9ae
AulooI
13
Learn Metrics Toolkit (Accelerator/Framework) NOW!!!
10 Minutes:
https://www.youtube.com/watch?v=OstxT-_
vTt0

https://help.mulesoft.com/s/ideas#0872T000000Lrk8QAC
https://github.com/mulesoft-catalyst/metrics-toolkit
14
IDEA: Out of the box Metrics Accelerator/Framework/Toolkit

⅔ were not self serve and for some, very complex to set up
Increase Existing Connectivity Options
VPN VPC Peering
Direct Connect
Self Serve

Self-serve
● Setup network
connections to customer
environments without
ﬁling tickets with support
Simpliﬁed connectivity
● With one connection,
customer is able to provide
access to their on-prem
and Cloud resources
Scale more quickly
● AWS recommended
approach for handling
network connectivity at
scale
AWS Transit Gateway support
Released 13th Sep, 2021

An Extension and NOT a deprecation
Transit VPC vs Transit Gateway
● Allows for hub-spoke design for
inter-VPC Connectivity
● Overlay VPN Based transitive routing
● Requires 3rd party device
● Higher cost/Lower throughput
● Allows for hub-spoke design for inter-VPC
Connectivity
● AWS Managed HA/Scalability
● No reliance on EC2 or VPN overlay
● Higher throughput (burst up to 50 gbps
per AZ)

Transit Gateway main resources
● https://docs.mulesoft.com/runtime-manager/tgw-about
● https://docs.mulesoft.com/runtime-manager/tgw-attach-arm
● https://docs.mulesoft.com/runtime-manager/tgw-attach-arm
● https://docs.mulesoft.com/runtime-manager/tgw-troubleshoot

https://mulesoft-labs.dev/
19
NEW: MuleSoft Portal with GREAT tutorials to help you!
Including Transit Gateway Steps

Book your agenda here
MuleSoft focusing on best-in-class integration, API management,
and RPA capabilities with hyperscale resilience, universal
governance, end-to-end security, and reuse-at-scale to make the
composable business possible:
● MuleSoft and Slack embody the composable enterprise.
Empowers IT teams to deliver composable services, API
products, and bots at scale for the entire organization.
● MuleSoft Composer empowers business teams to
automate integrations to common systems without code, by
leveraging IT’s reusable assets, support, and governance.
● Servicetrace’s RPA solution will complement Composer by
automating repetitive and manual tasks with reusable,
personalized bots, allowing individuals to achieve
end-to-end automation.
Dreamforce 2021
20
Where is MuleSoft?

Mule 4 Migration
Planning
20th September
18:00 Europe CEST | 17:00 UK | 21:30 Delhi, India
9:00 San Francisco | 12:00 New York
Angel Alberici
Anu Vijayamohan
Integration
Challenges

Anu Vijayamohan
Customer Success Technical Architect, EMEA
Mule 4 Migration Planning
MuleSoft Meetup: 20 September 2021

Agenda
● Mule 4 Beneﬁts
● What are the Implications of Not Migrating?
● Where and How do I start?
● Migration Planning & Decision Guides
● Mule Migration Assistant
● Enablement & Resources
● Questions?

Mule 4 Beneﬁts

What Mule 4 means for your business
25
Unlock data quickly
Access data for parallel
consumption with better space
and size constraints
Run apps anywhere, better
Modernize application
deployments with power and
ease
Deliver integrations faster
Reuse functions, ﬂows, and
components to develop in a
standardized fashion
Simplicity
Power
Speed
3
1 2

Why move from Mule 3 to Mule 4
26
Accelerating digital transformation
Highly performant
A more powerful runtime
engine and signiﬁcant
performance improvements
New innovations
A host of new features to build
industry-leading APIs and
integrations
Future-proof
Latest in product
enhancement and
innovations and simpliﬁed
self-service upgrades

Integration development
Connect to legacy and modern systems easily
API development
Collaborate on APIs with all stakeholders
Runtime
Scale and process applications with speed
● More resilient and scalable non
blocking runtime
● Enhanced Streaming support
● Seamless upgrades of the Mule
Runtime and Connectors
● Design industry-standard APIs
with OAS or RAML
● Implement and optimize APIs
to run anywhere
● Facilitate critical integration
domains like connectivity,
transformation, validation, error
handling, and security
● Build tests automatically
Core
EXTENSIONS API
SA
P
LDAP
HTTP
MULE API
New with Mule 4
New and improved features

New and improved: Runtime
Enhanced streaming support Seamless upgrades
Mule 4 is the deploy-anywhere
runtime

Build integrations quickly
New and improved: integration development
MUnit test recorder
Auto-generate unit and integration tests
DataWeave
Transform data quickly

New and improved: API development
Build APIs with industry-adopted API
standards
Synchronize and coordinate
API delivery better
Design APIs for scale

Connect to modern apps and
legacy systems fast
New and improved: connectivity
Validate system ingestion
Test connections and conﬁrm payloads faster
Extend SaaS connectivity
Trigger connectivity to a wide-range of systems
See the whole list of Mule 4 connectors available here

What are the Implications of Not
Migrating?

33
Delaying often makes
migration more diﬃcult and
expensive
● More apps to migrate
● Increase maintenance costs
Increasing Technical Debt
The costs of delaying migration
As all organizations move to Mule
4, the ecosystem moves in
tandem:
● Available Developers, Partners
● Available Documentation
Weakening Ecosystem
Unable to leverage the latest
runtime and platform beneﬁts
● New features
● Performance enhancements
● Longer Support Timelines
Stability, Features, Support

Runtime Support & Timelines
34
Last updated September 1, 2020 (link)
Version Release Date End of Standard Support End of Extended Support
4.3 April 30, 2020 April 30, 2022 April 30, 2024
4.2 May 2, 2019 May 2, 2021 May 2, 2023
4.1 March 20, 2018 November 2, 2020 November 2, 2022
3.9
long term supported
October 9, 2017 March 20, 2021 March 20, 2024
3.8
long term supported
May 16, 2016 November 16, 2018 November 16, 2021
3.7 July 9, 2015 November 16, 2017 January 25, 2020
3.6 January 15, 2015 January 15, 2017 N/A
3.5
long term supported
May 20, 2014 July 15, 2016 * January 25, 2020 *

Support deﬁnitions
End of Extended Support
Typically, end of extended support means
that from this point onwards, no support
from the MuleSoft infrastructure or
organization will happen for applications
deployed on that version of the runtime.
- CloudHub apps will be stopped.
CloudHub apps will not be patched
- Apps will not be restarted if shut down
- Support will not debug or investigate
- Services will be asked to deﬂect
investigations on deprecated versions
What does this mean
From November 2021 onwards, customers
cannot deploy or run Mule 3.8.x apps on
CloudHub or Runtime Fabric
3.8

End of Standard Support
Typically, end of standard support means
that from this point onwards, no net new
applications can be deployed on that
version of the runtime. This applies to
CloudHub and Runtime Fabric only.
What does this mean?
From 20 March 2021 onwards, customers
will be unable to deploy net new Mule 3.9
applications to CloudHub and Runtime
Fabric
HOWEVER
Between March 2021 and November 2021
Customers can deploy new applications in
3.9.5 and 3.9.4
3.9

Extended Support
Extended Support includes:
● Technical support on and around the software for
production environments, including
troubleshooting, diagnosis and resolution of
issues which do not require source code
patches.
● MuleSoft will attempt to provide patches for
customers upon request, and where technically
possible, for the following: S1 issues; Critical
Security Vulnerabilities; S2 issues where the
product in a non-production environment is
nearing a critical milestone, is unusable, and no
workaround is available.
● Access to software downloads, technical content
and knowledge base articles.
3.9
What does this mean
From March 2021 until March 2024,
Mule 3.9 will be in extended support

EOS Reminder in Support Portal
You can check the end of support date of the Runtime you are using via
the Support Portal. For more information see our Product Support and
Maintenance Terms
38

All contents © MuleSoft, LLC 39
What happens to Sandbox & Production apps on Mule 3.8 & 3.9?
CloudHub
TYPE DATE SANDBOX/PRODUCTION
Mule 3.9
EoSS
March 20th,
2021
● Can customers restart apps after Nov 16th, 2021?: Yes.
● Can customers deploy new apps after Nov 16th, 2021?: No.
● Can customers deploy updated versions of existing apps on 3.9 runtimes after Nov 16th,
2021?: Yes (in place update ONLY - do not delete and new deploy).
● Can customers update an app using a 3.9 version (ie: 3.9.2) to the latest available 3.9
patch (ie: 3.9.2 to 3.9.5) after Nov 16th, 2021? No.
● Will MuleSoft interrupt and stop running apps after Nov 16th, 2021 and during Extended
Support?: No.
Mule 3.8
EOL
Nov 16th,
2021
● Can customers restart apps after Nov 16th, 2021?: No.
2021?: No.
● Will MuleSoft interrupt and stop running apps after Nov 16th, 2021?: Yes (No new starts
beyond this date).

What happens to Sandbox & Production apps on Mule 3.8 & 3.9?
Runtime Fabric
Mule 3.9
EoSS
March 20th,
2021
● Can customers deploy new apps after Nov 16th, 2021?: Yes.
2021?: Yes.
● Can customers update an app using a 3.9 version (ie: 3.9.2) to the latest available 3.9 patch
(ie: 3.9.1 to 3.9.5) after Nov 16th, 2021? Yes.
Support?: No.
Mule 3.8
EOL
Nov 16th,
2021
● Can customers restart apps after Nov 16th, 2021?: No.
2021?: No.
Support?: Yes (No new starts beyond this date).

What happens to Sandbox & Production apps on Mule 3.9?
MuleSoft-managed customer-hosted runtimes
registered through Anypoint Runtime Manager
Mule 3.9
EoSS
March
20th,
2021
● Can customers deploy updated versions of existing apps on 3.9 runtimes after Nov 16th, 2021?: Yes.
● Can customers update an app using a 3.9 version (ie: 3.9.2) to the latest available 3.9 patch (ie: 3.9.1
to 3.9.5) after Nov 16th, 2021? Yes.
● Will MuleSoft interrupt and stop running apps after Nov 16th, 2021 and during Extended Support?:
No.
● Will Mule Runtimes keep running after Nov 16th, 2021?: Yes.
● Can customers create and manage Servers after Nov 16th, 2021? Yes.

MuleSoft-managed customer-hosted runtimes
registered through Anypoint Runtime Manager
Mule 3.8
End of Life
Support
Nov
16th,
2022
Registered Mule 3.8 Runtime Servers can continue to run locally on the environments they are deployed post End of LifeSupport under
customer’s responsibility. Registered Mule 3.8 Runtime servers will be automatically deregistered, losing the ability to manage Mule 3.8
Runtimes on Anypoint Platform via Runtime Manager after Nov 16th, 2022.
● Can customers restart apps after Nov 16th, 2022?: Yes* (locally restart the Mule Runtime server, however it is not possible to restart
servers via Runtime Manager).
● Can customers deploy new apps after Nov 16th, 2022?: Yes* (locally, however it is not possible to leverage Runtime Manager).
● Can customers deploy updated versions of existing apps on 3.8 runtimes after Nov 16th, 2022?: Yes* (locally, however it is not possible
to leverage Runtime Manager).
● Will MuleSoft interrupt and stop running apps after Nov 16th, 2022?: No.
● Will Mule Runtimes keep running after Nov 16th, 2022?: Yes* (Mule Runtimes will keep running locally on the environment they were
deployed to under customer’s responsibility without support).
● Can customers create and manage Servers after Nov 16th, 2022? No. (after the End of Life Support, Runtime Servers will be
automatically deregistered).
● Can customers continue to connect with API Manager after Nov 16th, 2022? No.
● Can customers continue to monitor via Anypoint Monitoring after Nov 16th, 2022? No.
* For MuleSoft-managed customer-hosted runtimes registered through Runtime Manager:
Mule 3.8 End of Life Support will be provided for an additional 12 months after EOL (until November 16th, 2022) for On Premise registered Mule 3.8 Runtimes that customers decide to
run and manage on their own past the 3.8 EOL deadline. Registered 3.8 Runtime Servers will be automatically deregistered after reaching End of Life Support. On Premise registered
Mule 3.8 Runtimes will continue to connect to Anypoint API Management and Anypoint Monitoring until the End of Life Support period ends on November 16th, 2022.
End of Life Support will be limited to technical assistance ONLY during this period. Source code patches or new patch level releases will not be provided. If there is an issue that requires
a patch, you will need to upgrade to a newer version covered by the Standard or Extended Support windows. For additional details, please see Product Versioning and Back Support
Policy. Customers can continue to operate their Mule 3.8 Runtimes without support beyond End of Life Support’s deadline at their own responsibility.

Customer-managed customer-hosted runtimes
(standalone deployments)
Mule 3.9
EOSS
March 20th,
2021
● Can customers deploy updated versions of existing apps on 3.9 runtimes after Nov
16th, 2021?: Yes.
● Can customers update an app using a 3.9 version (ie: 3.9.2) to the latest available
3.9 patch (ie: 3.9.1 to 3.9.5) after Nov 16th, 2021? Yes.
● Will MuleSoft interrupt and stop running apps during Extended Support after Nov
16th, 2021?: No.

Customer-managed customer-hosted runtimes
(standalone deployments)
Mule 3.8
End of
Life
Support
Nov 16th,
2022
● Can customers restart apps after Nov 16th, 2022?: Yes.*
● Can customers deploy new apps after Nov 16th, 2022?: Yes.*
● Can customers deploy updated versions of existing apps on 3.8 runtimes after Nov
16th, 2022?: Yes.
● Will MuleSoft interrupt and stop running apps after Nov 16th, 2022?: No.
● Can customers continue to connect with API Manager after Nov 16th, 2022? No.
● Can customers continue to monitor via Anypoint Monitoring after Nov 16th, 2022?
No.
* For Customer-managed customer-hosted runtimes (f.k.a on-premises or standalone deployments):
Mule 3.8 End of Life Support will be provided for an additional 12 months after EOL (until November 16th, 2022) for On Premise unregistered
Mule 3.8 Runtimes that customers decide to run and manage on their own past the 3.8 EOL deadline. On Premise unregistered Mule 3.8
Runtimes will continue to connect to Anypoint API Management and Anypoint Monitoring until the End of Life Support
period ends on November 16th, 2022.
End of Life Support will be limited to technical assistance ONLY during this period. Source code patches or new patch level releases will not be
provided. If there is an issue that requires a patch, you will need to upgrade to a newer version covered by the Standard or Extended Support
windows. For additional details, please see Product Versioning and Back Support Policy. Customers can continue to operate their Mule 3.8
Runtimes without support beyond End of Life Support’s deadline at their own responsibility.

Where and How Do I Start?

● Go API-LED: This is an opportunity to build a API-LED
architecture ..
● Go Event Driven : This is an opportunity to true event driven
architecture ..
● Consider taking advantage of new Salesforce connector for
○ Streaming APIs
○ Platform Events
○ Triggers
How to start?
It is an opportunity to re-architect….
Take advantages of the latest integration trends and adopt a few ...
46

1. Uniform Error Handling Framework:
Mule 4 error handling is signiﬁcantly more enriched with try scope also every
component has list of possible errors.
2. Common JSON Logging Framework:
Output logger events as JSON. Use JSON layout instead of pattern layout adding
additional ﬁelds to the JSON output, via a list of KeyValuePair elements
3. Build and Use Common Reusable Templates
Most organizations have dominant repeatable integration patterns that can take
advantage of common reusable templates. Identify a few for your org, and build
them.
4. Build RAML Fragments and Reuse Them
Don't Jump The Gun..
1
2 3
This is also a great opportunity to re-align on a
better way to code and build apps...
47

Migration Planning & Decision Guides

Migration Plan for Projects
49
Now that you have taken a stock of your Mule 3 projects …
Classify the Mule 3 applications inventory into 3 categories
Simple Applications
Simple Apps in Mule 3 that can
be converted to Mule 4 easily
● No Custom
components.
● Simple Dataweave
transformations
● No Connector
incompatibility
Medium complexity
Applications with scope of
re-architecture and scope to
incorporate new features
● Medium to high usage
of MEL expression
● Limited usage of batch
scope , record vars and
watermarks
High complexity
Very complex application with
multiple ﬂows and connectors
and /or custom code.
● Heavy usage of
Dataweave .
● Usage of custom
object stores

Build
Foundation
Mule 4 Migration Decision Guide
50
Life-span < 3 yrs
Complexity
Imminent Biz Req
Biz Critical
Quick Wins/
Learning
Build
Momentum
Go to 3.9,
Avoid mig $$
Re-architect,
Rewrite
Scale Up
migration
Scale Up
migration
Tackle w/
expertise
3.8/3.9 apps
Proxies New 4.x apps
Common/
Custom
Components
1
L M H L M H
2 3 4 5 6 7 8

Mule Migration Assistant

Evaluate migration complexity
Generate an initial assessment of the manual
work required for migrating individual Mule
apps
Reduce manual eﬀort
Avoid rewriting from scratch by converting the
project structure and common connectors
(HTTP, DB, etc) into Mule 4 compatible
components
Customize with extensibility
Users can develop plugins on the open source
framework for repetitive migration tasks such
as custom connectors and complex MEL
Open
Source
Mule Migration Assistant (MMA)
Jumpstart Mule 3 to Mule 4 migrations

What Does MMA Translate?
Mule Migration Assistant will translate:
● Flows and their inner logic.
● Global Conﬁgurations
● Global Elements
● DataWeave 1 Scripts to DataWeave 2 Scripts
● MEL Scripts to DataWeave 2 Scripts
● MUnit Tests
● API Gateway Policies
● Connectors, Transports and Modules*
* Not all connectors/modules/transport are currently supported.

Contribution to the project
Where?
● https://github.com/mulesoft/mule-migration-assistant
Who?
● We welcome customers, partners and the community in general to make pull requests
and contribute to the project.
How?
● The MMA project in GitHub will have specific contribution rules and guidelines for those
contributors that want to submit fixes, enhancements and extensions
○ Very similar to those of Mule Community Edition
● There will a be clearly specified review process in the GitHub MMA project contribution
guidelines following a “Pull Request” mechanism and complying with a set of
pre-conditions to be merged.
54

Enablement & Resources

What we need to prepare?
● Training
● Reviewing of Documentation
● Review the Mule Migration Assistant (MMA)
● Build out new framework assets
○ Templates
○ Loggers
○ Policies
● Update your CI/CD pipeline
● Educate your internal organization
56
Enablement and readiness plans should include:

Training Courses for Mule 4
57
● Instructor-Led DELTA Course: Anypoint Platform Development: Mule 4 for Mule 3 Users (3 days)
This instructor-led course is for experienced Mule 3 developers who want to quickly uplevel their skills to Mule 4. Includes
voucher for MCD - Level 1 (Mule 4) exam or the MCD - Level 1 (Mule 4) DELTA exam. Datasheet.
● Instructor-Led FUNDAMENTALS Course: Anypoint Platform Development: Fundamentals (Mule 4) (5 days)
This instructor-led course is for developers and architects who want to get hands-on experience using Anypoint Platform to
build APIs and integrations. Includes voucher for MCD - Level 1 (Mule 4) exam. Datasheet.
● Mulesoft.U Self-Service/On-demand Courses:
○ MuleSoft.U Development Fundamentals for Mule 4
○ MuleSoft.U Getting Started with Anypoint Platform for Mule 4
○ MuleSoft.U Mule 4 for Mule 3 Users
● Other Courses Updated for Mule 4:
○ Anypoint Platform Development: DataWeave 2.0 (Mule 4)
○ Anypoint Platform Operations: Customer-Hosted Runtimes
○ Anypoint Platform Operations: API Management
○ Anypoint Platform Architecture: Application Networks
○ Anypoint Platform Architecture: Solution Design

Documentation
58
● Mule 4 User Guide: https://docs.mulesoft.com/mule4-user-guide/v/4.1/
● Mule 4 for Mule 3 Users: https://docs.mulesoft.com/mule4-user-guide/v/4.1/index-migration
● What's new in Mule 4: https://docs.mulesoft.com/mule4-user-guide/v/4.1/mule-runtime-updates
● Intro to Mule 4.1: https://docs.mulesoft.com/mule4-user-guide/v/4.1/intro-overview
● About Migration Process: https://docs.mulesoft.com/mule4-user-guide/v/4.1/migration-process
● Preparing for Migration to Mule 4: https://docs.mulesoft.com/mule4-user-guide/v/4.1/migration-prep
● Mule 4 Connectors: https://docs.mulesoft.com/mule4-user-guide/v/4.1/intro-connectors
● Intro to Studio 7: https://docs.mulesoft.com/mule4-user-guide/v/4.1/intro-studio
● Release Notes: https://docs.mulesoft.com/release-notes/mule-runtime/mule-esb
● Mule 4 performance benchmark report
● MMA GitHub repository: Migration Framework Tool, Documentation
● Migration Cheat Sheet https://docs.mulesoft.com/mule-runtime/4.3/migration-cheat-sheet

Resources
59
Webinars:
● https://www.mulesoft.com/demo/beta/mule-4 - This webinar series is available on demand. There are 3 webinars that
everyone is highly recommended to watch.
Friends of Max Videos:
● MMA Overview: https://www.youtube.com/watch?v=QxDNKaHttiE
● MMA Demonstration: https://www.youtube.com/watch?v=OsZ6QpovNdg
CONNECT 2018: Mastering Mule 4 & Studio 7
● https://videos.mulesoft.com/watch/gtWiNRFapPb9MAS4bxfevy

Mule 4 Documentation
60
Build
● Mule 4 User Guide
● Mule 4 Updates (What’s new in Mule 4)
● Mule 4 App Dev
● Mule 4 for Mule 3 Users
● Advanced Mule 4 Concepts
● Dataweave Language (DW 2.0+ for Mule 4.3)
● Mule 4 Connectors
● Intro to Studio 7
● Mule SDK
○ Module structure - Read this before you build/migrate framework components or customer connectors
● Mule Runtime Release Notes
Test
● Testing Mule 4 Apps
● Test Recorder
● MUnit 2.3
● Proﬁling Mule Apps
Deploy:
● Deploying Mule 4 Apps
● Maven Support in Mule
*Every Architect and Developer MUST start here

Important Mule 4 Blogs
● Studio 7 Beta Announcements
● Mule 4 Beta Announcement
● EA announcement
● GA Announcement
● Mule 4 Kit
● 10 ways Mule 4 will make your life easier
● Announcing DataWeave
● RC Announcement
● FTP connector
● Batch Module
● Database Connector
● File Connector
● ObjectStore connector
● DataWeave Syntax Changes Pt 1
● DataWeave Syntax Changes Pt 2
● Messaging Structure Pt 2
● Messaging Structure Pt 1
● Error Handling
● Automatic Streaming
● JSON logging in Mule 4: Getting the most out of
your logs
● How to build a streaming application in Mule 4
● Migrating DevKit projects to the new Mule SDK
Every Architect and Developer MUST read these!
++ a lot more Mule 4 blogs on blogs.mulesoft.com 61

Important Mule 4 Blogs
How-to:
● Scale your APIs with Mule 4
● JSON logging in Mule 4: Getting the most out of
your logs
● How to build a streaming application in Mule 4
● Mule 4 Error Handling Demystiﬁed
● Mule 4 Error Handling Deep Dive
● Use-case speciﬁc Error Handling
● MUnit testing made easy: Introducing the test
recorder
Recommended that architects and developers read these
++ a lot more Mule 4 blogs on blogs.mulesoft.com
Advanced:
● Messaging Structure Pt 1 & Pt 2
● Error Handling
● Automatic Streaming
● Mule 4 Logging
● Reactive programming in Mule 4
● Scalability Features in Mule 4 SDK
● Thread management and Auto-tuning in Mule 4
● Improved Thread modelling in Mule 4.3

Thank you!

10 Common Integration Challenges
Angel Alberici - MuleSoft Customer Success Architect

About the Speaker
65
Angel Alberici, Customer Success Architect at MuleSoft
● Born in Venezuela (not in the
Angel Falls)
● 10+ years in the IT ﬁeld
● Mostly development (full
stack), Delivery, Consultancy
and Customer Success
● IBM, OpenBravo, Cleardrop,
Addison Lee, Ricston, MuleSoft
● Started with Mule on 3.7.3 and
MMC (2016 can anyone
remember?)

What we are (not) going to see today
1. Versioning: The latest is the greatest, always... is it? Do not skip regression
testing! (3.7.3...3.9.5, 4.0, 4.1, 4.2, 4.3)
2. MuleSoft Training plan (yes, those 5+ days are worth the eﬀort)
3. Automation: Everything should be fully automated including tests and at least
80% tested code coverage for every project
4. MuleSoft Recommended Support Process: basic operational practice that
MuleSoft recommends customers follow when troubleshooting an
operational incident
5. Advanced troubleshooting of Application Performance : 90 minutes of Mule 4
on top of JVM (memory distribution, execution engine, dumps) and getting
applications ready (proactive troubleshooting)
66
But.. we know
this already!

1. “Case Studies” #1 to #3
2. Brieﬂy #4 to #6
3. Special Mentions (#7 to #10)
4. Closing words
Common Integration Challenges

Case Study #1: About
Performance

Situation
69
1. It is a normal day close to lunch time, you are working on a couple of tasks you need to ﬁnish
soon, but, you are suddenly pulled into a Production Outage thread, you have to drop
everything else you were doing, right now, and you have less than 3 hours to prepare for the
session for the Customer. It is not clear what is going on there
2. You had read through their case, there is not much information really; the case doesn’t have
anything attached either, there are some clues about APILed + SAP backend
3. From the Support case, it seems like a fairly new Customer that did not have a proper
onboarding with a Partner/Services and is not aware of the MuleSoft Recommended Support
Process
4. The issue has been going for a while now, 2 weeks and it haven’t been solved. Is this really
Critical? 🚩
5. You read the email chain 🚩: If we talk about architecture, @PersonA should be your contact. If
you want to analyzed the code, we should include @Partner1 in the loop, If you want to analyze
the SAP BAPI called through Mulesoft we should include @PersonB and if you want to review
the impact on the SAP resources or SAP behavior, @PersonC would take the lead.

Action
70
1. You oﬀer to schedule the meeting in 3 hours
2. You immediately create a diagram to validate
you understand the issue and you also share
two resources to be reviewed:
a. Guidelines on how to troubleshoot through Anypoint Platform:
https://help.mulesoft.com/s/article/MuleSoft-Recommended-S
upport-Process
Specifically please find there the topics on:
12. Problems in CloudHub
19. Problems with SAP
21. Problems with Connectors
b. A reference guide around general practices for performance
tuning (e.g. in case of threading pool needing to be tuned)
https://www.mulesoft.com/ty/wp/performance-tuning-guide-m
ule-3-8

Action
71
1. After a while you managed to
put a full list of questions that
need to be asked through the
call. You send them by email to
speed up the process
2. You are ready

Action
72

Progress
73
1. The call does not happens 🚩. PersonA and PersonB are on PTO.
2. It is postponed for one week. Was it really urgent? Intermittent on
Mondays
3. Diﬀerent people start replying to the email chain with answers for
questions on ‘their scope’
4. You get on a war-room with 15+ Customer employees. Conversation deviates a lot
from the main points, there is a lot of interaction between the Customer internal
teams.

Progress
74
1. There is an ongoing issue every Monday between 9:00-14:00 with unresponsive APIs.
2. There should a MuleSoft SystemAPI that makes requests to an SAP backend system
3. This SAP backend system is whether taking too long to process the request and provide responses OR does not
have enough capacity to handle the amount of concurrent connections/throughput, this is a known issue in the
Company with the SAP backend expected to be drastically improved in the future (6 months). The two potential
causes which were mentioned are:
a. A. The Client (Front-End app, MuleSoft APIs) not creating and handling the connection properly (zombie
calls, etc.). Not conclusive proofs were shared
b. B. The backend system is not equipped to manage the number of requests that are being sent,
consequently becoming unresponsive.
4. Lot of emails exchanges, meetings, new Support cases, etc….

Retrospective
75
Issue were solved by:
● Diagnose through dumps (Thread/Head)
● Performance Tuning, speciﬁcally Threading strategy + No. of threads (Mule 3)
● Implementing Cache. There was nothing before
● Improvements to their backend system (biggest bottleneck)
● Lack of Monitoring knowledge (e.g. when do workers automatically restart, etc.)
● Re-conﬁgure Timeouts in the Platform (e.g. System APIs)

● MuleSoft Recommended Support Process
● Considering Mule 4.3 this is the ﬁrst Whitepaper you should have read
(https://www.mulesoft.com/lp/reports/mule4-runtime-engine-performance)
● From the documentation there are some of the basic best practices
(https://docs.mulesoft.com/mule-runtime/4.3/tuning-recommendations)
○ Repeatable vs Non-repeatable Streaming
○ Back-Pressure and MaxConcurrency
○ Backend Server Response Time
○ Caching
○ Pooling Proﬁles
○ Domains
○ Logging
○ Batch Processing
○ Application Design Best Practices
Recommended resources
76

Key recommendations
77
1. Testing:
a. Unit Testing Best Practices documentation
b. General test plan template

Case Study #2: Surge day

Situation
79
1. After months of planning and dozens of sign oﬀ for budget, your Company is ready to roll
the most demanding Marketing Project it has done in the last 5 years. You are expecting
surges of 700% due to the powerful and aggressive advertising campaign the company is
leading
2. Your team has done the Performance testing in before and considered it ready
3. 🚩 BUT, two weeks before the rollout, one of the Senior Ops is getting alerts on the Test
environment about requests that are failing while doing pre-release routine of
stress/endurance tests
4. After some investigation you are able to ﬁnd you were a getting “429 Too Many Requests
HTTP response code”
5. Release is happening in 2 weeks. What is happening? What do we do? Where is it
happening?
6. Any Ideas?

Recap of a Shared Load Balancer
80
“Additionally, shared load balancers have lower rate limits that help ensure platform stability. MuleSoft
regularly monitors and scales these limits as necessary. Rate limits on shared load balancers are applied
according to region. If you are deploying an application to workers in multiple regions, the rate limit for
each region might be diﬀerent. If an application exceeds the rate limit for a shared load balancer, the
load balancer returns a 429 Service Unavailable response”
(https://docs.mulesoft.com/runtime-manager/dedicated-load-balancer-tutorial#shared-load-balancers)
“ There is no way to increase the limit that you will have over another customer; you can try a restart to
discard the worker, but if the load is on the Shared Load balancer that is the issue, it is not on the
worker. If you are seeing a 429, then you have exceeded the current rate limit“ (Support Advice)
Let’s consider the above:
- SLB performance varies in time (there is no guarantee), it also varies per region and it is used
for lower rate limits
- Shared Load Balancer Limits documentation (Numbers are not available)

As per the documentation:
● This enables you to host your applications under a single domain.
● Handle load balancing among the different CloudHub workers that run
your application.
● Define SSL configurations to provide custom certificates and optionally
enforce two-way SSL client authentication.
● Configure proxy rules that map your applications to custom domains.
Dedicated Load Balancers (DLB)
81

82
Additionally (Words of advice not a written rule):
● Consider DLBs whenever you reach more than 50-100 TPS per API.
(Guideline)
● Unless you have millions of transactions per month the OOTB 2 workers
DLB should be enough (e.g. if more you can consider using more
workers)
● Use the DLB for accessing the Process/System to remain in the VPC
instead of out to the SLB and back (reducing latency vs internal name,
e.g. network hairpinning)
● This is a low cost resource compared to the beneﬁts

83
As per the documentation:
● This enables you to host your applications under a single domain.
● Handle load balancing among the different CloudHub workers that run your application.
● Define SSL configurations to provide custom certificates and optionally enforce two-way
SSL client authentication.
● Configure proxy rules that map your applications to custom domains.
Additionally:
● Consider whenever you reach more than 50-100 TPS per API.
● Unless you have millions of transactions per month the OOTB 2 workers DLB should be
enough (e.g. if more you can consider using more workers)
● Use the DLB for accessing the Process/System to remain in the VPC instead of out to the
SLB and back (reducing latency vs internal name, e.g. network hairpinning)
● Low cost resource

Retrospective & Resources
84
1. Evaluate if using the SLB is a
valid use case for you. In many
scenarios you should get the
DLBs. This is not a sales pitch,
as Customer/Partner/CS/SE/PS
save the pain!
2. Cloudhub Physical Architecture
template (reference here)
3. Cloudhub Networking
(reference here)

Case Study #3: CICD

1. You receive a Request for BEST PRACTICES from a Customer. You read the email, the information
in the request and you have prepared again with the questions you have in mind, main doubts
seems about hard-coded values in transformation rules, ERROR HANDLING, LOGGING and CICD
2. You join a call to be received by a very angry and frustrated Customer, person is stating that
MuleSoft Anypoint solution is useless, that there is no way to do proper CICD and that the
behaviour is unacceptable, person keeps raising the voice (no insults, no personal offenses)
3. Customer voice tone sounds angry. It is understandable, probably they are under a lot of
pressure; it is definitely not the best way to express it but this is not a problem: you sit down and
listen, no defiance maintaining temper not taking anything personally (this is a technical session,
it is part of business), you want to drive a solution not to focus on a distraction
Situation
87

1. You have managed to tackle hard-coded values in transformation rules and
Error handling (101). Aesop Fables “...after seeing the lion’s paw, he removes a
large thorn...”
2. Diagnosis on what is missing:
a. Pipeline taking 10 hours to finish deploying into the Anypoint Platform!!!!
i. “If we mistype an URL, we amend the YAML/Configuration-File, we
push to GIT and the change takes up to 10 hours deploy to the
Anypoint Platform”
ii. “YAML files configuration changes (or adding companies) require
rebuild of the code and therefore requires deployment of all API’s of
that ERP genre (4 to 20 ERP’s)”
3. You follow up by email with some recommendations on what was discussed
and you get scheduled a second meeting to REVIEW what their current
process is
Situation
88
a pen and wash drawing by Baldassare Peruzzi, 1530s

Situation
89
a pen and wash drawing by Baldassare Peruzzi, 1530s
1. You join the meeting and... the Customer is at ease. It is definitely nothing
like the first encounter, Customer is asked by someone else and the reply is
“Yes, the information Angel shared was very useful”
2. There is still an issue so you all agree to run through the set up to find out
what is going on.
3. Then…

What is wrong in this picture?
90
1. If we mistype an URL, we amend the YAML/Configuration-File, we push to GIT and the
change takes up to 10 hours deploy to the Anypoint Platform
a. YAML files configuration changes (or adding companies) require rebuild of the code and
therefore requires deployment of all API’s of that ERP genre (4 to 20 ERP’s)

Recipe for Disaster
91
1. One API per Country
2. From 2 Countries total to more than 15 new Countries added (15 new APIs)
3. Azure DevOps pipeline set up was SEQUENTIAL for deployments
4. Using MULE MAVEN PLUGIN Default behaviour (which means -skipVeriﬁcation=FALSE)

92
Retrospective:
● Tactical: MMP -skipVeriﬁcation=true (it will start deploying to Cloudhub but it won’t wait until successful). After
everything is deployed, then do a post mortem analysis: retrieve status of each application and validate the
status per app/country. This is still sequential but from 6-18 minutes per app, it will be reduced to probably
around 5 mins). Can take longer to detect production errors
● Strategic: Parallel Branching per country and Property injection: Not storing the properties in the source code (at
least for environments higher than dev) but to instead inject them in the Maven command
(https://help.mulesoft.com/s/article/How-to-set-application-properties-using-the-Mule-Maven-Plugin) and to
set the values in the release pipelines and create one Release pipeline per country vs UmbrellaAPI (e.g. starts X
threads)
Resources:
● The Importance of CICD in 10 minutes | Friends of Max Overview : introduction into
Continuous Integration and Continuous Deployment (CICD)
● Or even better, back to the origin with Velocity 09: John Allspaw and Paul Hammond:
"10+ Deploys Per Day: Dev and Ops Cooperation at Flickr"

DevOps with MuleSoft
93
Designing, Development and Operations
Designing Development Operations
API
Designers
Design
Design Center
Collaborate
Exchange
Develop
Studio
Source Code
Management
Manage Source
Developers
Build/Deploy Automation
Automation
Server
Build
Automated
Testing
Test
Deploy
Runtime Manager
Artifact
Repository
Manage Release
Production
Sandbox
DEV QA
Runtime Runtime
PRD
Runtime
Monitoring & Alerting
Automated Testing

Case Study #4: Unable to use
CPU

Diagnosis
95
What is happening?
● It is observed that when running a CloudHub application, the CPU utilization does not
appear to use all the CPU capacity. It may be that the CPU maxes out at 50%. In some
scenarios it may appear CPU is locked to 10% or 20%.
What is causing it?
● Not enough concurrent load on the application
● The application is single threaded, using Synchronous ﬂows, using MaxConcurrency=1,
setting an inbound connector to only 1 thread or 1 consumer, or some combination of
these.
● 0.1 or 0.2 vCore where CPU burst capability is exhausted.

96
“...It is recommended that applications requiring infrequent bursts of high cpu, followed by low
or idle time, may work well on a 0.1 or 0.2 vCore worker.
For applications that require sustained high CPU (more than an hour continuously), a
non-fractional vCore worker is recommended. 1 vCore and above do not have burst capability,
and the full CPU capacity is available indeﬁnitely…”
● For all the details, check the Support Article

Case Study #5: Network

Common pitfalls
98
1. There are Organization with a dedicated Team for Network configurations, but, there is a
significant set of Orgs where during setup there is no Networking Subject Matter Experts
present.
2. For example, 5+ years ago as a Back-end Developer I was setting it up the Network for my
Org in a mid-size company, I was not the most qualified person for Network work
3. What happens later is:
a. The configuration will be re-visited a couple of semesters (or years) later due a ongoing review
or an ongoing issue
b. Employees will be drag into it with the alarms going on and try to sort whatever is happening
there

Common pitfalls
99
Examples of non-urgent situations:
1. We are running out of Static IPs! We are releasing in a week, Why? What do we do
now? Let’s create S1 S1 S1!!!
2. Our VPN is down! There is no traffic there!! There is no traffic!!!
3. It worked for our 50 TPS pilot but in our recent tests the VPC peering is not properly
working for our 1.000.000 TPS, we are also expecting a peak 10kk TPS on December
(Wait, what?)
4. We wanted vanity domains since the beginning, we are trying to set it up now
5. Our Testing environment is using a more economical Network option than our Production
environment, but now, (value realisation? Budget? priorities?) we want to change it so
that our Performance Tests are as mirrored to production as possible
In case you need it:
● All you need to know before creating your first VPC
● Static IP Best Practices
● Anypoint VPN (VPN/IPSec/DC)

Special Mentions (6,7,8,9)
What else could go wrong?

Special Mentions
101
● #6 - Common Service Definition
○ Logging
○ Auditing
○ Exception Handling
○ Caching
○ Security via Policy
○ Rate Limiting/Throttling via Policy
● #7 - SSO
○ Onboarding and Offboarding
○ Security, Governance and Control
○ MVP -> OOTB MFA for non Automation Accounts (reference)
○ Multiple IDP SSO for H2
● #8 - Conf/Secrets Management (Quimera mostly for FI/heavy-regulated industries)
● #9 - Redundancy of your VPN connection
○ Our VPN is down! There is no traffic!!
○ https://help.mulesoft.com/s/article/How-to-Configure-High-Availability-with-Anypoint-VPN

1. About Performance
2. Surge day Case Study
3. CICD Branching & Hardcoding
4. Unable to use CPU
5. Network
6. Common Service Deﬁnition
7. SSO
8. Secrets Management
9. Redundancy of your VPN connection
Summary

#10 What Integration challenge
you overcame in the past 2
months? How did you solve it?

Thank You!

Mule 4 migration + Common Integration Challenges : MuleSoft Virtual Muleys Meetups

More Related Content

What's hot (20)

Similar to Mule 4 migration + Common Integration Challenges : MuleSoft Virtual Muleys Meetups (20)

More from Angel Alberici (12)

Recently uploaded (20)

Mule 4 migration + Common Integration Challenges : MuleSoft Virtual Muleys Meetups