Talk from DigitalXChange 2023: Integration is a key element of any modernization project and is perhaps more relevant than ever. In our IT modernization project, we had to overcome a 30-year monolith and used state-of-the-art technologies to bring the old and new worlds together. We used Containers, API Gateway, Service Mesh and Event Hub as well as Kubernetes as the central runtime platform to increase flexibility, agility and developer productivity. This presentation shows how integrations are addressed today using modern technologies.

1. © OPITZ CONSULTING 2023 / Öffentlich
Integration Architectures based on µServices, APIs & Events 1
Gummersbach, June 3, 2023
Sven Bernhardt
INTEGRATION ARCHITECTURES BASED
ON ΜICROSERVICES, APIS & EVENTS

2. © OPITZ CONSULTING 2023 / Öffentlich
THAT‘S ME
Integration Architectures based on µServices, APIs & Events 2
Sven Bernhardt
Cloud-Native enthusiast, API & integration geek. Always curious how new
technologies and concepts can help to make things more valuable and efficient.
@sbernhardt
https://svenbernhardt.wordpress.com/
Chief Architect / Integration Evangelist
OPITZ CONSULTING Deutschland
GmbH

3. © OPITZ CONSULTING 2023 / Öffentlich
Integration Architectures based on µServices, APIs & Events 3
WHY INTEGRATION?
01
INTEGRATION: YESTERDAY
AND TOMORROW
02
OC INTEGRATION STORY
03
CONCLUSION
04

4. © OPITZ CONSULTING 2023 / Öffentlich
Integration Architectures based on µServices, APIs & Events 4
WHY INTEGRATION?
01

5. © OPITZ CONSULTING 2023 / Öffentlich Seite 5
This is what it looks like sometimes/often/always in application
landscapes...
https://www.youtube.com/watch?v=iXaw70X7wb4&t=2s
Integration Architectures based on µServices, APIs & Events

6. © OPITZ CONSULTING 2023 / Öffentlich
„STRUKTURELLE ZUKUNFTSUNFÄHIGKEIT“
6
¢ End of Support
¢ Lack of maintainability
¢ Declining benefits
¢ Dying knowledge
¢ Lack of agility / Implementation backlog
¢ Declining data quality
¢ Rising costs
¢ …
„Despite its benefits for companies, old
software is structurally unsustainable.
This is due to their characteristics and the
resulting risks for the actual business. “
(Lünendonk 2015)
Integration Architectures based on µServices, APIs & Events

7. © OPITZ CONSULTING 2023 / Öffentlich
20% Expiring sourcing contracts
with IT service providers.
71% Pressure to innovate in IT in order
to better support the business.
88% Integration capability of new digital
solutions into legacy software.
92% New technologies (e.g. AI, digital
marketing, etc.) are not supported
by legacy.
DRIVERS OF MODERNIZATION.
Integration Architectures based on µServices, APIs & Events 7
Fit for digital transformation
Status quo and goals for legacy
modernization and cloud
migration (Lünendonk Studie 2019)
31% Maintenance, care and further
development is no longer
guaranteed by a lack of skills.
DIGITAL
PRODUCT

8. © OPITZ CONSULTING 2023 / Öffentlich
STRATEGIC OPTIONS FOR MODERNIZATION
Integration Architectures based on µServices, APIs & Events 8
Current
situation
?
Continuation
Monolith
today tomorrow day after tomorrow
Retention
Platformshift /
Generator Concepts
?
Monolith Crushing
Reengineering
Custom off-the-Shelf
Relief
Change
Deprecated applications
(mostly monolithic):
• Legacy Java
(pre-cloud-native)
• Legacy App Server
• Cobol
• RPG
• Oracle Forms
• Database monoliths
Condition:
Structurally
unsustainable
Save time, postpone problem to later
Modernized applications
(mostly cloud-native):
• Containerized
• Aligned with Business
Capabilities
• Maintenance-friendly
• Design for Change
Condition:
Structurally
sustainable
Is there new, cost-optimized software on the market??
The "right" way
can only be decided
individually.
e.g. Lift&Shift, CI/CD, dyn. Runtime env

9. © OPITZ CONSULTING 2023 / Öffentlich 10
PRODUCT-CENTRIC APPLICATION WORLD DIGITAL
PRODUCT
Integration Architectures based on µServices, APIs & Events
Digital
Product
Monolith
Digital
Product
Business Domain
Shared Application Platform
Shared Services (Foundation) incl. integration platform
Hybrid, flexible infrastructure incl. Cloud Services
Standard
(On-Prem)
Business Domain
Business Domain
Digital Product
Standard
(On-Prem)
SaaS-Cloud
Standard
SaaS
integration is the foundation for it-modernization
-> Necessary to clean-up!
Gartner: Composable Enterprise

10. © OPITZ CONSULTING 2023 / Öffentlich
Integration Architectures based on µServices, APIs & Events 11
INTEGRATION: YESTERDAY AND
TOMORROW
02

11. © OPITZ CONSULTING 2023 / Öffentlich
INTEGRATION? IT’S JUST ABOUT BOXES AND LINES!
Goals:
¢ Enabling systems to talk to each other
¢ Ensure that data is where it is needed when it is needed in the form in which it is needed
Gregor Hohpe
A small line. That‘s where the beef is!
Integration Architectures based on µServices, APIs & Events 12

12. © OPITZ CONSULTING 2023 / Öffentlich
INTEGRATION AREAS
User-centric
apps („users”)
Real-life
objects
(„things”)
Cloud
On premise
Cloud to user
On premise to user
Cloud to thing
On premise to thing
Cloud to cloud
On premise to on premise
¢ There are seven typical integration areas in IT landscapes
Cloud to on premise
1
2
3
4
5
6
7
Integration Architectures based on µServices, APIs & Events 13

13. © OPITZ CONSULTING 2023 / Öffentlich
INTEGRATION CHALLENGES ARE MULTIDIMENSIONAL
Integration Architectures based on µServices, APIs & Events 14

14. © OPITZ CONSULTING 2023 / Öffentlich
BASED ON PATTERNS THAT ARE STABLE SINCE DECADES
Content-based Router
Message Filter
Splitter
Aggregator
Normalizer
Resequencer
https://www.enterpriseintegrationpatterns.com
Integration Architectures based on µServices, APIs & Events 15

15. © OPITZ CONSULTING 2023 / Öffentlich
USUAL STATE OF INTEGRATION ARCHITECTURE – IF THERE‘S ANY…
¢ Monolithic architecture
¢ Often classic JEE deployments
¢ Integrations share single execution process
¢ Limited freedom of technology choice
¢ Core component:
¢ ESB Platform (Oracle Service Bus, Sonic ESB)
¢ Reverse Proxy for handling external traffic
¢ Focus on horizontal integration challenges
Integration Architectures based on µServices, APIs & Events 16

16. © OPITZ CONSULTING 2023 / Öffentlich
FROM CENTRALIZED TO DECENTRALIZED ARCHITECTURES
Integration Architectures based on µServices, APIs & Events 17
Centralized
STATIC
ON-PREM
MONOLITH
VIRTUAL MACHINES
MANUAL CHANGE PROCESS
Decentralized
DYNAMIC
CLOUD / MULTI-CLOUD
MICROSERVICES / SERVERLESS
CONTAINERS, KUBERNETES
AUTOMATED CI/CD TOOL CHAIN
# Services & APIs
CONTROL AND VISIBILITY

17. © OPITZ CONSULTING 2023 / Öffentlich
PLATFORMS NEED TO REFLECT THIS CHANGE
18
Integration Architectures based on µServices, APIs & Events
CLASSIC API GATEWAY MODERN API GATEWAY
MONOLITHIC APP
API
API MANAGEMENT
CLIENT
PRIVATE
TRAFFIC
PUBLIC TRAFFIC
GW DP
CLIENT
PUBLIC TRAFFIC
GW
DP
API
APP
API
APP
API
APP
API
APP
API
APP
GW
DP
GW
DP
✓ High Performance
✓ Multi-cloud and Hybrid
✓ Kubernetes-native
✓ CI/CD automation
✓ Modern protocols

18. © OPITZ CONSULTING 2023 / Öffentlich
Adapter
Processes
Systems
APIs
INTEGRATION NEEDS TO CONSIDER SYSTEMS, DATA & PROCESSES
19
Integration Architectures based on µServices, APIs & Events
Vertical
Integration
Horizontal Integration
Data

19. © OPITZ CONSULTING 2023 / Öffentlich
CLOUD-NATIVE INTEGRATION – THIS IS THE WAY…
¢ Best-of-breed approach
¢ Cloud-native architecture
¢ Containerization
¢ Kubernetes as Core runtime platform
¢ Single execution process per integration
¢ Core components:
¢ Distributed, lightweight ESB
¢ Event Hub for async integration
¢ API Gateway for sync integration
¢ Focus on horizontal as well as vertical
integration challenges
Integration Architectures based on µServices, APIs & Events 20

20. © OPITZ CONSULTING 2023 / Öffentlich
Integration Architectures based on µServices, APIs & Events 21
TECHNOLOGY BASE - USE OPEN INDUSTRY STANDARDS
https://landscape.cncf.io/

21. © OPITZ CONSULTING 2023 / Öffentlich 22
OC INTEGRATION STORY
03
Integration Architectures based on µServices, APIs & Events

22. © OPITZ CONSULTING 2023 / Öffentlich
CRUSHING THE MONOLITH
Seite 23
Integration Architectures based on µServices, APIs & Events
Was completely transferred to SCS
Own microservices/SCSs with web UI, mobile UI and
chatbot interface (teams and mobile) with API
connection
Introduction of standard software (Salesforce, etc.)

23. © OPITZ CONSULTING 2023 / Öffentlich
AS-IS SITUATION – “SPAGHETTI” INTEGRATION
Integration Architectures based on µServices, APIs & Events 24
SharePoint
PVS
Doxis
FiBu
LoBu
Active Directory
DWH
assyst
OpenScape
HR4You
Oscar
Lohn Polen
Onesto
Exchange
FileServer
Steak
TLMT
Website +
Tracking

24. © OPITZ CONSULTING 2023 / Öffentlich
DESIRED SITUATION – STRUCTURED INTEGRATION
Integration Architectures based on µServices, APIs & Events 25
Doxis
Active Directory
DWH
assyst
OpenScape
Oscar
Lohn Polen
Onesto
Exchange
FileServer
Steak
Website +
Tracking
CRM
OC Integration
Platform (OCIP)
ERP
SharePoint
TLMT
3rd Party Cloud(s)
HR
Business Partners OC employees

25. © OPITZ CONSULTING 2023 / Öffentlich
GOALS OC INTEGRATION PLATFORM
Integration Architectures based on µServices, APIs & Events 26
¢ Ensure data integrity and quality during PVS replacement
¢ Simplification of the OC system landscape
¢ Traceability of OC integration paths
¢ Improved governance and compliance management (APIs, data structures, etc.)
¢ Creation of a technological basis for ensuring sufficient data quality in the future
¢ Increase operational excellence
¢ Ensuring an evolutionary architecture
¢ Future-proof
¢ Extensibility
¢ Robustness and Resilience
¢ Scalability and elasticity

26. © OPITZ CONSULTING 2023 / Öffentlich
VISION OC INTEGRATION
PLATFORM
Integration Architectures based on µServices, APIs & Events 27
¢ Ensuring an evolutionary architecture
(Design4Change)
¢ Future-proof
¢ Extensibility
¢ Robustness and Resilience
¢ Scalability and elasticity
¢ OCIP best-of-breed with characteristics
¢ Modern
¢ Automated
¢ Integrated
¢ Secure
“Als Rückgrat unserer
#zukunftswirksamen IT-
Landschaft setzen wir auf eine
dynamikrobuste
Integrationsplattform, um
unsere Systeme, Daten und
Prozesse zu integrieren:
OC|SPINE”

27. © OPITZ CONSULTING 2023 / Öffentlich
OC SPINE IS BUILT ON CLOUD-NATIVE PRINCIPLES …
28
Integration Architectures based on µServices, APIs & Events

28. © OPITZ CONSULTING 2023 / Öffentlich
… AND TECHNOLOGIES
Integration Architectures based on µServices, APIs & Events 29
¢ Open-Source tools from CNCF landscape
¢ Key technology components
¢ API Gateway (Kong)
¢ Event Hub (Confluent Cloud)
¢ Schema Registry (Confluent Schema Registry)
¢ Apicurio (API Registry)
¢ Secrets Management (HashiCorp Vault)
¢ Identity Management (Keycloak)
¢ Service Mesh (Kuma)
¢ Application-level monitoring is done with
Grafana Stack (Grafana Cloud)

29. © OPITZ CONSULTING 2023 / Öffentlich
KEY CONCEPTS
Integration Architectures based on µServices, APIs & Events 30
¢ Loosely-coupled service architecture
¢ System-independent domain model
¢ Transparent error handling
¢ Centralized observability

30. © OPITZ CONSULTING 2023 / Öffentlich
SERVICE CATEGORIES
31
¢ Presentation Services (Single-purpose APIs)
¢ Expose tailored information set for specific clients
¢ Use the information provided by Domain services
¢ Business Services (Multi-purpose APIs)
¢ Implement business resp. domain logic
¢ Operate on a domain object model
¢ Combine information provided by Adapters
¢ Connectivity Services (Adapter)
¢ Implement system-specific logic
¢ Convert data & Protocols
¢ Expose a domain model
Integration Architectures based on µServices, APIs & Events

31. © OPITZ CONSULTING 2023 / Öffentlich
EXAMPLE: SALES – PVS BACKWARD INTEGRATION
32
Integration Architectures based on µServices, APIs & Events

32. © OPITZ CONSULTING 2023 / Öffentlich
“LEGO”-LIKE COMPONENT ARCHITECTURE
33
Integration Architectures based on µServices, APIs & Events
EventHub
PVS
DWH
Other Systems
API Gateway
ChangedDataEvent
DomainEvent
DomainEvent
DomainEvent InsertChangedData
InsertChangedData
GetSalesforceObject
GetDomainObject

33. © OPITZ CONSULTING 2023 / Öffentlich
WITHOUT A SYSTEM-INDEPENDENT DOMAIN
MODEL
Integration Architectures based on µServices, APIs & Events 34
Präsentation 34
Account
• CId : String
• Name : String
• …
ErpKunde
• K_ID : Number
• C_ID : String
• K_NAME : String
• …
DwhKunde
• ID : Number
• C_ID : String
• FIRMEN_NAME : String
• …
Integration platform
CRM
ERP
DWH
Account -> DwhKunde
Account -> ErpKunde

34. © OPITZ CONSULTING 2023 / Öffentlich
CHALLENGES TO SOLVE
Integration Architectures based on µServices, APIs & Events 35
¢ Interdependencies between the systems
¢ Changes in data structures
¢ System-specific IDs for X-referencing
¢ X-referencing challenges may be addressed
by introducing X-Ref functionality
¢ Domain knowledge in Product teams

35. © OPITZ CONSULTING 2023 / Öffentlich
SOLUTION APPROACH
Integration Architectures based on µServices, APIs & Events 36
Account
• CId : String
• UUID : String
• Name : String
• …
ErpKunde
• K_ID : Number
• UUID : String
• K_NAME : String
• …
DwhKunde
• ID : Number
• UUID : String
• FIRMEN_NAME : String
• …
Integration platform
CRM
ERP
DWH
DomainAccount -> DwhKunde
DomainAccount -> ErpKunde
DomainAccount
• UUID : String
• Name : String
• …
Account
-> DomainAccount
Schema Management

36. © OPITZ CONSULTING 2023 / Öffentlich
CENTRALIZED OBSERVABILITY
Integration Architectures based on µServices, APIs & Events 37
¢ Challenge: Distributed integration
architecture that consists of multiple,
independent services
¢ Provide a centralized 360-degree view of
distributed integration architecture
¢ Infrastructure view / Platform health
¢ Application view / Integrations apps
¢ More efficient troubleshooting
¢ Key components:
¢ Grafana Stack, Datadog, EFK
¢ Service Mesh (Kuma, Istio, Consul Mesh)

37. © OPITZ CONSULTING 2023 / Öffentlich
TRACING EXAMPLE VIEW (WITH APP INSTRUMENTATION)
Integration Architectures based on µServices, APIs & Events 38

38. © OPITZ CONSULTING 2023 / Öffentlich
EXAMPLE ERROR DASHBOARD (BASED ON APPLICATION LOGS)
Integration Architectures based on µServices, APIs & Events 39

39. © OPITZ CONSULTING 2023 / Öffentlich
TRANSPARENT ERROR HANDLING
Integration Architectures based on µServices, APIs & Events 40
¢ Different exception types
¢ Retryable (Network errors, Rate limits, etc.)
¢ Non-Retryable (Data errors, Business errors,
etc.)
¢ Use of resilience patterns for auto-retry
exceptions
¢ Circuit breakers
¢ Timeouts
¢ Retries
¢ Central view on errors
¢ Proactive notifications to platform and
product teams

40. © OPITZ CONSULTING 2023 / Öffentlich 41
CONCLUSION
03
Integration Architectures based on µServices, APIs & Events

41. © OPITZ CONSULTING 2023 / Öffentlich
LEARNING: THINK IN
PLATFORMS!
Integration Architectures based on µServices, APIs & Events 42
¢ Integration is a shared responsibility
¢ Establish a platform team, responsible for:
¢ Maintain platform
¢ Operate platform
¢ Provide onboarding support (besides other
services)
¢ Ensure Developer Experience
¢ Decentralized integration implementation
¢ Responsibility: Product teams
¢ Separation of concerns

42. © OPITZ CONSULTING 2023 / Öffentlich
CONCLUSION
Integration Architectures based on µServices, APIs & Events 43
¢ Integration is the foundation for IT
Modernization
¢ Structural inability threatens!
¢ Need for meaningful data, system, and
process integration
¢ We need "Design4Change“
¢ Think platform! Think product!
¢ Not boring stuff… Highly exciting from a
technology perspective!
¢ A high degree of distribution. Everything is in
motion. Many new fields of learning.
¢ Building solutions for an increasingly complex
world needs professionalization
A small line.
That‘s where the beef is!
DIGITAL
PRODUCT

43. © OPITZ CONSULTING 2023 / Öffentlich
Integration Architectures based on µServices, APIs & Events 44
Q & A

44. © OPITZ CONSULTING 2023 / Öffentlich
IN CASE OF ANY QUESTIONS…
Integration Architectures based on µServices, APIs & Events 45
Sven Bernhardt
Chief Architect / Integration Evangelist |
Oracle ACE Director
OPITZ CONSULTING Deutschland GmbH
Kirchstrasse 6, 51647 Gummersbach, Germany
Phone: +49 172 2193529
Mail: sven.bernhardt@opitz-consulting.com
Twitter: @sbernhardt
Blog: https://svenbernhardt.wordpress.com