Declarative observability management for Microservice architectures

© OPITZ CONSULTING 2023 / Interner Gebrauch
Declarative o11y management for your µArchitectures 1
Gummersbach, June 3, 2023
Sven Bernhardt
DECLARATIVE O11Y MANAGEMENT
FOR MICROSERVICE ARCHITECTURES

THAT‘S ME
Sven Bernhardt
Cloud-Native enthusiast, API & integration geek. Always curious how new
technologies and concepts can help to make things more valuable and efficient.
@sbernhardt
https://svenbernhardt.wordpress.com/
Chief Architect / Integration Evangelist
OPITZ CONSULTING Deutschland
GmbH

CHALLENGES
01
SOLUTION STRATEGY
02
CONCLUSION
04
USE CASES
03

CHALLENGES
01

PREPARE FOR THE UNKNOWN
Things we are aware of
and understand
Things we understand,
but are not aware of
Things we are aware of
but we don’t understand
Things we are neither
aware of nor understand

MONITORING
¢ Historically: Up/Down-Status of a system
¢ Now: Depict the overall health status of a
system
¢ The monitoring system should answer
¢ What is broken?
¢ Why is it broken?
¢ Usually includes levels
¢ Network
¢ Machine
¢ Application
Quote from the SRE Book by Google
“Your monitoring system should address two
questions: what’s broken, and why? The “what’s
broken” indicates the symptom; the “why”
indicates a (possibly intermediate) cause. “What”
versus “why” is one of the most important
distinctions in writing good monitoring with
maximum signal and minimum noise.”

EXAMPLE: SYMPTOMS & CAUSE
Google SRE Book about the correlation between symptoms and causes:
“Note that in a multi-layered system, one person’s symptom is another person’s cause!”
Symptom (“What?”) Cause (“Why?”)
Webserver responds with 500 or 404 Database doesn’t accept new connections, as it is in
maintenance mode
Connection reset while initializing database connection
pool
Wrongly configured firewall rules (IP ranges)
Slow responses for web frontend Backend system is busy with wrongly configured batch job

OBSERVABILITY – JUST A FANCY
NEW TERM?
¢ Oftentimes defined as a superset of
monitoring
¢ Monitoring
¢ Alerting/visualization
¢ Distributed systems tracing infrastructure
¢ Log aggregation/analytics
¢ Goals:
¢ Enhance debuggability of a system
¢ Learn as much as you can about your system
environment
SRE book what isn’t the goal of monitoring:
”It can be tempting to combine monitoring
with other aspects of inspecting complex
systems, such as detailed system profiling,
single-process debugging, tracking details
about exceptions or crashes, load testing, log
collection and analysis, or traffic inspection.
While most of these subjects share
commonalities with basic monitoring, blending
together too many results in overly complex
and fragile systems.”

CHARACTERISTICS OF MODERN MONITORING SOLUTIONS
¢ Must be
¢ Understood as platform
¢ Simple to use
¢ Simple to onboard new applications to
¢ Should be built upon open standards (e.g.
Open Telemetry)
¢ Should only collect data from which one can
make sense of
This is where many traditional
monitoring solutions are strong

APPS / SYSTEMS MUST BE
MONITORABLE!
¢ Modern software today should be “able”
¢ “Able”-Criteria:
Verifi
Test
Oper
Read
Evolv
Maintain
Observ
Scal
Recover
Prov
Afford
Deploy
Audit
Impenetr
Debugg
Credits to Lucas Jellema: https://www.slideshare.net/lucasjellema/code-with-honor-on-
the-importance-and-beauty-of-software-programming-rmoug-2021-training-days
Able

FROM MONOLITHIC TO CLOUD-NATIVE ARCHITECTURES
Traditional approach
Monolithic architecture
Cloud-native approach
Microservice architecture

FROM CENTRALIZED TO DECENTRALIZED ARCHITECTURES
Centralized
STATIC
ON-PREM
MONOLITH
VIRTUAL MACHINES
MANUAL CHANGE PROCESS
Decentralized
DYNAMIC
CLOUD / MULTI-CLOUD
MICROSERVICES / SERVERLESS
CONTAINERS, KUBERNETES
AUTOMATED CI/CD TOOL CHAIN
# Services & APIs
CONTROL AND VISIBILITY

CONSISTENCE AND EFFICIENCY CHALLENGE
Declarative o11y management for your µArchitectures
¢ Multi-layered Observability
challenge
¢ Edge
¢ App
¢ In-App
15

SOLUTION STRATEGY
02

CONCEPT: API GATEWAY
¢ Single entry point for clients to access
Services
¢ No matter the implementation technology
¢ No matter the deployment model (Monolithic
or µService)
¢ Provides a consistent governance model
¢ Decouples Client and Service implementation
¢ Is deployed separately in its own instance
¢ Deployment models:
¢ Bundled data and control plane
¢ Independent data and control plane
Source: https://tinyurl.com/yxbds3cd

CONCEPT: SERVICE MESH
¢ Efficient implementation of cross-cutting
concerns with respect to service
integration challenges
¢ Everything is a service!
¢ Cloud-native apps deployed to Kubernetes
¢ Non Cloud-native workloads
¢ Should be independent of
¢ Architecture (e.g. Monolithic or µService)
¢ Platform (e.g. VMs, Containers, Kubernetes)
Dedicated infrastructure layer that
makes service-to-service communication
more reliable, secure and observable

API GATEWAY & SERVICE MESH TO EMPOWER E2E CONNECTIVITY
¢ API Gateway is integrated in the Mesh
¢ Is just another service
¢ Gateway proxy just handles outbound traffic
¢ Benefits:
¢ Increased Developer experience
¢ Seamless observability
¢ Consistent security
¢ Resilience & Reliable connectivity
¢ Flexibility
GW DP
CLIENT
PUBLIC TRAFFIC
GW DP
MESH CP
MESH 1 MESH 2

Business
Logic (B)
Policy (P)
SHARED POLICY LAYER
OTHER
MONOLITH
MICROSERVICES 3RD-PARTY
Routing
Load Balancing
Authentication
Authorization
Logs & Metrics
Rate Limiting
Caching
Validation
Transformation
MULTI-CLOUD & KUBERNETES
ON-PREMISE
HOSTING
B
Shared Policy Layer (Kong Gateway)
B B B B B B
B B B
Serverless
(e.g. AWS
Lambda)
More applications and services
More languages and protocols
REST}
{
More deployment types

COLLECTING DATA AT THE EDGE LEVEL
¢ Kong Plugins to emit respective data
¢ HTTP / TCP Log
¢ Prometheus
¢ Zipkin
¢ Open Telemetry
¢ Kong EE provides more information OOTB
(Vitals)
¢ # API calls (per API resource)
¢ # Errors / Successful requests
¢ Gateway might be deployed as
¢ Kubernetes Ingress Controller
¢ Standalone Gateway (on VM or Bare Metal)

COLLECTING DATA AT THE APP-LEVEL
¢ Kuma Observability policies are used to emit
needed data
¢ TrafficLog
¢ TrafficMetrics
¢ TrafficTrace
¢ Metrics data can be collected for Data and
Control plane
¢ Insights into Mesh Topology with Service Map
¢ Options for Mesh Gateway
¢ Kong
¢ Kubernetes Gateway API (if operated on K8s)

ANALYZING AND MONITORING THE DATA
Declarative o11y management for your µArchitectures
¢ Using Grafana Stack to create a 360-degree view
¢ Component usage:
¢ Visualization: Grafana
¢ Logging: Loki (Log Shipping: FluentD / FluentBit / Promtail)
¢ Metrics: Prometheus (for long-term storage Cortex / Thanos)
¢ Tracing: Tempo
¢ Alerting: Prometheus Alert Manager
¢ Operating models
¢ Self-managed on-prem
¢ Grafana SaaS offering
23

CONCEPTUAL O11Y ARCHITECTURE
¢ Flexible, cloud-agnostic approach
¢ Independent of architecture and platform
¢ VM / Bare Metal
¢ Containers / K8s
¢ Cloud / On-prem
¢ Easily extensible
¢ Completely based on Open Source
¢ Declarative approach (no code changes)

USE CASES
03

#1 DATA PLATFORM – CHALLENGE / GOALS
¢ Distributed on-prem architecture based on VMs and Containers
¢ Ensure SLAs and operability of the data platform
¢ Insights to data access and processing in a Data Lake scenario
¢ Using o11y data to being able to analyze and optimize data access and processing
¢ Ingestion
¢ Processing
¢ Analysis

#1 DATA PLATFORM – SOLUTION ARCHITECTURE
¢ On-prem only scenario
¢ Challenge for platform team
¢ Patching / Upgrading
¢ Backup / Recovery
¢ Extending platform
¢ Monitoring platform
¢ Host-level: Checkmk
¢ App-level: Grafana Stack (Self-hosted)

#1 DATA PLATFORM – SOLUTION ARCHITECTURE
¢ O11y data is delivered by agents:
¢ Host Metrics: Checkmk Agents
¢ Mesh / App Metrics: Envoy, Prometheus
Exporters, Spring Boot Actuator
¢ Logs: Envoy, Promtail, FluentD
¢ Traces: Envoy, Spring Boot / Micrometer
¢ Agent distribution:
¢ Automation of agent distribution (Ansible)
¢ Envoy proxies for apps are automatically
initiated (if applicable)

#2: INTEGRATION PLATFORM – CHALLENGES / GOALS
¢ Distributed hybrid architecture based on VMs and Containers
¢ Ensure SLAs and operability of the integration platform (critical system)
¢ Insights to cloud-native integration flows
¢ Support API lifecycle management
¢ Support platform team to proatively operate the platform
¢ Increase debugability of the distributed platform in case of application errors
¢ Transparency about data usage
¢ Monitor overall platform state (not only infra)
¢ Insight into data flows with respect to state & performance

#2: INTEGRATION PLATFORM – SOLUTION ARCHITECTURE
¢ Distributed, hybrid architecture
¢ Monitoring Platform:
¢ Grafana Stack (Grafana Cloud)
¢ O11y data is delivered by agents:
¢ Mesh / App Metrics: Envoy, Prometheus
Exporters, Spring Boot Actuator
¢ Logs: Envoy
¢ Traces: Envoy, Java agent

#2: INTEGRATION PLATFROM – TRACING EXAMPLE VIEW

#2: INTEGRATION PLATFROM – EXAMPLE ERROR DASHBOARD

CONCLUSION
04

HOPE IS NOT A STRATEGY!
¢ We need to build monitorable apps /
systems
¢ Having a consistent Monitoring strategy is
key to build sustainable solutions
¢ NOTE: Having good monitoring in place
doesn’t mean that we compeltly get rid of
failures
¢ Just collect important information from
which you make sense, e.g. in dashboards or
alerts
Source: Unsplash

THINK IN PLATFORMS!
¢ Declarative o11y with API Gateway & Service
Mesh
¢ Provide a shared policy layer to configure
respective aspects
¢ Think in platforms!
¢ Make it easy to use, operate and extend
¢ For product teams (Developer
experience)
¢ For platform teams

WHAT SHOULD BE MONITORED?
¢ The amount of monitoring data is almost
infinite
¢ Limit the amount of data collected to
relevant ones
¢ Focus on the four golden signals mentioned
in Google SRE Book:
¢ Latency
¢ Traffic
¢ Errors
¢ Saturation

Q & A

IN CASE OF ANY QUESTIONS…
Sven Bernhardt
Chief Architect / Integration Evangelist |
Oracle ACE Director
OPITZ CONSULTING Deutschland GmbH
Kirchstrasse 6, 51647 Gummersbach, Germany
Phone: +49 172 2193529
Mail: sven.bernhardt@opitz-consulting.com
Twitter: @sbernhardt
Blog: https://svenbernhardt.wordpress.com

Declarative observability management for Microservice architectures

Recommended

Recommended

More Related Content

Similar to Declarative observability management for Microservice architectures

Similar to Declarative observability management for Microservice architectures (20)

More from Sven Bernhardt

More from Sven Bernhardt (12)

Recently uploaded

Recently uploaded (20)

Declarative observability management for Microservice architectures