Volvo Cars uses InfluxDB to improve operations by empowering its developers. Self-service monitoring tools have enabled the company’s teams to become more agile and collaborative while ultimately improving product development lifecycle and incident management. Join this webinar to learn how Volvo Cars uses a time series database to become data-driven and accelerate innovation.
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Discover How Volvo Cars Uses a Time Series Database to Become Data-Driven
1. InfluxDB and TICK Stack @ Volvo Cars
InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
2. MAKSIM PUZYRKOV
InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
DevOps Engineer at Volvo Cars
Maksim has almost 20 years of experience in the IT field. During his career, he
worked with a broad range of technologies. Now his professional interests lie
within Enterprise Management and Monitoring solutions.
Maksim works as an Engineer at Volvo with the DevOps Enablement product;
the team adopts an agile way of working. He was responsible for the
development of Volvo’s monitoring approach over the years and contributed to
a number of transformations and platform changes based on growing
company needs.
Recently, Maksim has been working with integration of InfluxDB and other
solutions in the monitoring platform.
3. IBM Tivoli Framework &
Tivoli Enterprise Console
IBM Tivoli Monitoring &
Tivoli Netcool Omnibus
TICK Stack + Elastic Stack +
Grafana +
IBM Netcool Operations Insight
History of Monitoring At Volvo Cars: Monitoring tools evolution
InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
4. Challenges of Monitoring
InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
Low Transparency
Non-Collaborative Mindset
Lack of Self-Service
6. Empowered
Autonomous
Digital Teams
We Provide Tools that support Methodology
to accelerate DevOps culture for digital product teams
InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
7. What is DevOps Enablement at Volvo Cars?
It is about People, Processes and Tools. We offer technologies, ways of
working and thinking that support teams in their DevOps journey.
How do we help to get started?
We help to set up and support in how to structure, plan and operate in a
DevOps mode. As well we provide tool chains that include all functionalities
in order to perform as a DevOps organization.
DevOps Enablement at Volvo Cars
InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
What do we provide?
A toolchain with all the elements needed to develop great digital
products. From planning to release to operations.
What is the transformation we support?
At Volvo Cars we are moving to lean and agile teams. These teams
will work self-sufficient. The DevOps Enablement team provides all
necessary parts in order to perform this change.
8. DevOps Enablement Principles
InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
Best of breed
VS
One fits all
Enable Digital Teams
=
Provide Self-Service
Accelerate by
removing dependency
Don’t rule
Provide more value than
available on the internet
9. Logging
Elasticsearch Logstash Kibana
Monitoring &
Analytics IBM NOI Grafana Application Insights Apica
Quality
Performance Center Load Runner UFT Azure Test Plans SonarQube
CI/CD
Azure Repos Azure Pipelines Azure Artifacts
Planning
Azure Plans
Environment
Azure Cloud / On-Premise Containers Ingress and API Mgmt. Automation ServiceNow
DevOps Enablement Stack
InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
10. Logging
Elasticsearch Logstash Kibana
Monitoring &
Analytics IBM NOI Grafana Application Insights Apica
Quality
Performance Center Load Runner UFT Azure Test Plans SonarQube
CI/CD
Azure Repos Azure Pipelines Azure Artifacts
Planning
Azure Plans
Environment
Azure Cloud / On-Premise Containers Ingress and API Mgmt. Automation ServiceNow
DevOps Enablement Stack
InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
11. OR “Classic” VS “Modern”
InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
“TICK Stack is a loosely coupled yet tightly integrated set of open source projects designed to handle massive
amounts of time-stamped information to support your metrics analysis needs.”
Why InfluxDB and TICK Stack?
12. The TICK Stack is a set of Open-
Source projects designed to handle
massive amounts of time-stamped
information for metrics analysis
needs.
• Telegraf – agent for collecting
and reporting metrics.
• InfluxDB – datastore optimized
for time-stamped data.
• Chronograf – administrative and
visualization user interface.
• Kapacitor – real-time data
processing and alerting engine.
What is TICK Stack?
InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
13. • Grafana – multi-source
visualization interface.
• 3rd party integrations – push/pull
telemetry data from other
systems at DevOps Enablement.
• Custom agents and scripts –
purpose-built agents and scripts
to gather data from cloud
providers, SaaS and in-house
tools.
TICK Stack satellites at DevOps Enablement
InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
14. “Classic” Monitoring Solutions VS InfluxDB or time-series
InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
SELECT used FROM mem WHERE “env”=‘prod’ GROUP BY “app”, “location”
measurement=mem,field=used,app=“app A”,location=“emea”,env=“prod” 23,25,22…
measurement=mem,field=used,app=“app B”,location=“emea”,env=“prod” 45,64,80…
measurement=mem,field=used,app=“app C”,location=“americas”,env=“prod” 36,65,49…
10:10 10:20 10:30 10:40 10:50
app A
emea
americas app C
prod
dev app A
app B
prod
memory usage, %
16. InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
Data Collectors & Connectivity
API Management (3scale)
‒ Auth keys generated through self-service portal
‒ Unique keys helps users maintain application
groups
Kafka Gateways
‒ Very easy, only network connection required
‒ Adjustable buffer that can keep data for days
‒ Supported by Telegraf outputs
Agent of User Choice
‒ Telegraf, Fluentd, Prometheus exporter, custom
script or App built-in metric reporter
‒ Protocols and data formats:
InfluxDB Line Protocol | JSON | CSV | Collectd | Dropwizard | Graphite
| Logfmt | Nagios | OpenTSDB | Prometheus | Value | Wavefront
Data Mover
InfluxDB
Kafka Cluster
Host
Cloud App
Firewall
Containers
API Management
17. InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
Telegraf Installation and Configuration
• Ansible role for Telegraf deployment
• Jump-start configuration packages
• Recommended configuration files structure
C:Program Filestelegraf
| telegraf.conf
| telegraf.exe
|
---telegraf.d
aggregators.conf
inputs_system.conf
inputs_application_a.conf
inputs_application_b.conf
outputs_kafka.conf
processors.conf
19. InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
Event Management and Alerting Self-Service
Alert Management API via 3scale
‒ Auth key through self-service portal
‒ Extended logging and stats per application
Alerts from many sources
‒ Correlation and de-duplication
‒ Situation awareness at a glance
Auto ticketing via TICK script config or central config
‒ Alerts can be extended with ticketing attributes
‒ Extra rules defined on a central event router
Elasticsearch as historical database
‒ Trend analysis for alerts and incidents
Not restricted to “standard” alert management process
‒ Users can use bundled alert handlers
TICK Scripts Blueprints and Templates InfluxDB
3scale
API Management
Kapacitor
IBM NOI
ServiceNow
Elasticsearch
20. InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
Keys For Efficient Alerting: Tagging Strategy & Self-Services
InfluxDB
Group “EMEA” in Critical status!
IBM NOI
Application A, EMEA
Application A, APAC
Application B, AMERICAS
Application C, EMEA
Group by “Region”
Kapacitor
• Tagging strategy
• Soft enforcement through review
and recommendations
• Learning and knowledge sharing
• Templates and blueprints
21. |from()
.database('telegraf')
.retentionPolicy('autogen')
.measurement('disk')
.where(lambda: "environment" == ’prod’ and "path" =~ /data.*/ )
.groupBy('host', 'path')
|mean('used-percent')
.as('stat’)
|alert()
.id('{{ index .Tags "host"}}/disk-used/{{ index .Tags "path" }}')
.message('{{ .ID }}:{{ index .Fields "stat" }}')
.warn(lambda: "stat" > 75)
.crit(lambda: "stat" > 90)
.details(
"TicketFlag": "true",
"TicketPriority": "3",
"TicketGroup": "DevOps Enablement")
This example automatically detects
new metrics received from
production servers.
Warning and Critical alerts triggered
if average disk usage crossed 75 or
90% threshold accordingly.
Separate alerts generated for unique
combination for path and host name.
Alert rule example in TICK Script language
InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
23. TICK Stack at Volvo Cars
InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
Telegraf & other
data collectors
3scale
API Management
Kafka Clusters
Application
Gateway
Ingest Telegrafs Short Term DB
RAW Data
Long Term DB
Pre-filtered Data
Ingest Kapacitors
Replicate measurements
Alerting Kapacitors
HA Cluster
Private Kapacitors
Dev env for users
TICKscript
Replicator
PROD
DEV
Watcher
InfluxDB
& Kapacitor
Data Mover
24. TICK Stack at Volvo Cars: filter data
InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
Ingest Telegrafs
Short Term DB
RAW Data
Long Term DB
Pre-filtered Data
Ingest Kapacitors
Replicate measurements
Alerting Kapacitors
HA Cluster
Private Kapacitors
Dev env for users
TICKscript
Replicator
PROD
DEV
Azure Pipeline Azure Repo
Watcher
InfluxDB
& Kapacitor
Permission List
25. TICK stack at Volvo Cars: Kapacitors – Blue/Green deployment
InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
Azure Kubernetes Service
Application Gateway
service service service
Traefik Ingress Controller
serviceserviceservice
Traefik Ingress Controller
Azure
File share
26. TICK stack at Volvo Cars: Kapacitors – Infrastructure as Code
InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
Azure Kubernetes Service
Application Gateway
service service service
Traefik Ingress Controller
serviceserviceservice
Traefik Ingress Controller
Terraform
Azure
File share
27. “Classic” VS “Modern” Monitoring Life-Cycle
InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
28. InfluxDB and TICK Stack @ Volvo Cars. Maksim Puzyrkov, 2020
Monitoring Life-Cycle: New Monitoring Rule or Graph
Create new Monitoring Template
Assign Monitoring template
to Group of Objects (servers or apps)
Alerts Generated
by New Template Rules
Dashboards with Data
from new Template
“Classic”
Create new Monitoring Rule
Alerts Generated
by Kapacitor service
Dashboards with Metrics
from Agent configuration
No need to apply or reconfigure
agent to gather additional metrics.
Most of the metrics
collected by Telegraf by default.
Agent Discover and Apply
Monitoring Template
“Modern”