SlideShare a Scribd company logo
1 of 48
Download to read offline
EdgeCast
Performance Engineering Group

Third Party Monitoring
Use Cases

Nathan Moore, Web Performance Engineer
nmoore@edgecast.com
Web Performance Group

Analyze performance trends and come up with
ideas to make the CDN faster

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

2
Performance Optimizations
•
•
•
•

Network Stack
Routing
Kernel
Application

© 2013 EDGECAST NETWORKS

TCP Optimizations

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

3
Performance Optimizations
• We monitor as much as we can to find these:

Needles in haystacks

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

4
Why monitor?

You can’t manage what you don’t measure

-- Deming
-- Drucker
-- Traditional
© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

5
What can you monitor with Third Party
Applications?
• Performance
– Latency
– Bandwidth

• Availability
– Uptime

• Rate of Change of Internet
– Snapshot of Internet as a whole
© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

6
Why care?

Web performance drives KPIs

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

7
Why care?

KPIs:
• Fast websites drive Sales and Engagement
• Reliable websites drive Sales and Engagement

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

8
Not just a made-up statistic

http://sites.google.com/site/glinden/Home/StanfordDataMining.2006-11-28.ppt?attredirects=0
© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

9
Why monitor?

What you can measure, you can optimize
-- Ilya Grigorik

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

10
Simplified Internet Traffic Flow

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

11
Categorizing the World

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

12
Monitoring the World

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

13
What are Third Party Monitors for

Synthetic monitoring is
needed to dynamically
monitor the External
Internet
© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

14
Monitoring Hooks for Synthetic Monitoring

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

15
Synthetic Monitoring
Synthetic monitoring is clean
•
•
•
•

Small number of nodes
Well defined network
Highly reliable links between nodes and networks
Shows health of the backbone of the Internet

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

16
State of the Internet

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

17
What are Third Party Monitors For

Real User Monitoring
(RUM) is needed to
measure what your end
users really see
© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

18
From Synthetic to Real User Metrics (RUM)

Reporting
Server

CDN 1

© 2013 EDGECAST NETWORKS

CDN 2

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

CDN 3

19
Real User Data / RUM

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

20
Real User Metrics (RUM)
RUM is messy. Rum is the real observed data generated
by real users on the real Internet on real web browsers
Combinatorics:
• Hundreds of Millions of TCP connections
• Tens of Millions of end users
• Thousands of ISPs
• Tens of backbone providers
• One of your web servers
• Data Distribution issues:
• Power law distributions => averages lose explanatory power
• Quantiles and Cumulative Density Functions (CDF) gain importance
• Aggregating data and mining for correlations becomes more difficult

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

21
Real User Data / RUM benefits
Data aggregated across country and displayed across world

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

22
Use Case 1:
What if you only have synthetic monitors?
New
platform
launch:
EdgeCast
Transact

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

23
Use Case 1:
Launch of Transact
Directive from CEO:
Performance must match the existing matured network!

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

24
Use Case 1:
Launch of Transact
Transact is a brand new product from EdgeCast, designed to
accelerate e-commerce performance on the web.
Featuring full PCI compliance, it has its own dedicated and highly
reliable network, its own dedicated servers, and a fully
integrated security layer.
The challenge is to deploy a brand new network, and make it
perform like an existing, fully matured network.
Third party monitors are the key to accomplish this, and to know
when we’ve accomplished it.
© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

25
Use Case 1:
Launch of Transact
Established monitors across the major third parties to observe
actual latencies worldwide

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

26
Use Case 1:
Launch of Transact
Compared performance between Transact and EdgeCast ADN at all nodes
Used the delta to determine where issues were

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

27
Use Case 1:
Launch of Transact
Iterated: As issues were resolved, updated with new monitoring data
Repeat process!

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

28
Use Case 1:
Launch of Transact
Result: Worldwide average performance within 1.5ms
EdgeCast ADN

© 2013 EDGECAST NETWORKS

Transact

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

29
What are Third Party Monitors For

Third party monitoring
allows A/B comparison

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

30
Use Case 2: RIPE/ATLAS
Using RIPE/ATLAS to evaluate the network for IPv6 launch
for World IPv6 Day, 2012

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

31
Use Case 2: RIPE/ATLAS
For World IPv6 Day, we deployed IPv6 across our entire
network.
The Challenge:
How does the IPv6 Internet perform? We used RUM data
provided by RIPE/ATLAS to find out.

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

32
Use Case 2: RIPE/ATLAS
http://atlas.ripe.net/about
RIPE Atlas is a global network of probes that measure Internet connectivity and
reachability, providing an unprecedented understanding of the state of the Internet in
real time.

http://www.ripe.net
The RIPE NCC is one of five Regional Internet Registries (RIRs) providing Internet
resource allocations, registration services and coordination activities that support
the operation of the Internet globally.
© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

33
Use Case 2: RIPE/ATLAS
JSON based API

© 2013 EDGECAST NETWORKS

Ability to run multiple tests

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

34
Use Case 2: RIPE/ATLAS
We launched a beacon dedicated to IPv6 measurements
which tested:
• IPv4 only
• IPv6 only
• Dual IPv4 & IPv6

To reduce failures, the beacon also checked connectivity
to ipv6.google.com

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

35
Use Case 2: RIPE/ATLAS

What did we learn from Atlas about our
IPv6 performance and availability?

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

36
Use Case 2: RIPE/ATLAS
IPv6 Reach Analysis

AS path analysis
© 2013 EDGECAST NETWORKS

AS path of failed traces
2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

37
Use Case 2: RIPE/ATLAS
Visualization of Country-of-Origin traffic to EdgeCast POP

Ex: Ukraine is served by both POP7 and POP1, and POP7 has higher latency
© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

38
Use Case 2: RIPE/ATLAS
Availability predictions before launch

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

39
Use Case 2: RIPE/ATLAS
Performance improvements during pre-launch
Conclusions: IPv6 still has room for improvement

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

40
Use Case 3: Analytics within Third Party Monitors
Using Third Party Monitors to answer real-world questions
Often, we’re asked questions by different groups of the company.
Sales, for instance, always wants to prove that we’re the fastest worldwide.
But what to do when some regions are faster than others?

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

41
Use Case 3: Analytics within Third Party Monitors
Sales wanted to know why a prospective client was performing really well
in Asia, but not nearly as well in Europe.

EdgeCast

Asia
Mystery Competitor X

© 2013 EDGECAST NETWORKS

EdgeCast

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

Europe
Mystery Competitor X

42
Use Case 3: Analytics within Third Party Monitors
There was a lot of variance within countries
EdgeCast

© 2013 EDGECAST NETWORKS

Mystery Competitor X

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

43
Use Case 3: Analytics within Third Party Monitors
The scatterplot shows some odd signs of horizontal banding.
EdgeCast

© 2013 EDGECAST NETWORKS

Mystery Competitor X

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

44
Use Case 3: Analytics within Third Party Monitors
Good clue: Looking at the total downloaded bytes instead of response time,
the banding becomes really obvious, and wildly different in size

EdgeCast

© 2013 EDGECAST NETWORKS

Mystery Competitor X

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

45
Use Case 3: Analytics within Third Party Monitors
A histogram shows the magnitude of this difference

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

46
Use Case 3: Analytics within Third Party Monitors
Conclusion:
Client was returning different data depending on the geoIP lookup
of the end user. So end users in different countries were
deliberately being returned different objects.
However, as caches, both EdgeCast and Mystery Competitor X were
caching the index.html page, so whatever was on that page was
cached and returned on every request. New data was only returned
once every day, when the caching rule expired, and a fresh request
to origin was made.
The extreme difference in size was largely a function of random
chance- we got lucky to return smaller objects during this sample
time.

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

47
Questions?

© 2013 EDGECAST NETWORKS

2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA

48

More Related Content

Similar to Synthetic and Real World Web Performance Monitoring

Tech Talk: Leverage the combined power of CA Unified Infrastructure Managemen...
Tech Talk: Leverage the combined power of CA Unified Infrastructure Managemen...Tech Talk: Leverage the combined power of CA Unified Infrastructure Managemen...
Tech Talk: Leverage the combined power of CA Unified Infrastructure Managemen...CA Technologies
 
Case Study: O2/Telefonica Transitions From CA eHealth® to CA Performance Mana...
Case Study: O2/Telefonica Transitions From CA eHealth® to CA Performance Mana...Case Study: O2/Telefonica Transitions From CA eHealth® to CA Performance Mana...
Case Study: O2/Telefonica Transitions From CA eHealth® to CA Performance Mana...CA Technologies
 
The Aruba Tech Support Top 10: WLAN design, configuration and troubleshooting...
The Aruba Tech Support Top 10: WLAN design, configuration and troubleshooting...The Aruba Tech Support Top 10: WLAN design, configuration and troubleshooting...
The Aruba Tech Support Top 10: WLAN design, configuration and troubleshooting...Aruba, a Hewlett Packard Enterprise company
 
Driving Down Costs of z Systems™ Storage
Driving Down Costs of z Systems™ StorageDriving Down Costs of z Systems™ Storage
Driving Down Costs of z Systems™ StorageCA Technologies
 
What is expected from Chief Cloud Officers?
What is expected from Chief Cloud Officers?What is expected from Chief Cloud Officers?
What is expected from Chief Cloud Officers?Bernard Paques
 
Designing Fault-Tolerant Applications with DataStax Enterprise and Apache Cas...
Designing Fault-Tolerant Applications with DataStax Enterprise and Apache Cas...Designing Fault-Tolerant Applications with DataStax Enterprise and Apache Cas...
Designing Fault-Tolerant Applications with DataStax Enterprise and Apache Cas...DataStax
 
Forward Networks - Networking Field Day 13 presentation
Forward Networks - Networking Field Day 13 presentationForward Networks - Networking Field Day 13 presentation
Forward Networks - Networking Field Day 13 presentationAndrew Wesbecher
 
Hands-On Labs: Identifying Application Delivery Performance Problems with CA ...
Hands-On Labs: Identifying Application Delivery Performance Problems with CA ...Hands-On Labs: Identifying Application Delivery Performance Problems with CA ...
Hands-On Labs: Identifying Application Delivery Performance Problems with CA ...CA Technologies
 
Mobile Experience Management and Network Services Health Check with Aruba Air...
Mobile Experience Management and Network Services Health Check with Aruba Air...Mobile Experience Management and Network Services Health Check with Aruba Air...
Mobile Experience Management and Network Services Health Check with Aruba Air...Aruba, a Hewlett Packard Enterprise company
 
Forward Networks - Networking Field Day 13 presentation
Forward Networks - Networking Field Day 13 presentationForward Networks - Networking Field Day 13 presentation
Forward Networks - Networking Field Day 13 presentationForward Networks
 
Live traffic capture and replay in cassandra 4.0
Live traffic capture and replay in cassandra 4.0Live traffic capture and replay in cassandra 4.0
Live traffic capture and replay in cassandra 4.0Vinay Kumar Chella
 
Monetizing Big Data with Streaming Analytics for Telecoms Service Providers
Monetizing Big Data with Streaming Analytics for Telecoms Service ProvidersMonetizing Big Data with Streaming Analytics for Telecoms Service Providers
Monetizing Big Data with Streaming Analytics for Telecoms Service ProvidersCubic Corporation
 
EXTENT-2016: Network Instrumentation Challenges and Solutions
EXTENT-2016: Network Instrumentation Challenges and SolutionsEXTENT-2016: Network Instrumentation Challenges and Solutions
EXTENT-2016: Network Instrumentation Challenges and SolutionsIosif Itkin
 
Ciena - the journey to the adaptive network
Ciena - the journey to the adaptive networkCiena - the journey to the adaptive network
Ciena - the journey to the adaptive networkJisc
 
Enabling Cloud Storage Auditing with Key Exposure Resistance
Enabling Cloud Storage Auditing with Key Exposure ResistanceEnabling Cloud Storage Auditing with Key Exposure Resistance
Enabling Cloud Storage Auditing with Key Exposure ResistanceIRJET Journal
 
Internet of Things DC: July 13 Meetup - Link Labs
Internet of Things DC: July 13 Meetup - Link LabsInternet of Things DC: July 13 Meetup - Link Labs
Internet of Things DC: July 13 Meetup - Link LabsInternet of Things DC
 
A New Approach to Continuous Monitoring in the Cloud
A New Approach to Continuous Monitoring in the CloudA New Approach to Continuous Monitoring in the Cloud
A New Approach to Continuous Monitoring in the CloudNETSCOUT
 

Similar to Synthetic and Real World Web Performance Monitoring (20)

Tech Talk: Leverage the combined power of CA Unified Infrastructure Managemen...
Tech Talk: Leverage the combined power of CA Unified Infrastructure Managemen...Tech Talk: Leverage the combined power of CA Unified Infrastructure Managemen...
Tech Talk: Leverage the combined power of CA Unified Infrastructure Managemen...
 
Case Study: O2/Telefonica Transitions From CA eHealth® to CA Performance Mana...
Case Study: O2/Telefonica Transitions From CA eHealth® to CA Performance Mana...Case Study: O2/Telefonica Transitions From CA eHealth® to CA Performance Mana...
Case Study: O2/Telefonica Transitions From CA eHealth® to CA Performance Mana...
 
The Aruba Tech Support Top 10: WLAN design, configuration and troubleshooting...
The Aruba Tech Support Top 10: WLAN design, configuration and troubleshooting...The Aruba Tech Support Top 10: WLAN design, configuration and troubleshooting...
The Aruba Tech Support Top 10: WLAN design, configuration and troubleshooting...
 
Driving Down Costs of z Systems™ Storage
Driving Down Costs of z Systems™ StorageDriving Down Costs of z Systems™ Storage
Driving Down Costs of z Systems™ Storage
 
What is expected from Chief Cloud Officers?
What is expected from Chief Cloud Officers?What is expected from Chief Cloud Officers?
What is expected from Chief Cloud Officers?
 
Designing Fault-Tolerant Applications with DataStax Enterprise and Apache Cas...
Designing Fault-Tolerant Applications with DataStax Enterprise and Apache Cas...Designing Fault-Tolerant Applications with DataStax Enterprise and Apache Cas...
Designing Fault-Tolerant Applications with DataStax Enterprise and Apache Cas...
 
3.2_Securing Microgrids, Substations, and Distributed Autonomous Systems_Lawr...
3.2_Securing Microgrids, Substations, and Distributed Autonomous Systems_Lawr...3.2_Securing Microgrids, Substations, and Distributed Autonomous Systems_Lawr...
3.2_Securing Microgrids, Substations, and Distributed Autonomous Systems_Lawr...
 
Forward Networks - Networking Field Day 13 presentation
Forward Networks - Networking Field Day 13 presentationForward Networks - Networking Field Day 13 presentation
Forward Networks - Networking Field Day 13 presentation
 
Hands-On Labs: Identifying Application Delivery Performance Problems with CA ...
Hands-On Labs: Identifying Application Delivery Performance Problems with CA ...Hands-On Labs: Identifying Application Delivery Performance Problems with CA ...
Hands-On Labs: Identifying Application Delivery Performance Problems with CA ...
 
Mobile Experience Management and Network Services Health Check with Aruba Air...
Mobile Experience Management and Network Services Health Check with Aruba Air...Mobile Experience Management and Network Services Health Check with Aruba Air...
Mobile Experience Management and Network Services Health Check with Aruba Air...
 
Forward Networks - Networking Field Day 13 presentation
Forward Networks - Networking Field Day 13 presentationForward Networks - Networking Field Day 13 presentation
Forward Networks - Networking Field Day 13 presentation
 
Live traffic capture and replay in cassandra 4.0
Live traffic capture and replay in cassandra 4.0Live traffic capture and replay in cassandra 4.0
Live traffic capture and replay in cassandra 4.0
 
Monetizing Big Data with Streaming Analytics for Telecoms Service Providers
Monetizing Big Data with Streaming Analytics for Telecoms Service ProvidersMonetizing Big Data with Streaming Analytics for Telecoms Service Providers
Monetizing Big Data with Streaming Analytics for Telecoms Service Providers
 
EXTENT-2016: Network Instrumentation Challenges and Solutions
EXTENT-2016: Network Instrumentation Challenges and SolutionsEXTENT-2016: Network Instrumentation Challenges and Solutions
EXTENT-2016: Network Instrumentation Challenges and Solutions
 
Packets never lie: An in-depth overview of 802.11 frames
Packets never lie: An in-depth overview of 802.11 framesPackets never lie: An in-depth overview of 802.11 frames
Packets never lie: An in-depth overview of 802.11 frames
 
Ciena - the journey to the adaptive network
Ciena - the journey to the adaptive networkCiena - the journey to the adaptive network
Ciena - the journey to the adaptive network
 
Enabling Cloud Storage Auditing with Key Exposure Resistance
Enabling Cloud Storage Auditing with Key Exposure ResistanceEnabling Cloud Storage Auditing with Key Exposure Resistance
Enabling Cloud Storage Auditing with Key Exposure Resistance
 
Internet of Things DC: July 13 Meetup - Link Labs
Internet of Things DC: July 13 Meetup - Link LabsInternet of Things DC: July 13 Meetup - Link Labs
Internet of Things DC: July 13 Meetup - Link Labs
 
DNS: EdgeCast Route - Technical DNS Service Overview
DNS: EdgeCast Route - Technical DNS Service Overview DNS: EdgeCast Route - Technical DNS Service Overview
DNS: EdgeCast Route - Technical DNS Service Overview
 
A New Approach to Continuous Monitoring in the Cloud
A New Approach to Continuous Monitoring in the CloudA New Approach to Continuous Monitoring in the Cloud
A New Approach to Continuous Monitoring in the Cloud
 

Recently uploaded

UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?SANGHEE SHIN
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdfJamie (Taka) Wang
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Babel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptxBabel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptxYounusS2
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 

Recently uploaded (20)

UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Babel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptxBabel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptx
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 

Synthetic and Real World Web Performance Monitoring

  • 1. EdgeCast Performance Engineering Group Third Party Monitoring Use Cases Nathan Moore, Web Performance Engineer nmoore@edgecast.com
  • 2. Web Performance Group Analyze performance trends and come up with ideas to make the CDN faster © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 2
  • 3. Performance Optimizations • • • • Network Stack Routing Kernel Application © 2013 EDGECAST NETWORKS TCP Optimizations 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 3
  • 4. Performance Optimizations • We monitor as much as we can to find these: Needles in haystacks © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 4
  • 5. Why monitor? You can’t manage what you don’t measure -- Deming -- Drucker -- Traditional © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 5
  • 6. What can you monitor with Third Party Applications? • Performance – Latency – Bandwidth • Availability – Uptime • Rate of Change of Internet – Snapshot of Internet as a whole © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 6
  • 7. Why care? Web performance drives KPIs © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 7
  • 8. Why care? KPIs: • Fast websites drive Sales and Engagement • Reliable websites drive Sales and Engagement © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 8
  • 9. Not just a made-up statistic http://sites.google.com/site/glinden/Home/StanfordDataMining.2006-11-28.ppt?attredirects=0 © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 9
  • 10. Why monitor? What you can measure, you can optimize -- Ilya Grigorik © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 10
  • 11. Simplified Internet Traffic Flow © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 11
  • 12. Categorizing the World © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 12
  • 13. Monitoring the World © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 13
  • 14. What are Third Party Monitors for Synthetic monitoring is needed to dynamically monitor the External Internet © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 14
  • 15. Monitoring Hooks for Synthetic Monitoring © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 15
  • 16. Synthetic Monitoring Synthetic monitoring is clean • • • • Small number of nodes Well defined network Highly reliable links between nodes and networks Shows health of the backbone of the Internet © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 16
  • 17. State of the Internet © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 17
  • 18. What are Third Party Monitors For Real User Monitoring (RUM) is needed to measure what your end users really see © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 18
  • 19. From Synthetic to Real User Metrics (RUM) Reporting Server CDN 1 © 2013 EDGECAST NETWORKS CDN 2 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA CDN 3 19
  • 20. Real User Data / RUM © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 20
  • 21. Real User Metrics (RUM) RUM is messy. Rum is the real observed data generated by real users on the real Internet on real web browsers Combinatorics: • Hundreds of Millions of TCP connections • Tens of Millions of end users • Thousands of ISPs • Tens of backbone providers • One of your web servers • Data Distribution issues: • Power law distributions => averages lose explanatory power • Quantiles and Cumulative Density Functions (CDF) gain importance • Aggregating data and mining for correlations becomes more difficult © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 21
  • 22. Real User Data / RUM benefits Data aggregated across country and displayed across world © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 22
  • 23. Use Case 1: What if you only have synthetic monitors? New platform launch: EdgeCast Transact © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 23
  • 24. Use Case 1: Launch of Transact Directive from CEO: Performance must match the existing matured network! © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 24
  • 25. Use Case 1: Launch of Transact Transact is a brand new product from EdgeCast, designed to accelerate e-commerce performance on the web. Featuring full PCI compliance, it has its own dedicated and highly reliable network, its own dedicated servers, and a fully integrated security layer. The challenge is to deploy a brand new network, and make it perform like an existing, fully matured network. Third party monitors are the key to accomplish this, and to know when we’ve accomplished it. © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 25
  • 26. Use Case 1: Launch of Transact Established monitors across the major third parties to observe actual latencies worldwide © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 26
  • 27. Use Case 1: Launch of Transact Compared performance between Transact and EdgeCast ADN at all nodes Used the delta to determine where issues were © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 27
  • 28. Use Case 1: Launch of Transact Iterated: As issues were resolved, updated with new monitoring data Repeat process! © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 28
  • 29. Use Case 1: Launch of Transact Result: Worldwide average performance within 1.5ms EdgeCast ADN © 2013 EDGECAST NETWORKS Transact 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 29
  • 30. What are Third Party Monitors For Third party monitoring allows A/B comparison © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 30
  • 31. Use Case 2: RIPE/ATLAS Using RIPE/ATLAS to evaluate the network for IPv6 launch for World IPv6 Day, 2012 © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 31
  • 32. Use Case 2: RIPE/ATLAS For World IPv6 Day, we deployed IPv6 across our entire network. The Challenge: How does the IPv6 Internet perform? We used RUM data provided by RIPE/ATLAS to find out. © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 32
  • 33. Use Case 2: RIPE/ATLAS http://atlas.ripe.net/about RIPE Atlas is a global network of probes that measure Internet connectivity and reachability, providing an unprecedented understanding of the state of the Internet in real time. http://www.ripe.net The RIPE NCC is one of five Regional Internet Registries (RIRs) providing Internet resource allocations, registration services and coordination activities that support the operation of the Internet globally. © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 33
  • 34. Use Case 2: RIPE/ATLAS JSON based API © 2013 EDGECAST NETWORKS Ability to run multiple tests 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 34
  • 35. Use Case 2: RIPE/ATLAS We launched a beacon dedicated to IPv6 measurements which tested: • IPv4 only • IPv6 only • Dual IPv4 & IPv6 To reduce failures, the beacon also checked connectivity to ipv6.google.com © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 35
  • 36. Use Case 2: RIPE/ATLAS What did we learn from Atlas about our IPv6 performance and availability? © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 36
  • 37. Use Case 2: RIPE/ATLAS IPv6 Reach Analysis AS path analysis © 2013 EDGECAST NETWORKS AS path of failed traces 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 37
  • 38. Use Case 2: RIPE/ATLAS Visualization of Country-of-Origin traffic to EdgeCast POP Ex: Ukraine is served by both POP7 and POP1, and POP7 has higher latency © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 38
  • 39. Use Case 2: RIPE/ATLAS Availability predictions before launch © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 39
  • 40. Use Case 2: RIPE/ATLAS Performance improvements during pre-launch Conclusions: IPv6 still has room for improvement © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 40
  • 41. Use Case 3: Analytics within Third Party Monitors Using Third Party Monitors to answer real-world questions Often, we’re asked questions by different groups of the company. Sales, for instance, always wants to prove that we’re the fastest worldwide. But what to do when some regions are faster than others? © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 41
  • 42. Use Case 3: Analytics within Third Party Monitors Sales wanted to know why a prospective client was performing really well in Asia, but not nearly as well in Europe. EdgeCast Asia Mystery Competitor X © 2013 EDGECAST NETWORKS EdgeCast 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA Europe Mystery Competitor X 42
  • 43. Use Case 3: Analytics within Third Party Monitors There was a lot of variance within countries EdgeCast © 2013 EDGECAST NETWORKS Mystery Competitor X 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 43
  • 44. Use Case 3: Analytics within Third Party Monitors The scatterplot shows some odd signs of horizontal banding. EdgeCast © 2013 EDGECAST NETWORKS Mystery Competitor X 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 44
  • 45. Use Case 3: Analytics within Third Party Monitors Good clue: Looking at the total downloaded bytes instead of response time, the banding becomes really obvious, and wildly different in size EdgeCast © 2013 EDGECAST NETWORKS Mystery Competitor X 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 45
  • 46. Use Case 3: Analytics within Third Party Monitors A histogram shows the magnitude of this difference © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 46
  • 47. Use Case 3: Analytics within Third Party Monitors Conclusion: Client was returning different data depending on the geoIP lookup of the end user. So end users in different countries were deliberately being returned different objects. However, as caches, both EdgeCast and Mystery Competitor X were caching the index.html page, so whatever was on that page was cached and returned on every request. New data was only returned once every day, when the caching rule expired, and a fresh request to origin was made. The extreme difference in size was largely a function of random chance- we got lucky to return smaller objects during this sample time. © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 47
  • 48. Questions? © 2013 EDGECAST NETWORKS 2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA 48