Attend this presentation covers how Disney, Intel, Appnomic Systems, and the ODCA Infrastructure Work Group delivered a study of how to measure Standard Units of Measure (SUoM) across multiple cloud providers. You will see application and platform performance results of deploying the same application across three platforms including public and private. This approach of technology, tools, and methods can be used to hold your own cloud providers accountable for performance and cost management – using ODCA usage models.

1. MAXIMIZING YOUR CLOUD
SERVICE QUALITY AND COSTS
1
Ed Simmons
- Executive Director, Platform Engineering, UBS
- ODCA Infrastructure Workgroup Co-Chair
Ray Solnik
- President, Appnomic Systems, Inc.
- ODCA Infrastructure Workgroup Member

2. OBJECTIVE AND AGENDA
Objectives:
1. For you to understand how Proofs of Concept Work at the ODCA and why
you should be interested in participating
2. For you to learn and benefit from one of the POCs implemented this year
on behalf of ODCA
Agenda:
1. The ODCA Usage Models and the POC
2. POC Overview
3. POC Real Life Results
4. Q&A
2

3. PUBLISHED USAGE MODELS
Master Usage Models
Information as a Service 1.0
Finer Grain Usage Models
• Virtual Machine Interoperability
• Long Distance Migration
• IO Control
• Service Catalog
• Standard Units of Measure
• Provider Assurance
• Security Monitoring
• Regulatory Framework
• Security Data Framework
• Data Security
• Data Management
• Carbon Footprint
• SaaS Interoperability
• PaaS Interoperability
• Interoperability across Clouds
• Identity Management Interoperability
• Cloud Based Identity Provisioning
• Cloud Based Identity Governance and Auditing
• Single Sign on Authentication
• IaaS Privileged User Access
• SW Entitlement Management Framework
Usage Model
Contents:
• Exec
Summary
• Problem
Statement
• Use Cases
• RFP
Questions
• Industry
Action
Compute IaaS 1.0, Scale-out Storage 1.0,
Software-Defined Networking 1.0
Service Orchestration 1.0
Commercial Framework 1.0

4. PROOF OF CONCEPTS
ODCA Proof of Concepts (POC) Objectives:
1. Validate & improve UMs through real world testing
2. Lead to implementation: trigger real adoption of UM based solutions
3. Influence product roadmaps: promote innovation in line with ODCA
requirements
4. Showcase provider solutions & services
5. Increase visibility for the alliance & participants through press & publications
4
PoC Type Description
SP/UM Alignment Paper evaluation or lightweight review performed in lab/online, generate Cloud Finder data,
publish paper
Biz Case Study Set up usage scenario(s), execute tests, collect and analyze data, provide recommendations,
publish raw internal report and public best practices paper

5. POC PROCESS
1 2 3
A
D
OPTION
• Identify UM
• Define PoC scope &
objectives
• Secure WG alignment &
sponsor
• Understand governance
• Create project plan
• Identify team roles &
responsibilities, tasks
Planning
• Scope Resource
Requirements (people,
technology)
• Create PoC Team
CONCEPT RESOURCE ALIGNMENT PROJECT PLAN
• Deliver project work
• Document results (raw
results/report)
Execution
4 5
• Develop public collaterals
• Socialize with stakeholders
• Publish through ODCA
Marketing with PR activities
PROJECT EXECUTION RESULT COLLATERALS
5

6. POC CASE STUDY: COMPUTE IAAS / SUOM
Measured:
• Actual end user transaction volumes
• Transaction response times
• CPU capacity utilization
• Memory utilization
Learned:
• Explored methods for comparing multiple
cloud IaaS providers
• Insights can be used to tune workloads
and costs for particular cloud providers
• New operations & troubleshooting
techniques
• Industry gap in comparison shopping -
recommending “Cloud Facts” labelling
Value of Participating:
• Disney
• First-hand experience and
learnings from new ITOA solution
• Influence maturity of cloud using
real-world experience
• Appnomic
• Accelerate solution & adoption
• Build awareness to generate sales
6

7. CASE STUDY: HOW DO I GET AND ENSURE ENTERPRISE
GRADE CLOUD PERFORMANCE AT THE BEST PRICE?
Participants: Appnomic, Disney, and Intel
Goal: Establish a clear, equitable means of rating cloud service
providers in respect to performance and quality of service
Business Value: Provide method for enterprise decision-makers
to evaluate competing services, including their own
production application environments, achieve the best
price/performance available
Approach: Run synthetic transactions through a Disney app in
an enterprise private cloud and two external providers; profile
environments using Appnomic AppsOne IT Operations
Analytics platform; “rinse repeat.”
Full report available at:
http://www.appnomic.com/ODCA_ITOA_POC
7

8. THE COMPETITORS: EQUIVALENT COST
8
$1.06/hr $1.12/hr

9. THE RESULTS: NOT EQUIVALENT PERFORMANCE
9
Transaction volumes completed at 3 different tiers of load
Hour of the Day Hour of the Day Hour of the Day
Transaction Count
CIaaS Platform 1
Completed Transactions
CIaaS Platform 2
Completed Transactions
CIaaS Platform 3
Completed Transactions

10. DASHBOARD SUMMARY
Metric
CIaaS
Platform 1
CIaaS
Platform 2
CIaaS
Platform 3 Comments
End‐User
Metrics
Peak Transaction Volume (TPM) 75,000 47,000 25,000
The difference between 75,000 and the values in this
table are transaction requests that were not completed.
Peak Load Transaction Response
Time (milliseconds)
750 1,200 720
CIaaS
Measures
Peak App Server CPU Utilization 2.4% 13.0% 8.5%
Peak App Server Memory Utilization 26.0% 99.0% 11.5%
While CIaaS Platform 1 is higher (worse than) CIaaS
Platform 3, it remains in an acceptable range.
Cost Per Hour
See
Comment
$1.06 $1.12
CIaaS Platform 1 is an internal infrastructure and does
not have a per‐user cost structure associated with it.

11. PRODUCTION APPLICATION USAGE PATTERNS
(AUPS) CAN ACT AS THE BASELINE CONTROL

12. FIGURING OUT WHAT TO PROVISION FROM WHICH
PROVIDER IS CURRENTLY VERY CHALLENGING

13. THE “CLOUD FACTS” LABEL
ODCA has evaluated
other industry solutions
and is investigating the
equivalent of the food
industry solution with
helpful labelling

14. THANK YOU
Ed Simmons, Executive Director, Platform Engineering, UBS
Edward.Simmons@ubs.com
Ray Solnik, President, Appnomic Systems, Inc.
Ray.S@Appnomic.com

15. ADVANCED IT ANALYTICS DEFINED
For very complex IT environments, the automated or
computing-machine organization and interpretation of
massive volumes of metrics representing the IT environment
into meaningful, actionable insights or automated actions to
improve IT systems’ performance and/or security.
Often, outcomes can have material positive impact on business results
through prevention of IT operations incidents like slowness and outages as
well as enabling many other use cases based on the various mathematical
algorithms and techniques that may be utilized.
h t t p : / / d o u g m c c l u r e . n e t / b l o g / d e f i n i n g - i t - o p e r a t i o n s - a n a l y t i c s /
M u l t i p l e t e r m s : I T O p e r a t i o n s A n a l y t i c s ( G a r t n e r ) , I T A n a l y t i c s
( F o r r e s t e r ) , A d v a n c e d A n a l y t i c s ( E M A ) . N o t A P M – A P M i s a d o p t i n g I T O A .

16. 16

17. © 2 0 1 4 O p e n D a t a C e n t e r A l l i a n c e , I n c . A L L R I G H T S R E S E R V E D .