WSI32 - IBM WebSphere Performance Fundamentals

IBM European WebSphere
Technical Conference
14 – 18 November 2008, Barcelona, Spain
© 2008 IBM Corporation
Conference materials may not be reproduced in whole or in part without the prior written permission of IBM.
WebSphere Performance FundamentalsWebSphere Performance Fundamentals
Session Number: WSI32Session Number: WSI32 Hendrik van RunHendrik van Run –– hvanrun@uk.ibm.comhvanrun@uk.ibm.com

WebSphere Technical Conference and Transaction & Messaging Technical Conference
2
What is Performance?What is Performance?
Performance is defined as a non-functional requirement (NFR)
Defines an operational capability of the application and system
Integrating performance requirements into a project requires balance
Too much functional emphasis leads to poor operational behavior
Too much performance emphasis may inhibit fundamental utility
Performance requires lifecycle attention and specialized verification
Requires consideration, management, and tracking through all phases
Functional and integration verification cannot predict operational performance
Performance EmphasisFunctional Emphasis

3
Performance LifecyclePerformance Lifecycle
Capacity Planning
Project Planning
Architecture and Design
Development
Verification
Operations

4
Capacity PlanningCapacity Planning
Defining Performance RequirementsDefining Performance Requirements
Response Time
Time required to process a request
Typically more critical for systems supporting direct user interaction
Throughput
How many things (requests, messages, etc.) the system can produce in a unit of time
Often the key metric expressed for message-based systems
Throughput is interdependent with response time for systems supporting end users
• These systems may run under their maximum throughput in order to meet responsiveness
goals
Users
A metric for systems supporting direct user interaction, such as web sites
Users often imply a memory footprint requirement for tasteful systems
In some cases, the only metric available
• Throughput and response time must be interpolated from desired user traffic
• Metrics based on understanding of system flow
All these requirements are typically included in Service Level Agreements (Slaps)

5
Determining Load and Response Time TargetsDetermining Load and Response Time Targets
Always develop requirements around peak load
If the system doesn’t support peak, it does not function
Often a significant difference between “peak” and “average” loading
Web Site Traffic Arrival
0
20
40
60
80
100
Time
Requests/Second
Average
Peak

6
Availability and Recovery RequirementsAvailability and Recovery Requirements
High availability and recovery requirements represent another class of
non-functional requirements (NFRs)
Define criteria for resiliency to various failures
This includes the user’s experience during certain classes of failures
Also the timeframe for recovery (seconds? Minutes? Hours? Days?)
Results in requirements that impact all areas of the system
Software, hardware, physical locale, personnel, etc.
Important to identify core needs before writing requirements
Traffic patterns and existing maintenance cycles may make extreme requirements unnecessary
In many cases, doing nothing is often a viable and reasonable option
Address availability and recovery their own initiatives
Integrated with the overall project plan, of course
Usually ongoing after the delivery of the initial software system

7
Building an Initial Capacity Plan for the WebSphere TierBuilding an Initial Capacity Plan for the WebSphere Tier
Obtain an initial capacity estimate during contract phase
Revisit the estimate during the design phase
Are the initial assumptions still applicable?
Is there a fundamental change in assumptions that could impact
the capacity required?
Plan for the capacity required for the purchase lifecycle of the systems
Growth projections
Failover requirements
Include test environments as well as the production systems
Minimum 25-50% of production infrastructure for performance verification
Other systems for function and integration testing
Remember the estimate is only as good as the data provided!
Also, remember to validate any estimates with testing prior to deployment

8
IBM Techline Sizing SupportIBM Techline Sizing Support
Techline provides software and server sizing recommendations
Across the entire IBM brand* portfolio
Covering a full range of server platforms
SOA Sizing Methodology and support developed by Techline with an Express
offering for SMB customers
Techline may be engaged by IBM Field Specialists and Business Partners to
provide remote Client support that includes:
Consultation throughout the sizing process
Detailed Client deliverable with projected resource requirements
Review of final recommendations
Techline provides leadership to the IBM Sizing Board and is an advocate in
driving sizing tool requirements
* Pending tooling for Rational Sizing

9
Capacity Planning
Project Planning
Development
Verification
Operations

10
Performance is not a point-in-time activity at end of project
Many project plans limit performance activities to end of cycle
Significant performance requirements unmanaged for the bulk of the product lifecycle
Successful projects manage performance throughout
Performance “swim-lane” added to project plans
• Coordinates performance with early phase activities
• Keeps performance visible on the project plan throughout
Performance design and implementation expertise engaged throughout
Regular activities to:
• Track key metrics against targets
• Prepare for verification phase(s) of the projects
• Develop resources, skills and processes for management of code, environments and
content
Validate performance criteria at each phase of the project
A performance test effort alone does not guarantee meeting Service Level Agreements
(Slaps)!
Project PlanningProject Planning
Managing for PerformanceManaging for Performance

11
Example of Lifecycle Performance ActivitiesExample of Lifecycle Performance Activities
Performance Activities
During Design
Performance Activities During
Coding and Unit Testing
Design
Development
Verification
Delivery
Performance Testing
Review and validate NFRs
with IT and Business
Assist with application
flow design
Performance Education
for Developers
Component design
reviews focused on
performance
System maintenance
strategies
Phase 1 of Performance
Validation begins: Developer
performed application
profiling
Identify initial cache strategy
Implementation of system
maintenance strategies
Build performance
verification environment
(including data)
Finalize Performance
Testing Plan
Prioritization and risk
mitigation planning with
Business
Performance Test Prep Steps:
Script creation, test data
creation, environment
validation, etc
Multiple phased test approach
applied
Single system and scale-up
testing
Endurance runs
Availability testing at end of
this phase
Prioritization and risk
mitigation planning with
Business
Launch and Post-
Launch Performance
Activities
Migration of tuned
settings to production
Production system
monitoring &
troubleshooting
Analysis of production
access logs >> advise
on script rework
Performance testing of
initial round of post-
launch fixes
On-going interlock with
Business to understand
future plans

12
Key Performance RolesKey Performance Roles
Develops & Maintains
Workload
Scripts
Test data reporting
Code profiling analysis &
recommendations
Analysis of results, debug &
tuning of middleware
applications
Guidance on performance
test design
Leader for Performance team
Overall leadership in performance aspects of
solution architecture & design
Ownership of Performance Related NFRs
Leads Performance Verification Activities
Performance Analysis &
Recommendations for:
SQL queries
Database indexes
Database configuration
Etc.
Overall coordination of performance engagement
activities
Reporting of results
Drives issue resolution
Links performance with PM for overall project
Cache Design &
Implementation
Performance Architect Project Manager
Performance Specialist --
Test & Script
Performance Specialist –
Middleware performance
Database
Caching (optional)

13
Engaged throughout project lifecycle
Engaged during the Verification phase
Performance StaffingPerformance Staffing
1. Design
2.Development
3. Verification
4.Delivery
Product
SMEs
Performance
Lead
Senior Project
Architect
Performance
Analysis
Script
Development
Test Execution &
Data Collection

14
Performance and Iterative DevelopmentPerformance and Iterative Development
Agile and other iterative models
Deep slices of functionality delivered at frequent intervals
Stakeholder approval sought at each delivery point
Include performance criteria as part of approval process
Delivery not final until functionality proven to perform
Exercising iterative model requires:
Building a test environment early in anticipation of early deliveries
Bringing scripter's and other performance staff on-board early
Regression coverage between iterations
Maintaining development coverage until final testing complete

15
Capacity Planning
Project Planning
Development
Verification
Operations

16
Architecture and DesignArchitecture and Design
Performance ConsiderationsPerformance Considerations
Providing functionality while meeting operational objectives
Application considerations
Efficient implementation of use cases
Identifying potential caching opportunities for future exploitation
Selecting technologies appropriate to the requirements
Topology considerations
Balancing “enterprise flexibility” with deployment reality
Identifying potential dataflow bottlenecks
Resiliency considerations
Streamlining core function
Isolating areas of instability or potential unreliability

17
Capacity Planning
Project Planning
Development
Verification
Operations

18
DevelopmentDevelopment
Common Application IssuesCommon Application Issues
Concurrency/Throughput Prohibitors
Coding techniques may keep the system from full concurrency
• Synchronized code blocks
− Note that Java libraries can contain synchronised code blocks too!
• Long code paths
• Build log messages only if logging is enabled
Efficiency Issues
Often tied to interaction with remote systems and database
• Consider using Prepared Statements instead of Statements when using JDBC for database
access
• Keep the size of the ResultSet as small as possible
− Only return the data you need from the database!
• Work with Database Administrators to ensure well formed SQL statements
Memory Issues
Memory budget for each transaction/user extremely high
Only create HttpSessions when you need them
Handling XML documents in Java can require a lot of memory
• XML parsing and applying XSL Transformations can be expensive in Java
• Consider using an XML appliance such as IBM DataPower for XSL Transformations
− In some cases the browser can be used to handle XSL Transformations

19
HttpSession ManagementHttpSession Management
HttpSession objects hold a user’s state information between visits
Session data is stored in the Java heap, together with application objects
Large HttpSessions impact scalability
Require more Java heap and potentially a larger heap size
• Result in more complex and expensive deployments (clustering and tuning)
Impact on HttpSession persistence
• Serialisation overhead increases with the size of the session object
Best Practices
Keep session data minimal per user
Proactively manage sessions
• Remove data as it becomes unneeded
• Use the session.invalidate() method to remove HttpSessions after logouts, etc.
Use transient keyword to mark data you do not need to persist
Only return updated data values to the session
Avoid using cyclic data graphs and non-serialisable items for persistent sessions

20
EJBsEJBs
Use the façade bean pattern
Stateless Session Bean to interact with Entity Beans
Results in a single transaction when the beans are running in the same application
server
Avoid extremely fine-grained EJB models
Extreme scenarios with hundreds of EJBs created for each request
Redesign this layer to accommodate a more granular model
• Use tooling to build a reasonable model
Proactively manage Stateful Session Beans
Use the ejb.remove() method when no longer needed
Avoid passivation
Memory implications across the cluster with new failover feature
• Similar to managing HttpSessions

21
JMSJMS
JMS messaging runs in-process in WebSphere 6.0 and higher
Reduces messaging overhead for light-weight messaging
Non-persistent messaging requires no external process interaction
• Reduces overhead as Java objects do not have to be serialised
Correctness always the most important consideration
• Use transactional support for multiple, inter-dependent messages
• Otherwise, non-transactional messaging (default) more efficient
Use light-weight acknowledgement, if possible
• DUPS_OK_ACKNOWLEDGE least expensive/greatest risk of re-delivery,
duplicates
• AUTO_ACKNOWLEDGE most expensive/no duplicate messages
Cache JMS objects that are thread-safe
Queue, QueueConnection, QueueConnectionFactory

22
Web ServicesWeb Services
Enables communication between heterogeneous applications
For example integrating IBM WebShere with Microsoft ASP.NET
Not a communication mechanism for application layers within a JVM
Use normal application protocols for better performance, fewer objects
Add distributed communications logic only if you distribute
General Web Services tips
Keep documents small and simple
• Large, complex documents increase parsing and serialization
• Also, avoid deeply nested payloads
Keep metadata vs. data ratio small
RMI/IIOP vs. Web Services
Performance of some Web Services payloads approaches RMI/IIOP performance

23
WebSphere Dynamic Cache ServiceWebSphere Dynamic Cache Service
Consider using the WebSphere Dynamic Cache Service (DCS) during application design
Part of design rather than retrofitting later
Organize page layout and design for DCS policy based caching
Servlet/JSP caching to hold repetitive, expensive page elements
Stateful elements vs. general use elements
Consider using DCS API based caching
Leverage DCS within the application
Distributed Map API usually most obvious choice
Leverage new caching structure
Multiple internal caches now supported
Split objects into different caches to more efficiently distribute cache data
Consider WebSphere Extreme Scale for applications that make extensive use of caching
Can provide significant benefit over DCS for certain scenarios
More details in WSI35 - WebSphere Extreme Scale Customer Scenarios and Use Cases

24
Capacity Planning
Project Planning
Development
Verification
Operations

25
VerificationVerification
Four Phases of Performance TestingFour Phases of Performance Testing
Phase 1
Single User
Testing
• Detailed code analysis
• Pathlength
• Memory footprint
• Basic SQL structure
• Application architecture
Phase 2
Single System
Concurrency
Testing
• Single system
• Drive full workload mix
• Iterative test and fix/tune
• Establish tuned reference
Phase 3
Incremental
Scale
Testing
• Incrementally grow farm
• Apply single sys. Tuning
• Iterative fix/tune
• Failover
• Long runs
Phase 4
Stability
Testing

26
Phase 1Phase 1 –– Single User TestingSingle User Testing
Pathlength Analysis
Limited runtime view of the application
IBM Rational Application Developer or other code profiling tool
Useful for:
• Identifying inefficiencies, some locking and memory problems
Not useful for:
• Subtle locking and memory problems, resource consumption issues, etc.
Static Analysis
Code reviews, performance “lint” tools, pattern identifiers, etc.
Static view of code
Useful for:
• Identifying anti-patterns, poor programming practices
Not good for:
• Identifying concurrency issues, memory issues, etc.

27
Phase 2Phase 2 –– Iterative Testing and TuningIterative Testing and Tuning
Concurrency testing
Testing under representative
loading
Custom-crafted using load testing
tool
Essential for understanding
application behavior during
production
Useful for finding:
• Subtle, large-scale problems
and concurrency/resource
issues
Establish maximum throughput
Maximum throughput typically
reaches a plateau before falling
Response times and number of
users should be captured as well
Throughput(tr/s) Concurrent Users
Throughput plateau
Light Load Zone Heavy Load Zone Buckle Zone

28
Phase 2Phase 2 –– Iterative Testing and TuningIterative Testing and Tuning
1. Test
Sufficient duration
Measure at steady-state
2. Observe
Look at all systems, logs, etc
Record results!
3. Minimal adjustment
Plan for next test and make adjustments
Limit change between runs!
1. Test
2.Observe
3.A
djust
First establish a baseline
What the system does now
Essential for measuring improvement

29
Entrance CriteriaEntrance Criteria
Align functional testing exit criteria with performance entrance criteria
Criteria to consider:
No outstanding high-severity problems
Modest quantity of mid-severity problems
Decreasing rate of new problem discovery
• Also an overall metric of stability
• Discourages developers from re-categorizing severities to meet a deadline
Impact to performance scripts from flow/layout changes understood
Clean, production-level logs
Develop a regression test for iterative drops/bug fixes
Drops should not break existing functionality
New code should operate near the performance of old code

30
Exit CriteriaExit Criteria
Performance requirements satisfied
Solution meets non-functional requirements (NFRs)
Infrastructure demonstrates scalability
Hardware scales to capacity plan
• Predicted loads execute within CPU budget
• System can make full use of CPU available
• Applications operate within memory budget
Bottlenecks identified and understood
System is stable
Repeatable, predictable behavior
No pathological indicators (paging, etc.)
System sustains performance over operational period
Clean logs

31
Capacity Planning
Project Planning
Development
Verification
Operations

32
OperationsOperations
Crossing Organisation SilosCrossing Organisation Silos
Systems Networks Software
Data
Management
Custom
Application
Performance VerificatiPerformance Verification/Testing/Tuning/Analon/Testing/Tuning/Analysisysis
System management
Crosses traditional I/T silos
More difficult to manage/support
Help team manage and identify
Manage areas under their control
Identify issues in components outside their control

33
Moving to DeploymentMoving to Deployment
Move tuning settings to production
Apply optimal settings to the production environment
• WebSphere/J2EE configuration settings
• Tuned application
• OS settings
• Updates to other systems (databases, WebSphere MQ, etc)
Implies compatibility between test and production
• Hardware and software versions
Use test “normals” for initial monitoring settings
CPU, I/O, network traffic, logs, etc.
Use metrics to describe abnormal situations for monitoring
Iterative process
• Re-evaluate after first weeks of live deployment
• Develop more accurate monitoring settings based on live traffic patterns

34
Other ConsiderationsOther Considerations
Limited rollout
Reduces risk for new sites
Validate assumptions about use cases and workload mix
Difficult to setup for public sites
Performance Test Environment
After deployment, maintain a test environment for the site
Similar in hardware and design as the production site
Useful for
• Testing updates, fixes, and new application releases/features
• Recreating production problems and troubleshooting
Troubleshooting in the live site problematic
• Tracing and iterative debug often intrusive
• Test environment supports debug without impacting user experience

35
SummarySummary
Successful projects focus on performance early and continuously
Designed, written, and tested for performance
Optimizations considered and exploited throughout the lifecycle
Cross-organizational approach taken
Verification requires planning, skill, and time
Just two weeks at the end of a major project is insufficient to ensure successful
launch
Larger projects require more testing
• Scale up testing
• Validation of backend components
Team must cross organisational silos
• Tuning not only for web and application tier, but also for other systems
Use historical information to predict new site behavior
Confirm with testing and limited roll-out, if possible

36
ReferencesReferences
Kyle Brown and Keys Botzum: Improving HttpSession Performance with Smart
Serialization
http://www.ibm.com/developerworks/websphere/library/bestpractices/httpsession_perfor
mance_serialization.html
Many thanks to Stacy Joines for her contributions to this presentation.

37
Other SessionsOther Sessions
WSI33 – Advanced Performance Tactics for IBM WebSphere Application Server
WSI35 – WebSphere Extreme Scale Customer Scenarios and Use Cases

38
Further ReadingFurther Reading
Performance Analysis for Java Web Sites
Joines, Willenborg, Hygh
IBM WebSphere Deployment and Advanced Configuration
Barcia, et al
IBM Web Sphere System Administration
Williamson, et al
WebSphere Application Server: Step by Step
Turaga, et al
Persistence in the Enterprise
Barcia, Hambrick, et al

IBM European WebSphere
Technical Conference
14 – 18 November 2008, Barcelona, Spain
Conference materials may not be reproduced in whole or in part without the prior written permission of IBM.
Questions ?Questions ?
Thank you for attending! Please complete your session evaluationThank you for attending! Please complete your session evaluation, s, session number is WSI32ession number is WSI32..

WSI32 - IBM WebSphere Performance Fundamentals

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to WSI32 - IBM WebSphere Performance Fundamentals

Similar to WSI32 - IBM WebSphere Performance Fundamentals (20)

More from Hendrik van Run

More from Hendrik van Run (19)

Recently uploaded

Recently uploaded (20)

WSI32 - IBM WebSphere Performance Fundamentals