My presentation (OUG Harmony 2012 in Tallin) on how to use SQL Plan baselines to assess the completeness of testing. I also have a white-paper on this topic, see the whitepapers section.

1. Using SQL Baselines for
Performance Testing
Maris Elsins
Oracle Applications DBA
30.05.2012

2. © 2012 Pythian
Few words about me...
• 10y  Oracle
• 3y – PL/SQL Developer
• 7y – Oracle [Apps] DBA
• Certificates:
• 10g OCM, 9i/10g/11g OCP,
• 11i Apps DBA OCP, 11i System Administrator OCE
• Employed by Pythian since July 2011
• Speaker at conferences: 5* , 3* , 3*
• How to find me?
• Blog – http://www.pythian.com/news/author/elsins/
• Earlier blog posts – http://appsdbalife.wordpress.com/
• LinkedIn – http://lv.linkedin.com/in/mariselsins
• Twitter – @MarisElsins
• Email – elsins (at) pythian.com
2

3. © 2012 Pythian
Why Companies Trust Pythian
Recognized Leader
Global industry-leader in remote database administration services and consulting
for Oracle, Oracle Applications, MySQL and SQL Server
Work with over 150 multinational companies such as Forbes.com, Fox Sports,
Nordion and Western Union to help manage their complex IT deployments
Expertise
One of the world’s largest concentrations of dedicated, full-time DBA expertise.
Employ 8 Oracle ACEs/ACE Directors.
Hold 7 Specializations under Oracle Platinum Partner program, including Oracle
Exadata, Oracle GoldenGate & Oracle RAC.
Global Reach & Scalability
24/7/365 global remote support for DBA and consulting, systems administration,
special projects or emergency response
3

4. © 2012 Pythian
Performance
SQL Baselines
Testing
4

5. © 2012 Pythian
PERFORMANCE
5

6. © 2012 Pythian
Performance
• Performance is important because users talk a lot about it
• Oracle is slow
• Application hangs
• The report was not ready by 8AM ... again
• DBAs don’t usually hear much about «good» performance
• How to know if the performance is «good»?
• 0 complaints? May be users are tired of reporting problems?
• 0 broken keyboards reported by IT Service staff?
• DB Server Load Average < 2 ?
What really is good or bad performance?
Or should we ask if the performance is sufficient?
6

7. © 2012 Pythian
Performance
• Response time of the application
• Jackob Nielsen «Response Time Overview»
• 0.1 second - Instant reaction of the system
• 1 second - Limit where the flow of though stays uninterrupted
• 10 seconds - Users gets the focus on something else
• Users can define what response times would be acceptable for the
application to consider it performing well
• Response time allows to define «Oracle is slow» and «Application
hangs» problems
• Produced results per unit of time
• Why «work / time» is not a good measure?
• Results – amount of processed units (orders, rows, etc)
• Users can define what amount of results need to be provided and in
what time it has to be completed
• Results/time allows to define the «The report was not ready by 8AM»
problem in detail
7

8. © 2012 Pythian
The goal of the Presentation
• Ensuring the sufficient performance is not easy
• Applications change
• Data grow
• HW changes
• Usage of applications change
• It’s a big chellange to make sure the performance does not
degrade
• Performance testing is complicated
• How to simulate production workload
• How to make sure everything is tested?
• Big issues are easily revealed by testing, what about smaller problems?
• Let’s find out how SQL Baselines can help
8

9. © 2012 Pythian
SQL BASELINES
9

10. © 2012 Pythian
Introduction to «SQL Plan Management»
• New feature introduced in 11gR1
• Ensures only verified execution plans are used
• Prevents performance regressions caused by execution plan changes
• Allows controled introduction of the new execution plans
• «Stored outlines on steroids»
• How it works?
• SPM evaluates and records SQL execution plans over time
• SPM creates SQL plan baselines – sets of plans known to perform well
• Only approved plans in SQL plan baselines are used for execution
• If SQL plan baseline does not exist execution plan provided by the
optimizer is used
10

11. © 2012 Pythian
Introduction to «SQL Plan Management»
• Typical use cases?
• Database upgrades to 11g
• Prevent SQL execution plan changes due to ongoing system and data
changes
• Transfer tested Execution plans from DEV DB to PROD
• Forcing execution plan for particular SQL without changing the
statement
• Availability of SPM
• Available in Enterprise Edition starting from 11gR1
• SPM does NOT require Diagnostic or Tuning pack licenses
• Capturing the SQL Plan Baselines
• Automatic - OPTIMIZER_CAPTURE_SQL_PLAN_BASELINES parameter
• Manual – use DBMS_SPM to load plans from SQL Tuning Set (STS) or
Cursor Cache
• STS has more options to load SQLs from, but requires Diag+Tuning licences
11

12. © 2012 Pythian
Introduction to «SQL Plan Management»
• Evolving the SQL Plan Baselines
• Evolving = process to verify the new execution plans
• DBMS_SPM.EVOLVE_SQL_PLAN_BASELINE
• Execute statement using the best accepted baseline and the questionable plan
• Compare resource usage
• Accept the new plan if it’s better then the current (threshold 1.43 - 1.76)
• SQL Tuning Advisor task that runs during maintenenace window
• Load new plans manually
• Purging of SQL Plan Baselines
• DBMS_SPM. DROP_SQL_PLAN_BASELINE
• Automatic
• plan_retention_weeks :=53 – deletes only unused execution plans
• space_budget_percent :=10 (SMB size in SYSAUX, gives warning in alert log)
12

13. © 2012 Pythian
SQL Baselines Demo 1
13
OE@TEST:36> CREATE TABLE test2 AS SELECT ROWNUM n, Lpad('a', 100, 'a')
v FROM dual CONNECT BY LEVEL <= 500000;
Table created.
OE@TEST:36> exec dbms_stats.gather_table_stats('OE', 'TEST2');
PL/SQL procedure successfully completed.
OE@TEST:36> ALTER SESSION SET optimizer_capture_sql_plan_baselines = TRUE;
Session altered.
OE@TEST:36> SELECT * FROM test2 WHERE n = 3453;
N V
---------- ----------------------------------------------------------------------------------------------------
3453 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
OE@TEST:36> /
N V
---------- ----------------------------------------------------------------------------------------------------
3453 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
OE@TEST:36> ALTER SESSION SET optimizer_capture_sql_plan_baselines = FALSE;
Session altered.
SYS@TEST:1> SELECT sql_text, sql_handle, plan_name, accepted FROM dba_sql_plan_baselines;
SQL_TEXT SQL_HANDLE PLAN_NAME ACC
---------------------------------------- ------------------------------ ------------------------------ ---
select * from test2 where n=3453 SQL_4024b7fb6cadf7db SQL_PLAN_4095rzdqavxyv99963deb YES

14. © 2012 Pythian
SQL Baselines Demo 2
14
OE@TEST:36> SELECT * FROM TABLE(dbms_xplan.Display_sql_plan_baseline('SQL_4024b7fb6cadf7db',
'SQL_PLAN_4095rzdqavxyv99963deb'));
PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
SQL handle: SQL_4024b7fb6cadf7db
SQL text: select * from test2 where n=3453
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
Plan name: SQL_PLAN_4095rzdqavxyv99963deb Plan id: 2576760299
Enabled: YES Fixed: NO Accepted: YES Origin: AUTO-CAPTURE
--------------------------------------------------------------------------------
Plan hash value: 300966803
---------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
---------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 106 | 2126 (1)| 00:00:26 |
|* 1 | TABLE ACCESS FULL| TEST2 | 1 | 106 | 2126 (1)| 00:00:26 |
---------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter("N"=3453)
24 rows selected.
OE@TEST:36> CREATE INDEX test2_ix ON test2(n);
Index created.

15. © 2012 Pythian
SQL Baselines Demo 3
15
OE@TEST:36> SET AUTOTRACE ON EXPL
OE@TEST:36> SELECT * FROM test2 WHERE n = 3453;
N V
---------- ----------------------------------------------------------------------------------------------------
3453 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
Execution Plan
----------------------------------------------------------
Plan hash value: 300966803
---------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
---------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 106 | 2126 (1)| 00:00:26 |
|* 1 | TABLE ACCESS FULL| TEST2 | 1 | 106 | 2126 (1)| 00:00:26 |
---------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter("N"=3453)
Note
-----
- SQL plan baseline "SQL_PLAN_4095rzdqavxyv99963deb" used for this statement
SYS@TEST:1> SELECT sql_text, sql_handle, plan_name, accepted FROM dba_sql_plan_baselines;
SQL_TEXT SQL_HANDLE PLAN_NAME ACC
---------------------------------------- ------------------------------ ------------------------------ ---
select * from test2 where n=3453 SQL_4024b7fb6cadf7db SQL_PLAN_4095rzdqavxyv99963deb YES
select * from test2 where n=3453 SQL_4024b7fb6cadf7db SQL_PLAN_4095rzdqavxyvb4aa7de4 NO

16. © 2012 Pythian
SQL Baselines Demo 4
16
SYS@TEST:1> var rep clob
SYS@TEST:1> exec :rep := dbms_spm.evolve_sql_plan_baseline(sql_handle =>
'SQL_4024b7fb6cadf7db');
PL/SQL procedure successfully completed.
SYS@TEST:1> print :rep
Inputs:
-------
SQL_HANDLE = SQL_4024b7fb6cadf7db
PLAN_NAME =
TIME_LIMIT = DBMS_SPM.AUTO_LIMIT
VERIFY = YES
COMMIT = YES
Plan: SQL_PLAN_4095rzdqavxyvb4aa7de4
------------------------------------
Plan was verified: Time used .49 seconds.
Plan passed performance criterion: 1926.48 times better than baseline plan.
Plan was changed to an accepted plan.
Baseline Plan Test Plan Stats Ratio
------------- --------- -----------
Execution Status: COMPLETE COMPLETE
Rows Processed: 1 1
Elapsed Time(ms): 29.928 .05 598.56
CPU Time(ms): 29.335 0
Buffer Gets: 7703 4 1925.75
Physical Read Requests: 0 0
Physical Write Requests: 0 0
Physical Read Bytes: 0 0
Physical Write Bytes: 0 0
Executions: 1 1
-------------------------------------------------------------------------------
...

17. © 2012 Pythian
SQL Baselines Demo 5
17
OE@TEST:36> SELECT * FROM test2 WHERE n = 3453;
N V
---------- ----------------------------------------------------------------------------------------------------
3453 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
Execution Plan
----------------------------------------------------------
Plan hash value: 430623757
----------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
----------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 106 | 4 (0)| 00:00:01 |
| 1 | TABLE ACCESS BY INDEX ROWID| TEST2 | 1 | 106 | 4 (0)| 00:00:01 |
|* 2 | INDEX RANGE SCAN | TEST2_IX | 1 | | 3 (0)| 00:00:01 |
----------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - access("N"=3453)
Note
-----
- SQL plan baseline "SQL_PLAN_4095rzdqavxyvb4aa7de4" used for this statement

18. © 2012 Pythian
TESTING
18

19. © 2012 Pythian
What do we have?
19
SYS@TEST:1> desc DBA_SQL_PLAN_BASELINES –- 11.2 version, 11.1 does not have «REPRODUCED» column
Name Null? Type
----------------------- -------- ----------------
SIGNATURE NOT NULL NUMBER -– SQL Baseline identifier
SQL_HANDLE NOT NULL VARCHAR2(30) -– dec_to_hex(SIGNATURE)
SQL_TEXT NOT NULL CLOB
PLAN_NAME NOT NULL VARCHAR2(30) –- «plans» (hints) are stored in SQLOBJ$DATA
CREATOR VARCHAR2(30)
ORIGIN VARCHAR2(14)
PARSING_SCHEMA_NAME VARCHAR2(30)
DESCRIPTION VARCHAR2(500)
VERSION VARCHAR2(64)
CREATED NOT NULL TIMESTAMP(6)
LAST_MODIFIED TIMESTAMP(6)
LAST_EXECUTED TIMESTAMP(6) –- This is not exactly what we think it is (6.5 days)
LAST_VERIFIED TIMESTAMP(6) –- Last time DBMS_SPM.EVOLVE_SQL_PLAN_BASELINE verified it
ENABLED VARCHAR2(3)
ACCEPTED VARCHAR2(3) -- only ACCEPTED baselines are used by the optimizer
FIXED VARCHAR2(3)
REPRODUCED VARCHAR2(3) -- New in 11gR2. NO = plan is not valid (e.g. indexes missing)
AUTOPURGE VARCHAR2(3)
OPTIMIZER_COST NUMBER -- potentially very misleading value, CBO does not use it
MODULE VARCHAR2(64) -- it’s so much easier if the application is instrumented
ACTION VARCHAR2(64) -- it’s so much easier if the application is instrumented
EXECUTIONS NUMBER -- not available or not up to date
ELAPSED_TIME NUMBER -- not available or not up to date
CPU_TIME NUMBER -- not available or not up to date
BUFFER_GETS NUMBER -- not available or not up to date
DISK_READS NUMBER -- not available or not up to date
DIRECT_WRITES NUMBER -- not available or not up to date
ROWS_PROCESSED NUMBER -- not available or not up to date
FETCHES NUMBER -- not available or not up to date
END_OF_FETCH_COUNT NUMBER -- not available or not up to date

20. © 2012 Pythian
What do we have?
• For each SQL baseline we have:
• SQL Statements and execution plans
• Creation and Last execution dates
• Other properties like ACCEPTED, REPRODUCED, ENABLED
• We don’t have:
• Any statistics stored in baselines (elapsed/cpu time, Logical/Pysical IO
counts, ...)
• Any reliable information about cost of the SQL statement
So how does it help to test the performance?
• What is the typical involvement of a DBA in testing?
• SPM allows DBAs to have higher control over testing
• Assess the completeness of testing (are all modules tested?)
• Assess the impact of schema changes (e.g. dropping/creating an index)
• Impact of application change (identify new SQLs)
20

21. © 2012 Pythian
Main steps for performance testing
using SQL baselines
• Capture SQL Baselines in PROD DB
• Prepare the TEST DB for performance testing
• Import SQL Baselines from PROD into TEST DB
• Calibrate the TEST DB
• Make the planned changes in TEST DB
• Run preformance tests in TEST
• Compare states of SQL baselines before and after the
testing to find:
• SQLs not executed during the test – LAST_EXECUTED, SIGNATURE
• new SQLs executed during the testing – CREATED, SIGNATURE
• Non-reproducible execution plans (11gR2)– REPRODUCED, SIGNATURE, PLAN_NAME
• new execution plans found – CREATE, SIGNATURE, PLAN_NAME
21

22. © 2012 Pythian
Capture SQL Baselines in PROD DB
• Baselines have to be created for all queries for this to work
• Which way of capturing SQL Baselines to choose?
• optimizer_capture_sql_plan_baselines parameter
• + Catches most of the execution plans
• + Can be set at session level, consider a logon trigger
• – Ongoing performance overhead
• – Serious performance overhead if Bug 9910484 not fixed (before
11.2.0.3)
• – can exhaust SYSAUX tablespace quickly
• Frequent capture of execution plans from cursor cache
• + Basic filtering available (schema, module, action, sql text)
• + overhead can be «forecasted» (at least when it strikes)
• + More control on space usage in SYSAUX
• – Might not capture some queries
• – Need to be done manually or need to be automated by yourself.
22

23. © 2012 Pythian
Capture SQL Baselines in PROD DB
• Problems with the capture process
• If you’re not using bind variables in your OLTP application
• Do you care about the performance at all?
• SQL Baselines are useless
• SQL Baselines will clog your SYSAUX tablespace
• will cause more performance overhead
• 2 Executions !!!
• Quite a few bugs in SPM
• 11.1.0.6 – First release with SPM
• 0 bugs fixed in 11.1.0.7
• 7 bugs fixed in 11.2.0.2
• 13 bugs fixed in 11.2.0.3 (including the 9910484)
• Not usable for DWH applications as bind variables are
typically not used
23

24. © 2012 Pythian
Prepare the TEST DB for performance testing
• Quality of testing depends on how well your test
environment represents the production
• Test DBs differ from PROD DBs
• hardware
• software versions - RDBMS version and patches
• statistics and optimizer parameters
• Object definitions
• changes to system settings
• SQL profiles
• data that is stored in tables
• Most of the mentioned things can impact optimizer's
calculations
These differences can make your test results unreliable
24

25. © 2012 Pythian
Prepare the TEST DB for performance testing
• Some problems arise because
• testers and developers want to max performance on the DEV/TEST DBs
• DBAs support them and «tune» the DBs making different from PROD
• These differences make performance testing harder and gives room for
surprises «after going live»
• Things you can do:
• clone TEST from PROD as late as possible
• don't tune TEST DB unless same changes are planned in PROD
• don't gather statistics in TEST, import them regularly from PROD
• import baselines from PROD shortly before testing
• Differences are inevitable
25

26. © 2012 Pythian
Import SQL Baselines from
PROD DB into TEST DB
• Simple process
• Pack baselines into a staging table
• Export from PROD, import into TEST
• Unpack Baselines into TEST
• «6.5 days» problem with LAST_EXECUTED column
• Bug 8864741 : last_executed date in dba_sql_plan_baseline is not
getting updated. All versions are affected.
• LAST_EXECUTED date is updated once every ~6.5 days,
• Oracle’s response, it would be too expensive to update it every time
• We need to track usage of all plans, accurate data is important
• Workaround: update LAST_EXECUTED to NULL before importing the
baselines in TEST
26

27. © 2012 Pythian
DEMO – Importing the baselines
27
/* EXPORT FROM PROD */
/* Create staging table */
SQL> exec DBMS_SPM.CREATE_STGTAB_BASELINE(table_name=>'STAGE_BASELINES',table_owner=>'SYSTEM');
/* Pack baselines into the staging table*/
SQL> var n number
SQL> exec :n
:=DBMS_SPM.PACK_STGTAB_BASELINE(table_name=>'STAGE_BASELINES',table_owner=>'SYSTEM');
SQL> print n
N
----------
42
/* Export the staging table */
SQL> create directory BASELINES_EXPORT as '/tmp/';
SQL> !expdp tables=SYSTEM.STAGE_BASELINES directory=BASELINES_EXPORT
dumpfile=STAGE_BASELINES.dmp logfile=expdpSTAGE_BASELINES.log
SQL> drop directory BASELINES_EXPORT;
/* IMPORT INTO TEST */
/* Import the staging table */
SQL> create directory BASELINES_IMPORT as '/tmp/';
SQL> !impdp tables=SYSTEM.STAGE_BASELINES directory=BASELINES_IMPORT
dumpfile=STAGE_BASELINES.dmp logfile=impdpSTAGE_BASELINES.log
SQL> drop directory BASELINES_IMPORT;

28. © 2012 Pythian
DEMO – Importing the baselines
28
/* Drop all current baselines*/
declare
n number;
cursor c1 is select distinct sql_handle from dba_sql_plan_baselines;
begin
for c in c1 loop
n:=DBMS_SPM.DROP_SQL_PLAN_BASELINE(c.sql_handle);
end loop;
end;
/* reset the LAST_EXECUTE date to NULL*/
SQL> update system.STAGE_BASELINES set last_executed=null;
SQL> commit;
/*Import the baselines*/
SQL> var n number
SQL> exec :n
:=DBMS_SPM.UNPACK_STGTAB_BASELINE(table_name=>'STAGE_BASELINES',table_owner=>'SYSTEM');
SQL> print n
N
----------
42
/*Check the baselines were imported*/
SQL> select count(*) from DBA_SQL_PLAN_BASELINES;
COUNT(*)
----------
42

29. © 2012 Pythian
Calibration
• The purpose of Calibration
• To assess the quality of the performance test (completeness)
• To verify how differences between PROD and TEST affect the queries.
• To distinguish changes introduced by configuration changes on TEST
DB from changes introduced planned to test
• The calibration process
• The planned changes should not be installed yet
• Save the state of the SQL Baselines before you start
• Execute full performance tests
• Collect data after testing and assess the results
• Prepare SQL Baselines for final performance testing
• Export into staging table
• Update LAST_EXECUTED again to NULL
• Import all baselines from staging table
• Save current state of the SQL baselines
29

30. © 2012 Pythian
DEMO – Calibration
30
/* Capture the state of SQL Baselines before calibration test */
SQL> create table SPM_PRE_CALIB as select * from DBA_SQL_PLAN_BASELINES;
SQL> create index SPM_PRE_CALIB_I1 on system.SPM_PRE_CALIB(signature);
SQL> exec dbms_stats.gather_table_stats(user,'SPM_PRE_CALIB', cascade=>true);
/* Make sure the baselines are captured and used */
SQL> alter system set optimizer_capture_sql_plan_baselines=true;
SQL> alter system set optimizer_use_sql_plan_baselines=true;
/* !!! RUN THE PERFORMANCE TESTS !!! */
/* No need to collect baselines while you’re checking the results */
SYS> alter system set optimizer_capture_sql_plan_baselines=false;
/* !!! COLLECT THE DATA, I WILL SHOW SAMPLES LATER !!! */
/* Prepare SQL baselines (reset the LAST_EXECUTED) */
SQL> exec DBMS_SPM.CREATE_STGTAB_BASELINE(table_name=>'STAGE_BASE2',table_owner=>'SYSTEM');
SQL> exec :n:= DBMS_SPM.PACK_STGTAB_BASELINE(table_name=>'STAGE_BASE2',table_owner=>'SYSTEM');
SQL> update system.STAGE_BASE2 set last_executed=null;
SQL> commit;
SQL> exec :n:= DBMS_SPM.UNPACK_STGTAB_BASELINE(table_name=>'STAGE_BASE2',table_owner=>'SYSTEM');
/*Save the satae of the baselines again*/
SQL> create table SPM_POST_CALIB as select * from DBA_SQL_PLAN_BASELINES;
SQL> create index SPM_POST_CALIB_I1 on system.SPM_POST_CALIB(signature);
SQL> exec dbms_stats.gather_table_stats(user,'SPM_POST_CALIB', cascade=>true);

31. © 2012 Pythian
Run the Final performance tests
• Implement the planned changes
• DB side (change schema, parameters, statistics, configuration, ...)
• Application side (new modules, fixes to code, ... )
• Make sure baselines are used and capture is on
• Use any tools you have
• Load testing tools
• Automated testcases
• Manual testing
• ...
• Considerations
• Aim for full functionality testing
• Remember! Run every test twice!
• Creating a PROD-like workload is NOT required for this method to work
31

32. © 2012 Pythian
DEMO – Final performance testing
32
/* Implement the changes*/
SQL> DROP INDEX tpcc.orders_i1;
SQL> CREATE INDEX tpcc.c_fn ON tpcc.customer(Upper(c_first));
SQL> ALTER TABLE orders ADD (ocn NUMBER);
SQL> CREATE OR replace TRIGGER tpcc.cust_ord_counter
BEFORE INSERT ON tpcc.orders FOR EACH ROW
BEGIN
SELECT Count(*)
INTO :new.ocn
FROM tpcc.orders
WHERE o_c_id = :new.o_c_id;
END;
/
/* Make sure the baselines are captured and used */
SQL> ALTER SYSTEM SET optimizer_capture_sql_plan_baselines=TRUE;
SQL> ALTER SYSTEM SET optimizer_use_sql_plan_baselines=TRUE;
/* !!! RUN THE PERFORMANCE TESTS !!! */
/* No need to collect baselines while you’re checking the results */
SYS> ALTER SYSTEM SET optimizer_capture_sql_plan_baselines=FALSE;
/* !!! COLLECT THE DATA !!! */

33. © 2012 Pythian
RESULTS
COLLECT THE DATA
33

34. © 2012 Pythian
SQLs not executed during the test
34
SELECT DISTINCT sql_handle
,dbms_lob.Substr(sql_text, 100, 1) sql_text
--,module
--,action
FROM spm_post_calib c
WHERE c.enabled = 'YES'
AND c.parsing_schema_name = 'TPCC'
AND NOT EXISTS (SELECT 1
FROM dba_sql_plan_baselines b
WHERE b.signature = c.signature
AND b.last_executed IS NOT NULL)
AND NOT EXISTS (SELECT 1
FROM dba_sql_plan_baselines b
WHERE C.signature = B.signature
AND b.created > c.created);
SQL_HANDLE SQL_TEXT
------------------------------ --------------------------------------------------------------
SQL_fc5efaa8ffabe508 SELECT I_PRICE, I_NAME, I_DATA FROM ITEM WHERE I_ID = :B1

35. © 2012 Pythian
new SQLs executed during the testing
35
SELECT DISTINCT sql_handle
,dbms_lob.Substr(sql_text, 100, 1) sql_text
--,module
--,action
FROM dba_sql_plan_baselines b
WHERE enabled = 'YES'
AND parsing_schema_name = 'TPCC'
AND NOT EXISTS (SELECT 1
FROM SYSTEM.spm_post_calib C
WHERE C.signature = B.signature
AND C.enabled = 'YES');
SQL_HANDLE SQL_TEXT
------------------------------ ---------------------------------------------------------------
SQL_141e8b480198d134 SELECT COUNT(*) FROM TPCC.ORDERS WHERE O_C_ID=:B1

36. © 2012 Pythian
Non-reproducible execution plans
36
SELECT sql_handle, plan_name,
dbms_lob.Substr(sql_text, 100, 1) sql_text
FROM dba_sql_plan_baselines b
WHERE enabled = 'YES'
AND reproduced = 'NO'
AND parsing_schema_name = 'TPCC'
INTERSECT
SELECT sql_handle,plan_name,
dbms_lob.Substr(sql_text, 100, 1) sql_text
FROM spm_post_calib c
WHERE enabled = 'YES'
AND reproduced = 'YES'
AND parsing_schema_name = 'TPCC';
SQL_HANDLE PLAN_NAME SQL_TEXT
------------------------------ ------------------------------ ----------------------------------
SQL_1d3eb12408a63da1 SQL_PLAN_1ugpj4h4acgd12e067175 UPDATE ORDERS SET O_CARRIER_ID =
:B4 WHERE O_ID = :B3 AND O_D_ID =
:B2 AND O_W_ID = :B1
SQL_f7db40080b18fe6a SQL_PLAN_ggqu0105jjzma6d5a2ea5 SELECT O_C_ID FROM ORDERS WHERE
O_ID = :B3 AND O_D_ID = :B2 AND
O_W_ID = :B1

37. © 2012 Pythian
New execution plans found –
candidates for evolving
37
SELECT sql_handle, plan_name, dbms_lob.Substr(sql_text, 100, 1) sql_text
FROM dba_sql_plan_baselines b
WHERE enabled = 'YES‘ AND parsing_schema_name = 'TPCC'
AND EXISTS (SELECT 1
FROM SYSTEM.spm_post_calib C
WHERE C.signature = B.signature
AND C.enabled = 'YES')
AND NOT EXISTS (SELECT 1
FROM SYSTEM.spm_post_calib C
WHERE C.signature = B.signature
AND C.plan_name = B.plan_name
AND C.enabled = 'YES');
SQL_HANDLE PLAN_NAME SQL_TEXT
------------------------------ ------------------------------ ----------------------------------
SQL_1d3eb12408a63da1 SQL_PLAN_1ugpj4h4acgd12ce83b16 UPDATE ORDERS SET O_CARRIER_ID =
:B4 WHERE O_ID = :B3 AND
O_D_ID = :B2 AND O_W_ID = :B1
SQL_8659e8b608cffc3c SQL_PLAN_8cqg8qs4czz1wb1714af0 SELECT COUNT(*) FROM TPCC.CUSTOMER
WHERE UPPER(C_FIRST) LIKE
:B1 ||'%'
SQL_f7db40080b18fe6a SQL_PLAN_ggqu0105jjzma4bfb13fc SELECT O_C_ID FROM ORDERS WHERE
O_ID = :B3 AND O_D_ID = :B2
AND O_W_ID = :B1

38. © 2012 Pythian
RESULTS
ASSESS THE RESULTS
38

39. © 2012 Pythian
Completeness of the performance test
• Assessing the completeness of testing is usually not easy
• You have
• SQLs coming from PROD stored in SQL Baselines
• You know which of them are not executed during testing
• You can
• Identify module/action which the untested SQLs belong to
• Provide information to developers / testers about it
• Probably it was a planned application change
• Probably the functionality was not tested
• Take a justified decision on the next steps
39

40. © 2012 Pythian
The Impact of schema changes
• Find the queries which have new new execution plans
recorded in the SQL baselines
• Use DBMS_XPLAN.DISPLAY_SQL_PLAN_BASELINE to check the new
execution plan
• Evolve the baseline to activate it if it provides better performance
• Import the new baselines in production with the implementation of the
change so the best execution plan can be used immediately
• Find the baselines which are not reproducible after the
change
• Be careful with DBMS_XPLAN.DISPLAY_SQL_PLAN_BASELINE
• It does not show the plan for non-reproducible SQL baselines
• It shows best available execution plan instead
• You might find the original execution plan in AWR tables
• You will find the original execution plan in Production DB
40

41. © 2012 Pythian
The Impact of application changes
• Identify new SQL statements
• Easy to find and view all new execution plans
• Much harder to find real execution statistics
• V$SQL.EXACT_MATCHING_SIGNATURE =
DBA_SQL_PLAN_BASELINES.SIGNATURE
• sqlobj$.PLAN_ID = PLAN_HASH_VALUE, look up AWR or STATSPACK
tables for PLAN_HASH_VALUE
41

42. © 2012 Pythian
Summary
• DBAs are rarely actively involved in performance testing
• DBAs are usually ones who get called first when there are
performance issues
• Using SQL Baselines allows to involve DBAs in performance
testing more actively
• DBAs can take part in testing
• DBAs can see the scope and completeness of testing
• DBAs can partially assess the impact of the changes before they are
implemented in production
• DBAs can alert developers about issues before they happen
If this does not sound good enough
use the SQL Plan Management
at least for what it was designed
It’s absolutely worth it!
42

43. © 2012 Pythian
?
Tweet about the event: @MarisElsins and #ough2012
Give your feedback!
Ask more questions: elsins@pythian.com, @MarisElsins
Blog posts on this topic will follow: http://www.pythian.com/news/author/elsins/
Follow Pythian on twitter @pythian and LinkedIn http://linkd.in/pythian
43