Title: PostgreSQL Schema Changes with Minimal Downtime using pg_osc Speaker: Aakash M, Mydbops Event: PGConf India, 2024 Description: This presentation explores pg_osc, a tool that enables efficient schema changes in PostgreSQL tables with minimal downtime and locking. It addresses the challenges of traditional ALTER statements and provides a smoother alternative. Key points covered: • Introduction to pg_osc and its benefits. • Limitations of ALTER statements and how pg_osc overcomes them. • Step-by-step explanation of the pg_osc process. • Prominent features and considerations for using pg_osc. • References and resources for further exploration. Target Audience: • Database administrators • Developers working with PostgreSQL • Anyone interested in optimizing schema changes This presentation provides valuable insights for anyone seeking to streamline schema modifications in PostgreSQL while minimizing disruptions.

1. PostgreSQL Schema Changes with
pg_osc
PGConf India, 29th Feb 2024
- Aakash M

2. About Us

3. Our Services
Focus on MySQL, MongoDB, PostgreSQL, TiDB, Cassandra
Consulting
Services
Consulting
Services
Managed
Support
Services
24*7
DBA Team
Targeted
Engagement

4. What is pg_osc ?
● Tool for making schema changes
● Minimal lock and zero downtime
● Similar to pt_osc (MySQL) and pg_repack
● Not an extension

5. Problems with Alter
● ALTER - Requires locks, downtime
● CDC - Complex to use

6. Requirement to Use
● PostgreSQL version >= 9.6
● DB User with trigger access or a Superuser

7. Installation
● Using gem command
● Docker Installation

8. How does it work ?
1. Create an Audit table
Audit table

9. 1. Create an Audit table
2. ACCESS EXCLUSIVE lock to add a trigger
C1 C2 C3
Primary table
How does it work ?

10. 1. Create an Audit table
2. ACCESS EXCLUSIVE lock to add a trigger
3. Create a new shadow table
C1 C2 C3
C1 C2 C3
Primary table
Shadow table
How does it work ?

11. 1. Create an Audit table
2. ACCESS EXCLUSIVE lock to add a trigger
3. Create a new shadow table
4. Execute ALTER on shadow table
C1 C2 C3 C4
ALTER TABLE SHADOW
TABLE ADD COLUMN C4 ;
Shadow table
How does it work ?

12. 1. Create an Audit table
2. ACCESS EXCLUSIVE lock to add a trigger
3. Create a new shadow table
4. Execute ALTER on shadow table
5. Copy Existing Data
C1 C2 C3
C1 C2 C3 C4
Shadow table
Primary table
How does it work ?

13. 1. Create an Audit table
2. ACCESS EXCLUSIVE lock to add a trigger
3. Create a new shadow table
4. Execute ALTER on shadow table
5. Copy Existing Data
6. Create indexes
C1 C2 C3 C4
Shadow table
CREATING INDEXES
How does it work ?

14. 1. Create an Audit table
2. ACCESS EXCLUSIVE lock to add a trigger
3. Create a new shadow table
4. Execute ALTER on shadow table
5. Copy Existing Data
6. Create indexes
7. Replay changes recorded on Audit table
C1 C2 C3 C4
Shadow table
Audit table
How does it work ?

15. 1. Create an Audit table
2. ACCESS EXCLUSIVE lock to add a trigger
3. Create a new shadow table
4. Execute ALTER on shadow table
5. Copy Existing Data
6. Create indexes
7. Replay changes recorded on Audit table
8. Finally Swap table inside a transaction
How does it work ?
C1 C2 C3
C1 C2 C3 C4
Shadow table
Primary table

16. 1. Create an Audit table
2. ACCESS EXCLUSIVE lock to add a trigger
3. Create a new shadow table
4. Execute ALTER on shadow table
5. Copy Existing Data
6. Create indexes
7. Replay changes recorded on Audit table
8. Finally Swap table inside a transaction
9. Analyze on new table
How does it work ?
C1 C2 C3 C4
ANALYZE ON NEW PRIMARY TABLE

17. 1. Create an Audit table
2. ACCESS EXCLUSIVE lock to add a trigger
3. Create a new shadow table
4. Execute ALTER on shadow table
5. Copy Existing Data
6. Create indexes
7. Replay changes recorded on Audit table
8. Finally Swap table inside a transaction
9. Analyze on new table
10. Drop old table(optional)
How does it work ?
C1 C2 C3
DROPPING THE OLD PRIMARY TABLE

18. Flow Chart

19. Prominent Features
● Minimal Lock throughout the process
● No chance of data loss
● Table rebuild, reduce bloat
● Finetune Replay
● Custom SQL to copy data

20. Few Considerations
● Mandate to have primary key
● No support for Partitioned Table
● Need minimal locks
● Need of more storage
● Index, constraint, Sequence name lose their naming
● Avoid parallel DDLs

21. References
● GitHub : shayonj/pg-osc : https://github.com/shayonj/pg-osc
● Revolutionizing PostgreSQL Schema Changes with pg_osc :
https://www.mydbops.com/blog/postgresql-schema-changes-with-pg_osc/

22. Thank You !
PgConf India 2024