Don't you (forget about me) - PHP Meetup Lisboa 2023

DON'T YOU
(FORGET
ABOUT ME)
PHP Meetup Lisboa 2023
Things developers forget to
remember not to forget
Talk by Bernd Alter

Bernd Alter
Co-CTO at Turbine Kreuzberg GmbH
Twitter: @bazoo0815
Experience:
years developer
years PHP
years eCommerce
years Spryker
25+
20+
10+
7+

DEVELOPERS
ONLY CARE ABOUT
FUNCTIONALITY

"WORKS ON MY MACHINE"
DATABASE
Local development
❏ small dataset
❏ queries are fast
❏ no memory issues
❏ no latency
❏ indexes don't really matter
Production system
❏ large(r) dataset
❏ queries are slower
❏ memory/load issues
❏ large(r) latency between services/components
❏ missing indexes (can) slow down queries

"WORKS ON MY MACHINE"
TRAFFIC
Production system
❏ (huge) traffic
❏ concurrency
❏ blocking requests
❏ lack of resources
Local development
❏ no traffic (it's just you!)
❏ single requests/processes
❏ no concurrency
❏ all resources just for you

POSSIBLE SOLUTIONS
❏ SCALING
❏ PERFORMANCE OPTIMIZATION
❏ CACHING

SCALING
Easy solution:
Throw more resources at it!
Downside:
❏ more complex orchestration
❏ increasing infrastructure costs
(resources + devops ⇒ money)

What do you consider a large dataset
in your day-to-day business?
(quick show of hands)
100.000 entries?
1.000.000 entries?
10.000.000 entries?
100.000.000 entries?
more?
DATABASE PERFORMANCE

Consider payload size of query result set
❏ transfer time matters
❏ only fetch data you need
❏ limit columns in SELECT
Example: table with ~1.200 rows, total size 42 MB, avg row size ~40kB
250ms
3.5 s
vs
(=14x faster)
select * from myTable;
select id from myTable;
PAYLOAD SIZE

Set proper indexes
❏ analyze your queries, not your tables
(EXPLAIN is your friend!)
❏ use composite indexes
INDEX ANALYSIS/OPTIMIZATION
Remember: TANSTAAFL!
❏ 'no free lunch' principle: indexes come at a cost
❏ avoid large/many indexes on tables with many write operations
❏ avoid unnecessary indexes
CREATE INDEX myIndex ON myTable (lastname, firstname);
✔ SELECT * FROM myTable WHERE lastname = ‘Smith’ AND firstname = ‘John’;
✔ SELECT * FROM myTable WHERE lastname = ‘Smith’;
✖ SELECT * FROM myTable WHERE firstname = ‘John’ and lastname = ‘Smith’;
✖ SELECT * FROM myTable WHERE firstname = ‘John’;
Index used for queries:
← size order matters!

Common approach: LIMIT/OFFSET
❏ query performance is bad
Use keyset pagination (or seek pagination)
❏ no read-ahead needed
❏ only required rows are fetched/read
+ constant execution time
+ real pagination (no missed entries)
− only previous/next page (no page jumps)
KEYSET/CURSOR PAGINATION

KEYSET/CURSOR PAGINATION

Don't hit the database at all (or at least less)
❏ use bulk operations over single operations
❏ select as much data at once as possible
❏ Common Table Expressions (CTE)
❏ UPSERTs for write operations
❏ avoid unnecessary queries
INSERT INTO myTable (...) VALUES (...) ON DUPLICATE KEY UPDATE SET ...; (MySQL)
ON CONFLICT (...) DO UPDATE SET ...; (Postgres)

CACHING
For how long? ❏ as long as possible
❏ usually trade-off between up-to-dateness and performance
How? ❏ prepare data (aggregate/transform)
❏ in-process caching (in PHP: static (member) variables)
❏ browser-caching
❏ key-value storage (e.g. Memcache, Redis)
❏ reverse proxy (e.g. Varnish)
What? ❏ time-/resource-consuming operations
❏ often used/requested data

5 days?
5 hours?
5 min?
5 sec?
CACHING
What would you consider a reasonable
amount of time to cache for?
(quick show of hands)

Even short caching periods (TTL) can boost your application’s performance significantly
CACHING
HOW LONG?
Example: Universal Music community chat (~2008)
❏ simple iframe with latest 50 messages
❏ ajax-reload every 10s
❏ ~1.000 concurrent users
⟶ ~6.000 req/min

Even short caching periods (TTL) can boost your application’s performance significantly
CACHING
HOW LONG?
Solution:
❏ local file cache with TTL of 5s
Example: Universal Music community chat (~2008)
❏ simple iframe with latest 50 messages
❏ ajax-reload every 10s
❏ ~1.000 concurrent users
⟶ ~6.000 req/min
—> 12 req/min (500x less)

CACHING
❏ ensure fast, responsive experience for all users
❏ use stale-while-revalidate mechanism
(supported by all modern browsers)
❏ after first caching, all users get always
(sufficiently) up-to-date data
ADVANCED TECHNIQUES

Try to make all your non-production environments as production-like as possible:
DEVELOP PRODUCTION-LIKE
❏ replicate infrastructure in development, testing and staging environments
❏ use Docker
❏ ideal: use same components/services in exactly same version and resources (CPU, memory, etc.)
❏ at least: use same components/services in same major version and similar resources
❏ use production-like dataset for development and staging/testing
❏ ideal: complete anonymized/pseudonymized production dump (-->GDPR!)
❏ at least: similar amount of data for crucial entities, e.g. products, prices, customers in ecommerce

There is no such thing as 'another production system'
DEVELOP PRODUCTION-LIKE
DISCLAIMER
You cannot duplicate your production system.
Period.
Live with it …
❏ traffic is different
❏ state of data is different
❏ services/components have different versions
❏ …

FEATURE TRAP
❏ also work on:
❏ tech debt
❏ updates
❏ refactoring
❏ maintenance tasks
❏ no immediate, but long-term business value
❏ stand up to your PO or stakeholders!
"WE HAVE TO BUILD FEATURES,
WE DON'T HAVE TIME FOR OTHER STUFF"

LINKS
❏ Use The Index, Luke
❏ No offset: We need tool support for keyset pagination
❏ SQL CTE (WITH Clause): The Ultimate Guide - Database Star
❏ Staleness and revalidation
❏ GitHub - DivanteLtd/anonymizer: Universal tool to anonymize database
❏ Blackfire.io - PHP profiling tool

QUESTIONS?

FOR LISTENING!

Don't you (forget about me) - PHP Meetup Lisboa 2023

Recommended

Recommended

More Related Content

Similar to Don't you (forget about me) - PHP Meetup Lisboa 2023

Similar to Don't you (forget about me) - PHP Meetup Lisboa 2023 (20)

Recently uploaded

Recently uploaded (20)

Don't you (forget about me) - PHP Meetup Lisboa 2023