The Open Street Map project provides invaluable data that keeps driving users toward the PostGIS and PostgreSQL stacks. Loading today’s full Planet data set takes a 120GB XML file and unrolls it into over a terabyte of database data. Crunchy’s benchmark labs have followed the expansion of that Planet data over the last six database releases, as the re-ignition of the CPU wars combined with parallel execution features landing in the database. We’ll take a look at that data evolution, which server configurations worked, and which metrics techniques still matter in the all SSD era.

1. Speedrunning the
Open Street Map
osm2pgsql Loader

2. The Open Street Map and osm2pgsql Loader
● The Open Street Map project provides invaluable data that keeps
driving users toward the PostGIS and PostgreSQL stacks.
● Baseline Planet data set loader option osm2pgsql unrolls a 120GB
XML file (or 70GB PBM one) into up to a terabyte database.

3. Crunchy Benchmarking Labs
● Crunchy runs a set of benchmark labs focused on various
PostgreSQL community goals.
● I run a “Farm” focused on storage and regression testing, mainly with
comparisons of bare metal Linux and Mac laptops. Hardware visible
at https://browser.geekbench.com/user/232126
● Code published to pgbench-tools. Results and PG metrics written to
database. Graphs via various Python data analysis tools.
● PG15 testing : osm2pgsql is an alternate backend to pgbench.

4. Speedrunning: 5950X Ubuntu 22.04+PG15b4
test db GB hours change
7558 924 19 Baseline Ubuntu 20.04, osm2pgsql-1.2, ssd:Rocket 4 Plus 2TB
7564 924 17
7913 947 11 Ubuntu 20.04->22.04, osm2pgsql1.2->1.6
7914 948 12 shared_buffers:4194304->6291456
7915 947 11 Linux performance: power,read-ahead
7916 948 10
7917 1075 12 update planet
7919 980 13 update to PG15beta4
7921 980 14 checkpoint_timeout:300->3600;fsync:on->off;synchronous_commit:on->off
7922 980 15 checkpoint_timeout:3600->300;fsync:off->on;synchronous_commit:off->on
7923 980 15

5. osm2pgsql loading stages
Processing
Node(6959649k 663.5k/s)
Way(772748k 29.16k/s)
Relation(8931370 862.18/s)
● Derived or intermediate tables: COPY, CREATE TABLE AS, CREATE INDEX
(B-Tree).
● CREATE INDEX (PostGIS).
● Parallel index builds have greatly reduced the CPU bound periods.
● On SSD, single core performance predicts performance for over half
the runtime.
○ Intel’s Xeon line has long held the lead there.

6. AMD Socket AM4, Zen 1/2/3
3900X:
FOSSGIS e.V.
Ref CPU

7. Unicorn platform?

8. Speedrunning: under 8 hours
Full hstore tagging, single 7700X client/server, 64GB RAM, PG14.5
Processing: Node(7992268k 15399.4k/s) Way(895907k 215.41k/s) Relation(10329300 2008.4/s)
2022-11-12 17:07:32 [0] Reading input files done in 9822s (2h 43m 42s).
2022-11-12 17:07:32 [0] Processed 7992268937 nodes in 519s (8m 39s) - 15399k/s
2022-11-12 17:07:32 [0] Processed 895907075 ways in 4159s (1h 9m 19s) - 215k/s
2022-11-12 17:07:32 [0] Processed 10331346 relations in 5144s (1h 25m 44s) - 2k/s
2022-11-12 22:12:48 [0] Done postprocessing on table 'planet_osm_ways' in 14571s (4h 2m 51s)
2022-11-12 22:12:48 [0] Done postprocessing on table 'planet_osm_rels' in 98s (1m 38s)
2022-11-12 22:12:48 [0] All postprocessing on table 'planet_osm_point' done in 2237s (37m 17s).
2022-11-12 22:12:48 [0] All postprocessing on table 'planet_osm_line' done in 3387s (56m 27s).
2022-11-12 22:12:48 [0] All postprocessing on table 'planet_osm_polygon' done in 8340s (2h 19m 0s).
2022-11-12 22:12:48 [0] All postprocessing on table 'planet_osm_roads' done in 357s (5m 57s).
2022-11-12 22:12:48 [0] osm2pgsql took 28138s (7h 48m 58s) overall.
600 MB/s shared_buffers hits
100 MB/s PG->Linux reads
100 MB/s PG client backend writes
9 MB/s WAL writes

9. Disk I/O:
AMD 7700X 64GB
SSD: SK Hynix P41 Platinum 2TB
(7GB/s seq read, 6.5GB/s seq write, 1,300,000 IOPS)

10. Disk I/O: reads

11. Disk I/O: writes showing 1s bursts

12. Well known osm2pgsql tuning
https://osm2pgsql.org/doc/manual.html
https://wiki.openstreetmap.org/wiki/Osm2pgsql/benchmarks
● Use PBM input, shrinking download to 70GB.
● Use temporary flat file for Nodes
○ Currently 70GB and growing. Can leave on root SSD disk.
● Increase dedicated node cache memory in favor of almost
everything else.

13. Use PBM files instead of OSM [xml+bzip]
● Protocol buffer Binary Format vastly better.
● 7700X CPU adds AVX512 memory transfers.
○ Up to 74% better on simple memory-bound single core SELECT tests!!
CPU Memory k/s Secs Hours PBM k/s Minutes
3950X 128GB 474 16637 4.6 9759 14
5950X 128GB 682 11549 3.2 12468 11
7700X 64GB 784 10053 2.8 17077 8

14. osm2pgsql Memory Budgeting: 128GB RAM
Phase Dynamic memory Static memory
Nodes Node Cache: Up to 50GB Shared Buffers
Ways Same Up to 48GB
Relations Same
CTAS Work Mem: 1GB plenty
CREATE INDEX Maintenance Work Mem:
Up to 20GB

15. Sample import runs
128GB RAM:
osm2pgsql --create --slim -v -d gis
--flat-nodes /var/lib/postgresql/node.db
--cache 50000 --number-processes 2
--hstore --hstore-add-index
~postgresql/osm/planet-latest.osm.pbf
64GB RAM:
osm2pgsql --create --slim -v -d gis
--flat-nodes /var/lib/postgresql/node.db
--cache 36000 --number-processes 2
--hstore --hstore-add-index
~postgresql/osm/planet-latest.osm.pbf

16. Server postgresql.conf stanzas
Common settings:
min_wal_size = 4GB
max_wal_size = 16GB
work_mem = 1GB
maintenance_work_mem = 20GB
64GB Server:
shared_buffers = 8GB
effective_cache_size = 48GB
128GB Server:
shared_buffers = 32GB
effective_cache_size = 80GB

17. Remove standard seatbelts
checkpoint_timeout = 60min
synchronous_commit = off
fsync = off
No replication?
wal_level = minimal
max_wal_senders = 0

18. Controversial adjustments
autovacuum = off
full_page_writes = off
● My results turn autovacuum off for a 4% gain
○ Standard trade-off for bulk loading
○ Not all hosting platforms allow this?
○ Workaround is to starve it of as many resources as you can
● Full page writes are a platform specific storage optimization
○ Most likely outcome 1% gain
○ Shouldn’t advise regular users to ever tinker with this one

19. Advanced tuning: Background Writer
Tuned for heavy writes:
bgwriter_delay = 10ms
bgwriter_lru_maxpages = 500
bgwriter_lru_multiplier = 4.0
Or just disable:
bgwriter_lru_maxpages = 0
checkpoint_flush_after=0 [new parameter]

20. Performance vs. growth: 2016-2022
Table 2016 2022 Multiplier
Nodes 3,374,717K 7,992,268K 2.4
Ways 349,883K 895,907K 2.6
Relations 4,238,333 10,329,300 2.4
Node rate 258K 15,399K 60
Way rate 27K 215k 8.0
Relation rate 226 2,008 8.9

21. NVMe Trimming
$ sudo fstrim -av
/skp41: 1.8 TiB (1950602321920 bytes) trimmed on /dev/nvme2n1p1
Aggressive discarding sample:
$ iostat –mx 1 /dev/nvme2n1
Device d/s dMB/s drqm/s %drqm d_await dareq-sz
nvme2n1 1989.60 250651.89 0.00 0.00 0.49 129004.59

22. Linux tuning
$ sudo blockdev --setra 4096 /dev/nvme2n1
$ sudo cpupower frequency-set --governor=performance
● Boot PCI3.0 SSD plus single PCI4.0 NVMe SSD for tests, no RAID.
● Using ext4 filesystem
○ COW filesystems might safely disable full_page_writes
● Setup Linux swap.
○ For single server installs, osm2pgsql memory peaks can wipe out the
database server.
○ 32GB used here for full Planet.
● Huge pages can be useful too. Hard to measure with single workload.

23. Linux tuning: dd speedups
CPU Governor Read-ahead dd Read
Powersave 256 4.3 GB/s
Powersave 4096 4.9 GB/s
Balanced 256 5.3 GB/s
Balanced 4096 6.4 GB/s
Performance 256 5.7 GB/s
Performance 4096 6.6 GB/s
AMD 7700X
SSD: SK Hynix P41 Platinum 2TB
(7GB/s seq read, 6.5GB/s seq write, 1,300,000 IOPS)

24. Next steps
● Mac testing (M1 Pro, M2 Air)
● Benchmark work created 32GB of Time Series data