Upgrade mishaps

Last weekends Ubuntu upgrade to 18.04 LTS went mostly well, the Mapnik Python Binding problem from 17.10 has been fully fixed etc.

What didn’t go so well though was the database upgrade from PostgreSQL 9.6 / PostGIS 2.3 to 10 / 2.4.

irst of First of all the process for upgrading PostgreSQL and PostGIS to a new version at the same time is not as trivial as it could be, and requires some file name faking hacks before running the actual pg_upgrade:

http://www.bostongis.com/blog/index.php?/archives/268-Using-pg_upgrade-to-upgrade-PostGIS-without-installing-an-older-version-of-PostGIS.html

Once I found that information the actual pg_upgrade ran fine, and so did the extension upgrade for PostGIS, too.

Map rendering also looked fine, but then when I tried to reactivate import of minutely diff jobs that should only take secondes, or minutes at best, just stalled, and eventually made one postgresql process run out of memory after allocating over 60GB of RAM over some 20 minutes.

https://github.com/openstreetmap/osm2pgsql/issues/846

In the end we came to the conclusion that something must already have gone wrong on the previous full planet import, somehow not causing problems on the older versions but triggering somethng strange on the more recent ones.

I dind not put much more effort into debugging this at that point, and to prevent possible spread of bad data decided to cut my losses and start a new full import right away without pre-announcing a downtime notice a week ahead of time.

Fortunately that import went well, and so does import of minutely diffs now. The system has since caught up and is less than 15 minutes behind on averaege.

I also have a gut feeling that rendering jobs are processed a little faster now, but unfortunately didn’t properly preserve timing results from test runs yet.

Rendering maps again

The database import has finished and the rendering daemon is running again.

The Openstreetmap and German Carto stiles have been updated to their most recent releases, so rendererd maps should finally look the same as on openstreetmap.org and .de again. The rest of the styles will follow today.

Database updates still need some work, so for now map data is of last Thursday, the time of the last full planet export.