Last weekends Ubuntu upgrade to 18.04 LTS went mostly well, the Mapnik Python Binding problem from 17.10 has been fully fixed etc.
What didn’t go so well though was the database upgrade from PostgreSQL 9.6 / PostGIS 2.3 to 10 / 2.4.
irst of First of all the process for upgrading PostgreSQL and PostGIS to a new version at the same time is not as trivial as it could be, and requires some file name faking hacks before running the actual pg_upgrade:
Once I found that information the actual pg_upgrade ran fine, and so did the extension upgrade for PostGIS, too.
Map rendering also looked fine, but then when I tried to reactivate import of minutely diff jobs that should only take secondes, or minutes at best, just stalled, and eventually made one postgresql process run out of memory after allocating over 60GB of RAM over some 20 minutes.
In the end we came to the conclusion that something must already have gone wrong on the previous full planet import, somehow not causing problems on the older versions but triggering somethng strange on the more recent ones.
I dind not put much more effort into debugging this at that point, and to prevent possible spread of bad data decided to cut my losses and start a new full import right away without pre-announcing a downtime notice a week ahead of time.
Fortunately that import went well, and so does import of minutely diffs now. The system has since caught up and is less than 15 minutes behind on averaege.
I also have a gut feeling that rendering jobs are processed a little faster now, but unfortunately didn’t properly preserve timing results from test runs yet.
I’m going to disable the Space Station render style, it never really looked good on paper anyway, and rendering it consumes too much menory, causing errors with larger paper formats.
If you think it should still be offered in the future please let me know in the comments.
The OSM Carto default style has released its new version v4.10.0 today, the local installaction on this MapOSMatic instance has been updated to use this latest release.
Continue reading “OsmCarto updated to v4.10.0”
The database import has finished and the rendering daemon is running again.
The Openstreetmap and German Carto stiles have been updated to their most recent releases, so rendererd maps should finally look the same as on openstreetmap.org and .de again. The rest of the styles will follow today.
Database updates still need some work, so for now map data is of last Thursday, the time of the last full planet export.
I will have to do another full import again, causing about a day of outage and some more before the MapOSMatic service instance is fully up to data again.
Starting tonight (Friday April 13th, 2018) at 21:00 UTC the GIS database will be offline, so no new rendering jobs will be processed until for a while. The web frontend will still work most of the time, already rendered Maps will still be available, and submission of new jobs may be possible, but these will only be queued then, and will only be processed after the database import, and some additional changes like stylesheet updates, have finished.
The reason this time is not that I did something wrong on the previous import, or on the minutely diffs import to keep the database up to date. This time it is neAeded to apply the necessary schema changes needed to finally support current OSM Carto default style releases again. OSM Carto v4.0 changed the database requirements in a non-backwards-compatible way, so a re-import is needed to not be stuck on v3.x forever, and to make the default style look the same as on openstreetmap.org again.
To be able to still support all the other older styles again, and to provide better internatiolization and localization features in the near future, I’ll actually be using the hstore-only schema used by the current German OSM style maintained by Sven Geggus, with database views on top that then provide the right schema for each style.
All this works fine in my test environment already, it just needs to be applied to the big machine now, and the re-import is the biggest step of that. I’ll be working on this during the upcoming Berlin OSM Hackweekend, and hope to have everything up and running again Saturday evening, but it may take parts of Sunday, too.
So far street names that start or end in a number were sorted in simple textual order, e.g. 1-10-11-…-19-2-20-21-…
Now, with the help of the Python “natsort” package, these will be listed in “natural” order, like 1-2-…-10-11-…-19-20-21-…
At first this only worked for ASCII / Latin numbers, but with two little fixes to the “natsort” package it should now work for all numeric unicode characters (tested with Persian/Farsi so far).
There have been a few small, but very useful user interface changes recently:
- When accessing the service for the first time the map will be centered to your current location, when detactable. The old “Bielefeld” default will only be used when location data is not available
- From there on the map will be centered to the the area you last used for creating a map
- The layout, style, and overlay you had used in your last request will be remembered and used as default choices, instead of just pre-selecting the first choice.
I started to work on a RESTish/JSONish API for direct automated submission of rendering requests. This is all still work in progress, and subject to further changes, so I don’t want to make it publicly available yet.
If you are interested in draft documentation, example code, and a test account please drop me a mail on <firstname.lastname@example.org>
Umap overlay support hat a slightly bumpy start, so I may have put it on the public server too early, but should now be fully usable in single page mode.
Fixes and improvements applied during the last two days include:
- rendering doesn’t fail anymore when certain properties are not found
- per-layer defaults are now taken into account in addition to global ones
- the “circle” and “ball” markers are now rendered properly
- whether a marker icon should be black or white is now reliably detected
- external marker icons referenced by http: or https: URLs are now downloaded and displayed
- dash patterns are supported when rendering lines
- the map title field is pre-filled with the Umap map name
- when the umap file uses a tile layer style supported by this MapOSMatic instance it is auto selected
Things that are still missing or need to be fixed:
- Umap overlay rendering in multi page format
- handling of label display options
- vertical marker positioning may still be minimally too low or high
As an alternative to GPX uploads it is now also possible to uploaed saved Umap maps and render the contents as an overlay.
Continue reading “Umap overlay support added”