Upcoming new database import

I will have to do another full import again, causing about a day of outage and some more before the MapOSMatic service instance is fully up to data again.

Starting tonight (Friday April 13th, 2018) at 21:00 UTC the GIS database will be offline, so no new rendering jobs will be processed until for a while. The web frontend will still work most of the time, already rendered Maps will still be available, and submission of new jobs may be possible, but these will only be queued then, and will only be processed after the database import, and some additional changes like stylesheet updates, have finished.

The reason this time is not that I did something wrong on the previous import, or on the minutely diffs import to keep the database up to date. This time it is neAeded to apply the necessary schema changes needed to finally support current OSM Carto default style releases again. OSM Carto v4.0 changed the database requirements in a non-backwards-compatible way, so a re-import is needed to not be stuck on v3.x forever, and to make the default style look the same as on openstreetmap.org again.

To be able to still support all the other older styles again, and to provide better internatiolization and localization features in the near future, I’ll actually be using the hstore-only schema used by the current German OSM style maintained by Sven Geggus, with database views on top that then provide the right schema for each style.

All this works fine in my test environment already, it just needs to be applied to the big machine now, and the re-import is the biggest step of that. I’ll be working on this during the upcoming Berlin OSM Hackweekend, and hope to have everything up and running again Saturday evening, but it may take parts of Sunday, too.

Natural sorting

So far street names that start or end in a number were sorted in simple textual order, e.g. 1-10-11-…-19-2-20-21-…

Now, with the help of the Python “natsort” package, these will be listed in “natural” order, like 1-2-…-10-11-…-19-20-21-…

At first this only worked for ASCII / Latin numbers, but with two little fixes to the “natsort” package it should now work for all numeric unicode characters (tested with Persian/Farsi so far).

Remembering defaults

There have been a few small, but very useful user interface changes recently:

  • When accessing the service for the first time the map will be centered to your current location, when detactable. The old “Bielefeld” default will only be used when location data is not available
  • From there on the map will be centered to the the area you last used for creating a map
  • The layout, style, and overlay you had used in your last request will be remembered and used as default choices, instead of just pre-selecting the first choice.

Umap handling improvements

Umap overlay support hat a slightly bumpy start, so I may have put it on the public server too early, but should now be fully usable in single page mode.

Fixes and improvements applied during the last two days include:

  • rendering doesn’t fail anymore when certain properties are not found
  • per-layer defaults are now taken into account in addition to global ones
  • the “circle” and “ball” markers are now rendered properly
  • whether a marker icon should be black or white is now reliably detected
  • external marker icons referenced by http: or https: URLs are now downloaded and displayed
  • dash patterns are supported when rendering lines
  • the map title field is pre-filled with the Umap map name
  • when the umap file uses a tile layer style supported by this MapOSMatic instance it is auto selected

Things that are still missing or need to be fixed:

  • Umap overlay rendering in multi page format
  • handling of label display options
  • vertical marker positioning may still be minimally too low or high

Still some rough edges left

There are still a few small things not working well on the new machine yet, as due to the failure of the old machine ahead of time.

Problems that I’m aware of right now:

  • WayMarkedTrails overlays were missing most track symbols, and on the way regenerating them I accidentially deleted an important data file. So right now a full re-import of the routes database is ongoing, hopefully finished in a few hours
  • Selecting a geographic area in the web frontend is somewhat broken, I hope to figure out what went wrong with the Javascript code there soon …
    Rendering by city name is not affected.