When I buy something I like to own it. I'm not against rental or subscription services by any means, in fact in our consumerist world I'd like to see much more sharing of resources within communities, but when I buy something for myself I want to be able to (within reason!) do what I want with it.

Ordnance Survey, Great Britain's national mapping agency, sells its excellent paper maps with a digital download code that allows you to download the map onto your phone for use in their OS Maps app. This is an excellent feature, and their app is very good indeed, but I'd love to be able to browse these maps offline on my computer, or view them in a different mapping app, or change their compression slightly to improve their file size a little. After all, I've bought those digital maps, haven't I?

So I've been doing a little exploring of the OS Maps app to see if I can extract the map files from its storage.

Finding the Maps

First things first, I've be doing all this on an Android device because it's what I am most familiar with. The first hurdle that I encoutered is that the OS Maps app doesn't store its data in a user accessible folder, unsurprisingly. There are a couple of things we can try to get around this:

1. Use adb backup to try and pull the app data
2. Use a rooted device to explore the app data

The Android Debug Bridge (adb) developer tool has historically allowed users to take backups of their app data using the adb backup command. The user can then extract this backup file in order to inspect the app data stored in folders not accessible to the user. However, this isn't sure to work - developers can opt-out of backing up their app data using an attribute in the app manifest (which is what the OS Maps app developers have done), on top of which the adb tool has recently deprecated the backup tool entirely, and may remove it in future releases.

So instead we try inspecting the app data using a rooted android device. Now I happen to have a rooted android phone, but if you don't you can emulate an android device in Android studio and choose a system image with elevated priviledges. These images don't come loaded with Google Play, so you may have to download the OS Maps .apk file from elsewhere, but once it's installed you'll be able to use the file browser built-into Android studio to have a poke around its app data.

The first promising thing I noticed is the /data/data/uk.co.ordnancesurvey.osmaps/files/mbgl-offline.db file. The file name implies that it is an "offline" file saved using the MapBox Graphics Library ("mbgl") as an SQLite database (".db"). A quick search on GitHub confirms that Ordnance Survey works with these files, and the large file size seems plausible for lots of map tiles.

Working with Map Tiles

There are a number of file formats for storing map tiles offline, but I like the MBTiles format as it's open source, easy to work with (it's just an SQLite database), and is widely supported in mapping and QGIS applications. The mbgl-offline.db tile database isn't in a particularly accessible format, but as it too is SQLite based it was easy to write a python program to convert between them.

Introducing ostools.py, my python toolkit for working with these files.

Get ostools on GitHub

It comprises three(ish) useful functions:

1. extract which extracts the individual maps stored within the mbgl-offline.db file into seperate MBTiles files
2. convert which converts the PNG map tiles to the more space efficient WebP format
3. dedupe which removes the duplicated map tiles from adjacent MBTiles files

extract is pretty straightforward, mainly just changing the structure of the SQLite database and putting the right data in the right places. It makes two other substantial modifications. Firstly, it decompresses the zlib compressed PNG map tiles, as this isn't widely supported by mapping applications. Secondly, it converts the y-coordinate from the TMS to XYZ scheme required by the MBTiles spec; which is a straightforward operation: \( y_{xyz} = 2^{z}-y_{tms}-1 \).

convert just changes the format of the map tiles from PNG to WebP. It allows the user to set the desired quality parameter that controls the trade off between a small file size and a high quality image. It can be helpful to visualise how this quality parameter behaves to allow users to make an informed choice, but the default quality of 50 seems like a good compromise.

PNG	WEBP (100)	WEBP (75)	WEBP (50) default	WEBP (25)	WEBP (0)
26,912 bytes	23,668 bytes	16,726 bytes	14,476 bytes	8,820 bytes	2,362 bytes

dedupe is useful because paper OS Map sheets helpfully overlap at the edges to make map reading at the boundary betwen maps a little bit easier. However this isn't so important when loading adjacent maps into a viewer as the viewer handles the transition between MBTiles files seamlessly, so duplicated tiles are just wasting space. As such you can save a little space by removing the duplicate tiles between adjacent maps.

Viewing the Maps

The MBTiles files that this tool produces have decent support in viewing applications. The most striking exception is OSMand, which still doesn't support MBTiles format maps, however there are open source tools available that allow you to convert MBTiles maps to the SQLite format used by OSMand and RMaps.

My favourite viewing app is the Locus Maps app, with it's support for British Grid References. It's not open source 😢, but the Locus Maps Classic (Pro) version is available free online (choose the Amazon release if you didn't purchase the app through Google Play before the pricing structure changed).