This page describes how to install, setup and configure all the necessary software to operate your own tile server. The step-by-step instructions are written for Ubuntu Linux 12.04 LTS (Precise Pangolin), however they should transfer fairly straightforwardly to other versions of Ubuntu or Linux distributions.
This howto is based on an earlier version for Ubuntu Linux 10.04
The OSM tile server stack is a collection of programs and libraries that work together to create a tile server. As so often with OpenStreetMap, there are many ways to achieve this goal and nearly all of the components have alternatives that have various specific advantages and disadvantages. This tutorial describes the most standard version that is also used on the main OpenStreetMap.org tile server.
It consists of 5 main components: Mod_tile, renderd, mapnik, osm2pgsql and a postgresql/postgis database. Mod_tile is an apache module, that serves cached tiles and decides which tiles need re-rendering – either because they are not yet cached or because they are outdated. Renderd provides a priority queueing system for rendering requests to manage and smooth out the load from rendering requests. Mapnik is the software library that does the actual rendering and is used by renderd.
In order to build these components, a variety of dependencies need to be installed first.
Mapnik requires a recent version of Boost, which can be installed on Ubuntu like this:
sudo apt-get install python-software-properties sudo add-apt-repository ppa:mapnik/boost sudo apt-get update sudo apt-get install libboost-dev libboost-filesystem-dev libboost-program-options-dev libboost-python-dev libboost-regex-dev libboost-system-dev libboost-thread-dev
The remaining dependencies can be installed with the following instructions
sudo apt-get install subversion git-core tar unzip wget bzip2 build-essential autoconf libtool libxml2-dev libgeos-dev libpq-dev libbz2-dev proj munin-node munin libprotobuf-c0-dev protobuf-c-compiler libfreetype6-dev libpng12-dev libtiff4-dev libicu-dev libgdal-dev libcairo-dev libcairomm-1.0-dev apache2 apache2-dev libagg-dev liblua5.2-dev ttf-unifont
Installing postgresql / postgis
On Ubuntu there are pre-packaged versions of both postgis and postgresql, so these can simply be installed via the Ubuntu package manager.
sudo apt-get install postgresql-9.1-postgis postgresql-contrib postgresql-server-dev-9.1
Now you need to create a postgis database. The defaults of various programs assume the database is called gis and we will use the same convention in this tutorial, although this is not necessary. Substitute your username for username in the two places below. This should be the username that will render maps with Mapnik.
sudo -u postgres -i createuser username # answer yes for superuser (although this isn't strictly necessary) createdb -E UTF8 -O username gis exit
Set up PostGIS on the postresql database:
psql -f /usr/share/postgresql/9.1/contrib/postgis-1.5/postgis.sql -d gis
This should respond with many lines ending with:
... CREATE FUNCTION COMMIT ... DROP FUNCTION
Give your newly-created user permission to access some of the PostGIS extensions’ data. Make sure you replace username with your user’s name:
psql -d gis -c "ALTER TABLE geometry_columns OWNER TO username; ALTER TABLE spatial_ref_sys OWNER TO username;"
Although there might be a osm2pgsql package in the repository, it is likely rather old and so we need to compile a newer one from source
mkdir ~/src cd ~/src git clone git://github.com/openstreetmap/osm2pgsql.git cd osm2pgsql ./autogen.sh ./configure make sudo make install psql -f /usr/local/share/osm2pgsql/900913.sql -d gis
Install Mapnik library
Next, we need to install the Mapnik library. Mapnik is used to render the OpenStreetMap data into the tiles used for an OpenLayers web map.
Build the Mapnik library from source:
cd ~/src git clone git://github.com/mapnik/mapnik cd mapnik git branch 2.0 origin/2.0.x git checkout 2.0 python scons/scons.py configure INPUT_PLUGINS=all OPTIMIZATION=3 SYSTEM_FONTS=/usr/share/fonts/truetype/ python scons/scons.py sudo python scons/scons.py install sudo ldconfig
Verify that Mapnik has been installed correctly:
python >>> import mapnik >>>
If python replies with the second chevron prompt >>> and without errors, then Mapnik library was found by Python. Congratulations!
Install mod_tile and renderd
Compile the mod_tile source code:
cd ~/src git clone git://github.com/openstreetmap/mod_tile.git cd mod_tile ./autogen.sh ./configure make sudo make install sudo make install-mod_tile sudo ldconfig
Install Mapnik style-sheet
Next, we need to install the OpenStreetMap Mapnik tools, which include the default style file and tools to help Mapnik render OpenStreetMap data:
cd ~/src svn co http://svn.openstreetmap.org/applications/rendering/mapnik mapnik-style
Mapnik uses prepared files to generate coastlines and ocean areas for small scale maps since it is faster than reading the entire database for this information. Downloading the coastline data requires about 400Mb of download.
cd ~/src/mapnik-style sudo ./get-coastlines.sh /usr/local/share
Now that all of the necessary software is installed, you will need to configure some of the software to work correctly.
Configure mapnik style-sheet
In order for mapnik to find the correct postgis database and the coast line data, you will need to configure the mapnik style-sheet to your local settings. There are a number of different style-sheets publically available and you can of course create your own ones. Each style-sheet might be slightly different in how to configure it and so this page describes how to configure the “default” OpenStreetMap style-sheet that was installed with the mapnik-tools.
In your style-sheet directory (e.g. ~/src/mapnik-style) there should be a directory inc. There are a number of files you need to adapt in this directory:
cd inc cp fontset-settings.xml.inc.template fontset-settings.xml.inc cp datasource-settings.xml.inc.template datasource-settings.xml.inc cp settings.xml.inc.template settings.xml.inc
Now you need to modify each of these files:
<!ENTITY symbols "%(symbols)s">
<!ENTITY symbols "symbols">
<!ENTITY osm2pgsql_projection "&srs%(epsg)s;">
with (assuming you will be using the defaul projection 900913)
<!ENTITY osm2pgsql_projection "&srs900913;">
<!ENTITY dwithin_node_way "&dwithin_%(epsg)s;">
with (assuming you will be using the default projection 900913)
<!ENTITY dwithin_node_way "&dwithin_900913;">
<!ENTITY world_boundaries "%(world_boundaries)s">
with (assuming you installed the coast line data in /usr/local/share/world_boundaries)
<!ENTITY world_boundaries "/usr/local/share/world_boundaries">
<!ENTITY prefix "%(prefix)s">
with (assuming you are using the default database table prefix)
<!ENTITY prefix "planet_osm">
In this file you will need to enter your database settings. You are running postgresql on the same machine as the rendering stack, so you can comment out the parameters “password”, “host” and “port” with an HTML-style comment. This will enable mapnik to use the “unix local user” as an authentication method.
Change the “dbname” from “%(dbname)s” to “gis”, “estimate_extent” to “false”, and “extent” to “-20037508,-19929239,20037508,19929239″ so that the file now reads:
<!-- Settings for your postgres setup. Note: feel free to leave password, host, port, or use blank --> <Parameter name="type">postgis</Parameter> <!-- <Parameter name="password">%(password)s</Parameter> --> <!-- <Parameter name="host">%(host)s</Parameter> --> <!-- <Parameter name="port">%(port)s</Parameter> --> <!-- <Parameter name="user">%(user)s</Parameter> --> <Parameter name="dbname">gis</Parameter> <!-- this should be 'false' if you are manually providing the 'extent' --> <Parameter name="estimate_extent">false</Parameter> <!-- manually provided extent in epsg 900913 for whole globe --> <!-- providing this speeds up Mapnik database queries --> <Parameter name="extent">-20037508,-19929239,20037508,19929239</Parameter>
This file contains font definitions, and information about how to change the default font. It is recommended that you don’t edit this right now.
Change the the renderd settings by editing the /usr/local/etc/renderd.conf and change the following lines like so (remember to change username to your user’s name):
socketname=/var/run/renderd/renderd.sock plugins_dir=/usr/local/lib/mapnik/input font_dir=/usr/share/fonts/truetype/ttf-dejavu XML=/home/username/src/mapnik-style/osm.xml HOST=localhost
Create the files required for the mod_tile system to run (remember to change username to your user’s name):
sudo mkdir /var/run/renderd sudo chown username /var/run/renderd sudo mkdir /var/lib/mod_tile sudo chown username /var/lib/mod_tile
Next, we need to tell the Apache web server about our new mod_tile installation by creating the file /etc/apache2/conf.d/mod_tile and adding one line:
LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so
Also, Apache’s default website configuration file needs to be modified to include mod_tile settings. Modify the file /etc/apache2/sites-available/default to include the following lines immediately after the admin e-mail address line:
LoadTileConfigFile /usr/local/etc/renderd.conf ModTileRenderdSocketName /var/run/renderd/renderd.sock # Timeout before giving up for a tile to be rendered ModTileRequestTimeout 0 # Timeout before giving up for a tile to be rendered that is otherwise missing ModTileMissingRequestTimeout 30
Tuning your system
A tile server can put a lot of load on hard- and software. The default settings may therefore not be appropriate and a significant improvement can potentially be achieved through tuning various parameters.
The default configuration for PostgreSQL 9.1 needs to be tuned for the amount of data you are about to add to it. Edit the file /etc/postgresql/9.1/main/postgresql.conf and make the following changes:
shared_buffers = 128MB checkpoint_segments = 20 maintenance_work_mem = 256MB autovacuum = off
These changes require a kernel configuration change, which needs to be applied every time that the computer is rebooted. As root, edit /etc/sysctl.conf and add these lines near the top after the other “kernel” definitions:
# Increase kernel shared memory segments - needed for large databases kernel.shmmax=268435456
Reboot your computer. Run this:
sudo sysctl kernel.shmmax
and verify that it displays as 268435456.
Loading data into your server
Get the latest OpenStreetMap data
Retrieve a piece of OpenStreetMap data from http://planet.openstreetmap.org/. Since the whole planet is at least 18GB when compressed, there are links to smaller country or state sized extracts on that page. Since it is smaller and faster to interact with, the PBF file format is preferable when available. In this case we’ll download the entire planet file by issuing the following command:
cd planet wget http://planet.openstreetmap.org/pbf/planet-latest.osm.pbf
Importing data into the database
With the conversion tool compiled and the database prepared, the following command will insert the OpenStreetMap data you downloaded earlier into the database. This step is very disk I/O intensive and will take anywhere from 10 hours on a fast server with SSDs to several days for the full planet depending on the speed of the computer performing the import. For smaller extracts the import time is much faster accordingly, and you may need to experiment with different -C values to fit within your machine’s available memory.
osm2pgsql --slim -d gis -C 16000 --number-processes 3 ~/planet/planet-latest.osm.pbf
Let’s have a look at your data import as it progresses. The first part of the osm2pgsql output looks scary, but is normal:
Using projection SRS 900913 (Spherical Mercator) Setting up table: planet_osm_point NOTICE: table "planet_osm_point" does not exist, skipping NOTICE: table "planet_osm_point_tmp" does not exist, skipping Setting up table: planet_osm_line NOTICE: table "planet_osm_line" does not exist, skipping NOTICE: table "planet_osm_line_tmp" does not exist, skipping Setting up table: planet_osm_polygon NOTICE: table "planet_osm_polygon" does not exist, skipping NOTICE: table "planet_osm_polygon_tmp" does not exist, skipping Setting up table: planet_osm_roads NOTICE: table "planet_osm_roads" does not exist, skipping NOTICE: table "planet_osm_roads_tmp" does not exist, skipping Mid: pgsql, scale=100, cache=4096MB, maxblocks=524289*8192 Setting up table: planet_osm_nodes NOTICE: table "planet_osm_nodes" does not exist, skipping NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "planet_osm_nodes_pkey" for table "planet_osm_nodes" Setting up table: planet_osm_ways NOTICE: table "planet_osm_ways" does not exist, skipping NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "planet_osm_ways_pkey" for table "planet_osm_ways" Setting up table: planet_osm_rels NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "planet_osm_rels_pkey" for table "planet_osm_rels"
Don’t be concerned by the NOTICE: entries above. All normal. Next, osm2pgsql will start reading the compressed planet file.
Reading in file: /home/user/planet/planet-latest.osm.bz2
As osm2pgsql reads the planet file it will give progress reports. The line below will refresh every few seconds and update the numbers in brackets. This part of the import takes a long time. Depending on your server, it will take between hours and days.
Processing: Node(10140k 200.1k/s) Way(0k 0.00k/s) Relation(0k 0.00/s)
As the import proceeds, first the Node number will increase untill they are all imported, then the Way number and finally the Relation number. The speed at which the import occurs depends strongly on the hardware available and if the node cache fits into ram. That said, if your node cache is big enough, you can probably expect the speed of nodes to be on the order of a few hundred k per second for nodes, a few 10s of k per second for ways and a few hundred per second for relations. If the node cache does not fit into ram or you have set the cache value to low, the process can be an order of magnitude slower. If you are importing the full planet, the import can take days. If you are only importing single countries, the time should be cut down to minutes or hours. Do not interrupt the import process unless you have decided to start over again from the beginning.
Processing: Node(593072k) Way(45376k) Relation(87k) Exception caught processing way id=110802 Exception caught processing way id=110803 Processing: Node(593072k) Way(45376k) Relation(474k)
The exceptions shown above are due to minor errors in the planet file. The planet import is still proceeding normally.
The next stage of the osm2pgsql planet import process also will take between hours and days, depending on your hardware. It begins like this:
Node stats: total(593072533), max(696096737) Way stats: total(45376969), max(55410575) Relation stats: total(484528), max(555276) Going over pending ways processing way (752k)
The processing way number should update approximately each second.
Going over pending relations node cache: stored: 515463899(86.91%), storage efficiency: 96.01%, hit rate: 85.97% Committing transaction for planet_osm_roads Committing transaction for planet_osm_line Committing transaction for planet_osm_polygon Sorting data and creating indexes for planet_osm_line Sorting data and creating indexes for planet_osm_roads Sorting data and creating indexes for planet_osm_polygon Committing transaction for planet_osm_point Sorting data and creating indexes for planet_osm_point Stopping table: planet_osm_nodes Stopping table: planet_osm_ways Stopping table: planet_osm_rels Building index on table: planet_osm_rels Stopped table: planet_osm_nodes Building index on table: planet_osm_ways Stopped table: planet_osm_rels Completed planet_osm_point Completed planet_osm_roads Completed planet_osm_polygon Completed planet_osm_line Stopped table: planet_osm_ways Osm2pgsql took 86400s overall
The number of nodes, ways and relations processed will obviously differ by the size of the data extract you are using and the date of the data dump. The numbers shown here are not reflective of the full planet import, which is substantially larger.
Starting your tileserver
Now that everything is installed, set-up and loaded, you can start up your tileserver and hopefully everything is working. We’ll run it interctively first, just to make sure that everything’s working properly:
sudo mkdir /var/run/renderd sudo chown username /var/run/renderd renderd -f -c /usr/local/etc/renderd.conf
and on a different session:
sudo /etc/init.d/apache2 restart
If any FATAL errors occur you’ll need to double-check any edits that you made earlier.
If not, try and browse to http://yourserveraddress/osm_tiles/0/0/0.png to see if a small picture of the world appears. The actual map tiles are being created as “metatiles” beneath the folder /var/lib/mod_tile.
If it does, you can stop the interactive renderd process and configure it to run automatically at machine startup as a daemon.
sudo cp ~/src/mod_tile/debian/renderd.init /etc/init.d/renderd sudo chmod u+x /etc/init.d/renderd
Edit the /etc/init.d/renderd file as root – you’ll need to make a couple of changes to the DAEMON and DAEMON_ARGS lines so that they read:
DAEMON=/usr/local/bin/$NAME DAEMON_ARGS="-c /usr/local/etc/renderd.conf"
Also, you’ll need to change references to www-data so that they match your username – change “www-data” to what you changed “username” to in other files.
You should now be able to start mapnik by doing the following:
sudo /etc/init.d/renderd start
and stop it:
sudo /etc/init.d/renderd stop
Logging information is now written to /var/log/syslog instead of to the terminal.
Next, add a link to the interactive startup directory so that it starts automatically:
sudo ln -s /etc/init.d/renderd /etc/rc2.d/S20renderd
and then restart your server, browse to http://yourserveraddress/osm_tiles/0/0/0.png and everything should be working! You can also go to the page http://yourserveraddress/mod_tile which should give you some stats about your tileserver.
Next, you might want to have a look at something in the using tiles section to create a map that uses your new tile server.