Get the latest OpenStreetMap data
Retrieve a piece of OpenStreetMap data in PBF format from http://planet.openstreetmap.org/.
If you need the entire planet file, you can do it by issuing the following command (again substitute your user for username below):
sudo mkdir /usr/local/share/maps/planet sudo chown username /usr/local/share/maps/planet cd /usr/local/share/maps/planet wget http://planet.openstreetmap.org/pbf/planet-latest.osm.pbf
Since the whole planet is at least 18GB when compressed, there are links to smaller country or state sized extracts on that page. However, many people will only need one country or city; you can download PBF files for these (‘extracts’) from download.geofabrik.de. We would recommend that you test with a much smaller area, and only move up to the full planet when you are confident your setup is working.
** Here you can download the data from http://extract.bbbike.org/ . By selecting the bound box and do select the format as pbf or any other according to your choice. You will get the email with the link having the data of the region you want and then download it .
Note : 1.Data is only Available for 48 hours to download from link you got in mail .
2. Do use the sudo in front of wget.
Importing data into the database
With the conversion tool compiled and the database prepared, the following command will insert the OpenStreetMap data you downloaded earlier into the database. This step is very disk I/O intensive; the full planet will take anywhere from 10 hours on a fast server with SSDs to several days depending on the speed of the computer performing the import. For smaller extracts the import time is much faster accordingly, and you may need to experiment with different -C values to fit within your machine’s available memory.
osm2pgsql --slim -d gis -C 1500 --number-processes 4 /usr/local/share/maps/planet/planet-latest.osm.pbf
Let’s have a look at your data import as it progresses. The first part of the osm2pgsql output looks scary, but is normal:
Using projection SRS 900913 (Spherical Mercator) Setting up table: planet_osm_point NOTICE: table "planet_osm_point" does not exist, skipping NOTICE: table "planet_osm_point_tmp" does not exist, skipping Setting up table: planet_osm_line NOTICE: table "planet_osm_line" does not exist, skipping NOTICE: table "planet_osm_line_tmp" does not exist, skipping Setting up table: planet_osm_polygon NOTICE: table "planet_osm_polygon" does not exist, skipping NOTICE: table "planet_osm_polygon_tmp" does not exist, skipping Setting up table: planet_osm_roads NOTICE: table "planet_osm_roads" does not exist, skipping NOTICE: table "planet_osm_roads_tmp" does not exist, skipping Mid: pgsql, scale=100, cache=4096MB, maxblocks=524289*8192 Setting up table: planet_osm_nodes NOTICE: table "planet_osm_nodes" does not exist, skipping NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "planet_osm_nodes_pkey" for table "planet_osm_nodes" Setting up table: planet_osm_ways NOTICE: table "planet_osm_ways" does not exist, skipping NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "planet_osm_ways_pkey" for table "planet_osm_ways" Setting up table: planet_osm_rels NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "planet_osm_rels_pkey" for table "planet_osm_rels"
Don’t be concerned by the NOTICE: entries above. All normal. Next, osm2pgsql will start reading the compressed planet file.
Reading in file: /home/user/planet/planet-latest.osm.bz2
As osm2pgsql reads the planet file it will give progress reports. The line below will refresh every few seconds and update the numbers in brackets. This part of the import takes a long time. Depending on your server, it will take between hours and days.
Processing: Node(10140k 200.1k/s) Way(0k 0.00k/s) Relation(0k 0.00/s)
As the import proceeds, first the Node number will increase untill they are all imported, then the Way number and finally the Relation number. The speed at which the import occurs depends strongly on the hardware available and if the node cache fits into ram. That said, if your node cache is big enough, you can probably expect the speed of nodes to be on the order of a few hundred k per second for nodes, a few 10s of k per second for ways and a few hundred per second for relations. If the node cache does not fit into ram or you have set the cache value to low, the process can be an order of magnitude slower. If you are importing the full planet, the import can take days. If you are only importing single countries, the time should be cut down to minutes or hours. Do not interrupt the import process unless you have decided to start over again from the beginning.
Processing: Node(593072k) Way(45376k) Relation(87k) Exception caught processing way id=110802 Exception caught processing way id=110803 Processing: Node(593072k) Way(45376k) Relation(474k)
The exceptions shown above are due to minor errors in the planet file. The planet import is still proceeding normally.
The next stage of the osm2pgsql planet import process also will take between hours and days, depending on your hardware. It begins like this:
Node stats: total(593072533), max(696096737) Way stats: total(45376969), max(55410575) Relation stats: total(484528), max(555276) Going over pending ways processing way (752k)
The processing way number should update approximately each second.
Going over pending relations node cache: stored: 515463899(86.91%), storage efficiency: 96.01%, hit rate: 85.97% Committing transaction for planet_osm_roads Committing transaction for planet_osm_line Committing transaction for planet_osm_polygon Sorting data and creating indexes for planet_osm_line Sorting data and creating indexes for planet_osm_roads Sorting data and creating indexes for planet_osm_polygon Committing transaction for planet_osm_point Sorting data and creating indexes for planet_osm_point Stopping table: planet_osm_nodes Stopping table: planet_osm_ways Stopping table: planet_osm_rels Building index on table: planet_osm_rels Stopped table: planet_osm_nodes Building index on table: planet_osm_ways Stopped table: planet_osm_rels Completed planet_osm_point Completed planet_osm_roads Completed planet_osm_polygon Completed planet_osm_line Stopped table: planet_osm_ways Osm2pgsql took 86400s overall
The number of nodes, ways and relations processed will obviously differ by the size of the data extract you are using and the date of the data dump. The numbers shown here are not reflective of the full planet import, which is substantially larger.