Talk:Wikimaps

The Wikimaps page summarizes most of what has been done regarding maps in Wikimedia projects. Special interest should be paid to the Wikiteer discussion. Another promising article is titled WikiProject Geographical coordinates.

This page seems to be getting too large for the discussion. Any suggestions how to continue? Several mailing lists have already been proposed but none seems to really cover the whole topic.

Proposal for a Wikimaps Architecture

The first essential feature of Wikimaps will be the ability to address datasources outside Wikimaps. In the realm of geodata processing a comprehensive set of standards has been implemented by the Open Geospatial Consortium (OGC). Most of these standards have been implemented in several Free Software projects. Compatibility is very high due to the nature of geo data and reflects nicely in these software projects. This discussion describes the components of a geodata infrastructure, how to interact and what would be the job of the Wiki.

Geo Data

Everything starts with geo data. In a simplyfied sense maps are always generated from real world coordinates. These can be attributed with alphanumerical data, name, description, further links. Another source of data are raster images like aerial or satellite imagery. There are two different data sources of interest:

• Free Wikimaps - a geodata collection copyrighted and maintained by Wikipedians
• Proprietary or protected geodata hosted and maintained by third party - linked into Wikimaps

Both data pools can interoperate perfectly, if all systems agree to use the same interface specifications. Most larger Geographic Information System Software package nowadays supports the basic industry standards as specified by the OGC (Open Geospatial Consortium). Most specifications have been implemented by one or more Free Software projects. For more details about those software packages refer to [User:Tschirl my homepage].

point of interest / geo bookmark / geocach / geoblog

The smallest possible geodata is one set of coordinates and which reference system they belong to. A lot of data is not stored in degrees but in a projected spatial reference system. Therfore the source srs is crucial whenever geodata is adressed.

GML

One coordinate can be stored together with other attibutes of this location in an XML schema which has been formalized by the OGC as GML (geographic markup language). Although at first sight it seems a little overloaded this is probably the format which is best used to create new geo data. It can be extended to contain several coordinate tuples to form a line or polygon and can combine several features to describe a complexer object. This example shows the GML of the postal code area 53111 in, city of Bonn, Germany. This geo data format is supported by many Free Software GIS software components, client, server and application. PostgreSQL/PostGIS and in parts also MySQL implement the geometry format WKT (Well Known Text) and use WKB (Well Known Binary) for physical storage.

Spatial Reference System

The whereabouts are defined in a spatial reference system (coordinate system). The earth is neither flat nor a perfect sphere but a bumpy and crumbled ball. Therefore spatial reference systems have to be calculated on spheroids some of which are not compatible to each other. An interesting aspect about maps is the way they are oriented. This can have such a large impact on our psychology that we even call people from "down under" when they come from the other side of the globe. Pardon me for stressing this fact but no place on earth can be under, down or above the world, they all are on the earth.

How to display a map

The basic information required to obtain a map are the geographic coordinates and the type or content of geodata.

Map Contents

The second part is the "content" of geodata. Even the most obvious maps show that it will not do to stick with just one type of geodata. A world map cannot be displayed from geodata needed to display a road map (for navigation). On the other hand the coordinates of the world capital cities will not make sense to somebody having a look at the Eiffel tower. A biologist might want to document the occurrence of insects in a map and therfore needs topographical information. The location of a certain species does not make a lot of sense without the correct backdrop map. There are several different geodata elements, the most common are:

• point
• line
• polygon (area)
• grid (raster)

There is an essential difference between these types of geodata. Points and lines do not cover a lot of space. An area will always cover something (if it is not transparent, but then it essentially is a line). A grid or raster will definitely block anything lying underneath. So depending on the type of data it can be of great importance in which order a map is composed. A dynamic map interface should therfore be capable of displaying geodata of any kind, combine several geodatasets and switch projections (spatial reference systems).

Formalizing a Map Requests

A map request can be formalized using the format of an OGC WMS request (standardized specification) which defines a region with a set of coordinates and adds additional technical information like the spatial reference system, image format, error return code etc..

WMS getMap request

The following URL shows an OGC WMS getMap request for a map showing the outline of Germany. Several of these requests can be combined to create a map with overlaying data coming from different servers located all over the world.

Web Context Document

A combination of several servers, layers and map requests can be stored in an OGC standard format as defined in Web Context Documents. This XML document can be read and interpreted by a host of different GIS software packages making the information interoperable to the highest possible degree.

I believe that this would be the best possible way to include layouted maps in any Wiki article. As this approach is rather technical a wysiwyg interface would be helpful if not essential. Every author should be able to modify a map compilation without having to resort to other software. Mapbender provides PHP modules which can probably directly be integrated with [Mediawiki] software.

Where to start?

As a first step Mediawiki could learn to recognize a getMap request and resolve the underlying URL to show the requested result as an image - a map. This should be farily trivial as the result of a getMap request usually will be an image of the formats PNG or JPEG (also GIF, WBMP, Flash and several others). Maybe a new tag "Mapimage".

[[Mapimage:http://stadtplan.bonn.de/isserver/sdims/bin/wms/getMap.pl?dataSource=basic&WMTVER=1.0.0&REQUEST=map&LAYERS=1&STYLES=&SRS=EPSG:31492&BBOX=2575681,5622978,2575723,5623020&WIDTH=200&HEIGHT=200&FORMAT=PNG&BGCOLOR=0xffffff&TRANSPARENT=TRUE&EXCEPTIONS=WMS_XML getMap request]]

This link is dynamic and will always be requested from the originating service. Whenever the city administration of Bonn acquires new orthophotography it will automatically be displayed as soon as it is available on the Bonn server. The following request returns a larger image and less detail.

[[Mapimage:http://stadtplan.bonn.de/isserver/sdims/bin/wms/getMap.pl?dataSource=basic&WMTVER=1.0.0&REQUEST=map&LAYERS=1&STYLES=&SRS=EPSG:31492&BBOX=2575650,5622950,2575750,5623050&WIDTH=800&HEIGHT=800&FORMAT=PNG&BGCOLOR=0xffffff&TRANSPARENT=TRUE&EXCEPTIONS=WMS_XML getMap request]]

Anybody who can technically code a getMap request can then directly start to add them to Wiki articles.

There was a discussion on the mailing list Foundation-l whether Wikimaps should be a separate Wiki. Some coding has been done using SVG in the project Geo by [Magnus Manske]. This approach uses a special geo data format explicitly for use in Wikis. Maybe this data could be used as an overlay to the architecture discussed here.

A Wikitech mailing asked for development of a WYSIWYG editor because standard users wont know how to code all those wiki tags. This is so much more true for the above getMap requests. I admit that without a map editor the task of coding a getMap request is plainly unrealistic. I didn't chime in on the WYSIWYG discussion because I believe it to be next step (see below). Mabybe we should reorder priorities and directly start with the mapping frontend? I will set up a demo, link follows here.

The next steps

The task of manually coding getMap request to obtain maps will be a rather tedious and highly error prone process. It will be necessary to integrate OWS mapping software to create the getMap requests. The mapping module should be invoked by clicking on a button in the standard edit toolbar. It has a map interface with navigation buttons, a gazetteer and map catalog to choose map services from. The author can compose a map by combining and overlaying different services and navigating to the area of interest.

Static and dynamic maps

The map resulting from OGC WMS getMap requests can either be saved as a static image or the metainformation which led to this map can be stored. The static map will be useful as a document for the archives. Consider the current online maps showing the earthquake and tsunami damage of 2004. With the next larger disaster all those maps will be lost (hopefully only from the online servers), a copy of the resulting physical image therfore is the better choice. The dynamic links will reproduce the same map extent as in the static document but with the dynamic contents that are provided by the linked services. Imagine a highway map with the current pile ups changing every 60 seconds. A static map of this information is absolutely worthless (except for the documentation of highest congestion in a city). Consider the temporary aspect. A good axample are election results -select a year and show what people have voted.

Finding Places with Wikiteer

Most of the initial traffic on the geodata servers will be caused by erroneous navigation because people don't find what they are looking for (besides simple good old curiosity). Therefore one of the early features should be a robust and comprehensive naivgation. Wikiteer combines Wiki with gazetteer. A Gazetteer simplyfied means a geographic search engine for place names. The data for naviagtion is the same as for presentation but it is not accessed by selecting a geographic region but attributes of the geo data - in this case place names. These can be ordered hierarchically, the most obvious being national borders, possibly preceded by continents. Within one country it should be possible to list placenames - the more the better. I have not yet tested how to integrate Wikiteer geo data formats with OGC syntax.

Time Line Navigation

By means of a time input, graphical or otherwise, maps could be generated for different geological time periods. Also links to articles from maps could be fixed to times, so a map of Paris in 1940 would have different links than one in 2005. This could be a powerful navigational tie-in to other wiki resources.

Further Development

After setting up a navigation and gazetteer geo database and including map tags in Mediawiki the funcionality can be enhanced where it is needed most.

Managing Geodata - Catalogs

The map services and the maps they serve will have to be categorized, sorted and monitored. There already exists a host of catalog services, many of which are kept up to date and in shape with quite a remarkable amount of money and manpower. Still this doesn't help to make the data a lot more accessible to the general public, the complexity even of the catalogs preserve access to the specialists. This is a pressing problem for many data providers who have the duty to publish their data in a comprensive way but seem to not manage very well. The compilation of a usable catalog would be one of the more intriguing jobs for the Wiki community.

Monitoring Map Services

Web Map Servers are always in danger of beeing slashdotted. Geodata can become very heavy indeed. A badly configured orthophoto mapserver will not work at all, no matter how large the hardware cluster might be. It can also happen that a service disappears altogether. Therefore careful monitoring of the services should be considered. This will also provide valuable data for a meta infromation system.

Meta Information Systems

If a web map server disapperars that can be fatal for dynamic maps. Somebody will have to start from scratch, and add new data to show what the previous service contained. In this context metadata is of high value. Metadata will describe aspects about the geodata - not the individual geo objects themselves. Whenever a map is linked into an article some metadata should be referenced so that in case of a failing or disappearing service ersatz can be found.

Routing

At a later stage it might be possible to access routing data with street names and housenumbers, but usually this data is proprietary, expensive and/or out of date. The algorithms calculating shortest paths are well known and implemented in several free software packages.

How to create new geodata

Whenever a place name could not be found it should be possible for the Wikipedian to add this information to the free dataset. If it is possible to interactively find the place by navigating manually, the newly found location could then easily be entered into the gazetteer database. Ideally either the database or the Wiki software will be able to show geographical diffs just like like it would show diffs in the articles. This is adding point data only, in a next step it could be possible that people enter complete geometries like a road section, house, park or whatever. Coordinates can be taken from car navigation systems or other GPS equipped gadgets. Anything more than a single Point (Line, Polygon) should be created with a topology. This is a lot harder to implement as the features have to be validated with geometric rules. I assume that the growing GPS enlightened geocaching community will also be a valuable source of free geo data.

There is an open source project at http://mapgeneration.berlios.de that works on automatically generating road maps from GPS data.

Software and Technology

As data store we would strongly recommend to use PostgreSQL because it includes the very powerful geodata interface PostGIS. (Authors note: I hope that this does not cause a long debate between MySQL and PostgreSQL disciples. Believe me, when I say that I had enough of it. We use both databases, one for administration data one for geo data and we live to tell - so it cannot be that difficult. I personally favor diversity, so why not just use both?). On top of this central datastore several services run. WMS for maps (UMN MapServer), WFS-T (GeoServer) for features and gazetteer and deegree as building blocks for both. Editing and management can be done with the Mapbender framework which is implemented in PHP and therefore is probably a nice interface to Mediawiki.

Demo server

For a starter we can happily provide manpower, know-how, test environments, OWS testbed and discussion. We have several services up and running which we can also modify and adjust in any way necessary. Additionally we have been collecting online geo data services all around the world as a starter for the catalog.

Performance Issues

Our internet access will definitely not cope with what traffic Wikimaps will produce. It has to be clear that Wikimaps in the proposed way will be quite a processor killer. It will need strong hardware infrastructure and a fast internet backbone. Whether or not this whole idea is worth pursuing will be a lot more transparent as soon as we can show some demos. I have heared of 200 to 300 pages per second on the classic WIkipedia Apache setup. Talking about web mapping systems you rather think the other way round in seconds per request - even using the best hard- and software you can get. A lot of thought should therfore go into performance issues. A comprehensive planning of the initial data structure and semantics will help fine tune the system later.

My Home location

This is the aerial photography of the location of my office.

lat:7.072499 lon:50.73881

--Arnulf Christl 20:31, 23 Jan 2005 (UTC)

Data sources

I would be willing to make and contribute data. I have been making maps of roads and railroads, specifically the NYC subway, and these could in theory be included in the database. I would need a way to convert it - I can output a PDF with USGS quad lines marked, if that could somehow be converted (automatically or manually). I could also mark latitude and longitude from the quads. Once I get it into GIS data (if that is what we are to use), I could then add fields, like opening date, closing date, etc. Maybe this could be cross-referenced with the station infoboxes. --SPUI 21:37, 6 Mar 2005 (UTC)

GMT

The standard map-making tools in the world of geophysics are the Generic Mapping Tools. They ship with a detailed database of coastlines and political divisions (they are not so good on major rivers and lakes). The tools are command-line and scriptable, which is nice for producing maps out of data sources.

A simple example using GMT

Very complex maps may be produced if one has access to information such as height fields (but the tools do not ship with height or so information; one has to acquire it from other sources). David.Monniaux 18:19, 23 May 2005 (UTC)

It seems that height/bathymetry information may be obtained from NOAA. (global relief data) David.Monniaux 18:37, 23 May 2005 (UTC)

Trying with GTOPO30 (30"):

Another example with GMT

30" data, smoothed, legend etc...