Chapter 2. Web Mapping Basics
Many people making web-based maps do not come from a geographical information system (GIS) or traditional mapmaking background, and can go years without knowing what a map projection is. This chapter will introduce map projections, data storage, and the challenges of making maps online.
If you just want to start working through the tutorials and making maps, feel free to skim over this chapter and come back to it later. That said, even trained geographers often have to pause to remember which is latitude and which is longitude. While it is not necessary to understand these concepts to make maps with Drupal, there will be a time when it is really helpful. Is all the GPS data that you imported showing up a few meters out of place? You’re probably using mismatched projections. Is your map of Washington, DC, showing up in Antarctica? Your latitude and longitude are probably reversed. The first bug reported for the GMap module in Drupal in October 2005 was because the developer got this wrong.
Note
There are many mnemonics to help remember which is which. A common one is “lat is flat”: on a normal map, the lines of latitude run east-west across the map and look horizontal or flat.
Projections and Coordinate Systems
The world is roughly spherical, but computer screens are flat. How we represent the world on a two-dimensional surface and how we locate ourselves and objects on it are difficult problems, and there are many different ways to handle these issues. A map projection is a way of representing the Earth on a flat surface, and a coordinate system is a way of describing a place on a map. These are difficult concepts to fully understand, but they affect how your maps look, how they get used, and how you create maps from a technical standpoint.
Map Projections
A map projection is any method of representing the surface of a sphere or other three-dimensional body on a plane.
This representation is not a simple task and it has consequences depending on the method used. Ideally, we want to depict the territory of, for example, Europe on a flat piece of paper or screen so as to preserve shapes, directions, and bearings, and uniformly reduce area and distance. Sadly, there is no way to preserve all these properties. The variety of map projections are different ways of preserving some of these properties while sacrificing others.
For example, the Mercator projection has the property of preserving shapes and directions. However, it does this at the cost of enlarging all lands the further they are from the equator. Many people therefore get the impression that Greenland is about the same size as South America. Developing nations tend to be close to the equator, and their inhabitants often resent the Mercator projection, which makes Europe and North America appear much more significant in relative area than they are. Some of these objectors prefer the Equal-Area or Gall-Peters Projections, which are two different ways of representing the Earth that maintain the correct areas; the trade-off being distorting shapes—Iceland, for instance, looks like a stretched-out lens. Depending on what your map wants to communicate, it is important to choose a projection that will help your message. It is impossible to say what projection is best for your case, but it is important to consider what the projection communicates. No projection will be perfect, but one could be more appropriate for your audience than another.
In web mapping you are relying on the maps provided by Google, Bing, OpenStreetMap, or a similar service, and as a consequence there are only one or two projections available to use. The Spherical Mercator projection is the most common web mapping projection. This is probably what you will be using for your maps, but keep in mind the effect that the projection has on your map. If you need to map Antarctica, the Mercator projection assigns this continent an infinite area!
Some different map projections of the Earth are shown in Figure 2-1, Figure 2-2, Figure 2-3, and Figure 2-4. These are just a sample of projections to illustrate the impact that this choice has on a map.
Spherical Mercator
The Mercator projection is one of the most common projections. A variant of it, Spherical Mercator, is used by all major web-based maps (including Google, Bing, MapQuest, and OpenStreetMap). It makes the assumption that the Earth is a perfect sphere (though it is not). As shown in the illustration (Figure 2-5), the method of flattening the globe is to spread out the globe onto a cylinder. Imagine a translucent world with a very bright lightbulb at the exact center and its image projected onto the cylinder. This means that areas close to the equator are fairly accurate, but as you get further toward the poles, the areas are very enlarged. This illustrates the problem with all map projections: it is impossible to have all four aspects of a map be accurate (direction, distance, area, shape). In fact, you can’t get more than two. Mercator is good for direction and shape.
Note
If you are working with geographic data sets, you may come across numbers that refer to their data projection. Projections have been given identification numbers by the European Petroleum Survey Group (EPSG; now the International Association of Oil & Gas Producers). These identification numbers are often used by different tools to designate projections. The Spherical Mercator projection, temporarily given EPSG:900913, is now officially EPSG:3857. (If you are familiar with leet speak, you would notice that 900913 looks like the word Google). Another common projection is the projection that translates to latitude and longitude, which is EPSG:4326.
More resources
Although Spherical Mercator is the projection that is most common in web mapping, there are a theoretically limitless number of other map projections. It is important to understand a bit about projections from a technical standpoint and the effects they can have on your maps and your map users. Other map projections are covered in more depth in Appendix B. In addition to this, there is some useful information online:
Wikipedia has a thorough article on map projections.
Mapthematics has a really interesting visualization of the effects of projection.
Mapthematics also has an in-depth look at projections.
Mapthematics provides a list of projections and details about each.
Kartoweb has an even more in-depth look at projections.
Wikipedia has a good overview of the World Geodetic System projection.
Further reading is listed in Appendix A.
Coordinate Systems
A coordinate system is a way of referring to a location with a set of numbers. The coordinates that are most important for web-based maps are latitude and longitude. Latitude is the angular distance north or south from the equator. Latitudes north of the equator are positive; latitudes south of the equator are negative. Longitude is the east-west position of a point, and is also expressed as an angle. The latitude of zero runs north-south through the Royal Observatory at Greenwich in the UK. Places east of this have positive values and west are negative.
When setting up maps in Drupal, developers run into a surprising number of problems caused by mixing up latitude and longitude or mixing positive and negative values for latitude. If you ever come across your map of public toilets in Paris unexpectedly showing you some ocean west of the continent of Africa (Figure 2-6), this is the point where both latitude and longitude are zero, which usually means that your map is not receiving the location data or there is some projection issue.
Latitude and longitude are normally measured in degrees, minutes, and seconds. A sphere has 360 degrees. Each degree can be divided into 60 minutes (referring to the angle, not time). Each minute can be divided into 60 seconds. For example, New York City is at 40° 43′ 0″ North, 74° 0′ 0″ West. This is often turned into a pair of decimal numbers for web mapping or mathematical calculations: 40.716667, -74.
Longitude first
Most of us usually recite coordinates as latitude first, then longitude second. This order is arbitrary and makes things harder for web-based mapping, so often you will see APIs using longitude first.
Web mapping, specifically, tile-based mapping (see Mapping Terms for more details on
tile-based mapping) presents a unique problem: how does one take into
account, map tile positions, viewport positions, and actual map
coordinates? Computer applications, specifically with HTML and CSS,
make the assumption that the position that is most top and left is a
(0,0) point, and that positive
numbers represent going down and right respectively. This means that
tile sets are usually formed like Z/X/Y.png. The Z is the zoom level, and the X and the Y represent where the tile image fits into
the map starting from top left. This then leads to web mapping
libraries basing a lot of calculations on this top/left or x,y system.
Also, when you start to look at latitude and longitude, the longitude
value would be on the x-axis and latitude would be on the y-axis, and
computers usually use a Cartesian x,y coordinate system. So, that is
why we often see longitude first in web mapping.
Data Storage
There are a number of open source and proprietary storage systems for geospatial data, and there are just as many standards for how to represent the data. We will go through some of the basics here, but this is a large topic that we cannot cover completely. The Open Geospatial Consortium (OGC) is a well-established body that helps define many of the open standards used in geographic information systems. It is helpful to understand these concepts as you decide on the best way to gather and store information to display on your map.
Data Types
There are two main data types when talking about geospatial data: vector (such as points and lines) and raster (such as satellite imagery).
Vector data
Most often, geospatial data is represented as geometric shapes and stored sets of numbers representing these shapes; these are most commonly points, lines, or polygons.
Vector data standards
Three of the main geographic data storage types are shown in the table below (Table 2-1) for point data and polygon data (a shape), along with a text version. When we think about a place, typically we refer to its name, for example, New York. But this is not very useful when we try to do calculations or comparisons. What do we mean when we say New York? Are we talking about the city? The state? The borough of Manhattan? And nothing in the names Shanghai and Sydney is any help in determining where they are or what the distance is between them. Representing a location with numbers (for example latitude and longitude) removes this ambiguity.
A location as a latitude and longitude is not always useful, though. It is just a single point. Sometimes we need to represent a shape, normally referred to as a polygon in mapping. Only polygon data can answer questions like “Is Manhattan an island?” Whereas point data can be described as two separate fields in a database, polygons require more complicated data storage types, such as Well Known Text (WKT) and Keyhole Markup Language (KML).
WKT is a versatile format, because it is text that can be read and stored very easily. It is supported in Drupal by the Geofield module, and is used in the tutorials in Geofield Module. As is shown in Table 2-1, WKT is flexible in the type of geographic data it can handle: it can be used to represent a single point with just one latitude and longitude; it can represent a line (a series of points); or it can represent a polygon (a line that connects at each end).
KML is a file format used to display geographic data using a structure that is based on XML. It is widely used because of its support in Google Maps and Google Earth. As a file format it is not suited to being stored in a database. Instead, it is useful when you want to share geographic information with other websites and applications, especially if you do not need to update this information frequently. Using KML files on your maps is covered in Creating an OpenLayers Map Layer from KML, and creating KML files is described in the tutorial in Creating Feeds.
There are many other geographic data formats. GPX and TCX are based on XML. They are formats used by GPS (global positioning systems) devices to track routes and way points. GeoJSON is based on JSON (JavaScript Object Notation), which is generally more compact than XML. Like WKT, GeoJSON is able to represent many different geographic data types.
Raster data
The other main way of representing and storing geospatial data is in raster format. This is the idea of storing data as a continuous surface, a grid of pixels. Raster data is most often a digital image, or set of digital images; it can be satellite imagery, or it could be an image of a street map, but the pixels can represent any value. Elevation is a good example of the difference between vector and raster data; using vector format, elevation is represented with contour lines, but in raster format, it is a digital elevation model (DEM), a continuous surface with each pixel equal to the elevation averaged across the pixel (see Figure 2-7). Digital imagery provides a very efficient way of displaying a lot of data in a fast format. It is also very stable; it looks the same in any browser.
Almost all modern web mapping uses a system of tiles (images) to provide a “Slippy Map” (pioneered by Google Maps). A “Slippy Map” is an interface where a map is displayed inside a rectangle of a particular fixed size, a viewport. You can drag the map around while remaining in that viewport and the page does not need to reload to display new sections of the map. The mechanics of this interface are based on map information stored as a set of sliced-up images (map tiles). These tiles can be loaded on demand depending on which part of the map the user is viewing, and at what zoom level. A tile set can be huge; a world map supporting a typical 256×256 image size with 18 zoom levels (at which point a baseball or football stadium fills a typical laptop screen) leads to billions of tiles. Map tiles for Drupal are discussed and illustrated later in Map Tiles.
Raster tile sets are also often generated from vector data. Tile sets from map providers like Google or Bing use their database of roads and points of interest to create raster tile sets for users to put into their application. Another example is OpenStreetMap, which provides a wiki for users all over the world to add geospatial data describing the world around them to a single vector database. OpenStreetMap is actually just the vector data—the roads and paths and points of interests, all of which is released under a license for anyone to reuse. Anyone can use this data to create raster images. The OpenStreetMap organization also produces some tile sets for people to use in their applications.
Databases
Mapping applications that need to perform spatial calculations (for example to produce a map that shows all whiskey bars within one mile of my office) will use a database that stores data in a spatial format. A spatial database is optimized to store and query geographic information, so it will be faster, for instance, to measure a distance between two points, to return all points within an area or radius, or check if two areas intersect. PostGIS does this for PostgreSQL databases, and MySQL has spatial extensions available.
Currently no Drupal 7 modules support spatial databases. Instead they store geographic data in the database as text or integers (numbers). This is a good solution for a typical Drupal installation, as most mapping websites do not need the most demanding operations. This does make it difficult and cumbersome to do spatial calculations, such as finding what lines intersect, or what points or in a polygon, or what whiskey bars are close to you.
Drupal 7’s database abstraction layer makes it easier to support databases other than MySQL, but there has been only a little work done to develop geographic modules that take advantage of this. The PostGIS module integrates with the OpenLayers module and allows geographic features to be stored natively in a spatial database, but it does not attempt to provide a way to do spatial queries. The Geo module provides both support for spatial databases and queries in Drupal 6, but it has not yet been upgraded to Drupal 7.
Note
Despite the limitations of not having deep integration with spatial databases, Drupal can handle most mapping storage and calculations. The main limitation is doing calculations with complex shapes. Calculating what places are within a rectangle or circle is possible, but determining what falls within an arbitrary shape is more difficult. Similarly, calculating whether a shape that a user draws on a map overlaps with another shape (such as a voting district) is beyond the current capabilities of mapping in Drupal 7.
Given the lack of support for spatial databases in Drupal 7, most mapping applications in Drupal just use a standard MySQL database. This is suitable in most cases, but creates some challenges.
Challenges of Web-Based Mapping
The difficulties of making a map to be used online can be grouped into a couple of areas: there are limits to how much data a web browser can display on a typical interactive map, and there are usability problems when trying to display large amounts of information on a map.
Browser Capabilities
Most interactive maps use JavaScript very heavily. The image tiles that make up the map are loaded using JavaScript as the user zooms in and out and moves around the map. The points, lines, and polygons displayed on the map are added with JavaScript (usually interacting with HTML Canvas or SVG). Even modern web browsers have a point at which they get overwhelmed and become unresponsive because the client cannot handle all of the processing. A good rule of thumb for the limitation of points (including vertices in a polygon) is around 200–500. But, as this is client-side processing, performance varies depending on the user’s browser and hardware, so you should take into account your audience when determining these limitations.
Note
There is no hard limitation, as the limitation comes from the client (the browser and hardware of the user’s computer, phone, tablet, etc.). Modern browsers, such as Chrome, can handle upwards of 10,000 points, while Internet Explorer 7 is more like 300. Unfortunately, there have been no benchmarks for this, as there are so many variables to consider.
Client-side clustering
One approach to lessen the load on the client is to use a form of clustering. If there are a large number of points in close proximity, they will be grouped together, and this grouping will normally be represented by a different style of marker. What happens when this “group” marker is clicked depends on the clustering tool. Either the map may zoom in to show all the separate points in that area, or the group marker may load a pop up displaying a list of all the points that it contains. This helps the browser not have to render all the markers or features, and it will save some resources. However the clustering is done by JavaScript in the browser, so this will not lead to a huge increase in performance.
Library size
A second approach is to use less demanding JavaScript. By lessening the amount of JavaScript that the client has to download, and minimizing the amount of processing on the client in order for you to achieve the map you want, you can squeeze some more performance out of your maps.
One example is Modest Maps. This is a very lightweight JavaScript API for displaying map tiles. By focusing on displaying map tiles and not providing any default markers or buttons for navigating the map, the Modest Maps JavaScript file is 28 KB, compared to over 700 KB for the OpenLayers JavaScript and around 150 KB for the Google Maps JavaScript API. However, the trade off is that you do not have a very full-featured API available.
One company that decided to make this trade off is Development Seed. Development Seed has worked with open source web-based mapping for years and contributed a lot to many Drupal modules. They switched from primarily using the OpenLayers JavaScript library to using Modest Maps. They wrote their own library for the interaction and map widgets to go on top of this, called Wax, and in doing so they reduced file size by 70%. Although there is this considerable improvement in file size, the significant trade off is that Modest Maps and most of the other newer mapping APIs do not yet have integration with Drupal.
Note
Recently, CloudMade released an open source JavaScript mapping library called Leaflet. It is simple and lightweight. There is also a new Leaflet module for Drupal that provides some basic integration between Drupal and Leaflet. It is something to keep an eye on.
Tile rendering
The other way to work around web browser limitations in creating maps is to render your data as raster imagery (see Raster data).
Until recently this required a thorough understanding of complex and/or expensive GIS software. This barrier has been reduced by two recent new technologies. The first of these is TileMill, which is open source software for designing maps and creating map tiles that integrate with various mapping APIs.
TileMill allows you to load in a large dataset containing spatial data and turn it into map tiles. The data could be points, such as locations of health centers, or shapes, such as outlines of states in the US. They can be rendered on the tiles in different colors to represent the data. And despite rendering the data into the map tiles, people can still click on the map to get more information about the data. TileMill and other ways of creating tiles are discussed in Map Tiles.
While using TileMill allows web maps to display massive data sets, the trade off is that the tiles that are generated need to be hosted (an extra expense) and the maps can only contain the information that is available when the tiles were created. It is difficult to display current information in the same way that would be possible if the information was being pulled straight from a database. Every time you update the data you must rerender all the tiles. If you are mapping incidents of violence during civil unrest, you do not want to wait a day or a week to update your map.
The other interesting new technology that uses tile rendering to display large data sets is Google’s Fusion Tables. Using the Google Maps JavaScript API, you can query data and map tiles are rendered on the fly to display that data. Currently, though, there is no integration in Drupal for Fusion Tables.
Usability
Large amounts of data are a problem for the web browser, and are also a problem for people looking at the map. A map with hundreds of markers covering it is overwhelming. Too much information is as bad as too little. Indeed, it is worse, as it takes longer to load!
Another problem happens when the map is trying to display several pieces of information at the same location, for example, showing three businesses in the same building on different floors. With a standard configuration it is impossible to access more than one marker. Clustering, discussed earlier, is one way to deal with this. The other way is to manage what data is displayed and to give users different ways to refine this, for example by using exposed filters in views, or detecting a user’s location and showing them what is nearby. This is discussed in Chapter 4.
As more people access websites through mobile browsers on tablets and smartphones, it is important to consider the user experience for this audience. In other words, keep the maps you build simple, so that the information can be seen on small screens and so that the maps load quickly. The latest versions of the APIs for both Google Maps (V3 Maps API) and OpenLayers (2.11) have added a lot of improvements for using maps on mobile devices. They have made the maps faster, have added support for touch screens (such as zooming in and out by pinching the screen), and integrated geolocation to show where a user is on the map.
In some cases a map is not the best interface for geographic data, whether because of the size of the data set or because people can not access visual information due to a visual impairment. For both accessibility and search engine optimization, it is often worth creating alternative text-based ways of displaying the same data. The Views module in Drupal makes this straightforward. A map is just one display type in Views; you can provide the same information as a list or table. This is covered in Other Ways of Displaying Spatial Data.