Dynamically loading thousands of markers in Google Maps

UPDATE (September 17, 2011): If you want to effectively handle many thousands of points in a Google Map then the only real solution is using Google Fusion Tables. I hit on that as a solution a few months ago and it’s been working perfectly for The Broadband Map and it’s over 400,000 loadable points. Blog post with details coming soon.

When you’re making a Google Map with more than about a hundred markers, there are two established solutions: MarkerManager and MarkerClusterer.

MarkerManager:

Loads/unloads markers in a map as you pan the viewport
Allows adding sets of markers to specific zoom ranges

The restriction of markers to zoom levels means that you can use MarkerManager to roll your own clustering. That’s great for the end user because you serve up a completely customized experience. The downside is that you have to completely customize the experience. That means you’ll have to have some metric by which to split out your points into zoom levels. Say you have five thousand points: you’ll need to split them into zones so that no more than a few hundred (at the most) are visible at once.

MarkerClusterer

Automatically clusters markers

That point is a big deal. Clustering isn’t cake, and marker cluster performs the tricky task with aplomb. The icons and styling is a little eh out of the box, but it gets you to the point where you actually care about icons and styling immediately.

The problem with both

Both solutions really want to work off a fixed array of markers that you’ve already pulled in. Feed either solution up to a couple thousand points of data and they’ll handle it (either automatically or with some work on your part) and you can move on.

But neither solution really deals well with loading almost a hundred thousand points of data. Obviously we won’t feed every map every point all at once, and neither MarkerManager nor MarkerClusterer handles dynamically adding and removing points that well since they are both doing a fair bit of calculation that requires knowing about every point.

My solution

My solution, so far, is to:

dynamically load markers until we reach a threshold
keep a hashtable of markers that have already been added
after the threshold has been reached, remove markers that aren’t currently within the viewport
remove all markers from the map when the user has zoomed out, and don’t load any markers until the user zooms back to a reasonable level

In my case, the point data is really only useful at the city level. This allows me to completely ignore the higher zoom levels.

Here’s the essentials from the code I have so far:

function requestMarkers() {
  // no loading markers when zoomed way out
  if (map.getZoom() < 11) { return };
  // start dropping markers before the browser bogs down
  if (markers.length > 50) {
    dropSuperfluousMarkers();
  }
  $.ajax({
    // [snip typical ajax data request]
    success: populateMarkers,
  });
}

// add the markers from pointsData if they haven't already been placed
function populateMarkers(pointsData, status, xhr) {
  for (var i = 0, ii = pointsData.length; i < ii; i++)  {
    var lat = pointData.geom.x;
    var lng = pointData.geom.y;
    var marker = new google.maps.Marker({
      position: new google.maps.LatLng(lat, lng),
      draggable: false,
      animation: google.maps.Animation.DROP, // whee!
      flat: true, // disables some styling DOM elements for a faster marker
    });

    // hash the marker position
    coordHash = calculateCoordinateHash(marker);

    /*
    if we haven't seen this hash, add the marker and mark as seen

    without this, the markers array quickly grows unwieldy as duplicate points
    are loaded
    */
    if(seenCoordinates[coordHash] == null) {
      seenCoordinates[coordHash] = 1;
      markers.push(marker);
      marker.setMap(map);
    }
  }
}

// turn marker coordinates into a hash key
function calculateCoordinateHash(marker) {
  var coordinatesHash = [ marker.getPosition().lat(),
                          marker.getPosition().lng() ].join('');
  return coordinatesHash.replace(".","").replace(",", "").replace("-","");
}

// remove markers that aren't currently visible
function dropSuperfluousMarkers() {
  mapBounds = map.getBounds();
  for (var i = 0, ii = markers.length; i < ii; i++)  {
    if (!markers[i]) {continue};
    if (!mapBounds.contains(markers[i].getPosition())) {
      // remove from the map
      markers[i].setMap(null);

      // remove from the record of seen markers
      coordHash = calculateCoordinateHash(markers[i]);
      if(seenCoordinates[coordHash]) {
        seenCoordinates[coordHash] = null;
      }

      // remove from the markers array
      markers.splice(i, 1);
    }
  }
}

// clear all markers from the map, empty the markers array and seen markers
function clearMarkers() {
  for (var i = 0, ii = markers.length; i < ii; i++)  {
    markers[i].setMap(null);
  }
  markers = [];
  seenCoordinates = {};
}

Tags: javascript googlemaps programming

XyzzyB