Blog

• A love letter to OpenStreetMaps

  Monday, Dec 11, 2023

  I think we have begun to take maps for granted - they seem woven into the fabric of our world. We expect to be able to look up directions, and scour places of interest all around the world. We think we know what Greenland looks like, and we barely register that you can keep scrolling horizontally on the map, and Antartica looks like a single contiguous, unbroken landscape. We benefit from how easy it is to use modern mapping tools, but we also cede a lot of power to technology companies.

  Just like how many publishers are choosing the self-host and publish on less centralized systems, this revolution is also happening with maps, and OpenStreetMap is leading the front. OSM is the Wikipedia of geospatial information. Volunteer-led, meticulously tagged, it’s an internet artifact whose existence seems to defy common-sense. I’ve only encountered it previously as the ‘map provider’ for Mapbox, and now I understand their relationship a lot more.

  Maps are expensive to serve and render - some back-of-the-envelop calculations have shown that it can take up to 50TB of data if it is rendered naively, which is why organizations like Mapbox have been able to build a business being a map tooling provider.

  This is the magic - OpenStreetMap also hosts map tiles, free for non-commercial use. If you go to openstreetmaps.com you’ll find a map like experience that is delightful. You get to see the parks in New York City or the tiny alleys in Istanbul. There are tiles that are translated into multiple languages. There is also a host of open-source software that allows people to work with OpenStreetMaps data, including tagging, querying, and serving map tiles. There is even software to allow you to render your own map tiles. Other than making a donation, it’s not clear to me how we can make these enterprises more self-sustaining.

  When I think of fundamental technologies of our era, I think of encryption, networking, databases, technologies that we largely don’t have a tactile relationship with. However, maps are something we continue to interface day in and day out, yet we have forgotten that maps are, like with all of our technologies, developed with a set of cultural and political assumptions and encode the biases of their creators. OpenStreetMaps really is a labor of love that we all benefit from, and we should do our best to keep it alive.

---

• Speeding up Leaflet Markers

  Wednesday, Nov 1, 2023

  I recently published a map of the collection of wastebaskets in NYC. Initially, I was curious about whether I can show that the density and availability of wastebaskets vastly differs depending on which borough you are in. It was a project that helped me understand how to work with geospatial data, everything from downloading OSM layers and objects, querying PostGIS, and understanding the conversion between different mapping projections. But right now I’d just like to discuss the performance of map markers on Leaflet.

  The dataset I was displaying contained over 20k points, and displayed terrible drag performance when being added in the canonical way:

  datapoints.forEach(({lat, lng}) => {
      L.circle([lat, lng], {
        color: 'blue',
        radius: 0.001
      }).addTo(map);
  });
  

  

  Notice how the the image lags behind the pointer by several seconds. After digging through the source code in vain, I realized that the issue lay in a ‘style flush’. Looking at the profiler more closely, we see that the style flush was caused by adding a class in the _onMove handler of the layer.

  

  This profiling was done retrospectively - while I was working on my visualization, I wanted to speed up the interactivity of map. A tried and true way to do this is to reduce the number of points I was showing on the map. I tried heatmaps, clustering, and even a novel approach by showing the roads that were ‘serviced’ by wastebaskets. I considered sampling but it would need to be fairly intelligent, since I would only want to reduce the density of points only in already dense parts of the map. Nothing felt as clear as displaying the points directly on the map. So what I ended up doing instead was rendering a raster image of the points.

  Rendering map points

  To do this, we first calculate the extent of our displayed data. This gives us a lat-lng rect that we can then use to compute the aspect ratio of the image we want to generate:

  const latLngToContainerBounds = function(map: Map, b: LatLngBounds) {
    return bounds(
      map.latLngToContainerPoint(b.getNorthWest()),
      map.latLngToContainerPoint(b.getSouthEast())
    );
  }
  
  const getExtent = function(arr: LatLngLiteral[]) {
    let initialBounds = latLngBounds(arr[0], arr[1]!);
    return arr.reduce((bounds, basket) => bounds.extend(basket), initialBounds);
  }
  
  const targetWidth = 10000; // this is how large of a raster image we want to generate
  const radiusInM = 50; // this is how big we want to draw each marker
  
  const latLngBounds = getExtent(arr);
  const containerBounds = latLngToContainerBounds(map, latLngBounds);
  const origin = containerBounds.getTopLeft();
  
  const { x, y } = containerBounds.getSize();
  const width = x;
  const height = y;
  
  const aspectRatio = height / width;
  const scaleFactor = targetWidth / width;
  

  We then take two random points, in this case we just take the first two points:

  const p1 = map.containerPointToLatLng([0, 0]);
  const p2 = map.containerPointToLatLng([0, 1]);
  
  const pxInMeter = map.distance(p1, p2);
  
  const radiusInPx = Math.max(Math.ceil(radiusInM/pxInMeter), 1);
  

  The variable pxInMeter is the distance in meters, in the map’s CRS, between the two points. This is used as a scale factor, which we use to determine the size(in pixels) of each marker.

  
  const canvas = document.createElement('canvas');
  canvas.width = 500;
  canvas.height = 500;
  const ctx = canvas.getContext('2d')!;
  
  canvas.width = targetWidth;
  canvas.height = Math.floor(targetWidth * aspectRatio);
  
  for (const latlng of datapoints) {
      const pt = map.latLngToContainerPoint(latlng).subtract(origin).multiplyBy(scaleFactor);
      ctx.beginPath();
      ctx.arc(pt.x, pt.y, radiusInPx, 0, 2 * Math.PI);
      ctx.fill();
      ctx.closePath();
  }
  

  And last but not least, we render the image using the canvas.toBlob function and we return as a url which we then display on the map at the extent that we calculated earlier:

  canvas.toBlob(b => {
    if (b == null) {
      reject('error generating image');
      return;
    }
  
    let url = URL.createObjectURL(b)
    imageOverlay(url, latLngBounds).addTo(map);
  });
  

  This technique works pretty well, and manages to draw the load the image within 3s, with no issues with drag performance.:

  

  Try it here. Now this obviously introduces some other issues, namely that we now have to hit test for any interaction, and that since the marker is a raster, we will see pixelations as we zoom in. In the future I think this can be combined with Leaflet.LayerGroup.Conditional could allow us to show actual markers when zoomed in enough such that the number of markers are drastically smaller, or we could only render the points that are onscreen at any given point, and updating as the map bounds changes.

---

• Animating leaflet points

  Wednesday, Oct 4, 2023

  This is a naive way of animating points in leaflet.

  In a css file, add the keyframes and details of the css animation:

  @keyframes fadeInOpacity {
  	0% {
  		opacity: 0;
  	}
  	100% {
  		opacity: 1;
  	}
  }
  
  .fadein {
  	animation-name: fadeInOpacity;
  	animation-iteration-count: 1;
  	animation-timing-function: ease-in;
    /* animation-duration is set in code to a random number */
  }
  

  Then, where you create the Leaflet circle:

  let c = L.circle([0, 10]);
  c.addTo(map);
  // add animation to circles
  let e = c.getElement() as HTMLElement
  e.classList.add("fadein");
  e.style.animationDuration = `${Math.random() * 2}s`
  

---