Helping those on Free School Meals during half-term
Earlier in the year, footballer Marcus Rashboard started a successful campaign to make sure those who get free school meals had access to food during lockdown and over the summer holidays. He was awarded an MBE for his services to vulnerable children.
A few weeks ago, with the autumn-half-term holidays approaching, Marcus launched a petition to #EndChildFoodPoverty which has now had over 1 million signatures (see the geographic distribution of the petition on our constituency hex map). On Thursday 22nd he started receiving offers of free meals from businesses and organisations via Twitter and Facebook and he retweeted many of them. We were asked on Twitter if we could put these offers on a map.
Crowd-sourcing the data
The first thing to do was get some structured data. The tweets/Facebook-posts that Marcus was sharing were hand-typed offers of support from businesses, organisations, and individuals and tagged with a city/town name by Marcus Rashford. Just collecting those offers by town/city would only be of so much use - cities are big places and travel is hard if you have little income and there are pandemic-related travel restrictions. You really want people to be able to quickly find the closest offers of support. It felt like the best way to organise the curation of this data was via a collaborative Google Sheet. Thankfully John Vesey let us know that someone had already started one.
The Google Sheet was started by someone named Anjali who we'd never met (either virtually or in real life). Anjali had already done a couple of vital things: she had structured the data with the sorts of columns we needed and she had made the Sheet open so that we could access it cleanly via software. That meant I could start working on a map entirely independently of her and as she continued to add to the data.
Getting the data on a map
I wanted the map to be as up-to-date as possible and, preferably, just a stand-alone webpage that didn't need anything fancy running on a server. Thankfully, publicly shared Google Sheets allow you tobuild a link to export the sheet as a CSV file and, because Google enable cross-origin resource sharing, I can make a webpage somewhere else that grabs that CSV output. I knocked-up a webpage with an interactive map using Leaflet.js. The next problem was knowing how to connect the CSV data with the map.
Ideally the Google Sheet would collect the latitude/longitude of each offer but it was unreasonable to expect many people to be able to add that. The best, most-structured, geographic information we had in the Google Sheet was the postcode. Helpfully, the ONS publish a postcode lookup file which gives you lots of information about every postcode in the UK. That is a 1GB file! I didn't want server-side code so I needed to make a cheap-and-cheerful "search" system that just uses static files of reasonable size. I just needed a way to split up all the UK's postcodes into searchable, useable, chunks.
|Outward code||Inward code|
The table shows how a postcode is structured. Once you get down to Sector you've got down to a few hundred postcodes at most. So I processed the ONS file into a directory structure of the style
LS/9/8.csv contains 264 postcodes along with their latitudes and longitudes (and LSOA for use later). In total the directory contains around 100MB of data but the biggest file is only 15kB. We now have a cheap-and-cheerful postcode lookup using static files.
My code could now load the Google Sheet directly, lookup the postcodes (a few hundred kB of file requests), put them on a map, and then update from the Google Sheet automatically every 5 minutes (or when you reloaded the page). It was all ready by lunchtime on 23rd October. Anjali then linked to the map from the top of the Sheet which temporarily broke my map as my code had been expecting the column headings to be on line 1! That was a quick thing to fix.
Adding more data
As the Friday went on it became clear that several people/groups had had similar ideas and there were several projects curating data independently and showing them on their own maps. By the Saturday, another curation project - AllOfUsTogether from TechForUK - had amassed more offers than Anjali (they were able to add Anjali's data to their data because her Sheet was public) but weren't yet sharing their Sheet back. By early the next week they'd made their Google Sheet open and that meant I was able to fairly quickly add it to my map - my map now loads both Google Sheets.
As a result of them being open and being able to put it into my tool I was able to find around 15 invalid postcodes in their dataset (a very small fraction but effectively "lost" offers). Because they had links to the original offers, I was able to look through each of them and work out the correct postcode. In some cases the original businesses had the wrong postcodes on their own websites! I made sure to send the AllOfUsTogether folks the postcode corrections so that they could improve their dataset.
Alternatives to a map
Not everyone is comfortable with maps so I'd been showing a list underneath the map too. Adding in AllOfUsTogether's data massively increased the size of the list (from around 300 to around 1800 at that point) so I needed a way to help people look through it. I quickly wrote a simple type-ahead search that used the name/town/city/postcode fields in the Sheets to sort the list. So, for instance, you can type "Leeds" and narrow the list down to 53 offers or type "LS12" and find the two offers in that postcode district.
In just a few hours, people who didn't know each other were able to create a practical and useful interactive map. This was possible thanks to collaborative tools (Google Sheets), by being "open by default" with the data collection/curation, and by being able to split the effort into self-contained tasks that could be worked on in parallel.
Ultimately it didn't matter if people used our map or found offers of support through the other tools people developed e.g. Freemeals.uk, AllKidsMatter's map, or AllOfUsTogether's map. What mattered was people finding the offers when they needed them.
In future, I'd recommend any similar projects that crowd-source data do their best to share the data back to the community right from the start. Don't leave sharing the data (not just a visualisation of the data) as a something to think about later down the line. Being open helps reduce duplication, it means others can use the same data to make alternative visualisations (letting you focus on the curation task), and you then have more people who can spot mistakes and help improve the data.