My colleague Alex has already written about looking forward from this election, so here I am going to look back at the technical work that was involved for the election, and in getting all the new MPs into TheyWorkForYou.
Boundary changes
This election was the first UK Parliament election with boundary changes since 2010. Due to the long-running nature of TheyWorkForYou, which has been around now for over 20 years, this can throw up some interesting challenges. In this particular case, it turned out we were using two different JSON data lists of constituencies – both containing the same data, but one also included the other Parliaments and Assemblies, whilst the other included alternative names for some constituencies. I took the opportunity presented to merge these together and update the bits of code to use the one consolidated dataset, and then added in the 650 new constituencies to the JSON data.
Loading the new constituency data into TheyWorkForYou then threw up another historical problem – the constituency table was still using the very old Latin-1 character set encoding, rather than a more modern encoding such as UTF-8, that almost everything we have uses. This had been fine until now, with even Ynys Môn covered by that encoding, but the new constituency of Montgomeryshire and Glyndŵr contained a letter that Latin-1 could not cope with, leading to a quick emergency upgrade of the table to UTF-8 (thankfully this is a backwards compatible encoding, so worked without issue).
We had already generated data of the new constituencies and loaded these into our lookup service MapIt before Christmas. Ordnance Survey more recently published the official dataset of the boundaries, which we could then import via our usual processes, though even this raised a small issue to be resolved. It turned out in the last data release OS had given the parts of two county council electoral divisions with detached parts (Lightwater, West End and Bisley and Thorpe St Andrew) different identifiers, which they had reverted in their new release, causing our import script to get a bit confused – resolved with a small manual script.
Displaying on TheyWorkForYou
In the period before the election, we knew people would be using our site as a postcode lookup, perhaps to look up their previous MP but perhaps also expecting something useful for the upcoming election, which we wanted to provide, and so we used Democracy Club’s API to show election candidates and link to their WhoCanIVoteFor and WhereDoIVote services. We also displayed your boundary changes using the new constituency data mentioned above.
TheyWorkForYou isn’t just the UK Parliament, though, it also covers the Scottish and Welsh Parliaments, and the Northern Ireland Assembly, so we also had to maintain the provision of that information to people – email alerts for those bodies continued throughout as usual, and the postcode lookup kept showing people their representatives in the devolved nations.
Once the election closed, we automatically updated our messaging, and the next day switched back to our normal behaviour of taking you directly to your MP page in England, and showing you your MP and other representatives elsewhere.
We had a fun issue where some people were getting their new MP, whereas some were getting the old MP – during the period of dissolution, when there are no MPs, we have a configuration flag to enable the site to know it should return the latest result even if it’s not current (you don’t want this all the time, when e.g. an MP has resigned or died), but once new data was being loaded in, one database query was returning results in a random order; fixed by adding some sorting by descending end date.
Election result data
At the last election in 2019, we took a live feed of election results from Democracy Club, who have collected all the candidate information for their Who Can I Vote For service – which all began as the result of a mySociety project back in 2010.
Democracy Club were performing the same service this time, and gratifyingly it was quite a small change to have our 2019 code work with any 2024 changes to the source information (incidentally, there aren’t a lot of narrative doctests in our codebase, but I quite like the one in use there!).
This script would do half the job, of taking in some source data (who has been elected, and including their TheyWorkForYou identifier if they already had one due to being a previous representative of some sort) and amending our source JSON data to add the newly elected representative.
The other half is loading that source data into the TheyWorkForYou database for display on the site. Our normal loading script works fine, but looks through all the source data to see if there have been any changes to take account of. For the election, we don’t need it to do all that, so I tweaked the script to only do the minimal necessary to load in newly created information.
These two scripts were then added to a cron on our server, running every few minutes through the night. I did stay up long enough to check that the first few worked okay, before leaving it to itself from then on. I also set it up to pipe its output to our Slack channel, so people could see it operating:
This also meant as the final few trickle through, it’s popping up reminding us it’s still doing its job:
All the results (bar the one we’re still waiting for) are now committed to the repository, joining all our other open data.
Support TheyWorkForYou and our work
TheyWorkForYou and WriteToThem are run by mySociety, a small UK charity. We’re a very efficient operation and do a lot with a small team; if we had bit more money, we could achieve a lot more.
We want to see a transparent, resilient democracy, with equal access to information, representation and voice for citizens. If you believe in this vision please donate today to enable greater transparency and accountability of the next government.
—
Image: Moritz Kindler