Hey look, someone’s writing a Python library to analyse US electoral spending from a network theoretic viewpoint. A good introduction to the problems and also why it’s so intractable in terms of nice charts.
Read More someone else has a #lobsterSo I spent part of my weekend tinkering with robots. I should actually have committed more but I was conflicted and a bit overscheduled, but anyway I got to the UK NAO user group’s hackathon out at Queen Mary College. This was an interesting experience; I’ve been to a few events like this before in…
Read More A weekend with robotsSo, did ya ever want to know social network analysis metrics on the Department for Culture, Media, and Sport’s special advisers during the spring of 2011, as Rupert Murdoch’s interests in the UK feverishly lobbied Jeremy Hunt and his staff and we raced towards the epochal eruption of Hackgate? ‘Course you did. In June, 2011,…
Read More Hunt, through a non-sexist prismSo, Wired writes up three West Point professors and their algorithm to decide which members of a terrorist network to zap. Apparently they implemented it in 30 lines of Python. The paper is here, with some pseudocode and the tantalising hint that they used NetworkX, but no Python. However, even the Wired piece tells us…
Read More Lib Dems: not quite uselessOK, so I found out why data was apparently disappearing. Or rather David Jones from ScraperWiki did. It turns out the ScraperWiki datastore works rather differently to what you might expect. I theorise this has something to do with the fact it was a key-value store before it was an SQLite database, as it tastes…
Read More A ScraperWiki snippetSo I completely forgot I needed to register for OKFN’s Open Interests Europe hackathon last weekend, which even had a lobbying track, and just round the corner from the office, too. I decided to have my own lobbying hackathon by eating pizza and caffeine pills and being misogynistic spending my weekend finishing the Lobster Project’s…
Read More A Project Lobster progress report!The question isn’t so much “did Eric Pickles eat all the pies?”, it’s “who paid for the pies, and how many did he declare in the register of members’ interests?”. TBIJ is on an absolute tear on Tory lobbying stories at the moment, and the combination of photo and caption for the Eric Pickles one…
Read More it flies! well, that’s what it was bloody well designed to doMy lobbying project has been entered in the Open Data Challenge! Someone posted this to the MySociety list, with rather fewer than the advertised 36 hours left. I was at a wedding and didn’t read it at the time. After my partner and I had tried to invent a tap routine to the back end…
Read More meet Project LobsterSo it was OpenTech weekend. I wasn’t presenting anything (although I’m kicking myself for not having done a talk on Tropo and Phono) but of course I was there. This year’s was, I think, a bit better than last year’s – the schedule filled up late on, and there were a couple of really good…
Read More OpenTech washup, and an amended resultAfter this post and the outstanding response to it, I’ve just been working on the lobby project’s underpinnings, specifically to backport some data cleaning from the analyser script into the original scraper, and to fix the one-edge-per-row version of the scraper. As a result I’ve had to flush the datastore and also search out some…
Read More Routine maintenance