Best lobby metrics (lobbylyzer? lobster, for LoBbying Social Topology ExploreR?) result yet. Today I implemented my gatekeeper vs. flakcatcher metric – it averages the edge weights of all the neighbours of a minister, and returns a ratio of the difference between this and the average weight and the difference between the minister’s weighting and the…
Read More lobbyists lobbying for lobbyingAfter this post and the outstanding response to it, I’ve just been working on the lobby project’s underpinnings, specifically to backport some data cleaning from the analyser script into the original scraper, and to fix the one-edge-per-row version of the scraper. As a result I’ve had to flush the datastore and also search out some…
Read More Routine maintenanceI’m beginning to make some progress with the lobbying project. Last week I got it spitting out data; in mid-week, I optimised the process of loading the meetings from the ScraperWiki API into NetworkX. Hint: the obj_hook keyword argument in python’s json.load() function is really useful! This weekend it’s producing information about lobbies, ministers, and…
Read More lobby: updateI’m beginning to make some progress with the lobbying project. Last week I got it spitting out data; in mid-week, I optimised the process of loading the meetings from the ScraperWiki API into NetworkX. Hint: the obj_hook keyword argument in python’s json.load() function is really useful! This weekend it’s producing information about lobbies, ministers, and…
Read More lobby: updateSo, a free, jetlagged afternoon by the pool in Palo Alto, after this experience. What to do? Obviously, hack on some code. I dragged out the lobby analyzer project and got it to actually spit out ministers, lobbyists, and MPs, with their weighted degrees in the network, onto the command line. The conclusions are dreadful…
Read More I cannot begin to theorise what may have caused such a catastrophic malfunctionAbsolutely, certainly, without a doubt the most important bit in the News of the World case: Last Friday, a high court judge ordered NoW to make available Mulcaire’s notes to the growing list of people suing the paper. Justice Geoffrey Vos, who is in charge of the hacking cases, ordered “rolling disclosure” to all claimants.…
Read More BritiLeaks: the best website that doesn’t exist yetAmusingly for a comment on scalability, I couldn’t post this on D^2’s thread because Blogger was in a state. Anyway, it’s well into the category of “comments that really ought to be posts” so here goes. So various people are wondering how the New York Times managed to spend $50m on setting up their paywall.…
Read More Scaling and scoping the NYT paywallThings to get out of the data in this scraper of mine: for each lobby, the monthly meeting counts, degrees in the weighted multigraph, impact factor (i.e. graph degree/meetings to give an idea of productivity), most met ministers, most met departments, topics. For each ministry, meeting counts, most met lobbies, most discussed topics. For each…
Read More Self-binding note: lobby metricsOne thing that is perhaps being overlooked by people discussing whether or not it would be wise to impose a no-fly zone over Libya is exactly what such a zone would set out to prevent. When it was first suggested, it was inspired by the general horror that the Libyan government was having crowds of…
Read More And it’s one jet airliner, for ten prisoners…Pulling together various resources, I’m beginning to get a picture of what happened with the cut-off and restoration of the Internet in Egypt. First up, at least in some senses, it may be valid to say that the Internet played a role – Arbor Networks observed that traffic to and from Egyptian networks (and between…
Read More A Little Bit of Egyptian Internet Twaddle