I’m beginning to make some progress with the lobbying project. Last week I got it spitting out data; in mid-week, I optimised the process of loading the meetings from the ScraperWiki API into NetworkX. Hint: the obj_hook keyword argument in python’s json.load() function is really useful!
This weekend it’s producing information about lobbies, ministers, and government departments. I’ve got implementations nearly ready for a couple more dimensions of data – providing each actor’s network degree by month, and trying to measure the extent to which ministers act as gatekeepers or flak-catchers. The first of those involves reimplementing a bit of NetworkX – you can’t ask for node properties excluding certain edges by attribute, or at least you can’t do so without creating a new subgraph, which seems ugly. The second, at the moment, counts the edges of a node if they have a higher weight than that of the node itself and expresses the sum of those edges’ weights as a percentage of the total meetings that minister had. That doesn’t take any account of time, yet.
I’m thinking of using Google App Engine to deploy it, running the data generator as a cron job and using the bulk uploader utility to slurp the results.
As a taster, the biggest single private interest lobbying Government is Barclays Bank, followed by Shell, the World Bank, the London Stock Exchange, BP, RBS, BAE, Standard Chartered, Lloyds, and Ratan Tata. This may not be that surprising. Neither is it very surprising, if somehow comforting in an old-fashioned way, that the two biggest lobbies of all are the Confederation of British Industry and the TUC, which is achieving about two-thirds the lobbying effort of the CBI and about twice that of Barclays. I was surprised to find that lobby 26 is Facebook, above Tesco, Microsoft, or UNISON. (Google is far, far down the list.) The highest placed individual trade union is the CWU at 24, between HSBC and the Electoral Commission. The littlest lobby is a nursery school in Leeds that got herded into a Big Society meeting with Nick Hurd MP.
I’m not so sure about using this model to assess the ministers, as we’re using a priori weightings on them. But the decision to lobby a given minister must contain some information about the lobbyist’s perceptions of their power and influence. Britain’s most lobbied minister is Chris Grayling MP, Minister of State for Employment, who achieves a weighted degree of 4.2, not far off twice the prime minister. David Willetts, Vince Cable, Nick Clegg, and Francis Maude are the next four before the prime minister. They range between 2.8 and 2.6 with the PM on 2.3. Britain’s least influential minister appears to be Baroness Warsi, minister without portfolio, on a score of 0.057.
BIS is the most lobbied department on 12.42, followed by the Department for Work and Pensions on 9.65, the Treasury on 7.065, the Cabinet Office on 5.62, and the DCLG on 3.825. Delight to the econophysicists (are they still around?): the distributions seem to show a nice power-law relationship! Which tells us what precisely? Well….not much except that it’s a social network and they usually have them!