Geeks try their hand at mining…data – Guest Blog by Luke Balleny, Thomson Reuters Foundation

A man types on a computer keyboard in Warsaw, on February 28, 2013. REUTERS/Kacper Pempel

A man types on a computer keyboard in Warsaw, on February 28, 2013. REUTERS/Kacper Pempel

Stepping into the cavernous room in the heart of London’s West End last Saturday, I felt very much out of my comfort zone. While everyone there spoke fluent English, it was filled with people who spoke a different language than I do – the language of computer code.

I had come to report on and judge an event called “Follow the Data Hack Day: London”, part of a much-needed effort to help extractive transparency campaigners and the media to make sense of the impending tsunami of data that will come out of the oil, gas and mining industry in 2014. (If you’re wondering, there’s more on what a “hack day” is just a few paragraphs down.)

The U.S has passed a law that forces oil, gas and mining companies listed on the U.S. stock exchange to publish the royalties and taxes that they’ve paid to the governments of the countries in which they operate. The payments will be listed on a project-by-project basis and cover anything over $100,000. The European Union is expected in June to sign off on a similar law that will cover European extractive companies.

While transparency campaigners have lobbied long and hard for these two laws, most would acknowledge that they are not yet ready for what is expected to be an enormous amount of data. In what format will the data arrive? Will it be easily comparable across countries and across projects? Are the campaigners numerically literate enough to be able to interpret the data? “Follow the Data Hack Day”, organised by professional hack day coordinators Rewired State was set up to address these very issues.

For the uninitiated, the concept of a hack day, or “hackathon”, is to bring together a group of technology developers for an intense 24-hour period (sometimes longer) during which the developers are tasked with creating a piece of software or an app that addresses a particular challenge. The objective of the “Follow the Data Hack Day” was to create something that would either help alleviate poverty in resource-rich countries or highlight discrepancies in the extractive data that could point to possible corruption.


The day started with introductions from Rewired State and the UK’s Department for International Development (DFID) and was quickly followed by short speeches from the judges as to what they were looking for in the apps. It should be noted that the developers were not campaigners or governance experts, so in order to help them understand why their apps would be important, they received a crash course in extractive transparency, corruption and the resource curse from Marinke van Riet, the international director of the Publish What You Pay (PWYP) coalition and a member of the judging panel.

Following the introductions and the setting of the challenge, I left to enjoy the rest of my Saturday while the developers toiled away on their projects.

A little over 24 hours later I returned to find the hackers finishing up their projects. In line with the cliché surrounding late-night computer programming, large quantities of pizza had been consumed as they had worked long into the night. And while I wasn’t there to see if for myself, I was assured that some of the developers hadn’t even left the room, simply laying sleeping bags down on the floor in order to save crucial time as they raced against the clock.

The 10 hackers had initially split themselves into two teams of five, but one of the hackers split off to pursue his ideas by himself.

Perhaps surprisingly, the hackers created three very different products that they presented to the audience of judges, transparency campaigners and fellow hackers.

The first was a website that allowed the user to compare and contrast data from the oil industry across countries and included figures such as oil output, price and estimated barrels of oil still to be explored. While not visually particularly impressive, the technical aspects behind the front end of the site were what wowed the plaudits.

The second team to present their app was the only team to directly attempt to meet the challenge set by the organisers, in that they made an effort to analyse the relationship between resource wealth and poverty via a number of interesting and insightful graphs.

The third presentation was by the individual who had split off from one of the other teams. His skills were in web design as opposed to coding and developing, and this showed in the result. He had designed a website that would allow campaigners to upload pithy facts about the extractive industry and either share the entire website or tweet each fact individually. While there was no specific data component to it, it looked good and was the only project which could have been used right away.

The results of the hackathon may not have solved the resource curse, but they were a fascinating look at what a group of talented hackers could do given 24 hours and a few spreadsheets-worth of data. If their efforts and their creativity are representative of what is possible once extractive companies are forced to publish what they pay in 2014, the extractive transparency movement should have no problem holding governments and extractive companies to account.


2 thoughts on “Geeks try their hand at mining…data – Guest Blog by Luke Balleny, Thomson Reuters Foundation

  1. Pingback: Open data in extractives: meeting the challenges | Tim's Blog

  2. Pingback: Open Data in Extractives: Meeting the Challenges- Guest Post by Tim Davies, Practical Participation | Follow the Data Blog

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s