Guest post by James McKinney, open data hacker and creator of PatinerMontreal.ca.

The amount of public civic data is exploding, but much remains closed or unavailable. Here’s a canvas of existing projects, highlighting opportunities and challenges for open data.

Most civic projects target national governments. mySociety (UK) runs TheyWorkForYou, which allows you to track parliament’s MPs, bills, and debates. Canadian equivalents are Apathy is Boring‘s Citizen Factory and Michael Mulley‘s OpenParliament. In the US, Participatory Politics runs OpenCongress, which it has recently generalized for any jurisdiction with the OpenGovernment project. Some projects target voting records specifically, like The Public Whip in the UK, and How’d They Vote? in Canada. Americans go further and connects the dots between votes and money with MapLight.

Most of these projects are open-source, meaning anyone can get the code and download the data.

It is important to note that most projects do not use open data. In Canada’s open cities, most of the open data relates to transportation, recreation, and geography; very little of it would embarrass anyone. The Government of Canada discloses expenses, contracts, grants and contributions, but the data is spread across 132 departmental web sites and is available only in HTML, which is not an open format. The City of Toronto publishes its council’s meeting attendance and voting record in open formats, but not its operating budget or awarded contracts. In order to extract this data, developers must write scrapers, a time-consuming and error-prone process.

Whether it’s Resto-Net (Montreal), Eat Sure (London), or Eat Safe (US) for restaurant inspections; LondonTrash,VanTrash, or Déchets Montréal for garbage collection; MyTTC (Toronto), hbus.ca (Halifax), or Where is my bus? (Ottawa) for public transportation; or  CommonSpace (Philadelphia), Rinkside (Ottawa), or Patiner Montréal for recreation – these projects show that making data available is of enormous value to citizens.

It’s not only politically-harmless data that should be open, however. To build a strong democracy, politically-sensitive data should be open, too. In the US, LittleSis tracks the ties between powerful people and organizations; Transparency Data tracks money in politics from lobbying to campaign contributions; and PolitiFact monitors campaign promises and fact-checks politicians. These projects demonstrate the importance and usefulness of this data.

In many cities, civic projects are very difficult to complete, because the relevant data is often either difficult to retrieve or unavailable. Where it’s available, Openly Local (UK) makes municipal governments more accessible, and EveryBlock (US) tracks what’s happening in your part of town — from news and events to crimes and license revocations. Police services are often uneasy about publishing crime data, but in Oakland police cooperate with developers to produce Crimespotting.

Newsrooms around the world are making use of closed data despite the challenges. ProPublica sets an example, with its stories on which doctors are receiving money from drug companies and which nurses currently working in California were previously sanctioned out-of-state. And when the data is unavailable, services are popping up to make freedom to information requests easier to track and share, like WhatDoTheyKnow (UK) and MuckRock (US).

Governments collect their data on behalf of their constituencies. Unless privacy or security issues are involved, governments should release this data to the public that it represents. Making it public is a first step. But how reliable is it? Is it consistent, complete, and timely? Is it open? What data isn’t being made available? Governments at all levels are still only beginning to adopt open data, but we should remember that even after a city or country makes it policy, the push for open data isn’t over.

Data enthusiasts may also enjoy governmentexpenses.ca, punditsguide.ca, disclosed.ca, emitter.ca, vote.ca, representme.ca, and mycelium.ca.