Note: A version of this paper has been submitted for publication in the February

2009 issue of Open Source Business Resource.-- http://www.osbr.ca/

Government Transparency via Open Data and Open Source

Being a citizen in Canada is currently a passive act of faith. We trust that money is
being wisely spent on the systems that run our country. We trust that the people
governing us have all the skills, time, and information they need to make the best
decisions. We trust that bureaucracies are well-designed, and that the people in
them are motivated to make those bureaucracies better. Unfortunately, it’s hard to
trust what you can’t see. By publishing information in open, machine-readable
formats, governments can take a powerful step towards building public trust. By
sharing information, governments can start to channel the expertise of the crowds
outside the gray cubicles of the civil service to build more effective and inclusive
ways of running the country.

As part of his campaign platform, President-elect Obama has promised ground-

breaking steps towards openness and transparency. When will Canada follow?

Obama’s Promise

The Obama campaign made international headlines for tapping into a wellspring of
online contributions. The campaign raised $742M1 dollars in total, almost twice as
much as the more traditionally-funded McCain campaign. The majority of this
money came through online contributions. The Washington post2 identifies Lynne
Bailey as a typical online donor. Bailey is a 52-year-old mother of two in Riverside,
California, who gave a total of $120.40 in mostly $10 increments. Like Bailey, 90%
of Obama’s donors were ‘small scale’ donors, contributing less than $200 each. In
total, these small scale donations accounted for 57% of funds raised in Obama’s
presidential campaign3. This wealth of donations from ordinary Americans could not
have been tapped without the internet.

The campaign also made headlines for creating online tools that allowed their army
of on-the-ground volunteers to self-organize. The Obama volunteer website, called
the most important video game of 20084, allowed campaigners to post their
activities, organize house meetings, share their experiences, and distribute
responsibility for making phone calls encouraging voters to getto the polls. The
tools showcased in the Obama campaign have forever changedthe way politicians
are elected. What may be more interesting is how the online tools for governing the
country instigated by Obama’s team and policies will change the way governments
around the world work.

Well before he was a presidential nominee, Obama was defining strategies for using
the internet to improve government openness and accountability. In 2006, Obama,

1
“Banking on Becoming President” http://www.opensecrets.org/pres08/
2
“Obama Raised Half a Billion Online” http://voices.washingtonpost.com/the-
trail/2008/11/20/obama_raised_half_a_billion_on.html
3
Center for Responsive Politics, Contributions By Size,
http://www.opensecrets.org/pres08/donordems.php?sortby=S
4
“MyBO, the Video Game”,
http://www.techpresident.com/blog/entry/33178/mybo_the_video_game
along with senators Tom Coburn, Tom Carper and John McCain filed the ‘Federal
Funding Accountability and Transparency Act’, which mandated creation of a
searchable website of all government spending by January 1, 2008. After initially
refusing, citing that the website would be too costly to build, President Bush signed
the bill on September 26, 2006. The website, fedspending.org, was released ahead
of schedule in December, 2007. Not only does the website provide an easily
navigable interface, it provides an API for external developers to access that
information and build tools of their own. Since the release of the website, according
to the US watchdog group OMBWatch, 11 states have created similar state-
spending websites, and 24 other states are working towards such a goal.

Building on these themes of openness and accountability, Obama included the

following in his Technology Platform5under the heading “Create a Transparent and
Connected Democracy”.

“Obama will integrate citizens into the actual business of government by:

• Making government data available online in universally accessible formats

to allow citizens to make use of that data to comment, derive value, and take
action in their own communities. Greater access to environmental data, for
example, will help citizens learn about pollution in their communities, provide
information about local conditions back to government and empower people
to protect themselves.”

• Establishing pilot programs to open up government decision-making and

involve the public in the work of agencies, not simply by soliciting opinions,
but by tapping into the vast and distributed expertise of the American
citizenry to help government make more informed decisions.”

Making government data available online in accessible formats is a powerful idea:

one that industry analyst Gartner has describes as having “a much greater potential
effect on the ability to transform government than anything else in the Web 2.0
world”.6 For this transformation to take place, however, governments also have to
be willing to accept feedback and analysis that services based on this information
generate. Thus, the second point of a willingness to ‘tap into the vast and
distributed expertise of citizenry’ becomes vitallyimportant.

Open Systems Allow External Contribution

Wikinomics, by Dan Tapscott, opens with the story of Goldcorp, an Ontario mining
company who faced bankruptcy in 1999. A young mutual fund manager, Rob
McEwan, had become majority owner of Goldcorp after a messy takeover battle
several years earlier. While early test drilling in Gold Corps property had indicated
substantial gold deposits, after years of searching, Goldcorps engineers and

5
“BARACK OBAMA: CONNECTING AND EMPOWERING ALL AMERICANS THROUGH
TECHNOLOGY AND INNOVATION”
http://www.barackobama.com/pdf/issues/technology/Fact_Sheet_Innovation_and_Technology.
pdf
6
“E-Government Meets Web 2.0: Goodbye Portals, Hello Web Services”
http://www.readwriteweb.com/archives/e-government_meets_web_20.php
geologists had been unable to find the gold’s exact location. At a young president’s
meeting McEwan heard the story of Linux, where Linus Thorvalds co-ordinated the
development of a world-class operating system over the internet. McEwan,
inspired, decided to publish Goldcorp’s geological data and announce a ‘Goldcorp
Challenge’, with half a million dollars of prize money. The results of the challenge:
the fruit of analysis submitted by geologists from all over the world, created almost
9 billion dollars of market value.

As Tapscott emphasizes, opening up that data took tremendous bravery. As a first

step, the company had to admit it did not know how to find its own gold. McEwan
“realized that the uniquely qualified minds to make new discoveries were probably
outside the boundaries of his organization, and by sharing some intellectual
property he could harness the power of collective genius and capability.7” What if
governments in Canada were to take the same attitude?

There have been several examples in the US and UK of web sites that use
government data, opened up through APIs, gathered by scraping, or compiled by
citizens themselves that allow citizens to contribute analysis, expertise, or local
knowledge for public benefit. A short list8 of these sites include:

Maplight.org: Produced by the Sunlight Foundation, this website analyzes the

relationshipbetween contributions and votes in the US congress. The site shows
simple histograms, per bill, of donations by groups for and against the bill, against a
histogram of votes. The site allows citizens to look for trends between contributions
and votes by bill, and by law-maker. By making this information visible, the site is
effectively crowd-sourcing the function of a contribution watchdog, enabling issue
detection and discussion by both bloggers and the main-stream media.

PeerToPatent.org: This goal of this site, a project of the New York Law School, is to
relieve some of the burden of over-worked officers at the US Patent and Trademark
Office by tapping in to an online community of civilian experts. These experts
search for and explain prior art, as well as vote on the strength of patent
applications. A year into the pilot, the PeerToPatent system reported over 2000
citizen reviewers. The average citizen reviewer spent six hours reviewing each
patent.9

FixMyStreet.com: This site, produced by the UK non-profit MySociety, allows

citizens to report public safety and nuisance issues in their neighbourhood such as
graffiti, potholes, or bad lighting. Citizens can then subscribe to an RSS feed related
to a particular issue to receive updates on the problem by the town council. This
website not only provides citizens with an easy way to report and monitor problems,
it reduces the burden on authorities who are less likely to have to handle repeated
complaints submitted individually. Further, the number of subscribers to particular
problem can be used as a rough indication of public interest in an issue.

7
Wikinomics: How Mass Collaboration Changes Everything, By Don Tapscott, Anthony D.
Williams
Published by Portfolio, 2006, page 10
8
A more thorough list can be found of example sites from around the world can be seen in
the links section of VisibleGovernment.ca’s website: http://visiblegovernment.ca
9
Peer-to-Patent, Wikipedia http://en.wikipedia.org/wiki/Peer_to_Patent
These sites are early experiments in the field, and represent the iceberg tip of what
may be possible.

Open Systems Make Failure Less Costly

Finding the best ways to analyze government information and collect value from
public feedbackis going to take a lot of experimentation. The probability of a
successful solution is what author Clay Shirkey10might qualify as a scalar distribution
pattern: one where there’s a very large number of failures, some modest
successes, and a few solutions that will do amazingly well. Being prepared to
accept a lot of failures is the key to finding the successes.

Government bureaucracies are failure-averse for very good reasons. Public scrutiny
and the spectre of being accused of wastingtax payer funds make for a cautious
environment, where money is only spent on guaranteed successes. By publishing
data in open, standardized formats, governments can off-load the costs and stigma
of failure to external organizations. Like Goldcorps, governments can take the open
approach to innovation by challenging advocacy groups, the nascent community of
armchair egovernment-geeks, and the for-profit market to ‘build a better way’. The
government can then take advantage of the value created by the best solutions.
Solutions that don’t work can die quietly, without any tax dollars having been spent.

Open Systems Create New Markets for Innovation

Unrestricted access to government data will create new markets for innovative ways
of presenting, analyzing, and combining that data. Some creative companies will
eventually find profitable ways of using this information to generate value. Some of
the ways that these companies discover will inevitably be even more valuable as
public goods.

For the last 10 years, Cisco has grown by accepting the fact that, even though it
employs some of the most brilliant people in the industry, there is a very low
probability of the future’s most successful idea being generated in-house. Instead,
Cisco’s model is to buy the cream of innovation, effectively outsourcing their
technology R and D to the venture capital market and the start-up diaspora. If the
government were to get in the Cisco-like habit of buying out the most successful
innovations in displaying and using government data, and turning them into public
goods, it would create a thriving market for more of the same. If the government
were to, further, open source the tools that it buys, it would create an expanding
base of software components for building increasingly sophisticated tools.

An Architecture for Open

The UK’s Power of Information Task Force has proposed an application framework for
implementing government transparency. In a thoughtful blog post this June11,
Richard Allen proposed the following re-visioning of the way that the data in a
government website is used. Instead of a closed model, where the presentation,

10
See Clay Shirkey’s ‘Here Comes Everybody’, Penguin, 2008 for examples of scalar
distribution solutions sets that benefit from Web 2.0 experimentation.
11
“More Architecture”, Power of Information Task Force Blog
http://powerofinformation.wordpress.com/2008/06/19/more-architecture/
analysis, and data layers are locked together, Allen presents a model with access
layers between data, analysis, and presentation, andan ‘interaction layer’ laid over
top. These access layers give 3rd parties the flexibility to hook into the data directly
to provide their own analysis, or to use information from the government’s analysis
layerto provide their own presentation interfaces. Finally, the interaction layer
allows people to discuss the information and provide feedback.

Old New

As a concrete example of how this model can be applied, Allen presents the
evolution of tools around the UK’s parliamentary Hansard, a record of parliamentary
proceedings:

Old New

Originally, according to Allen, www.parliament.uk took an integrated approach to

presenting the Hansard, where the Hansard data is “wrapped up with Parliament’s
own analysis output and presented to the public in an official website.” The
innovation of a click-use license for copyright allowed a citizen-managed project
called publicwhip.org.uk to begin scraping the data and providing it for public use.
Allen describes the process this way:

“An access layer has been created for Hansard with a screen scraper and
Click-Use license to address both technical and copyright issues.
 The scraped data goes through an analysis process at publicwhip.org.uk.
Access to the output of this analysis process is offered by means of XML data
under a Creative Commons license. An API has been produced to make it very
easy to get this data.

TheyWorkForYou.comprovides a very good and popular presentation layer for

this content. The data as reworked by TheyWorkForYou is also commonly
presented in many other places on the web such as MPs’ personal sites.
There is a comment facility built into TheyWorkForYou to provide a layer of
interaction around the content. It is also cited in many blogs that generate
their own interaction as well as featuring in mainstream media stimulating
further discussion.

The new architecture now provides a platform for more innovation around the
Hansard data set with very low barriers to doing this.”

David Robinson, of Princeton’s Center for Information Technology Policy, takes the
concept of fitting access layers into existing architectures one step further. In his
paper ‘Government Data and the Invisible Hand’, Robinson argues that the
government intra-departmental reporting channels should be exposed to the public,
who can provide an external validation to complement internal checks and
balances.

If this model were followed by the Canadian federal government, data provided to
the Auditor General so that it can fulfill its mandate of “holding the federal
government accountable for its stewardship of public funds”12 would be opened up
to access by external agencies as it is being received. Like the Peer-to-Patent
model, the Auditor General would begin to benefit from scrutiny of the data by
external bodies. Systems may well evolve that relieve the burden of oversight from
the staff of the Auditor General altogether, allowing them the leisure to pay
attention only when issues are reported. With a system built on openness, the
public may also start to trust that the government in Canada is in fact well run,
instead of being required to take it on faith.

Open Source Tools for Open Data

Open source licensing for the tools that present and use government information
takes the concept of transparency to the level of the source code. It enables public
scrutiny of the presentation and analysis methods and allows many eyes to look for
errors. In the words of one of OMBWatch’s recommendations for President Obama,

“...agencies should have a policy to exercisea preference for open source

software for government activities as a means to improve stability,
transparency, metadata quality, and cost-efficiency. Open formats for
government information and open software applications will enable

12
Office of the Auditor General Website. http://www.oag-
bvg.gc.ca/internet/English/admin_e_41.html
 collaboration between agencies and will increase civilian oversight,
participation, and use of taxpayer-funded resources.”13

South Africa, Brazil, and Chinahave begun to adopt policies favouring open source.
Early in 2007, the South African cabinet “approved a policy and strategy for the
implementation of free and open source software (FOSS) in government”14.
According to a 2007 report in the open-source online magazine Tectonic,

“all new software developed for or by the government will be based on open
standards and government will itself migrate current software to FOSS. This
strategy will, among other things, lower administration costs and enhance
local IT skills.”14

The call for open, standardized APIs for government data create incentive for
governments to provide new systems for publishing that data. Many government
bodies will be facing the same problems around producing reliable data streams
complete with sensible meta-data.

In the Canadian federal government, departments are typically left to come up with
their own IT solutions, paid for out of each departmental budget. This leads to a
proliferation of redundant systems built according to the preferences of each
department’s IT consultant. Were departments to pool their resources into a joint
open-source effort to create data-publishing systems, it would save costs, and
create a foundation of inter-departmental co-operation for breaking free from this
model. The software produced would also benefit from the public scrutiny,
oversight and contribution of Canadian citizens. Further, it would be free for
governments around the world to adopt, creating the potential for contribution from
citizens world-wide.

Open System Roadblocks

“No one was ever promoted for disclosing information.”

13
– US Government Employee

The rewards of a civil service career are asymmetrical. As quoted in a recent

OMBWatch report, ‘Towards a 21st Century Right-To-Know Agenda’, civil servants
often feel they live in a fish-bowl. This fish bowl is made of a particular type of
filtered glass: onewhere only the bad light gets through. Overwhelmingly, the
disclosed information that gets publicized by the media is the negative, career-
destroying kind; information that points to success and improvement are rarely
publicly celebrated. This is something that has to change.

Recognizing that the incentives against transparency outweigh the incentives for,
the OMBWatch report devotes several sections to institutionalizing ‘open’. These
include:

13
OMBWatch Report: ‘Towards a 21st Century Right To Know Agenda’, p 60
14
“SA Government to Switch to Open Source” http://www.tectonic.co.za/wordpress/?p=1377
 • Having the president instruct agencies to request sufficient resources –
funding, personnel, and technical capacity, to implement the vision of a more
transparent government.
• Making transparency part of federal job evaluations where it is part of the job
description
• Implement directives protecting whistle-blowers who disclose waste, fraud, or
abuse within an agency
• Creating a system of transparency scorecards for rating agencies.
• Giving out transparency awards to celebrate achievements and best
practices.
Beyond these recommendations, external bodies that use government information
should, as much as possible, build systems that create heroes rather than
scapegoats. Individuals who find ways to save money, increase efficiency, or
deliver a valuable service in an innovative way should be publicly rewarded, either
through externalfinancial compensation or public recognition.

Public service was, at one time, thought of as a calling. If civil servants who
improve the way government functions are celebrated with the same media
reverence granted successful businessmen, perhaps it may become one again.

VisibleGovernment.ca: Promoting Citizen Services Based on Open

Government Data

The non-profit VisibleGovernment.ca was officially incorporated in December, 2008

to promote online tools for government transparency. One of the founding
principles of VisibleGovernment.ca is that, while there is a sound case for open
government data, a 3rd party organization is needed to raise awareness of the issues
and marshal public support.

VisibleGovernment.ca’s strategy has been to build a limited number of pilot projects

to gain visibility in Canada for the power of open government datato transform the
relationship between citizen and government. Our one pilot released so far: ‘I
Believe in Open’ (http://ibelieveinopen.ca) , challenged candidates in the 2008
federal electionto pledge to five aspects of government transparency. The site also
collected signups from voters, organized into ridings, so that MP candidates could
see support level in their area. 400 MP candidates signed our online pledge, 38 of
whom were elected.

Another VisibleGovernment.ca pilot is a tool forvisualizing federal government travel

and hospitality expenses. The project gathers data from federal travel and
hospitality expenses from tables published in a variety of formats spread over 100
different department websites, and creates an interface that citizens can use to
visualize this information, compare departments,and see trends over time. Further,
the project providesan RSS feed so that other groups can use the data to create
tools of their own. With help from volunteers from the Montreal high tech
community, over 30,000 records have been collected so far. A Montreal web
development company specializing in data visualization has volunteered to do the
visualization website, which we intend to launch at the conference 'Social Media for
Government' in Ottawa this February.
The long term goal of VisibleGovernment.ca is, like the Sunlight Foundation in the
US, to direct money and attention to external projects that further our mission
viagrants and contests. By being a catalyst driving public support of open
government data, and pushing the envelope of innovation for ways of analyzing and
presenting that data, we hope to create a network of active citizens who believe
that ‘open’ should be the normal state of governments.

Ways Forward

In the last two weeks, more than one grass-roots forum has appeared to advance a
new era in civic participation. ChangeCamp15, originating in Toronto, and
Communeautique’s Forum Ouvert, in Montreal, are two such movements calling for
an open exchange of ideas around using technology to re-define the role of the
citizen. The spontaneous emergence of these groups shows the demand for new
ideas and tools in the Canadian government.

The non-profit VisibleGovenrment.ca seeks the expertise and participation of grass-

roots groups, advocacy organizations, and citizens across the country to make
online tools for civic participation based on open government data a reality. If you
share this goal, here are some concrete steps for action:

– Host a ‘Change Camp’ or ‘Forum Ouvert’ in your city.

– Research resources and strategies on the VisibleGovernment.ca,
Sunlight Foundation, or MySociety websites.
– Contribute to a VisibleGovernment.ca project.
– Start a dialogue with your public representatives on how they can be
more open.
For more information, visit the VisibleGovernment.ca website, join our online
discussion group16, or email [email protected].

Further Reading:

‘Hack Mash and Peer, Crowdsourcing Government Transparency’, a working paper

by Jerry Brito, Mercatus Center at George Mason University,
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1023485

‘Government Data and the Invisible Hand’, David Robinson et al., Princeton Center
for Information Technology Policy,
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1138083

15
“What is ChangeCamp?” http://groups.google.com/group/changecamp/web/what-is-
changecamp-1-page-brief
16
http://groups.google.com/group/visiblegovernment-discuss