Transparency Through Open Data and Open Source-Scribd
Transparency Through Open Data and Open Source-Scribd
Being a citizen in Canada is currently a passive act of faith. We trust that money is
being wisely spent on the systems that run our country. We trust that the people
governing us have all the skills, time, and information they need to make the best
decisions. We trust that bureaucracies are well-designed, and that the people in
them are motivated to make those bureaucracies better. Unfortunately, it’s hard to
trust what you can’t see. By publishing information in open, machine-readable
formats, governments can take a powerful step towards building public trust. By
sharing information, governments can start to channel the expertise of the crowds
outside the gray cubicles of the civil service to build more effective and inclusive
ways of running the country.
Obama’s Promise
The Obama campaign made international headlines for tapping into a wellspring of
online contributions. The campaign raised $742M1 dollars in total, almost twice as
much as the more traditionally-funded McCain campaign. The majority of this
money came through online contributions. The Washington post2 identifies Lynne
Bailey as a typical online donor. Bailey is a 52-year-old mother of two in Riverside,
California, who gave a total of $120.40 in mostly $10 increments. Like Bailey, 90%
of Obama’s donors were ‘small scale’ donors, contributing less than $200 each. In
total, these small scale donations accounted for 57% of funds raised in Obama’s
presidential campaign3. This wealth of donations from ordinary Americans could not
have been tapped without the internet.
The campaign also made headlines for creating online tools that allowed their army
of on-the-ground volunteers to self-organize. The Obama volunteer website, called
the most important video game of 20084, allowed campaigners to post their
activities, organize house meetings, share their experiences, and distribute
responsibility for making phone calls encouraging voters to getto the polls. The
tools showcased in the Obama campaign have forever changedthe way politicians
are elected. What may be more interesting is how the online tools for governing the
country instigated by Obama’s team and policies will change the way governments
around the world work.
Well before he was a presidential nominee, Obama was defining strategies for using
the internet to improve government openness and accountability. In 2006, Obama,
1
“Banking on Becoming President” http://www.opensecrets.org/pres08/
2
“Obama Raised Half a Billion Online” http://voices.washingtonpost.com/the-
trail/2008/11/20/obama_raised_half_a_billion_on.html
3
Center for Responsive Politics, Contributions By Size,
http://www.opensecrets.org/pres08/donordems.php?sortby=S
4
“MyBO, the Video Game”,
http://www.techpresident.com/blog/entry/33178/mybo_the_video_game
along with senators Tom Coburn, Tom Carper and John McCain filed the ‘Federal
Funding Accountability and Transparency Act’, which mandated creation of a
searchable website of all government spending by January 1, 2008. After initially
refusing, citing that the website would be too costly to build, President Bush signed
the bill on September 26, 2006. The website, fedspending.org, was released ahead
of schedule in December, 2007. Not only does the website provide an easily
navigable interface, it provides an API for external developers to access that
information and build tools of their own. Since the release of the website, according
to the US watchdog group OMBWatch, 11 states have created similar state-
spending websites, and 24 other states are working towards such a goal.
“Obama will integrate citizens into the actual business of government by:
Wikinomics, by Dan Tapscott, opens with the story of Goldcorp, an Ontario mining
company who faced bankruptcy in 1999. A young mutual fund manager, Rob
McEwan, had become majority owner of Goldcorp after a messy takeover battle
several years earlier. While early test drilling in Gold Corps property had indicated
substantial gold deposits, after years of searching, Goldcorps engineers and
5
“BARACK OBAMA: CONNECTING AND EMPOWERING ALL AMERICANS THROUGH
TECHNOLOGY AND INNOVATION”
http://www.barackobama.com/pdf/issues/technology/Fact_Sheet_Innovation_and_Technology.
pdf
6
“E-Government Meets Web 2.0: Goodbye Portals, Hello Web Services”
http://www.readwriteweb.com/archives/e-government_meets_web_20.php
geologists had been unable to find the gold’s exact location. At a young president’s
meeting McEwan heard the story of Linux, where Linus Thorvalds co-ordinated the
development of a world-class operating system over the internet. McEwan,
inspired, decided to publish Goldcorp’s geological data and announce a ‘Goldcorp
Challenge’, with half a million dollars of prize money. The results of the challenge:
the fruit of analysis submitted by geologists from all over the world, created almost
9 billion dollars of market value.
There have been several examples in the US and UK of web sites that use
government data, opened up through APIs, gathered by scraping, or compiled by
citizens themselves that allow citizens to contribute analysis, expertise, or local
knowledge for public benefit. A short list8 of these sites include:
PeerToPatent.org: This goal of this site, a project of the New York Law School, is to
relieve some of the burden of over-worked officers at the US Patent and Trademark
Office by tapping in to an online community of civilian experts. These experts
search for and explain prior art, as well as vote on the strength of patent
applications. A year into the pilot, the PeerToPatent system reported over 2000
citizen reviewers. The average citizen reviewer spent six hours reviewing each
patent.9
7
Wikinomics: How Mass Collaboration Changes Everything, By Don Tapscott, Anthony D.
Williams
Published by Portfolio, 2006, page 10
8
A more thorough list can be found of example sites from around the world can be seen in
the links section of VisibleGovernment.ca’s website: http://visiblegovernment.ca
9
Peer-to-Patent, Wikipedia http://en.wikipedia.org/wiki/Peer_to_Patent
These sites are early experiments in the field, and represent the iceberg tip of what
may be possible.
Finding the best ways to analyze government information and collect value from
public feedbackis going to take a lot of experimentation. The probability of a
successful solution is what author Clay Shirkey10might qualify as a scalar distribution
pattern: one where there’s a very large number of failures, some modest
successes, and a few solutions that will do amazingly well. Being prepared to
accept a lot of failures is the key to finding the successes.
Government bureaucracies are failure-averse for very good reasons. Public scrutiny
and the spectre of being accused of wastingtax payer funds make for a cautious
environment, where money is only spent on guaranteed successes. By publishing
data in open, standardized formats, governments can off-load the costs and stigma
of failure to external organizations. Like Goldcorps, governments can take the open
approach to innovation by challenging advocacy groups, the nascent community of
armchair egovernment-geeks, and the for-profit market to ‘build a better way’. The
government can then take advantage of the value created by the best solutions.
Solutions that don’t work can die quietly, without any tax dollars having been spent.
Unrestricted access to government data will create new markets for innovative ways
of presenting, analyzing, and combining that data. Some creative companies will
eventually find profitable ways of using this information to generate value. Some of
the ways that these companies discover will inevitably be even more valuable as
public goods.
For the last 10 years, Cisco has grown by accepting the fact that, even though it
employs some of the most brilliant people in the industry, there is a very low
probability of the future’s most successful idea being generated in-house. Instead,
Cisco’s model is to buy the cream of innovation, effectively outsourcing their
technology R and D to the venture capital market and the start-up diaspora. If the
government were to get in the Cisco-like habit of buying out the most successful
innovations in displaying and using government data, and turning them into public
goods, it would create a thriving market for more of the same. If the government
were to, further, open source the tools that it buys, it would create an expanding
base of software components for building increasingly sophisticated tools.
The UK’s Power of Information Task Force has proposed an application framework for
implementing government transparency. In a thoughtful blog post this June11,
Richard Allen proposed the following re-visioning of the way that the data in a
government website is used. Instead of a closed model, where the presentation,
10
See Clay Shirkey’s ‘Here Comes Everybody’, Penguin, 2008 for examples of scalar
distribution solutions sets that benefit from Web 2.0 experimentation.
11
“More Architecture”, Power of Information Task Force Blog
http://powerofinformation.wordpress.com/2008/06/19/more-architecture/
analysis, and data layers are locked together, Allen presents a model with access
layers between data, analysis, and presentation, andan ‘interaction layer’ laid over
top. These access layers give 3rd parties the flexibility to hook into the data directly
to provide their own analysis, or to use information from the government’s analysis
layerto provide their own presentation interfaces. Finally, the interaction layer
allows people to discuss the information and provide feedback.
Old New
As a concrete example of how this model can be applied, Allen presents the
evolution of tools around the UK’s parliamentary Hansard, a record of parliamentary
proceedings:
Old New
“An access layer has been created for Hansard with a screen scraper and
Click-Use license to address both technical and copyright issues.
The scraped data goes through an analysis process at publicwhip.org.uk.
Access to the output of this analysis process is offered by means of XML data
under a Creative Commons license. An API has been produced to make it very
easy to get this data.
The new architecture now provides a platform for more innovation around the
Hansard data set with very low barriers to doing this.”
David Robinson, of Princeton’s Center for Information Technology Policy, takes the
concept of fitting access layers into existing architectures one step further. In his
paper ‘Government Data and the Invisible Hand’, Robinson argues that the
government intra-departmental reporting channels should be exposed to the public,
who can provide an external validation to complement internal checks and
balances.
If this model were followed by the Canadian federal government, data provided to
the Auditor General so that it can fulfill its mandate of “holding the federal
government accountable for its stewardship of public funds”12 would be opened up
to access by external agencies as it is being received. Like the Peer-to-Patent
model, the Auditor General would begin to benefit from scrutiny of the data by
external bodies. Systems may well evolve that relieve the burden of oversight from
the staff of the Auditor General altogether, allowing them the leisure to pay
attention only when issues are reported. With a system built on openness, the
public may also start to trust that the government in Canada is in fact well run,
instead of being required to take it on faith.
Open source licensing for the tools that present and use government information
takes the concept of transparency to the level of the source code. It enables public
scrutiny of the presentation and analysis methods and allows many eyes to look for
errors. In the words of one of OMBWatch’s recommendations for President Obama,
12
Office of the Auditor General Website. http://www.oag-
bvg.gc.ca/internet/English/admin_e_41.html
collaboration between agencies and will increase civilian oversight,
participation, and use of taxpayer-funded resources.”13
South Africa, Brazil, and Chinahave begun to adopt policies favouring open source.
Early in 2007, the South African cabinet “approved a policy and strategy for the
implementation of free and open source software (FOSS) in government”14.
According to a 2007 report in the open-source online magazine Tectonic,
“all new software developed for or by the government will be based on open
standards and government will itself migrate current software to FOSS. This
strategy will, among other things, lower administration costs and enhance
local IT skills.”14
The call for open, standardized APIs for government data create incentive for
governments to provide new systems for publishing that data. Many government
bodies will be facing the same problems around producing reliable data streams
complete with sensible meta-data.
In the Canadian federal government, departments are typically left to come up with
their own IT solutions, paid for out of each departmental budget. This leads to a
proliferation of redundant systems built according to the preferences of each
department’s IT consultant. Were departments to pool their resources into a joint
open-source effort to create data-publishing systems, it would save costs, and
create a foundation of inter-departmental co-operation for breaking free from this
model. The software produced would also benefit from the public scrutiny,
oversight and contribution of Canadian citizens. Further, it would be free for
governments around the world to adopt, creating the potential for contribution from
citizens world-wide.
Recognizing that the incentives against transparency outweigh the incentives for,
the OMBWatch report devotes several sections to institutionalizing ‘open’. These
include:
13
OMBWatch Report: ‘Towards a 21st Century Right To Know Agenda’, p 60
14
“SA Government to Switch to Open Source” http://www.tectonic.co.za/wordpress/?p=1377
• Having the president instruct agencies to request sufficient resources –
funding, personnel, and technical capacity, to implement the vision of a more
transparent government.
• Making transparency part of federal job evaluations where it is part of the job
description
• Implement directives protecting whistle-blowers who disclose waste, fraud, or
abuse within an agency
• Creating a system of transparency scorecards for rating agencies.
• Giving out transparency awards to celebrate achievements and best
practices.
Beyond these recommendations, external bodies that use government information
should, as much as possible, build systems that create heroes rather than
scapegoats. Individuals who find ways to save money, increase efficiency, or
deliver a valuable service in an innovative way should be publicly rewarded, either
through externalfinancial compensation or public recognition.
Public service was, at one time, thought of as a calling. If civil servants who
improve the way government functions are celebrated with the same media
reverence granted successful businessmen, perhaps it may become one again.
Ways Forward
In the last two weeks, more than one grass-roots forum has appeared to advance a
new era in civic participation. ChangeCamp15, originating in Toronto, and
Communeautique’s Forum Ouvert, in Montreal, are two such movements calling for
an open exchange of ideas around using technology to re-define the role of the
citizen. The spontaneous emergence of these groups shows the demand for new
ideas and tools in the Canadian government.
Further Reading:
‘Government Data and the Invisible Hand’, David Robinson et al., Princeton Center
for Information Technology Policy,
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1138083
15
“What is ChangeCamp?” http://groups.google.com/group/changecamp/web/what-is-
changecamp-1-page-brief
16
http://groups.google.com/group/visiblegovernment-discuss