n.

1 2012

The Open Data Choosing Framework

Processes, tools and thecniques for selecting the right data

A PIECE OF CONTENT OR DATA IS OPEN IF ANYONE IS FREE TO USE, REUSE, AND REDISTRIBUTE IT - SUBJECT ONLY, AT MOST, TO THE REQUIREMENT TO ATTRIBUTE AND SHARE-ALIKE.
> opendefinition.org

NEW GOLD, VEINS, MINES AND MINERS

WHAT THIS PAPER IS ABOUT

CHOOSE THE RIGHT DATA

THE OPEN DATA CHOOSING FRAMEWORK

w
I D E E E

paper

R I F L E S S I O N I

D I

E N G I N E E R I N G

P E R

L O P E N

D A T A

The Open Data Choosing Framework / Processes, tools and techniques for selecting the right data

New gold, veins, mines and miners

An article by the World Wide Webs creator Berners Lee appeared in The Times at the end of 2011 reported that data is becoming the brand new raw material of the XXI century and claimed that all our data worth gold.

() When the data has been released, applications have quickly followed, from mobile apps to find an NHS dentist to companies that use the open data on spending to advise local authorities on how to get the best value for money. These open data apps are creating new businesses for their developers and great resources for us all. Take, for example, bus finders (see London Bus Stop Live or BusMate London) these were developed within weeks of the datas release and did not cost the taxpayer a penny. () In the US, where weather data is already freely available, the private sector weather market is worth $1.5 billion a year (). Fragment from: Theres gold to be mined from all our data. Tim Berners-Lee and Nigel Shadbolt (www.eprints.ecs.soton.ac.uk)

If data is the new gold of our time, the public sectors information systems holding it are huge veins. As in the past the gold seekers did, today we should take our backpacks and move to explore these new large gold veins. Do you know what you have to put in your backpacks before leaving in search for the new gold? First of all you need a map for orientation during the search. An enormous amount of different types of data is generated and stored everyday within Public Sectors databases. It is only waiting to be discovered, extracted and enhanced. Once a potential vein has been identified, the most important thing to do is to check if mines for collecting, extracting and transporting this new gold exist, or if some actions are needed to create new or modernize existing ones. The mines, in this Open Data metaphor, are the Public Sectors applications. Sometimes these applications might be modern and efficient, other times they might be outdated and in need of maintenance and renewal. The miners are the Public Sectors business processes. In particular, the quality and quantity of new gold depend on the business processes performance in terms of effectiveness and efficiency. Searching, extracting and bringing to the surface data stored in the Public Sectors information systems and make it available to citizens, no profit organizations and business communities for creating more accountability in government, developing new products and services and bringing them successfully to market, is the inner meaning of Open Data. This paper is the first of a series through which Engineering aims at contributing to the dissemination and development of an Open Data culture in the Public Sector.
2012 Engineering

The Open Data Choosing Framework / Processes, tools and techniques for selecting the right data

What this paper is about

This paper deals with various aspects related to what data to publish and not to how to publish it.
This paper provides a framework which can be used by any organization in the Public Sector in order to support its datasets selection within an open data project. The objectives of this framework are: helping in the identification and assessment of information (veins) which might be useful to publish; establishing and measuring, for each application (mine) of the information system, the capabilities to extract this information and make it available following a few basic open data rules.

Exhibit 1 shows the Open Datas implementation path reported in the Open Data Manual by Open Knowledge Foundation and highlights the scope of this paper. There are four main processes in making data open, each of which can be done simultaneously. The aim of this paper is to propose, from an internal perspective, processes, tools and techniques for choosing and explaining to the community which datasets the organization plans to make open and why.

Papers objective CHOSE DATA SET (Asking community, Cost basis, Ease of release, Observe peers) APPLY OPEN LICENSE MAKE DATA AVAILABLE MAKE DATA DISCOVERABLE

- Asking community - Cost basis - Ease of release - Observe peers

- On line method - Off line method

- CKAN.net - Data catalogue

Exhibit 1 - What this paper is about.

Some of the methods and tools suggested in this paper can also be used to facilitate datasets comparison within the Public Sector (Observe peers) and initiatives aimed at involving citizens and companies (Asking the community) in the release of Open Data. The main purpose of this work, however, is to provide a framework for implementing a datasets choice process and to define a publication roadmap based on publishers perspective and focused on publishers requirements. This paper doesnt provide any suggestion about the technical and operational ways through which, after identifying the appropriate data, an organization can proceed to its publication and dissemination. In any case, this paper deals with various aspects related to what data to publish and not to how to publish it.
2012 Engineering

The Open Data Choosing Framework / Processes, tools and techniques for selecting the right data

Choose the right data

Open Data is a great opportunity for improving the relationship between the administration and the community but, like every communication process, it need to be managed carefully.
Open Data is a very topical issue. Publishing is becoming a priority for many politicians and administrators. In other words, it sometimes seems they need to publish some data as quickly as possible. No matter why, what, how and when: just publish it and do it fast (!). Currently, the ability of Public Sector organizations to resist to the temptation to put on their website anything that can be published quickly, without confusing the medium (published data) with the end (to provide the community with useful and usable data) requires a lot of strength and a certain discipline. Publishing data and making it open is a communication process. As is common knowledge, communication is always a critical process, mostly when it is performed by the Public Sector. Therefore, Open Data is a great opportunity for improving the relationship between the administration and the community but, like every communication process, it needs to be managed carefully and with a deep knowledge of: Awareness, regarding the quality of data available and the risk associated with the publication and use by the community of unreliable data; Imagination, regarding the ability to simulate different scenarios of data usage and determine which datasets will have more impact than others in terms of opportunity to create economic growth, accountability and trasparency, culture and progress once freely available.

Choosing the right data is a business process involving many different issues and requiring specific methods and dedicated tools. Public Sectors organizations should implement and perform this process systematically and not as a discret activity. One of the most important goals of implementing the choosing data business process is helping public organizations to give themselves a real agenda for the Open Data policies in terms of strategic sequences of data publication and updating, based on measurable criteria of usabilty potential and contents appropriateness of each dataset.

2012 Engineering

The Open Data Choosing Framework / Processes, tools and techniques for selecting the right data

The open data choosing framework

The consulting practice of Engineering Public Sector Department has established a framework to implement a comprehensive program of research and selection of data to be published as Open data. This framework consists of four processes, a five tools toolkit and seven management techniques. The four processes of this framework are as follows:
OPEN DATA VEINS & MINES MAPPING

It is the process through which you can provide the coordinates to be followed within the organization to research and locate the places where veins of data are supposed to be found. It is the process through which you can verify the consistency and richness of the new gold (data) veins previously been discovered, identified and taken into account. it is the process through which, once the presence of new gold (data) has been detected, you can establish whether there are mines (applications) capable of handling it and, if so, assess the complexity of data extraction and surface transportation (publishing) processes. it is the process through which it is possible, as a result of the exploration and the subsequent assessment, to associate with each mine a value that allows to evaluate and rank them in order of usability and therefore to establish a sequence for extracting and bringing data to the surface (publishing). The list of processes and their sequence are not prescriptive. As a matter of fact, in applying the framework, each organization should identify, select and tailor the processes which best fit its specific requirements and needs. The toolkit that can be used as a part of a framework to support the above described processes consists of the following tools: Open Data Fields and Mines Map: it is a model which allows to systematize and generalize the application architecture of the organization so as to make it easier to identify and compare possible Open Data veins and mines; Open Data Applications Potential Questionnaire: it is a questionnaire through which you can collect information about the data processed by an application as well as opinions and suggestions on possible uses of such data by the users of the applications, but also by individuals outside the organization such as, for example, citizens; Open Data Applications Appropriateness Check list: it is a questionnaire through which it is possible to assess the level of readiness of each application to support a program of Open Data from both the technical and organizational point of view;
5

OPEN DATA VEINS EXPLORATION

OPEN DATA MINES ASSESSMENT

OPEN DATA MEASUREMENT

2012 Engineering

The Open Data Choosing Framework / Processes, tools and techniques for selecting the right data

Open Data Application Scorecard: it is a scorecard which, for each application, gathers, synthesizes and measures on homogeneous areas the results of the analisys of usabilty potential and contents publishing appropriateness; Open Data Application Index: it is an indicator calculated for each application by attaching weights to the scores obtained and reported in the applications scorecards, which allows to define a ranking within an applications portfolio according to the Open Data readness and provides guidelines for defining a strategy and a roadmap for publishing datasets.

Data is everywhere, but the ability to arrange it into intelligent information and purposeful architectures NO. The value of information engineering lies in this difference.

These tools are needed to perform the four processes of the framework. However there is no need to implement all these tools in every instance. In fact, the most effective tools should be selected in each context according to the requirements and specific objectives, and - if needed integrated with other tools not provided as part of this paper. The management techniques that can be used in the development of the four processes and in the application of the tools described above are: Oral & Written Interview / Workshop / Focusgroup / Brainstorming / Benchmarking / Business Process Management / Enterprise Architecture Management / Project Management. The proposed management techniques can be employed within the various processes and support tools presented in a manner which can vary depending on the circumstances. Moreover, this list of techniques is not exhaustive, as you can add more, combine and tailor them in different ways depending on the specific needs and requirements of each Public Sectors organization.

OPEN DATA MANUAL PROCESSES CHOOSING DATA PROCESSES CHOOSING DATA TOOLKIT CHOOSING DATA TECHNIQUES

CHOSE DATA SET

(Asking community, Cost basis, Ease of release, Observe peers)

APPLY OPEN LICENSE

MAKE DATA AVAILABLE

MAKE DATA DISCOVERABLE

OPEN DATA FIELDS & MINES MAPPING OPEN DATA REPOSITORY & APPLICATIONS MAP

OPEN DATA FIELD EXPLORATION

OPEN DATA MINES ASSESSMENT

OPEN DATA MINES MEAUSERMENT

OPEN DATA APPLICATION POTENTIAL QUESTIONNAIRE

OPEN DATA APPLICATIONS APPROPRIATENESS CHECK LIST

OPEN DATA APP. SCORECARD OPEN DATA APP.LICATION INDEX BENCHMARKING

ORAL AND WRITTEN INTERVIEW EA MANAGEMENT BPM BRAINSTORMING WORKSHOP PROJECT MANAGEMENT

FOCUSGROUP

Exhibit 2 - The Open Data Choosing Framework

2012 Engineering

The Open Data Choosing Framework / Processes, tools and techniques for selecting the right data

Engineering Group
Engineering is a global player and Italys largest systems integration group and a leader in the provision of complete integrated services throughout the software value chain: design, development, outsourcing services, products and proprietary industry solutions, IT and strategy consultancy, tailored to the business models of our clients in all markets. With 6,500 employees and 43 branch offices, Engineering is present throughout Italy, has a direct commercial presence in the EU, in Ireland and Belgium, and outside the EU in Brazil, Latin America and USA. With value of production of about 775 million euros at the end of 2011, Engineering has a consolidated presence in several industries: Public Sector, Manufacturing, Finance, Telco and Utilities. The holding company, Engineering Ingegneria Informatica, has been listed on the Milan Stock Exchange since Decem-ber 2000 and today ordinary shares are traded in the FTSE Italia STAR segment for stocks with the highest industrial and financial requirements. Of the total 12.5 million shares, 33% are traded on the market and 67% are held by company founders. Thanks to the purchasing plan and the ability to open new markets, the Group has a global production capacity in more than 50 different countries, managing IT initiatives linked to the development of projects in the industrial and telecommunications sectors. It has strengthened its presence in the managed operations sector thanks to its data centre integrated network, located in Pont St. Martin (near Aosta), Turin and Padua, supplying business continuity services in the management of IT infrastructures. The Engineering Group operates through 5 business units: Finance, Public Administration & Healthcare), Industry & Services, Utilities & Telco, Training Services, which are supported by 350 highly specialized researchers, and by its Central Office for Research & Innovation, with the double objective of promoting the role of software research at international level as well as bringing the innovation to the business units productive cycles.

Contacts
For more information about this paper and Engineerings Open Data consulting services please contact:

www.eng.it

Davide Lipodio Public Sector Business Consulting Department Director / [email protected] For more information about Engineering please contacts: Costanza Amodeo Communication & Marketing Director / [email protected]

2012 Engineering