Open data quality
Download as PPTX, PDF•13 likes•12,281 views
The document outlines a training module on open data quality, emphasizing its importance for effective operations and decision-making. It details key data quality dimensions such as accuracy, completeness, and timeliness, and provides best practices for publishing high-quality open data and metadata. Funded by the European Commission, the training aims to enhance understanding and implementation of open data standards across organizations.
Open data quality
- 1. Training Module 2.2 OPEN DATA SUPPORT Open Data & Metadata Quality PwC firms help organisations and individuals create the value they’re looking for. We’re a network of firms in 158 countries with close to 180,000 people who are committed to delivering quality in assurance, tax and advisory services. Tell us what matters to you and find out more by visiting us at www.pwc.com. PwC refers to the PwC network and/or one or more of its member firms, each of which is a separate legal entity. Please see www.pwc.com/structure for further details.
- 2. This presentation has been created by PwC Presentation metadata Open Data Support is funded by the European Commission under SMART 2012/0107 „Lot 2: Provision of services for the Publication, Access and Reuse of Open Public Data across the European Union, through existing open data portals‟(Contract No. 30-CE0530965/00-17). © 2013 European Commission Authors: Makx Dekkers, Nikolaos Loutas, Michiel De Keyzer and Stijn Goedertier Disclaimers 1. The views expressed in this presentation are purely those of the authors and may not, in any circumstances, be interpreted as stating an official position of the European Commission. The European Commission does not guarantee the accuracy of the information included in this presentation, nor does it accept any responsibility for any use thereof. Reference herein to any specific products, specifications, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favouring by the European Commission. All care has been taken by the author to ensure that s/he has obtained, where necessary, permission to use any parts of manuscripts including illustrations, maps, and graphs, on which intellectual property rights already exist from the titular holder(s) of such rights or from her/his or their legal representative. 2. This presentation has been carefully compiled by PwC, but no representation is made or warranty given (either express or implied) as to the completeness or accuracy of the information it contains. PwC is not liable for the information in this presentation or any decision or consequence based on the use of it.. PwC will not be liable for any damages arising from the use of the information contained in this presentation. The information contained in this presentation is of a general nature and is solely for guidance on matters of general interest. This presentation is not a substitute for professional advice on any particular matter. No reader should act on the basis of any matter contained in this publication without considering appropriate professional advice. OPEN DATASUPPORT Slide 2
- 3. Learning objectives By the end of this training module you should have an understanding of: • What (open) data quality means. • The open data quality determinants and criteria. • Good practices for publishing high-quality (linked) open data. OPEN DATASUPPORT Slide 3
- 4. Content This modules contains... • A definition of data quality; • An overview of the dimensions of data and metadata quality; • A selection of best practices for publishing good quality data and metadata. OPEN DATASUPPORT Slide 4
- 5. What is data (and metadata) quality? Data is of high quality "if they are fit for their intended uses in operations, decision making and planning." Or more specifically: “High quality data are accurate, available, complete, conformant, consistent, credible, proces sable, relevant and timely.” OPEN DATASUPPORT Slide 5
- 6. What is metadata? “Metadata is structured information that describes, explains, locates, or otherwise makes it easier to retrieve, use, or manage an information resource. Metadata is often called data about data” -- National Information Standards Organization • We observe that metadata is a type of data. • The same quality considerations apply to data and metadata alike. OPEN DATASUPPORT Slide 6
- 7. The data quality dimensions What are the main dimensions to be taken into account for delivering good quality (meta)data? OPEN DATASUPPORT Slide 7
- 8. Data quality dimensions • Accuracy: is the data correctly representing the real-world entity or event? • Availability: Can the data be accessed now and over time? • Completeness: Does the data include all data items representing the entity or event? • Conformance: Is the data following accepted standards? • Consistency: Is the data not containing contradictions? • Credibility: Is the data based on trustworthy sources? • Processability: Is the data machine-readable? • Relevance : Does the data include an appropriate amount of data? • Timeliness: Is the data representing the actual situation and is it published soon enough? OPEN DATASUPPORT Slide 8
- 9. Accuracy The accuracy of data is the extent to which it correctly represents the characteristics of the real-world object, situation or event. For example: • Precision in representation of roads on a map in an area • Correct measurement of weather conditions (temperature, precipitation). • Correct indication of re-use conditions of the dataset. Recommendations: • Balance the accuracy of your data against the cost in the context of the application; it needs to be good enough for the intended use. • Make sure that there is organisational commitment and investment in procedures and tools to maintain accuracy . OPEN DATASUPPORT Slide 9
- 10. Accuracy by example Higher accuracy Less accuracy OpenStreetMap, Utrecht (2011 vs. 2007) Cloud cover = 75% Partly cloudy Image licence = Creative Commons AttributionShare Alike 3.0 Unported license. Attribution: Garytx at en.wikipedia OPEN DATASUPPORT Slide 10
- 11. Availability The availability of data is the extent to which it can be accessed; this also includes the long-term persistence of data. For example: • A Dataset that is identified by a http: URI that resolves persistently to the right resource (and does not give back 404 Not found). • A description of the dataset that is included in the search engine of a data portal. Recommendations: • Follow best practices for the assignment and maintenance of URIs. • Make sure that responsibility for the maintenance of data is clearly assigned in the organisation. See also: http://www.slideshare.net/OpenDataSupport/de sign-and-manage-persitent-uris OPEN DATASUPPORT Slide 11
- 12. Availability by example High availability OPEN DATASUPPORT Less availability Slide 12
- 13. Completeness The completeness of data is the extent to which it includes the data items or data points that are necessary to support the application for which it is intended. For example: • A Dataset that includes spending data for all ministries enables a complete overview of government spending. • A description of data that is generated in real time that includes the date and time of last modification. Recommendations: • Design the capture and publication process to include the necessary data points. • Monitor the update mechanisms on a continuous basis. OPEN DATASUPPORT Slide 13
- 14. Completeness by example High completeness OPEN DATASUPPORT Less completeness Slide 14
- 15. Conformance The conformance of data is the extent to which it follows a set of explicit rules or standards for capture, publication and description For example: • A Dataset that expresses coordinates in WGS84 and statistics in SDMX. • A description of a dataset according to the DCAT Application Profile. Recommendations: • Apply the most used standards in the domain that is most relevant for the data or metadata. • Define local vocabularies if no standard is available, but publish your vocabularies according to best practice (e.g. dereferenceable URIs). OPEN DATASUPPORT Slide 15
- 16. Conformance by example High conformance Less conformance See also: https://joinup.ec.europa.eu/asset/adms_foss/ne ws/just-released-admssw-validator-verify-andvisualise-rdf-software-metadata OPEN DATASUPPORT Slide 16
- 17. Consistency The consistency of data is the extent to which it does not contain contradictions that would make its use difficult or impossible. For example: • A dataset that combines data from different sources that has been processed to detect conflicting statements which have been resolved. • A description of a dataset that does not contain multiple licence statements or where the data of last modification is not before the creation date. Recommendations: • Process all data before publication to detect conflicting statements and other errors (in particular if data is aggregated from different sources). OPEN DATASUPPORT Slide 17
- 18. Consistency by example High consistency Less consistency OPEN DATASUPPORT Slide 18
- 19. Credibility The credibility of data is the extent to which it is based on trustworthy sources or delivered by trusted organisations. For example: • A dataset that contains data from processes that can be independently verified, e.g. election results or parliamentary proceedings. • A description of a dataset that is published by a government agency. Recommendations: • Base data on sources that can be trusted or on explicit Service Level Agreements where possible and appropriate. • Make appropriate attributions so that re-users can determine whether or not they can trust the data. OPEN DATASUPPORT Slide 19
- 20. Credibility by example High credibility Data coming from the Publications Office of the EU: Less credibility Data coming from Lexvo: Lexvo data may not be of less quality than EU Publications Office data, but the Publications Office is an authoritative source, while Lexvo is an initiative of an individual. OPEN DATASUPPORT Slide 20
- 21. Processability The processability of data is the extent to which it can be understood and handled by automated processes. For example: • A dataset that contains coded information based on publicly available controlled vocabularies and code lists. • A description of a dataset that expresses dates in W3C Date and Time Format (e.g. 2013-06-01) rather than as text (e.g. 1 June 2013). Recommendations: • Identify the source of terminology and codes used in the data in machine-readable manner. • Apply recommendations for syntax of data given in common standards and application profiles. OPEN DATASUPPORT Slide 21
- 22. Processability by example Higher processability OPEN DATASUPPORT Less processability Slide 22
- 23. Relevance The relevance of data is the extent to which it contains the necessary information to support the application. For example: • A Dataset that contains temperature measurements rounded to degrees Celsius for climate calculations; a dataset with precision of a thousandth of a degree for chemical reactions. • A description of a dataset that only contains temporal coverage data if necessary for its processing . Recommendations: • Match coverage and granularity of data to its intended use within constraints of available time and money. • However, also consider potential future usages of the data. OPEN DATASUPPORT Slide 23
- 24. Relevance by example High relevance Less relevance OPEN DATASUPPORT Slide 24
- 25. Timeliness The timeliness of data is the extent to which it correctly reflects the current state of the entity or event and the extent to which the data (in its latest version) is made available without unnecessary delay For example: • A dataset that contains real-time traffic data that is refreshed every few minutes. • A description of a dataset containing annual crime statistics that is made available within days of publication of the dataset. Recommendations: • Adapt the update frequency of data to the nature of the data and its intended use. • Make sure that processes and tools are in place to support the updating. OPEN DATASUPPORT Slide 25
- 26. Timeliness: examples High timeliness Less timeliness OPEN DATASUPPORT Slide 26
- 27. Best practices Best practices for publishing high-quality data and metadata. OPEN DATASUPPORT Slide 27
- 28. W3C: Best practices for publishing Linked Open Data OPEN DATASUPPORT See also: http://www.slideshare.net/OpenDataSupport/th e-linked-open-government-data-lifecycle Slide 28
- 29. Opquast: 72 Open data good practices Some examples See also: http://checklists.opquast.com/en/opendata OPEN DATASUPPORT Slide 29
- 30. What are the common elements in the best practices • Provide appropriate descriptions of data (i.e. metadata). • Use standard vocabularies for metadata and data whenever such vocabularies exist. • Specify the license under which the data may be re-used. • Adhere to legal requirements concerning protection of personal and other sensitive data. • Represent metadata and data according to the Linked Data principles using persistent URIs for identifying things. • Provide information about the source of the data. Maintenance of metadata and data is critical! See also: http://www.slideshare.net/OpenDataSupport /introduction-to-metadata-management OPEN DATASUPPORT Slide 30
- 31. Conclusions • The quality of data is determined by its fitness for (re-)use by data consumers. • Metadata is “data about data”, i.e. metadata is a type of data. The same quality considerations apply to data and metadata alike. • Data quality has multiple dimensions and is about more than the correctness of data. Accuracy, availability, completeness, conformance, consistency, credibility , processability, relevance, timeliness. OPEN DATASUPPORT Slide 31
- 32. Group questions In your opinion, which factors contribute the most to metadata quality? http://www.visualpharm.com http://www.visualpharm.com Improving quality can require time and resources. To which extent would your organisation be willing to invest in metadata quality? Could you provide an example of high quality metadata description for a dataset using the DCAT Application Profile? http://www.visualpharm.com OPEN DATASUPPORT Slide 32
- 33. Thank you! ...and now YOUR questions? OPEN DATASUPPORT Slide 33
- 34. References Slide 10: Slide 5: • Juran, Joseph M. and A. Blanton Godfrey, Juran's Quality Handbook, Fifth Edition, p. 2.2, McGraw-Hill, 1999 • Garytx at en.wikipedia Slide 6: Slide 11: • • National Information Standards Organization, http://www.niso.org/publications/press/UnderstandingMetadata. pdf Slide 8-26: • Kevin Roebuck. Data Quality: High-impact Strategies - What You Need to Know: Definitions, Adoptions, Impact, Benefits, Maturity, Vendors. Emereo Pty Limited, 2011. http://bit.ly/19Qb6Ov Slide 13: • Mark David Hansen. Zero Defect Data: Tackling the Corporate Data Quality Problem. 1991. http://dspace.mit.edu/handle/1721.1/13812 • • Thomas R. Bruce, Diane Hillmann. The Continuum of Metadata Quality: Defining, Expressing, Exploiting. ALA Editions, 2004. http://www.ecommons.cornell.edu/handle/1813/7895 • Sharon Dawes. Open data quality: a practical view. Open Data Roundtable. October 2012. http://www.slideshare.net/cityhub/sharon-dawes-ctg • Slide 28: • W3C. Best Practices for Publishing Linked Data. W3C Note 06 June 2013. https://dvcs.w3.org/hg/gld/raw-file/default/bp/index.html Slide 29: • OPQUAST. 72 Open data good practices. http://checklists.opquast.com/en/opendata Stefan Urbanek. Data Quality: What is It? January 2011. http://ckan.org/2011/01/20/data-quality-what-is-it/ • ISA Programme. 10 Rules for Persistent URIs. https://joinup.ec.europa.eu/community/semic/document/10-rules-persistenturis Joshua Tauberer. Open Government Data. Section 5.2 Data Quality: Precision, Accuracy, and Cost. June 2012. http://opengovdata.io/201202/page/5-2/data-quality-precision-accuracy-and-cost • European Commission. Telecommunication services: Access to networks (1 000). http://open-data.europa.eu/ Amrapali Zaveri, Anisa Rula, Andrea Maurino, Ricardo Pietrobon, Jens Lehmann, Sören Auer. Quality Assessment Methodologies for Linked Open Data. Semantic Web Journal (unpublished), 2012. http://www.semantic-webjournal.net/content/quality-assessment-methodologies-linked-open-data OPEN DATASUPPORT Slide 34
- 35. Further reading Joshua Tauberer. Open Government Data. http://opengovdata.io/ Juran, Joseph M. and A. Blanton Godfrey, Juran's Quality Handbook OPEN DATASUPPORT Slide 35
- 36. Related projects and initiatives Best Practices for Publishing Linked Data. https://dvcs.w3.org/hg/gld/raw-file/default/bp/index.html OPQUAST. Open data good practices. http://checklists.opquast.com/en/opendata OPEN DATASUPPORT Slide 36
- 37. Be part of our team... Find us on Join us on Open Data Support http://www.slideshare.net/OpenDataSupport Open Data Support http://goo.gl/y9ZZI Follow us @OpenDataSupport OPEN DATASUPPORT http://www.opendatasupport.eu Contact us [email protected] Slide 37
Editor's Notes
- #6: Data hoeftdusniet 100% juisttezijn maar goedgenoeg.
- #10: "Data imperfection is not an excuse for withholding data -- all data is imperfect to some extent, and openness around data can help improve it."
- #11: Thirdexampleto show accuracy. Dutch municipality Zwolle has made a map showingall trees withinitsterritory. If a tree is not on the map youcan cut it without applyingfor a permit first. That is a testament of how accurate theybelievetheir map is.http://www.computable.nl/artikel/praktijk/infrastructuur/4925954/2379248/kaart-stimuleert-samenwerking-in-zwolle.html
- #13: Dataavailablethrough:Existing websiteNew platform, canbehostedbythird partyOr preferably, an API (Application Programming Interface, like a SOAP or REST webservice)Otherexamples of less availability are:A requirementtofill in personal details beforeonecan download/get the dataOr legalimplicationsthatprohibit the use of data
- #15: The map shown is of of the post boxes (brievenbussen) in the UK. The Royal Mail has a list (shown on the left) of all the post boxes in the UK, this is anexample of high completeness. OpenStreetMaprelies on its users to plot the post boxes on the map. As youcansee the completeness of the mapping of post boxesvarieswildly in the UK. Sowhile OSM is not even close to complete, itstillprovides information on post boxes.The thirdexample is of actual missing (meta)data.Main issue here is that is has tobeclearto users of Open Data what is actually ‘nothere’, as this is often hard tosee.
- #19: There is a clear link betweenaccuracyandconsistency.
- #21: Sothisgets a little bit confusing: even though the quality of the data is notalwaysless, therecanbe a difference in credibilityassignedto different datasets and the ‘overall data quality’ maybelower.Consider a situationwhereyoureceiveconflicting information fromtwo sources, whichone do you trust? That is wherecredibilityplaysan important role. But theycouldbothbe right!Andfurthermore, there is always the risk that data from a credible source is not correct. See forexample the Dutch professor Diederik Stapel and Hogeschool InHolland.
- #23: http://www.happy-monkey.net/recipes/ is a website full of recipes in the RecipeBook XML markuplanguageto make eachrecipe machine readable.
- #25: In NL car tax depends on weightandobvisouslynot on color
- #27: Timeliness deals mainlywith the time betweenactual events andwhen data aboutit is beeingpublished. Old data canstillbevaluable, but data has tobepublished in time tobe of use. The more current data is, the less time thereshouldbebetween the moment of creationand the moment of publishing.
- #33: FOCUS ON GROUP QUESTION 2
- #34: See also: http://europa.eu/rapid/press-release_MEMO-11-891_en.htm
- #35: To be updated.