A guest blog post from Carlos Iglesias of the World Wide Web Foundation.
Open data holds the potential to build a prosperous, socially just world with more transparent, accountable, participatory and efficient gGovernments. Governments across the world are increasingly adopting open data policies and practices. From national portals, to municipal open data initiatives and sector specific efforts, open data is now widely been implemented as an important governance innovation.
At the same time, governments, researchers and civil society are continuously seeking for the necessary evidence, not only to be able to benchmark open data performance, but also to deepen their understanding of open data theory and practice as well as of its value chain. A proper coordination and collaboration between existing and emerging measurement analysis will lead to a broader, deeper and more relevant global evidence base, contributing to scale up previous actions that have already been tested and proved to work.
What do we mean by measuring?
There are many different kinds of measurement in practise. Measuring open data may involve the assessment and ranking of Open Government Data initiatives; quantitative metrics of outcomes and impacts; qualitative judgements on performance; case studies about use and results; technical analysis of datasets; or public views on Government use of data among others.
Why measure open data?
Clear, rigorous and relevant approaches to measuring progress towards open data, its implementation and impacts, are vitally important to drive general continuous improvement in practice. There is also a range of different motivations for measuring and assessing the various aspects of open data activities. Policy makers, activists, researchers and many others may be interested in measurement in order to understand, learn, benchmark, compare, prioritize, uphold, reinforce or improve open data.
What to measure?
Several projects are already measuring different aspects of open data and there are often overlaps between the questions and methods employed in all these studies. A proposed Common Assessment Framework for Open Data, developed through a workshop hosted by the Web Foundation and GovLab NYU, was a first step towards identifying different common measurement dimensions and components, including:
- Context and Environment: The circumstances and conditions within which open data is being provided.
- Data: The nature and qualities of open datasets. Including legal, technical, practical and social openness of data, as well as issues regarding data relevance and quality.
- Use: The context of use of an open dataset. Including categories of users accessing a dataset, purposes for which it will be used, and activities being undertaken.
- Impact: The potential benefits to be gained from using an open dataset. Including social, environmental, political, and economic dimensions.
The path towards improved measurement
Measuring open data is a complex task, given its novelty and the diversity of actors involved. Common and relevant methods are needed that can bring together quantitative and qualitative measurement from the global to the local level. This way we could shed light on the means by which open data can lead to actual behavioral changes delivering the expected social and economic change at the end.
We need to understand the requirements and audience for gathering evidence on open data first to be able to increase its beneficial impacts after. Then, we could be better serving the different open data stakeholders needs while merging demand and supply; identifying key measurement gaps and future opportunities; and making more responsive generalizations and insights from the growing amount of case-studies and real-world practices.
More collaborative and inclusive work will be necessary to enable coordination, exchange, transparency and agreement among the different measurement initiatives. The discussion is now open and your contributions really valuable in order to advance.
You can share your reflections on the importance and challenges of open data measurement in the comments below.