Photo/ Courtesy CBC.co

By Ana Brandusescu (Web Foundation) and Danny Lämmerhirt (Open Knowledge International).

The measurement and accountability working group is a space for those interested in assessing open data policy, publication, use, and outcomes. We bring together the researchers behind the main open data measurement tools, as well as other open data practitioners. The four key indices are:

Our work gives us a better understanding of how to support governments to open up data and sparks debate around challenges faced by governments implementing the Charter.

Our work so far

Over the past year, our group has shaped an agenda addressing three areas:

  • Producing an assessment guide for the Open Data Charter principles,
  • Discussing methodological strengths and weaknesses of existing open data indices and other assessment tools, and
  • Developing areas for primary research.

The group has looked at how easy it is to use the existing open data indices (listed above) to measure the progress governments have made towards implementing the Open Data Charter principles. This exercise has demonstrated that many Charter commitments are either not at all, or only partly, measured by existing measurement tools.

Our upcoming work — an assessment guide for the Open Data Charter

To deal with this challenge, the measurement and accountability working group will work on building an Open Data Charter Assessment Guide. The 2014 Common Assessment Framework (CAF) is used as a foundational point for this project and serves as an inspiration document.

The Charter Assessment Guide centers on the Charter principles and aims to answer the following questions:

  • Which Charter commitments are measurable?
  • Which Charter commitments are not measurable?
  • Where do we see measurement gaps?

The Assessment Guide explains what commitments can realistically be measured, what aspects cannot be measured, and where measurement gaps exist. The Guide will also feature a list of suitable existing measurement tools that can be used to keep track of the Charter adoption.

The Guide will not be a new measurement tool by which Charter adopters will be marked on their progress.

The Charter Assessment Guide will provide a framework to support open data implementation work and to keep track of progress by both government and civil society. It would also provide useful feedback and inputs for a future update of the Charter Principles themselves.

Photo/ Courtesy Computerworld New Zealand

Who is the Guide aimed at?

The Guide could be used by many open data stakeholders:

  • It could be linked directly to the Open Data Charter adoption and implementation process and help governments track their progress and identify areas of focus.
  • Civil society members could use it to flag whenever governments formulate weak, or vague, commitments.
  • Researchers and open data assessment practitioners could identify measurement gaps and develop new indicators.

Ultimately, this document will provide clarity to the entire open data community about what aspects of the Open Data Charter can be feasibly measured, and which ones cannot — and as such intends to address concerns of open washing and open wishing.

How to follow and engage with our work

We have set up a project plan that describes all relevant tasks over the coming months. This includes several work packages that we will work on:

Work package 1 (WP1) is a scoping study to understand what parts of the Charter commitments are measurable and what parts are not. This study will provide us with the necessary information to understand to what extent existing indicators cover Charter commitments.

Work package 2 (WP2) is a comparison of the four international open data indices (ODB, GODI, ODIN, OURdata) to identify similarities, differences, and gaps in measuring the Open Data Charter.

Once a first draft version of the document is completed, we will open it up to the entire open data community for feedback. Afterward, we will turn towards work package 3 (WP3) (developing a process to map existing indicators against the Charter). This, together with the outputs of WP1 and WP2, will form the Charter Assessment Guide.

This is an update on the Measurement and Accountability Working Group (MAWG) of the Open Data Charter. It was jointly written by the group chairs: Ana Brandusescu (Web Foundation) and Danny Lämmerhirt (Open Knowledge International). We invite practitioners, government officials, civil society members, researchers and others to join our work, by sending an email to the Charter’s Programme Officer Franka Vaughan: franka@opendatacharter.org or on Twitter @opendatacharter.

RELATED ARTICLESMORE FROM AUTHOR