Governments

Introduction

A decade ago, many in the open data movement primarily viewed governments as the institutions hoarding data, and they felt it was the responsibility of open data advocates to access and unlock that data. In a simple supply and demand model of open data, the government role was primarily as the data supplier; however, over the last ten years, there has been a recognition that governments can play a much broader role in the open data ecosystem. Not only have a large number of governments embraced open data as part of broader reforms of their digital infrastructure, many have worked to identify and optimise institutional arrangements, as well as the hard and soft policy levers needed to shape the implementation and impact of open data activities within their jurisdictions. In recent years, governments have also had to integrate open data activities into a much wider range of data governance issues that need to be addressed, including privacy, security, and algorithmic governance.

Since the breakthrough of open data onto the international stage in 2009 with President Obama’s Open Government Directive in the United States (US) and the initiatives subsequently launched by the United Kingdom (UK), France, Kenya, Canada, and many other countries, the leaders in power driving those activities have changed, more than once in most cases. The extent to which open data ideas have been sustained across different administrations provides evidence as to the strength of institutionalisation of open data policy within governments. There is evidence in many countries of mature coordination among institutions within and across levels of government, as well as beyond national borders, with work undertaken to sustain the implementation of open data commitments, drawing on initiatives such as the Open Government Partnership and the International Open Data Charter. Open data has also become central to wider agendas, such as monitoring the Sustainable Development Goals (SDGs)¹ or responding to the aftermath of the global financial crisis.² It is notable that government representatives have consistently made up the largest single stakeholder group at the International Open Data Conferences, demonstrating the significance of government officials and resources in popularising open data, as well as their own ongoing interest in exchanging ideas to create a shared global agenda.

In managing the development of (open) data ecosystems both domestically and internationally, clear governance frameworks are critical to supporting inclusive decision-making and the alignment of actions to implement policies which cross administrations and levels of government. Effective governance frameworks can enable the effective coordination of actors within the data ecosystem, including decision-makers, data producers, data users, data stewards, and intermediaries.³

This chapter will argue that clear governance helps provide greater transparency about the key stakeholders, their roles and responsibilities in the management of open data value chains, and who is accountable for open data decisions. This will sustain a more informed dialogue and better engagement across the entire ecosystem on the performance and impact of data publication and reuse.⁴ Even if different contexts for action require different governance models as there is no one-size-fits-all option, the maturing of open data policies requires stable governance models in order to secure long-term continuity and sustainable results.

The following sections will outline how government’s engagement with open data has developed over the last decade, explore common patterns of open data institutionalisation within government systems, set out ten elements of governance that governments need to address to fully realise social and economic value from open data, and consider which of these elements will need the most critical attention in the coming years.

The evolution of open data policy

There have been many strands and stages in the development of government open data policies and practices, starting long before the current wave of interest in opening government data. However, governments have generally had to navigate three main phases in the adoption and implementation of open data programmes, moving from an initial exploration of open data, through experimentation and consolidation, to, more recently, strengthening of international collaboration on common principles.

Initial exploration – making the case

The key drivers for open data have varied from country to country and over time.⁵ In many countries, the Right to Information (RTI) movement, which aims to promote the public’s right of access to information from a human rights and good governance perspective, has played a key role as the promoter of open data, and consequently has been reflected in various governance frameworks and priorities. For example, where open data policies have been driven primarily by RTI advocates, open data programmes have often emerged from building on or adapting access to information laws or equivalent normative instruments.⁶ This has been the case in Mexico and Spain, and evidence from the Organisation for Economic Co-operation and Development (OECD) Open government data report indicates that most OECD countries (21 out of the 33 surveyed in 2018 for the OECD Open Useful and Reusable Data Index – OURData Index⁷) have used freedom of information and/or access to information laws as a legal basis to support the release of open government data. In particular, in most European countries, this approach was accelerated and supported through the transposition into national laws of the European Union Public Service Information (EU PSI) Directive⁸ on the reuse of public sector information. Greece, for example, amended its existing law on the reuse of public sector information in 2014 based on the EU PSI Directive to include an open by default principle.⁹

This approach in Greece was in line with a general trend showing how, in response to open data advocacy, governments have adjusted existing legal and regulatory frameworks with the intent of progressively allowing access by the general public to an increasing amount and variety of data held by the state. For a number of years, the potential value of open data for improved good governance in terms of government transparency and accountability was the primary argument used to make the case for an open data agenda. This often resulted in a strong emphasis placed on the obligation of public bodies to respond to requests for information, rather than on creating the framework conditions, and the incentives necessary, for a proactive release of large volumes of data in ways that permit and encourage reuse.¹⁰

Although one could claim that the strong focus on open data as a tool for open government and public sector transparency meant that few countries initially explored its potential in other policy domains, such as open innovation or urban planning, more recently many public authorities have come to appreciate the relevance of accessible government data for producing social impact and stimulating data-driven economic growth. As a result, governments have started changing the way in which they value, manage, use, and share data both inside and outside their jurisdictions. This increasing focus on the potential economic and social value of open data has resulted in a growing emphasis on establishing conditions that can sustain the public accessibility of key datasets, representing a shift from only publishing reports, analysis, and documentation to substantially increasing the supply of machine-readable data.

A greater focus from the start of this decade on increasing open data availability has translated into efforts geared toward establishing and populating open data portals. Early in this decade, it was not uncommon for governments to benchmark their open data work against peers by comparing the number of datasets listed on their portals. The basic idea was to make as much data as possible accessible for technical users and to provide opportunities for less technical segments of the society to realise the value of data reuse through the provision of tools and guidance on data visualisation or by signposting applications created by third parties.

With time, governments started to understand the importance of going beyond just the release of datasets on data portals. With this recognition, they have increasingly focused on creating a critical mass of open data producers and users, often around specific datasets or particular sectors, in order to foster more sustainable value creation. By adopting and promoting common definitions and standards across the public sector, central units have built a wider common understanding of open data and increased the number of public sector organisations capable of releasing and working with open data. This has helped open data ideas and practices to gain momentum across government, rather than as something solely discussed within technology-focused or executive central departments. This has also been complemented by efforts to actively engage external actors and different communities across the ecosystem to raise awareness and spur reuse, working toward a critical mass of visible case studies of data reuse.¹¹,¹²

This change in focus has contributed to strengthening a more mature open data culture both across governments and the whole ecosystem. As a result, with more stakeholders coming to rely on government data, governments have become more aware of the need to establish sound governance frameworks and to move from experimentation toward making opening data the public sector’s regular way of doing business. Over the last few years, awareness has grown of the importance of creating or upgrading existing policies, institutional structures, and practices in order to produce, manage, and support the use of government data. This has resulted in a rapid maturation of open government data programmes in many countries that have moved from experimentation to consolidation.

From experimentation to consolidation

The 2018 United Nations (UN) E-Government Survey found that 139 countries (72% of UN member states) now have open government data portals, up from just 46 countries in 2014.¹³ Although many of these portals do not yet host predominantly machine-readable data, with PDFs remaining a common information format for dissemination, this trend illustrates the extent to which providing access to data has become an important international norm for governments. However, there is substantial variation in approach and level of investment. For some countries, their open data programmes represent a relatively minor project focused mainly on the operation of a portal, whereas, for others, open data has moved from a small programme to a major policy initiative.

However, even where open data initiatives have become well established, beyond simply the provision of portals, gains made are by no means automatically secure and sustainable. Many initiatives are still vulnerable to changing government priorities or subsequent retrenchment into old business-as-usual models that could undo any progress made to date. Making the most of the opportunities that open data provides requires changes in the organisational structure of governments to avoid existing rules, routines, norms, and power relations that will otherwise leave the potential of open data unfulfilled. Counteracting the tendency of governments to be silo-based, and more prone to inertia than innovation, is a requirement for the successful implementation of open data. Structural changes and new forms of organisation that allow communities to link across organisational boundaries need to be enabled for open data to mature and contribute to public value creation.¹⁴,¹⁵ In some cases, governments have managed to create new operational units to lead open data work, although not all such reforms have proven successful. For example, efforts at the wholesale reform of trading funds (government-owned companies responsible for key public datasets) in the UK, through the creation of a “Public Data Corporation” that was to facilitate public–private and civil society partnerships around strategic datasets, ultimately made little headway.¹⁶

Experience shows that consolidating open data progress requires support for data management across organisational silos, although without necessarily centralising ownership and responsibility for data within a single agency. This kind of governance framework is new for most governments around the world. Civil society activists and the private sector have not always realised the extent of institutional innovation required for government to respond to open data agendas effectively. Governments ultimately need a framework that can provide a friendly policy environment, adequate institutional structures and procedures, a solid and comprehensive legal framework, clear guidance on data licensing and standards, provision of incentives, and a solid commitment to define, steer, and support long-term action plans that include performance indicators and measures. These elements are recognised in World Bank research as critical to helping policy-makers create the enabling environment to move their open data initiatives from experimentation to consolidation.¹⁷

In order to embed new governance frameworks for open data, governments around the world have begun developing and implementing specific open data policies to facilitate data release and use, going beyond simple transparency agendas.¹⁸ However, it is notable that efforts of this type have been more prevalent in developed than in developing countries¹⁹ (e.g. the open data policies that first gained momentum in the UK and US with origins in the UK Power of Information Review²⁰ and the 2009 US Open Government Directive from President Obama²¹). These set the pattern later followed by many other governments, including Kenya, where efforts in 2011 were framed in terms of the power of information,²² and Mexico, where in 2015, the President published the Open Data Executive Decree.²³

In a number of countries, policies have gone through a series of iterations, building open data concepts into government business operations. Open data policy is an essential element of the governance of data ecosystems as it defines a common vision and clarifies goals and expected results, such as increased efficiency, higher transparency, and improved quality of public services. The vision established through policy can also embed key milestones and can propose indicators to measure progress. Having this common vision defined by policy also helps to target efforts toward shared objectives, set expected benefits, monitor achievements at the institutional level, and exploit potential synergies. Where they have been adopted, open data policies have set a process in motion toward political awareness not only among national public authorities, but also within the broader ecosystem of national actors.

Relatively robust data on the consolidation of open data policies and initiatives exists for OECD countries,²⁴ yet there are substantial gaps when it comes to comparable data on the features and functioning of open data policy for other countries around the world. There is also relatively limited information, at present, on how open data policies relate to other policy areas. However, as open data-related initiatives become part of broader efforts to advance the digital transformation of the public sector, it will become essential to understand the linkages between open data and other related policy areas as part of sound data governance. An improved understanding of this could, for instance, help to advance the adoption and implementation of personal data management policies that ensure that data is consistently and properly handled across the public sector, improving data security while also promoting the overall efficiency of the data value chain. Even consolidation efforts are not a one-time thing and require renewal as the wider policy context changes. It is notable that one of the biggest barriers for open data adoption noted in OECD countries in 2018 was the uncertainty currently created by privacy legislation, which has often been developed on an entirely separate track of activity from work on open data.²⁵

International efforts, shared policy, and common principles

The international community has been long aware of the critical value of a friendly policy context for both e-government and the handling of public sector information, although it is only comparatively recently that these elements have been framed in terms of open data. The European Commission (EC) formulated the foundations of open data policy via a Green Paper on Public Sector Information (PSI) in 1998,²⁶ acknowledging the fundamental role of accessible data for the proper functioning of the EU internal market and setting up the key framework conditions for successful open data policy within the EU. In 2003, with the aim of establishing the “systemic” prerequisites for effective use and reuse of data through legal and soft law measures, the EU adopted legislation to foster the reuse of open government data in member states via the PSI Directive 2003/98/EC.²⁷,²⁸

The European Directive established rules regarding availability, accessibility, and transparency of public sector datasets, and recommended that EU member countries both adopt a standard electronic licence for the reuse of data and provide tools to easily find relevant data sets. The PSI Directive was amended in 2013 by Directive 2013/37/EU,²⁹ which introduced the terminology of open data alongside PSI and the general principle that all information accessible under member state legislation should be, in principle, reusable, and that administrative charges should not exceed the marginal costs of making it available for such reuse.³⁰

With a more global focus, on 18 June 2013, the G8 leaders, chaired by the UK, signed the G8 Open Data Charter,³¹ which set out five strategic principles that all G8 members pledged to act on, including an expectation that all government data be published “open by default”, alongside principles to increase the quality, quantity, and reuse of the released data. When adopting this Charter, the G8 members also identified 14 thematic areas – from education to transport and from health to crime and justice – for which they would release “high value” datasets with the expectation that these would help unlock the economic potential of open data, support innovation, and provide greater accountability for governments. This indicated a shift to a more focused approach on specific datasets that should be released at a minimum.

Building on the G8 Open Data Charter, a framework for wider engagement and alignment was established as a result of collaboration between governments and civil society organisations working on open data, leading to a shared set of principles under the banner of the International Open Data Charter.³² This multi-stakeholder charter is intended to support the strengthening of open data around the world through its adoption by national, state, and municipal governments far beyond the G8. The International Open Data Charter, established in 2015, has ultimately proved to be a useful driver for the adoption of open data policies and the establishment of clearer governance frameworks. This is particularly true for countries that were latecomers in regarding government data as a valuable or vital resource for value creation. The Charter has also inspired the adoption of open data international commitments within policy specific global initiatives, such as the G20 Open Data Principles for Anti-Corruption.³³

Institutionalising open data

As shared global principles and clear visions established through policy have matured as a basis for open data activity, governments have had to now grapple with how to institutionalise open data as an integrated practice, and a number of common elements have emerged. With open data policies and programmes continuing to expand, the necessary supporting operational and administrative systems and programmes have become more complex and sophisticated. Certainly, given the complexity and cross-cutting nature of government data, each government needs to establish an organisational set-up that can appropriately support coherence and convergence of the actions of multiple departments responsible for hundreds of lines of business and is aligned with overarching strategic decisions.³⁴ Institutional arrangements as part of the governance framework for data are essential to reinforce governmental capacity to steer strategies and their implementation, as well as to create a collective commitment to the open data agenda across the entire public sector. This kind of collective committee requires a shift in organisational culture that can only happen if individual institutions buy into a government-wide vision that has been clearly articulated and operationalised.³⁵

Despite all the good initiatives underway to adopt policies aimed at improving open data availability, accessibility, and reuse, many countries still lack the internal coordination needed to move their open data agenda forward beyond simple data publication. As a result, many initiatives have been delivered as stand-alone projects, and the challenge remains of aligning the diverse efforts of multiple institutions so that governments can be as effective as possible in delivering public data for the benefit of citizens and businesses. This noted, the general trend across countries with more mature open data ecosystems is in the direction of establishing clearer governance frameworks that clarify responsibilities, capture the interests of all key actors, and respond to their demands (e.g. through the establishment of advisory groups).³⁶

Increased transparency of these governance frameworks themselves is also seen as key to improving accountability and increasing public trust; however, in many cases, too little is known about how open data is governed, how accountable data holders are to both data consumers and policy-makers, how data producers ensure the quality of government data, and who is specifically tasked to make data open.³⁷ We do not just need governance of open data; we also need open governance of data. Getting a better understanding of open data governance is not only important from an accountability point of view. With better insights into the different decision-making models and structures, the implementation of open data principles, such as those advocated by the International Open Data Charter, can be accelerated across countries.

The institutional arrangements put in place to steer strategic decisions and coordinate actions across the public sector often reflect national administrative cultures. Observing over a decade of national experiences has indicated that this often begins with tasking a government body, often the centre of government (such as the Prime Minister’s Office) or a line ministry, with responsibilities related to the government’s digital agenda or public sector modernisation, to champion, coordinate, and provide support for implementing open data programmes. For example, in France the open data agenda is led by Etalab within Prime Minister Services, while in Mexico, the open data programme is under the President’s Office, and in Canada, open data is led by the Treasury Board Secretariat. In Indonesia, the open data initiative is led by the President’s “Delivery Unit”.³⁸ The choice of a charismatic figure, or champion, to lead the coordinating institution also often plays a key role in creating the new mind-set required across the administration. As open data becomes institutionalised within governments, these organisations are increasingly becoming just one player in a new form of open-source governance and may often play the role of arbiter, coordinator, funder, or regulator for the activities of others in delivering public value through the use of open government data.³⁹ From this perspective, open data becomes the platform that can enable governments to establish new forms of collaborations and partnerships.

Embedding effective governance frameworks

How can governments move forward with institutionalising open data? In the box below, ten elements are set out that governments should consider in establishing long-term governance frameworks that can support the institutionalisation of open data activities. While many elements of a governance framework for open data are cross-cutting and intersect with agendas related to digital infrastructure, e-government, public engagement, data protection, and other emerging areas, such as artificial intelligence and algorithmic governance, it is vital for the future of open data that they also remain distinct within the open data governance framework. Without this, it will be challenging to sustain cross-border collaboration on open data into the future.

Although OECD surveys track many elements of an effective open data governance framework, to date, no government has every piece of the puzzle in place such that they can be assured of the sustainability and continuity of their open data work across time and changing political administrations. In particular, more work is needed to fully secure the right policy environments, the connection between data supply and demand, sustained funding for open data, and the mature organisational and partnership arrangements that support collaboration between stakeholders.

Even though, as this chapter argues, the most effective governance frameworks are those that focus on facilitating collaboration and the convergence and alignment of actors, in many contexts around the world, centralised and top-down decision-making processes still prevail. This provides limited mechanisms for different stakeholders to engage and strengthen the open data ecosystem. Notably, only around 40% of OECD countries currently have formal requirements for consultation with open data users to inform their open data planning,⁴⁰ and, as a result, open data is often still primarily supply, rather than demand, driven. The establishment of spaces and mechanisms as part of the overall governance framework that foster collaboration between governments and other stakeholders that leads to value co-creation needs increased emphasis.

When it comes to sustainable funding, there are some positive signs that governments are recognising its importance. Evidence shows that 20 out of the 36 OECD countries surveyed in 2018 for the OURData Index have assigned a distinct line of financing to open government data policies and strategies, and, in 16 of these countries, individual public sector organisations provide finance for their own specific open government data initiatives.⁴¹ However, evidence to date does not indicate how funding is changing year-on-year, nor is there much information on the resources available for open data policy and strategy beyond OECD countries.

Although some governments are well advanced in establishing legal and regulatory frameworks for open data, often building, as noted above, on existing legislation and regulation, in other countries, this remains a pressing gap to fill.⁴² Many developing countries taking part in Open Data Readiness Assessments appear not yet to have enacted and implemented the full range of necessary open data-related legislation, including access to information and data protection acts.⁴³ Even where laws and regulations are in place, training for relevant staff often remains ad hoc rather than part of normalised practices, and, in a number of cases, the independent commissioners required by their enacted legislation have not yet been appointed, creating a gap between the promise and the reality of the regulatory framework.

Similarly, many countries have not yet been able to take forward key policy reforms, such as establishing a clear fee regime for data access. These policies are frequently lacking in countries where open data efforts are led by institutions with limited political power, and the higher-level political leadership necessary to bring alignment and coherence across different data-related agendas is not present. Furthermore, in some countries, the institutionalisation of the responsibilities for coordinating open data activity has not yet been fully realised, creating strong risks for instability as a result of national elections.

In the years ahead, measurement and communications planning will also become increasingly critical parts of open data governance. Open data communities need to recognise shifting government priorities with global policy attention shifting away from open data. Having a clear way to communicate to stakeholders the value of open data in order to sustain awareness of a strategic open data vision will increase in importance. Communication is also vital to realising the benefits of making open data governance more transparent, allowing stakeholders to understand the touchpoints for engaging with government, and keeping track of the ongoing health of open data reforms.

Conclusion

For governments, open data is one of many reform agendas, yet it is one with the potential to transform policy-making and service delivery by fostering whole new models of collaboration. Through regular surveys, including OECD studies and the UN E-Government Report, government progress on open data has been tracked, and there are many promising signs of open data becoming more established and institutionalised. Institutionalisation requires clear and transparent frameworks for the governance of open data, making sure that open data is made integral to the business of government and that it can support engagement across traditional programming silos and stakeholder groups.

International collaboration among governments has made significant progress, especially in the development and adoption of shared principles. In the coming years, deeper collaboration on institutionalising open data will be critical around the development of shared or connected data infrastructures, capacity building, and the sharing of practices, knowledge, and tools.

Most importantly, governments need to continue to evolve more mature, consolidated, and embedded approaches to open data. Although open data reforms have proven reasonably resilient to shifts in political power, they are by no means automatically sustainable. Getting governance models right will be the key to ensuring we can still meaningfully discuss the state of open data for governments ten years from now.