Review: Government Online Engagement Service (GOES) Pilot (September 2017)

Executive Summary

“Public engagement is not just desirable; it is a condition of effective governance.”[Footnote 1]

The Government Online Engagement Service (GOES) is a multi-stage initiative that began in 2013. Its aim is to make it easy for people to provide feedback into policy discussions and the design of government services, so government can make better decisions with greater public confidence, trust and legitimacy.[Footnote 2]

To date the initiative has delivered:

• the all-of-government consultations listing (on Govt.nz)
• the online engagement guidance (on digital.govt.nz)
• a pilot for a minimum viable product (MVP) survey tool built using the Common Web Platform (CWP) and hosted on Govt.nz.

This review is part of the work Government Information Services (GIS) is doing with the State Services Commission on the Open Government Partnership (OGP). The OGP seeks greater civic participation to enable openness, transparency and accountability in government. GIS is tasked to deliver Milestone 1 of Commitment 5 of the OGP’s National Action Plan 2016–18, which looks to “improve government’s access to, and use of, digital public engagement tools” by June 2018.

Key Findings

The original focus of the GOES pilot was to test a low-cost survey tool with basic backend moderation for agencies. During the review, it became clear that it was necessary to look at the wider support system of education and capability for digital engagement. While these themes inform the review, only the tool was explicitly evaluated.

Initially, the GOES tool pilot was intended to comprise of a single consultation, but demand from agencies (specifically DIA) resulted in 11 consultations published by December 2016. As a minimum viable product (MVP) it showed that there is demand from agencies for both citizen engagement tools, and help on how to use them. The pilot also identified that some agencies want to have a higher value and more mature engagement with citizens, and to increase participation through a better digital experience.

The pilot review found that the help and advice offered by GIS was particularly valued by agencies. Often the staff leading the consultation lacked digital skills and needed hands-on help with the GOES consultation tool and training to use the moderation backend.

Like many survey tools, GOES focusses on 1-way communication; it is just a digitised version of the traditional paper-based consultations process. The review identified a broader need and appetite for a suite of tools that cover a variety of engagement methods e.g. co-design and participatory design. These user-centric methods support a range of system and citizen benefits, including better quality decision making, more successful policy implementations, social engagement and trust in government.

Another insight from the review is that agencies need educational support and training to be able to successfully engage citizens. Some support is currently available from government; however, there is some fragmentation in how it is provided. Responsibility for best practice public and community engagement sits in two different branches in DIA, as well as in DPMC, and is delivered across three websites. The review findings show that agencies would value a single source for advice and guidance about consultations and engagement, including channels, methods, tools and publishing.

This review outlines next steps for GIS, taking the insights from the review and looking at how we can provide improved access and use of digital engagement tools, advice and guidance in the future.

Recommendations

1. Discovery - citizen engagement with government

We recommend a service design discovery piece be done to identify and define options to extend the digital engagement service beyond the initial GOES pilot. A discovery piece should cover:

• citizens experience when engaging with government
• expertise to educate the system
• requirements for digital engagement tools
• improvements to the consultation listing on Govt.nz

2. Phase out the GOES software tool built on Govt.nz

The operational model for the GOES software tool, where the consultation is edited, made accessible and loaded by GIS, is unsustainable and not scalable because it is highly resource intensive and the tool’s limitations impact agencies’ responsiveness. Additionally, because GOES is a survey tool focusing on ‘broadcast’ 1-way communication, means it isn’t able to support system-wide transformative change. To achieve system change, GOES needs to be able to support 2-way citizen conversation and deliberative participation.

Introduction

Background

The Government Online Engagement Service (GOES) is a multi-stage initiative that began in 2013. It aims to make it easy for people to provide feedback into policy discussions and the design of government services, so government can make better decisions with greater public confidence, trust and legitimacy.[Footnote 3]

To date the initiative has delivered:

• the all-of-government consultations listing (on Govt.nz)
• the online engagement guidance (on webtoolkit.govt.nz)
• an online engagement community (now disbanded)
• a pilot for a minimum viable product (MVP) survey tool built using the Common Web Platform (CWP) and hosted on Govt.nz.

The initiatives went live at different times and within different projects. For example, the consultation listing was delivered as part of the Govt.nz work programme and has come under the Govt.nz product ownership. The online engagement guidance and online engagement community were commissioned by the Digital Engagement team[Footnote 4] with a focus on standards and guidance.

The GOES survey tool pilot is another component of the engagement service. The initial focus of this review was on the agencies use of the tool. However, in order to improve how citizens and government engage, the wider system requirements need to be considered. This review acknowledges the education and capability requirements needed for system change, while focussing specifically on the tool.

Piloting the GOES survey tool

The GOES survey tool leverages off standards which are ‘baked-in’ to CWP and Govt.nz. The CWP platform automatically uses responsive design and meets the New Zealand Web Standards for accessibility.

Data submitted through the survey tool is collected in a custom-built moderation module where agencies can review submissions before publishing. The module also provides some data analysis features.

Initially, the GOES tool pilot was intended to comprise a single consultation, but demand from agencies (specifically DIA) resulted in 11 consultations published by December 2016.

The tool was designed as an MVP to test the potential value of a low-cost survey tool with a core set of generic functionality for agencies.

A key driver for piloting the tool was that the majority of public service departments[Footnote 5] have low levels of maturity when it comes to engaging with the public and specialist stakeholders. Required legislative and regulatory consultations typically take the form of publishing a pdf document on a website and providing an email address for the submission (see Table 1 for an explanation of the current state of consultations in New Zealand).

In this context of low maturity, the survey tool combined with backend data moderation was a major improvement and made it easier for people to submit their feedback and for agencies to analyse the results.

Purpose of the review

The broad purpose of the review has been to:

1. assess the effectiveness of the GOES pilot in achieving its intended results as outlined in the Terms of Reference (Appendix D):

• making it easier for New Zealanders to participate in government
• enabling better government engagement with citizens

2. provide analysis and recommendations that will:

• identify areas for improvement
• inform future product development
• increase product effectiveness

3. gauge need/type of support agencies want

4. recommend the cost recovery model needed to enable further development and ongoing support of the engagement tool.

Scope of the review

The scope of the review covered the consultations published using the survey tool run in the pilot phase between June and December 2016.

Discussion has included other parts of the service (e.g. advice and guidance), as they contribute to the overall objective of making it easier for citizens to participate in government.

Insights and evidence for this review have been gathered from government agencies, NGOs and private sector engagement companies. Interviews with citizens were outside the scope of the current review. However, it is proposed to address this gap as part of the next stage of work, to ensure that government engagement is genuinely people-centred and it is easier to participate.

While the main focus has been on central, not local government, Wellington City Council and Auckland City Council were interviewed.

Methodology

The overall approach to the review included a desk review of all relevant project documentation, a wider environmental scan, and stakeholder interviews.

Further details are available in the Terms of Reference (Appendix D) and the list of interviewees (Appendix C).

Key findings

This is a summary of the GOES review criteria tested against the GOES pilot. See Appendix D for full details.

(a) Value of GOES pilot to agencies

Staff new to consultations

The GOES survey tool was valued by agencies and staff who either had little experience of digital tools or had not previously set up a consultation. Ease of use for agencies depended on the digital skill level of staff. Help with the tool was valued – “Wouldn't have been able to do it if just said ‘go use this tool’”.[Footnote 6]

Staff new to doing consultations also appreciated that GOES removed risks of using digital engagement tools. Using GOES meant that they did not have to worry about security, privacy and web accessibility standards.

Value of the all-of-government brand

Using Govt.nz, a website with all-of-government branding, was the preferred option for multi-agency consultations where it was not appropriate to be published on a single agency site. Some agencies noted that hosting on Govt.nz provided a greater market reach for niche policy consultations.

Quantitative vs qualitative

The GOES survey tool works well for consultations that have a majority of quantitative questions. A few agencies liked the ability to export to excel and do data visualisation on-the-fly[Footnote 7], as well as the rigour the tool provides for data analysis.[Footnote 8]

Value of GOES to citizens

Removing barriers to participation

There is clear evidence, both local and international, that consultations with easy to understand and well-structured information, increase participation for people as well as enable government transparency.[Footnote 9][Footnote 10] Consultations that use quantitative questions and chunk up their information into short sections make it easier and faster for people to submit their feedback. However, simplifying the content is difficult to achieve within the current policy process, with its tight time frames for managerial and ministerial sign-off. Also, survey tools are an example of 1-way communication, where the submission questions and structure focus on information government expects to receive, rather than facilitating a 2-way conversation, enabling people to have their say on issues that matter to them.

The government also needs to look at different ways of communicating to ensure diversity of opinion. The theme that people have the right to submit in the manner that suits them runs through the interviews with agencies,[Footnote 11] and international evidence shows that meaningful engagement with citizens should not take the form of “arms-length instruments such as surveys.”[Footnote 12] This is highlighted with the impact of social media on feedback to government. Staff know that social media is where the “conversation is already happening”, but the government is slow to acknowledge and participate.[Footnote 13] Video, infographics and memes are other methods that might lead to a diversity of opinion.[Footnote 14]

@GovtNZ needs to start taking services and info to citizens more directly, instead of privileging those who have time to seek them out.

Citizen feedback received via the Govtnz Twitter feedback by GIS

Operational insights - impact of GOES on GIS

It became clear during the pilot that the GOES operational model, where the consultation is edited, made accessible and loaded by GIS, is unsustainable and not scalable because of time and effort required.[Footnote 16]For example, hours worked (not including development time) for a recent consultation came in at 105 ($12,600 based on blended rate).

Along with an unsustainable cost model, the focus on the delivery of GOES as a survey tool meant it would never be able to support system-wide transformative change. To achieve system change, GOES would need to expand to include tools for citizen conversation and deliberative participation. Survey tools focus on 1-way, 1-off engagement, not on an ongoing conversation or engagement with citizens.

Broader foundations for citizen to government system change

Interviews with agencies identified areas where they needed more than what the GOES pilot delivered (see figure 1 for themes). For example, agencies want easy-to-understand, pragmatic advice and guidance on best practice methods to engage and consult with people. Additionally, what type of engagement method should be used, how to structure questions, the format for publishing submissions and what digital tool to use. While the online engagement guidance was seen as valuable, agencies want it to be more user-friendly. One agency described the guidance as “overcooked” and wanted a “cheat sheet”’ to make it easier to use.[Footnote 17]

In order to fully realise the benefits that digital technologies can bring to engagement with the public, we need to bring more of this capability into the public service. Engagement advice and support, along with tools and social media, is often contracted out, especially in central government. Advice on tool use is being offered by the private sector, for example the engage tech forums where government agencies can share their engagement experiences and learn about digital tools and social media.

Local government is generally ahead of central government with its focus on working with the community to design services and engage on local issues. Future work in GIS should look to leverage off the knowledge and valuable work that has been done at the local level. For example, in Auckland, The Southern Initiative has seen the development of UPsouth, with residents responding to South Auckland’s challenges by sharing ideas focused on the community’s strengths. The use of interactive simulation tools – including budget simulators – is growing. The Auckland Housing Simulator encouraged input on the city’s Unitary Plan and challenged Aucklanders to find space for another 400,000 homes.[Footnote 18] (See Appendix A for other examples of best practice engagement from New Zealand and other jurisdictions).

Figure 1 depicts key needs identified by agencies: Digital tools; standards and guidance; and skills/capability.

Figure 1: Key needs identified by agencies. Quotes from interviews with stakeholders, May to June 2017

Read detailed description of image

Government agencies were interviewed to find out about the problems they were having when engaging the public. Themes that came from the interviews were that agencies: needed help accessing digital tools; wanted guidance and standards for best practice public engagement; needed support and training for digital tools, social media and different methods of getting public participation (eg co-design, deliberative discussion).

Recommended next steps

1. Discovery - citizen engagement with government

We recommend a service design discovery piece be done to identify and define options to extend the digital engagement service beyond the initial GOES pilot. A discovery piece should cover:

• citizens experience when engaging with government
• expertise to educate the system
• requirements for digital engagement tools
• improvements to the consultation listing on Govt.nz.

2. Phase out the GOES software tool built on Govt.nz

The operational model for the GOES software tool, where the consultation is edited, made accessible and loaded by GIS, is unsustainable and not scalable because it is highly resource intensive and the tool’s limitations impact agencies’ responsiveness. Additionally, because GOES is a survey tool focusing on ‘broadcast’ 1-way communication, means it isn’t able to support system-wide transformative change. To achieve system change, GOES needs to be able to support 2-way citizen conversation and deliberative participation

Appendix A: Understanding the wider strategic context

Best practice in NZ and trends in other countries

“...we [in NZ] are not making much use of the new technologies that allow deep citizen participation and rapid government action”[Footnote 19]

Meaningful engagement with citizens is internationally recognised as a critical part of building a trusted government and stable democracy. Formally engaging the public when developing laws and regulations is one way to measure the extent to which people can become involved in government decisions.

In New Zealand the level of stakeholder engagement in developing regulations is 2.5 (on a scale between 0 and 4); the OECD average is 2.4.[Footnote 20] NZ also scores well in the United Nation’s e-participation index, coming 5th out of 50 countries.[Footnote 21] However, the UN acknowledges that the use of engagement tools by government is not easy to measure and it is challenging to assess the quality.[Footnote 22]

There is no study or data to show where New Zealand sits internationally when it comes to using more transformative methods such as system and design-based approaches. New Zealand has taken an approach in line with other countries with the creation of innovation labs such as the Auckland Co-Design Lab, which is funded by Auckland Council and sponsored by 9 government agencies. Many of the efforts made by innovation labs remain at the margins and do not touch the vast majority of engagements for government policy creation.[Footnote 23]

New Zealand is developing a customer-centric approach to service design and delivery under the Better Public Services (BPS) Programme. Capability development in human-centred design approaches, for example, has been central to the development of early-stage integrated services in the New Zealand context, based on life events (e.g. SmartStart).

The State Services Commission’s Open Government Partnership work has the potential to be a model for other government agencies to follow. A combination of online engagement, (using both GOES and Delib), in-person workshops with their focus on co-design, create a genuine citizen-centric outcome.

While New Zealand is yet to fully exploit the opportunities that digital tools offer, other countries are regularly using them to work with citizens, stakeholders and civil society groups. There are some inspirational examples:

• Taiwan used Pol.is to reach consensus through each stage of its consultations on subjects like regulating Uber and internet sales of alcohol. It is also radically transparent, with the entire consultation continuously summarised, transcribed and published in an open, structured and searchable format.[Footnote 24]
• Canada is co-creating with civic tech and the public an open source eco-system for citizens and government to engage online.[Footnote 25]

Other international trends in this area are for governments to help agencies access existing digital engagement tools. For example:

• the UK Government Digital Services (GDS) found no evidence for GDS to build a standalone consultations platform. Instead, they advise departments to “use the tool from the existing range which best fits their needs”[Footnote 26]
• the Scottish government has partnered with Delib to deliver their consultations hub. It features user-friendly summaries of consultations (e.g. ‘we asked’, ‘you said’, ‘we did’)
• experiments with new tools in parliaments are enabling citizens to contribute to draft legislation. Political parties such as Podemos in Spain and the Icelandic Pirate Party are using tools such as Loomio, Reddit and Discourse to enable party members and the general public to deliberate and feed into policy proposals.[Footnote 27]

Appendix B: Objectives from GOES Terms of Reference

Appendix C: Interviews for GOES review

Interviews were held over May and June 2017.

Appendix D: Terms of Reference: Review of the Government Online Engagement Service (GOES) Pilot

Background information and context

The Government Online Engagement Service (GOES) has been in a pilot phase since June 2016 with 11 consultations managed through the online engagement tool and several more in the pipeline for mid-2017.

Overall purpose

GOES is an all-of-government service that was established to meet increasing demand from government agencies for good practice guidance, access to a community of practice and new ways to engage with citizens online.

The service

The GOES service and support consists of:

• Guidance based on international good practice to help agencies plan, set up and run their consultations.
• Templates to help agencies develop their engagement strategy.
• Working with agencies on their engagement questions, using language that suits that target audience.
• Setting up the consultation within the GOES Content Management System.
• Training users.
• Providing advice on other consultation channels e.g. social media.

GOES consultation tool provides support for:

• web analytics – so agencies can obtain metrics in real time on how well their consultation is performing,
• Advice on data retention to support Public Records Act requirements and Official Information Act requests,

The tool is hosted on the Govt.nz website which is built on the Government’s Common Web Platform. The platform automatically:

• uses responsive design – the tool works easily on a smartphone, tablet or desktop
• meets the New Zealand Web Standards for accessibility – ensuring that this government information is available to everyone
• is easy to use.

Aims and objectives

This service aims to encourage agencies to engage New Zealander in meaningful public participation in policy development and service design, contributing to the development of user-centred services. GOES is a key enabler for fast-tracking the safe and trusted uptake of online engagement services by agencies and is reflected in Action Areas 1.3 and 8.3 of the Strategy and Action Plan.

GOES supports DPMC’s Policy Project which aims to improve the quality of policy advice by transforming the policy development process, and the Open Government Partnership by supporting greater civic participation and increasing access to new technologies for openness and accountability.

Making it easy for citizens to participate in engagement in government

Consultations built with GOES are hosted on Govt.nz and aim to make it easier for stakeholders to participate in online engagement because:

• listings are centrally located so people can find them easier
• responsive design supports different device types, browsers
• emphasis on coaching to help agencies on use plain English should make consultation content easier to understand
• the ability to set up re-director pages for consultations makes it easy to create and use short URLs, helping New Zealanders find consultations, and helping agencies market effectively
• consistent set of online consultation terms and conditions based on international best practice gives users a consistent guide to online engagement.

Enabling better government engagement with New Zealanders

For agencies, GOES provides a reusable, cost-effective option for engaging with their stakeholders to improve the quality of policies, processes or approaches being developed by government.

GOES assists agencies to be more responsive to public expectations of engagement by providing a suite of services and support that are reusable for all-of-government by:

• iteratively delivering a range of online engagement tools and services (through Govt.nz and the Web Toolkit), host on shared services (e.g. CWP, Govt.nz, and common web analytics),
• collaboratively developing good practice guidance and case studies with the NZ Online Engagement Community of Practice.

Purpose of the review

1. The broad purpose of the review is to assess the effectiveness of the GOES Pilot in achieving its intended results:

• making it easier for New Zealanders to participate in government
• enabling better government engagement with citizens.

2. To provide analysis and recommendations that will:

• identify areas for improvement,
• inform future product development,
• increase product effectiveness,

3. Gauge need/type of support agencies want.

4. Recommend the cost recovery model needed to enable further development and ongoing support of the engagement tool.

Scope

The scope of the review is expected to cover the consultations published between June – December 2016. Views will be sought from stakeholders.

Objectives

1. To assess the effectiveness of GOES in making it easier for citizens (stakeholders[Footnote 32]) to participate in government.

• Has GOES improved stakeholder engagement on government policy and decision making processes?

• Has GOES provided better visibility of and access to engagement opportunities compared to previous methods?

• Has GOES provided clear and easy to understand information about consultations open for submission?
• Was the submission process easy to use?
• How transparent was the resulting decision making process?

2. To assess the effectiveness of GOES in enabling better government engagement with citizens

• To what extent has GOES helped to improve the quality of government decisions and outcomes?
• To what extent has GOES provided greater cost-effectiveness compared with traditional methods of stakeholder engagement?

• Did GOES enable efficiency gains and a better quality submission analysis?
• Was GOES able to provide an improvement in the coordination of stakeholder engagement activities across government?
• To what extent were agencies satisfied with the GOES provision of access to online engagement knowledge and expertise.
• To what extent was the GOES tool easy to use?
• To what extent was the GOES tool standards compliant?
• Was training in the use of the tool sufficient? Could this be improved?
• To what extent would agencies pay to use the service in the future?
• How does the service compare with other similar services?

3. Provide Recommended actions noting:

• What could be done better or differently that will improve the GOES? E.g. content strategy, technical improvements?
• What could be done better or differently increase the effectiveness product? E.g. promotion, management, reporting, cost recovery model?

Methodology

The Reviewer should prepare a review plan which will be submitted to and approved by the Product Owner. The review plan should also be appended to the final report, and the main points of the review plan included in the methodology section of the report.

It is envisaged the Review would cover:

• Desk review of all relevant project documentation (see attached list in Appendix C);
• Interviews with key stakeholders
  • GIS Product Owners
  • GIS Principal Advisors
  • GIS Lead Content Designer
  • Partnership Manager
  • Agencies representatives owning consultations
  • Open Government Partnership Team (SSC as part of the National Action Plan commitments)
  • Individuals who made submissions (or a test group)

Deliverables

The Reviewer will produce the following deliverables:

1. Review plan. This will include a detailed approach and methodology
2. Draft evaluation report
3. Final evaluation report

Follow-up

Following submission of the final report, the Product Owner will use the recommendations to inform the strategic direction for GOES Product Roadmap and subsequent work proposals to be submitted to the PGG as required.

Supporting Documents

GIS Work Proposal: GOES Fixes and Enhancements, Susan Carchedi. November 2016.

Government Online Engagement Service (GOES) Stage 2 Business Case, 2014.

Government Online Engagement Service - Background and History - December 2014