Data management
Data management is about how your data moves through a data lifecycle: plan, collect, short-term storage, analysis, publish and preserve. At each stage you need to consider New Zealand data policies and adopt best practice when looking at how your data is stored, published and used.
New Zealand Data and Information Management Principles
There are 7 principles that help data, held by the NZ Government, to be managed to a high standard. Data should be:
- open
- protected
- readily available
- trusted and authoritative
- well managed
- reasonably priced
- reusable.
New Zealand Data and Information Management Principles — Data.govt.nz
Open data
The Declaration on Open and Transparent Government was approved by Cabinet in 2011. The declaration directs agencies to release high value public data, which is managed according to the New Zealand Data and Information Management Principles.
New Zealand has also adopted the International Open Data Charter principles, which support and build on the New Zealand Declaration on Open and Transparent Government and the supporting Data and Information Management Principles.
Declaration on Open and Transparent Government — Data.govt.nz
Benefits
Managing data appropriately:
- helps to facilitate open data, transparency and reuse
- enables creativity and growth
- preserves public knowledge.
Detailed advice
The data lifecycle
1. Plan
To save time, you should talk to your information or records management colleagues before creating a plan.
At the planning stage you can create your Data Management Plan (DMP).
A DMP is a simple and short document, usually around 1-2 pages, that:
- explains the plan to manage data
- includes what data will be created
- explains how data will be collected, stored, shared, preserved and described.
A DMP will:
- provide transparency and assurance to data suppliers
- identify policy and legislative requirements
- help to anticipate legal, ethical and commercial exceptions around releasing data.
Possible sections to include:
- data formats and storage
- data documentation/metadata
- data governance and access
- data reuse/sharing
- data retention and disposal.
2. Documentation
You need to describe your datasets by documenting:
- metadata, and
- procedures around management.
There are two types of metadata; discovery and technical. Both are required to find, identify and re-use data.
Discovery metadata includes:
- title
- creator
- subject.
Technical metadata includes:
- concepts
- variables
- type of data
- collection methodology.
When recording procedures around managing data, you can think about:
- versioning
- disposal of information, for example will the dataset be deleted or kept
- access information, for example is the dataset open and who decides who can access it
- long term locations and formats
- responsibility for long-term management.
3. Collect
At the collect stage you can begin to add technical metadata to datasets.
4. Short-term storage
Naming conventions will make it easier to identify datasets. When creating storage and naming conventions you should:
- keep different types of datasets in different folders (raw, processing and final)
- create a new dataset for each new version
- make the name of the dataset reflect its contents.
5. Analysis
At this stage you should update the metadata of the datasets.
6. Publish
You can now implement the data management plan.
7. Preserve
At this stage it is important to think about preservation and who is responsible for ongoing maintenance. You should:
- ensure the final location of the dataset is read only
- make sure there's someone responsible for ensuring the dataset does not become corrupt
- maintain long-term access and implement decisions around disposal.
Tools and templates
Related advice
Utility links and page information
Last updated
