FAQs

We have found a baking analogy is the easiest way to explain the discrepancies that can occur between local service data and the KPI Programme's data dashboards.

The process starts when an organisation submits their raw data collection to PRIMHD. The Ministry of Health (the Ministry) then takes the data, along with all the other raw data (ingredients) from other services and organisations across Aotearoa and combines them together.

From here, the Ministry applies appropriate filters, combines values, and de-identifies the data to make it safe to be shared. In other words, they bake the PRIMHD data cake in a way that is safe to be used by other organisations.

The KPI Programme is one of the organisations provided a slice of the PRIMHD data cake for the purposes of informing the relevant indicators and supporting national benchmarking activities. It is important to note that the slice used by the Programme is not the original raw data provided by every individual organisation. It is the 'baked' or processed version.

The consistency of the data slices are reliant on the raw data inputted by each organisation. For example, when PRIMHD data submissions are amended post error checking, the KPI Programme data dashboards will update also each month based on the most recent PRIMHD extract provided by the Ministry.

It is a culmination of all these processes which can impact how closely local data and the KPI Programme data dashboards match at any given time.

While data collection accuracy remains an important focus for all services, exact data matching is not critical for the purposes of benchmarking. This is because benchmarking puts attention on data patterns and trends over time to indicate progress.

If the data discrepancies in the KPI Programme's dashboards raise concern for you, we recommend connecting with your local PRIMHD site coordinator or data analyst to support a data investigation and cross-checking process.

A KPI (indicator) measures quality in service delivery in one of three domains – structure, process, or outcomes.

• Structure describes the resources and policies that enable people and whānau to access services.
• Process describes the evidence-informed treatment and support for people and whānau on their journey through services.
• Outcomes describes the impact of services on people and their whānau (Kilbourne et al., 2010).

Indicators generally measure quantitative (numbers) data. This data can be used to show patterns and create insights, including trends and progress over time. Indicators can help tell if a problem is getting better or worse. Qualitative data (words) tell the story behind the numbers and provide context.

Indicators can also help to simplify a complex range of information concerning the environment, practice and relationships between different things. They are important for informing the public, decision makers and our workforce about emerging and potential areas of concern, and the actions required to address them.

Indicator development is a collaborative process. It takes account of data collected alongside people’s perspectives including people with lived experience, their whānau, and service providers and staff; in conjunction with good practice evidence; and considers health system infrastructure.

Indicators are compiled in a KPI framework that describes specific aspects of people’s journey through the health system, including within and across services.

Benchmarking is a collaborative process between services to compare and evaluate their indicators and identify the underlying cause(s) of differences. The goal is to support all participants to understand good practice that can improve service quality.

Quality describes whether services increase the likelihood of desired health outcomes and are consistent with current evidence-based practice (Funk et al., 2003).

Quality improvement processes are generally internal to an organisation. The Health Quality and Safety Commission supports services to use indicators and evidence (lived experience, clinical, and research) to create sustainable change. In New Zealand, the Triple Aim for quality improvement guides activities across three areas.

1. Improving people and whānau experience of services, quality, and safety.
2. Improving health and equity for all populations.
3. Providing best value for health system resources (Health Quality & Safety Commission New Zealand, 2019)

Processes use the PDSA cycle (Plan, Do, Study, Act). Source: HQSC (Health Quality & Safety Commission New Zealand, 2019)

Funding for the KPI Programme comes from top slice budget that is allocated by DHBs and facilitated by General Managers Planning and Funding. The Ministry of Health (MOH) administer the funding through contracting with a Host organisation that is selected through a contestable process. Funding the Host organisation to deliver the Programme ensures the employment of a KPI Programme Team that supports the sector to deliver goals and activities based on the Programme’s strategic vision.

The KPI Programme team provides the mental health and addiction sector with operational support, and technical assistance to coordinate the key functions of the Programme so it is best positioned to achieve its strategic objectives.

The team is responsible for the ongoing development of the KPI Programme website and dashboard (data visualisation tool), providing guidance and expert advice to continually build sector literacy and capacity to use benchmarking as an effective tool for continuous service quality improvement.

In addition, the team is also responsible for growing and maintaining networks and relationships across the mental health and addiction, and wider social service sector to ensure it is aligned with current health system policies and priorities.