Ms Ang Bee Lian, 12 January 2018
Performance measurement is part of good service delivery as it helps promote improvements and ensures accountability to funders and service users. It also helps policy and decision makers to have a deeper understanding of the intent of the policy, the process of implementation and the deployment of resources.
A reasonable standard of service involves meeting or exceeding an acceptable level of performance. While practice guidelines and professional knowledge or training can ensure a reasonable standard of service, it is useful to measure and monitor indicators
as a proxy of the standard and to assure users of safe practice and improvements. Good monitoring will point out when a standard or situation falls to an undesirable level and hopefully trigger recovery without resorting to detailed investigations.
It is also important as it allows us to calibrate the amount of resources invested into a programme.
Given that users in the social sector tend to be less informed, have less access to good information and often lack the bandwidth to make demands of services or even accept services, what are the motivating factors for agencies to improve services for the wellbeing of clients?
In recent years, professionalism, regulation and advocates have made accountability more prominent in the discourse about social service delivery and social service interventions. Professionalism has placed expectations of standards on practitioners through
a system of governance, be it licensing, accreditation or registration. The state has also increased its influence over services through regulation such as establishing a certain code of practice or compliance to a service standard.
A Key Performance Indicator (KPI) is a common form of performance monitoring. It is a set of measures that an organisation uses to gauge its performance over time. While it is a common form of performance monitoring, it is not the only one. Other forms of performance monitoring include regulatory inspection, surveys of consumer experiences and third-party assessments .
KPIs cannot improve the quality of a programme or service but they can act as flags and alerts to identify good practice, and provide comparisons between similar services. They allow us to assess the efficacy of programmes, and can lead to proactive improvement or service recovery. At the agency level, measuring KPIs are useful to learn how to improve service provision, and at a national or system level, KPIs are useful as a tool to evaluate a provider.
When measuring performance through KPIs, it is important to note that the use of a single KPI, or even the use of a limited set of KPIs, may not provide sufficient information. Furthermore, having KPIs should not inadvertently encourage staff and agencies
to focus on the activity being measured to the detriment of a service or programme as a whole. This may lead to a situation where only “what gets measured gets done”.
KPIs should measure if a programme or intervention achieves the intended rationale or intent, the effectiveness of a programme or service delivery, as well as the results arising from the intervention or from receiving the service. KPIs can be characterised according to whether they are generic or specific1. Generic KPIs measure aspects of a service or intervention that apply to the bulk of service users. An example of this is the number of service users awaiting admission to an early intervention programme and the waiting period. Specific KPIs are related to a targeted service user population and measure particular aspects of the intervention related to those service users. For example, this includes the percentage of patients that have been referred for physiotherapy that have waited more than two months between a referral and an assessment. Understanding that there are many types of KPIs allows us to appreciate the deliberations needed for choosing the appropriate one. It will surprise many to know that there can be up to 8 types of KPIs. Hence, it is necessary for these to be discussed, negotiated and agreed upon.
Some common types of KPIs include the following2
Choosing what data to collect and when to collect is important as there is a cost to data collection. Focusing exclusively on programme delivery, such as meeting delivery targets or compliance with delivery standards, may be appropriate when programmes
are mature and the causal relationships between outputs and outcomes are well understood and established. However, this may not always be the case especially when programmes are new and based on uncertain or unverified underlying assumptions. In such
situations, we should consider how to measure the extent to which interventions actually contribute to observed results while taking external factors into account. This will help to deepen the understanding about how the input, output and results
interact to produce an eventual outcome. This is particularly so for a new service or programme where the causal relationship is not yet well established and there could be missing variables that a qualitative evaluation could surface at a later stage.
This is also to avoid having a service or programme claim undue responsibility for observed results - be it good, bad or neutral. Any KPI needs to be interpreted on the basis of the quality of the data and the definitions that constitute the KPI.
It is imperative that there are explicit definitions for each KPI and built-in data quality checks to verify that the required data is accurate, especially when the future of a service or programme is dependent on the data.
For KPIs to be useful and contribute well to policy, service or programme evaluation and the deployment of resources, we should deliberate and have consensus on what measurements and data collection are meaningful and worth the time. As social service sector professionals, this is part of our commitment to the best interests of our clients. We must not overlook the importance of performance measurement but should continually monitor and evaluate our programmes, policies and services to ensure that they are effective and best serve the needs of our clients.
(1)Health Information and Quality Authority. (2013, February). Retrieved from https://www.hiqa.ie/sites/default/files/2017-01/KPI-Guidance-Version1.1-2013.pdf
(2)ProjectManagement.com. (2017, February 6). Key Performance Indicators. Retrieved from Project Management.com: https://www.projectmanagement.com/wikis/345150/Key-Performance-Indicators
Download the full letter here
MS ANG BEE LIAN
Director-General of Social Welfare