Be wary of scams or phishing attempts (e.g. fake website on MSF services). MSF will never ask you to send money, give us your credit card information, or One-Time Passwords (OTP). Learn more from our scam advisory. Stay vigilant and protect yourself against scams.

MSF website will undergo scheduled maintenance on Saturday, 27 April, 10pm to Sunday, 28 April, 6am. During this maintenance period, users may experience intermittent access issues when accessing the website. We apologise for any inconvenience caused.

Have a question about MSF? Find quick answers with our chatbot Ask MSF.

Setting Key Performance Indicators

  • Practice issues

Ang Bee LianMs Ang Bee Lian, 12 January 2018


Dear Social Service Practitioners,

Performance Measurement as Part of Good Service Delivery

Performance measurement is part of good service delivery as it helps promote improvements and ensures accountability to funders and service users. It also helps policy and decision makers to have a deeper understanding of the intent of the policy, the process of implementation and the deployment of resources.

A reasonable standard of service involves meeting or exceeding an acceptable level of performance. While practice guidelines and professional knowledge or training can ensure a reasonable standard of service, it is useful to measure and monitor indicators as a proxy of the standard and to assure users of safe practice and improvements. Good monitoring will point out when a standard or situation falls to an undesirable level and hopefully trigger recovery without resorting to detailed investigations. It is also important as it allows us to calibrate the amount of resources invested into a programme.

Impetus for Accountability

Given that users in the social sector tend to be less informed, have less access to good information and often lack the bandwidth to make demands of services or even accept services, what are the motivating factors for agencies to improve services for the wellbeing of clients?

In recent years, professionalism, regulation and advocates have made accountability more prominent in the discourse about social service delivery and social service interventions. Professionalism has placed expectations of standards on practitioners through a system of governance, be it licensing, accreditation or registration. The state has also increased its influence over services through regulation such as establishing a certain code of practice or compliance to a service standard.

The Use of Key Performance Indicators

A Key Performance Indicator (KPI) is a common form of performance monitoring. It is a set of measures that an organisation uses to gauge its performance over time. While it is a common form of performance monitoring, it is not the only one. Other forms of performance monitoring include regulatory inspection, surveys of consumer experiences and third-party assessments .

KPIs cannot improve the quality of a programme or service but they can act as flags and alerts to identify good practice, and provide comparisons between similar services. They allow us to assess the efficacy of programmes, and can lead to proactive improvement or service recovery. At the agency level, measuring KPIs are useful to learn how to improve service provision, and at a national or system level, KPIs are useful as a tool to evaluate a provider.

When measuring performance through KPIs, it is important to note that the use of a single KPI, or even the use of a limited set of KPIs, may not provide sufficient information. Furthermore, having KPIs should not inadvertently encourage staff and agencies to focus on the activity being measured to the detriment of a service or programme as a whole. This may lead to a situation where only “what gets measured gets done”.

Types of Key Performance Indicators

KPIs should measure if a programme or intervention achieves the intended rationale or intent, the effectiveness of a programme or service delivery, as well as the results arising from the intervention or from receiving the service. KPIs can be characterised according to whether they are generic or specific1. Generic KPIs measure aspects of a service or intervention that apply to the bulk of service users. An example of this is the number of service users awaiting admission to an early intervention programme and the waiting period. Specific KPIs are related to a targeted service user population and measure particular aspects of the intervention related to those service users. For example, this includes the percentage of patients that have been referred for physiotherapy that have waited more than two months between a referral and an assessment. Understanding that there are many types of KPIs allows us to appreciate the deliberations needed for choosing the appropriate one. It will surprise many to know that there can be up to 8 types of KPIs. Hence, it is necessary for these to be discussed, negotiated and agreed upon.

Some common types of KPIs include the following2

  • Quantitative KPI - a measureable characteristic that is usually collected by counting, adding or averaging numbers. It is the most common form of measurement.

  • Qualitative KPI - a descriptive characteristic, an opinion, property or trait. Examples are user satisfaction obtained through survey which gives a qualitative report.

  • Input KPIs - a measure of the assets and resources invested in or used to generate business results. Examples include the amount of money spent on research and development, funds used for staff training and the quality and quantity of practitioners’ hours in interventions such as counselling.

  • Output KPIs - a measure of the financial and nonfinancial results of the input activities. Examples include the number of new users of clients, number of people who are reached or attendance at activities.

  • Process KPIs - a measure of the efficiency or productivity of an intervention or service delivery process. Examples are the number of days to respond to a first contact, months to complete an intervention or completion of a full procedure.

  • Leading KPIs - a measure of activities that have a significant effect on future performance. These act as a measure or predictor of success or failure. Examples include school completion rates and school performance.

  • Lagging KPIs - a measure or indicator that reflects the success or failure after an intervention or activity has been consumed. Examples include the number of users or clients who return to the system.

  • Outcome KPIs - a measure that reflects overall results or impact of an intervention or activity in terms of generated benefits as a quantification of performance. Examples include the rate of not returning to the rehabilitation system or emergency services.

Collecting and analysing data

Choosing what data to collect and when to collect is important as there is a cost to data collection. Focusing exclusively on programme delivery, such as meeting delivery targets or compliance with delivery standards, may be appropriate when programmes are mature and the causal relationships between outputs and outcomes are well understood and established. However, this may not always be the case especially when programmes are new and based on uncertain or unverified underlying assumptions. In such situations, we should consider how to measure the extent to which interventions actually contribute to observed results while taking external factors into account. This will help to deepen the understanding about how the input, output and results interact to produce an eventual outcome. This is particularly so for a new service or programme where the causal relationship is not yet well established and there could be missing variables that a qualitative evaluation could surface at a later stage. This is also to avoid having a service or programme claim undue responsibility for observed results - be it good, bad or neutral. Any KPI needs to be interpreted on the basis of the quality of the data and the definitions that constitute the KPI. It is imperative that there are explicit definitions for each KPI and built-in data quality checks to verify that the required data is accurate, especially when the future of a service or programme is dependent on the data.

For the Client's Best Interest

For KPIs to be useful and contribute well to policy, service or programme evaluation and the deployment of resources, we should deliberate and have consensus on what measurements and data collection are meaningful and worth the time. As social service sector professionals, this is part of our commitment to the best interests of our clients. We must not overlook the importance of performance measurement but should continually monitor and evaluate our programmes, policies and services to ensure that they are effective and best serve the needs of our clients.

References:

Download the full letter here

Ang Bee Lian
MS ANG BEE LIAN

Director-General of Social Welfare
Ministry of Social and Family Development

WHAT TO READ NEXT