Evaluations are at the heart of our services, ranging from small-scale rapid reviews to long-term, national projects. We work closely with our stakeholders to design and conduct evaluations that are fit for purpose, robust, and dynamic. But what happens at the end of an evaluation when final outcomes are shared? How can stakeholders make the most of the information to support their growth and development? And what happens if the outcomes aren’t quite as expected?
In this blog post, we will explore some of the ways in which evaluation outcomes can be used to drive improvements, whether expected or unexpected.
The impact of evaluation outcomes
Whilst outcomes are often shared at the end of an evaluation, they are not the endpoint. Evaluation outcomes are a valuable source of information that form part of a continuous improvement process – helping to inform developments and next steps, while providing an indication of impact at a given point in time. Depending on the context and scale of the evaluation, outcomes can inform and influence multiple aspects of health and care delivery. The impact of this can be viewed from different perspectives:
Innovators
Evaluations provide an opportunity to identify whether the intervention (i.e., a product or service) works in a specific setting or population, highlighting any issues that may be affecting factors such as effectiveness, value, or sustainability. These insights can be used to inform decision making around whether to adapt, scale-up, or re-think an intervention in the current context, and how to improve its delivery and design.
Outcomes can also highlight enablers and barriers to uptake, and identify other potential areas where the intervention could be adopted, providing valuable learnings to support future implementation. Being able to evidence the impact of an intervention can aid in securing funding to further grow and spread the solution, creating wider opportunities for innovators and those affected by the intervention.
Commissioners
Evaluation outcomes can add to the existing knowledge base, provide evidence to support or challenge current policies, and highlight gaps in current literature. Outcomes from health economic modelling can demonstrate the cost-effectiveness of an intervention, helping guide decisions on resource allocation and commissioning. System dynamics modelling provides insight into the impact of pathway changes, highlighting areas of improved efficiencies or areas requiring further resource.
Understanding the impact of an intervention on staff capacity and workload, alongside patient outcomes, enables commissioners to understand the utilisation needs of the service or setting and subsequent budget requirements.
Patients, public, and staff
For patients and the public, having oversight of the outcomes and impact of an intervention can help them to make informed decisions about their care and improve trust, particularly where there is evidence of patients’ experiences. Knowledge of outcomes and other patients’ experiences may improve the acceptability of an intervention, increasing uptake and improving reach.
Insight of evaluation outcomes can also increase staff engagement with an intervention. Where staff have been involved in an evaluation, there can be satisfaction that their perspectives and experiences have been acknowledged and utilised to improve service delivery.
What happens when outcomes aren’t as expected?
Evaluation outcomes are not always as anticipated. A recent example was an evaluation of a digital health intervention in a hospital setting, which aimed to support patients with their recovery and drive efficiencies for staff. A mixed-methods approach was used, combining quantitative data on uptake and outcomes alongside qualitative insights on patient and staff experiences. Following a challenging period of implementation, the evaluation outcomes were not as anticipated.
Patient uptake and adherence to the intervention was low, and staff did not perceive efficiency savings with how the intervention had been implemented into the pathway. The evaluation identified some challenges and barriers to the implementation and uptake of the intervention, such as technical issues, lack of confidence, and resistance to change. Despite this, some staff and patients saw the value in the intervention, with other clinical pathways highlighted as being a potential better fit.
These findings raised some important questions and implications for the innovator. In such cases, the recommendations are of key importance for stakeholders in order to facilitate next steps. In this example, the innovator pivoted their value proposition and relaunched a new product in response to the outcomes and feedback. They were able to use the insights gained to make meaningful change to their product, to more optimally suit their target market.
How can unexpected outcomes provide benefits?
Whilst unexpected outcomes may be less favourable, they provide valuable learning opportunities for future action. Some examples of how to progress with such outcomes include:
- Celebrate the positive impacts of the intervention and the collection of insights that can guide ongoing improvements.
- Review the recommendations presented in the evaluation and prioritise actions based on their anticipated impact and feasibility.
- Explore different approaches to facilitate adoption, such as implementing in other clinical areas, providing more initial support to users, or involving stakeholders in the co-design and delivery of the intervention.
- Reflect on factors that may have contributed to the outcomes and design strategies to overcome these.
- Revisit the logic model to refine each stage, considering the behaviours of the stakeholders and system context.
- Create opportunities for ongoing measurement to assess the impact of changes made to the intervention.
Summary
Whether as anticipated or unexpected, evaluation outcomes provide valuable insights to inform decision making, improve practices, and create positive changes. This blog has provided some examples of how to utilise outcomes to support next steps, whether favourable or unexpected. If you’re keen to learn more about our evaluation process, from design through to final recommendations, get in touch with our team below.
Get in touch: enquiries@unityinsights.co.uk