Independent evaluation partners in the REVISE HCC project

How we worked on the REVISE HCC project

Unity Insights were commissioned by The University of Manchester (UoM), Manchester University NHS Foundation Trust (MFT), and Roche Diagnostics as independent evaluators for the REVISE HCC project.

The project, funded by the NHS Cancer Programme Innovation Open Call with support from SBRI Healthcare and the Accelerated Access Collaborative, centred around the real-world validation of the navify® GAAD algorithm, developed by Roche Diagnostics, for the surveillance and detection of hepatocellular carcinoma (HCC) at MFT. Typical HCC surveillance incolves 6-monthly ultrasound (US) scans with optional alpha-fetoprotein (AFP) blood testing, a standard that is not universally adopted across the NHS and leaves desire for improvements around early-stage sensitivity. Additionally, the current structure of the HCC surveillance pathway has demonstrated poor adherence rates, necessitating exploration of factors affecting adherence to surveillance and what changes could be made to the pathway in maximising compliance.

As part of this trial, GAAD blood tests were conducted in conjunction with routine US and AFP blood tests, with different combinations of tests explored in identifying the optimal HCC surveillance pathway both for clinical and economic outcomes. The four surveillance pathway combinations explored were as follows:

• US
• US+AFP
• GAAD
• GAAD+US

As experts in independent evaluation, Unity Insights provided advice and external validation to work packages undertaken by various stakeholders in the consortium, spanning qualitative analysis, quantitative analysis, patient and public involvement, and health economics. Furthermore, we created a spread forecast analysis tool exploring potential spread and national impact, and developed the final project report exploring the findings across the project and answering key evaluation questions spanning process, impact, and economics.

What were the impacts of our work?

Independent review

Unity Insights developed an evaluation plan, summarising the approaches each work package was utilising to answer the proposed evaluation questions. Possible risks were also identified relating to project timelines, work package interdependencies and data collection.

Unity Insights also worked collaboratively with seven stakeholder groups to review study design and protocols across work packages, ensuring that project activities were conducted optimally:

Implementation study, led by MFT

Unity Insights reviewed the statistical analysis plan and interim analysis developed by MFT, spanning retrospective and prospective studies of the HCC surveillance pathway. Gaps were identified in the analysis plan with recommendations on what metrics and analyses to include as areas of interest to the project. The prospective study was also assessed against a quality appraisal checklist and scoring criteria guaging scope and robustness of study design.

Stakeholder engagement and value proposition, led by NIHR In Vitro Diagnostics

The protocols developed for patient and clinician engagement concerning gaps in the HCC surveillance pathway and the acceptability of GAAD among clinicians were assessed by Unity Insights. The Summary Report on Human Factors Studies was also reviewed, with the presentation of results and conclusions drawn assessed.

Health economics, led by NIHR In Vitro Diagnostics

Unity insights reviewed the designs of the health economic models created, a budget impact assessment and a cost-effectiveness model, generating feedback on assumptions and calculations used within the models. The budget impact analysis was also assessed against a framework of principles for good practice spanning data sources, methodology, and the presentation of results.

Development of a Spread Forecast Analysis

A spread forecast analysis, presented as a user-interactable tool, was developed to explore the potential impact of GAAD at prospective implementation sites. It considers how variations in key metrics by geographical region and healthcare site may affect the clinical, economic, and procedural outcomes of implementing GAAD. It provides a range of information, including contextual features of a given site, predicted economic and clinical outcomes of GAAD implementation, a predicted implementation timeline based upon a number of factors including digital maturity, and a list of requirements for implementation. It also provides a site comparison feature, allowing the user to input preference weightings for certain variables in assessing the most appropriate sites for future implementation of GAAD based upon their criteria.

Development of the final evaluation report

Unity Insights effectively planned the creation of the final report detailing the activities that had been undertaken as part of the REVISE HCC project, as well as results of all work packages. The results were utilised to answer the evaluation questions, assessing the process of implementing GAAD, the clinical and economic impacts of GAAD, and the scalability of the solution to other implementation sites. This required liaison with all seven other work packages in order to retrieve relevant outputs and assess key results and messages from the work undergone, as well as additional analyses conducted where required.

Evaluation findings

What value did our work add?

As a result of our work, activities contributing towards building an evidence base for various optimisations of the HCC surveillance pathway and the use of the navify® GAAD algorithm in HCC surveillance were optimised, improving the quality and scope of evidence produced. Outputs were also combined into a report so that a single point of reference may be used by commissioners when deciding on the proliferation of navify® GAAD throughout the NHS.