National Neurosurgery Audit Programme (NNAP): Modernising Mortality Benchmarking for NHS Neurosurgery Units

The challenge of fair comparison in a low-volume specialty

The National Neurosurgical Audit Programme (NNAP), established by the Society of British Neurological Surgeons in 2013, publishes annual benchmarking reports for 24 NHS neurosurgical units in England. These reports use HES and ONS mortality data to compare unit-level activity and mortality rates, with a risk-adjusted funnel plot of elective 30-day mortality serving as the primary outlier surveillance tool. However, the existing process was time-intensive, the risk adjustment methodology had not been revisited since inception, and procedure definitions needed updating. With a national elective mortality rate of just 0.54%, the statistical challenge of reliably identifying genuine outliers was considerable.

Rebuilding the audit’s analytical engine

Edge Health was commissioned to update and modernise the NNAP benchmarking programme across three workstreams:

1. Refining the activity metrics and aligning procedure definitions with national definitions from the Getting It Right First Time Programme (GIRFT) and the National Consultant Information Programme (NCIP)
2. Exploring improvements to the risk adjustment methodology for elective mortality
3. Producing unit-level PDF reports for each of the 24 participating sites on a repeatable basis.

A clinician-led, statistically rigorous approach

We adapted the risk adjustment framework developed for NCIP, training a logistic regression model on over 113,000 historical neurosurgical episodes to predict patient-level 30-day mortality. The model adjusted for age, sex, deprivation, comorbidities, frailty, and primary diagnosis using a clinician-led variable selection process. Clinical experts manually reviewed all elective deaths to identify and exclude cases where the underlying diagnosis or procedure indicated emergency pathology that had been misclassified as elective. Power calculations informed the selection of a two-year test window, balancing timeliness with statistical validity.

We also fitted a multilevel logistic regression as a methodological comparison. This showed near-identical unit rankings (r = 0.985), but the funnel plot method was retained as it produces Standardised Mortality Ratios, which are more familiar to clinical audiences and widely accepted in clinical audit.

Validated reports and an academic contribution

The final model achieved an AUROC of 0.857, demonstrating very good discriminative ability with strong calibration. It was used to produce 24 unit-level performance packs containing activity metrics, case-mix profiles, and risk-adjusted mortality funnel plots. An automated data pipeline means reports can now be produced on a regular cadence without the delays previously associated with data access.

The methodology is also being written up for publication in the BMJ, co-authored with clinicians from the University of Cambridge and the London School of Hygiene & Tropical Medicine. The paper provides a transparent, reproducible framework for benchmarking elective surgical mortality using administrative data, designed to be applicable to other smaller surgical specialties beyond neurosurgery.