Analytics & Measurement in Patient Safety

class: title-slide

# Analytics & Measurement in Patient Safety

### A wander through my personal experience 
### and work in the NHS

### <img src="./assets/img/crop.png" alt="Mainard icon" height="40px" />  Dr Chris Mainey

.footnote[Presentation and code available: **https://github.com/chrismainey/PS_measurement**]

.art_cap[R generative art - inspired by Antonio Sánchez Chinchón - @aschinchon]

---
class: inverse center middle

.pull-left[
<br>
## Background, and progression to this role
]

.pull-right[
<img src="https://media1.giphy.com/media/H5Ax4wLi7D7Elausl4/giphy.gif?cid=790b7611e64490feb1ebb99c89c34c151f68560728d983ec&rid=giphy.gif&ct=g" width="480" height="270"></img>
]

---

# Analytical  career

+ Didn't start as an analyst

+ Public Health - Cancer registry -> on to "Cancer Network."

+ Acute trust informatics - exposure to production databases

+ Progression to senior analyst
  + Responsible for statistics models
  + Started MPH course

+ PhD in applied statistics and ML, on how to use incident reporting data

+ Developed Senior Data Scientist role and Data Science team + external relationships

+ Taken role in National Patient Safety Team

---

#  Essential experience

.pull-left[
+ Excel
+ Relational Databases:
  + Structure
  + SQL
+ Summary Statistics
+ Data Visualisation
+ Dashboarding
+ Presentation to others

<br>
__It's OK to not understand everything, but work with people who do...__

]

.pull-right[

]

---
# Better Skills:

.pull-left[

+ Asking the right question
+ Sector knowledge / dataset knowledge
+ Study design / how to answer questions
+ Analytical programming language e.g. R or Python
+ Statistical analysis / Modelling / Machine learning methods
+ Collaborative working
  + With other disciplines / clinical colleagues

<br>
__You can't be good at everything, specialising is good__

]

.pull-right[
 
<img src="http://37.media.tumblr.com/tumblr_m0w7ccQomh1rpjhxuo1_400.jpg" height="350">
 
 ]

---

# Top skills:

.pull-left[
 
+ Designing measurement / analytical plans
+ Balancing detail / signal and story
+ Reproducible Analytical Pipelines (RAP) / MLops / DataOps
+ Communicating uncertainty in analysis
+ Ability to apply emerging methods
+ __Causal Inference__

<br>
__Too many specialties...can't keep up with everything...__

]

.pull-right[

]

---
class: inverse middle

# National Patient Safety Team at NHSEI

---

## The Patient Safety Strategy

.footnote[
https://www.england.nhs.uk/patient-safety/the-nhs-patient-safety-strategy/#patient-safety-strategy
]

---
class: inverse middle

## Workstream 1: Analysis of incident reporting data

---

## Using incident reporting data

### "Patient Safety I"

+ Harvard Medical Practice Study
+ IoM: To Err is Human
+ NHS: An Organisation with a Memory

> “Organisations should demonstrate that they have in place fully functional reporting systems for serious incidents, that staff know how to use them, that the systems are used, and that appropriate action is taken in response to incidents, including provision of appropriate support to the affected patients and their carers.” -Berwick, D (2013)

.footnote[Berwick D. A Promise to Learn—a Commitment to Act: Improving the Safety of Patients in England. London: Department of Health; 2013;6.]

<br>
+ Clinical (qualitative review) is major method
+ Quantitative approaches being reduced deliberately

+ Finding relevant incidents is a key question
+ Natural Language Processing (NLP)

---
class: inverse middle

## Workstream 2: Measurement for the Patient Safety Improvement Programmes (SIPs)
---

.pull-left[
## QI & SIPs

+ Quality Improvement now established as a discipline, with it's methods: PDSA cycle & variations: (Lean / Six-sigma etc.)

+ Difficult for analysts to appreciate differences in monitoring, improvement, assurance etc.

]

.pull-right[
<p align="center"><img src="http://www.apiweb.org/images/model.png" alt="IHI and API improvement process diagram"></p>

__IHI and API improvement process diagram: http://www.apiweb.org/__
]

+ 'Quality' and 'Patient Safety' something of a pseudo-science:
  + Psychology
  + Human Factors
  + Process control (Shewhart / Deming)

{{content}}
  
--

+ Contrasts with Donabedian model:
  + Structures
  + Process
  + Outcomes

---

# Methods

+ Depends the question!!!

.pull-left[

### Run charts & SPC

]

.pull-right[
### Funnel Plots

<img src="Chris_Mainey_presentation_files/figure-html/funnel-1.png" width="100%" />
]

---

## Methods

.pull-left[
+ __Statistical analysis:__
  + Hypothesis testing
  + Generalized Linear Modelling (e.g linear regression, logistic regression)
  + Multilevel Models (GLMM, random-intercept)
  + Time series (e.g. ETS or ARIMA)
  + Generalized Additive Models (GAMs)
]

.pull-right[
+ __Causal Inference:__
  + Consider causal pathway!!!
  + Temporal Order
  + Confounders, Mediators etc.
{{content}}
]
<br><br>
<p align = "center"><img src="https://imgs.xkcd.com/comics/correlation.png"><br>
<a href="https://xkcd.com/552/">https://xkcd.com/552/</a></p>

--
  + Difference-in-Difference
  + Regression Discontinuity analysis

---
class: middle inverse

# Example: NEWS2 Deterioration Scores and mortality

---

## Current progress on Causal Path

> _"...Since all models are wrong the scientist must be alert to what is importantly wrong. It is inappropriate to be concerned about mice when there are tigers abroad..."_ Box, GEP (1976)

.footnote[Box, G. E. P. (1976). "Science and Statistics". In: Journal of the American Statistical Association 71.356, pp. 791-799. DOI: 10.1080/01621459.1976.10480949. URL: https://www.tandfonline.com/doi/abs/10.1080/01621459.1976.10480949.]

---
# Summary

.pull-left[
### Developmental:
+ Get experience in different areas to build your skills

+ Takes time to get experience, don't move too fast
{{content}}
]

+ Think about your skill set, and whether new skills will deliver better results

+ Remember it is about the message, despite the importance of detail

.pull-right[
### NHS Patient Safety:
+ Using incident reporting data to learn and build safer systems
  + Clinical review is bottle-neck
  + NLP methods can help provide targets
{{content}}
]

--
+ Measuring improvement is an imperfect thing:
  + Change over time or comparison against standards are main methods
  + Over-reliance on SPC or run-chart
  + All success is interpreted as due to programmes

--
+ __We can do better:__
  + Properly designed study conditions: e.g. stepped-wedge trials
  + Causal Inference

---
class: inverse center middle
.pull-left[
<img src="https://imgs.xkcd.com/comics/curve_fitting.png" alt="XKCD Curve-fitting" height="600px" />
]

.pull-right[

## It's all about the message!!

<br><br><br><br>
__Source:__ xkcd: A webcomic of romance, sarcasm, math, and language

'Curve-Fitting Methods and the messages they send'

https://xkcd.com/2048/
]