Triage Score: A Counterfactual Risk Assessment Instrument

Risk assessment instruments, also known as "risk scores,'' are widely used in high-stakes decision-making settings such as medicine and the criminal justice system. A risk score predicts the likelihood of an undesired outcome if no intervention is made.  Thus, a sufficiently high score is often interpreted as a recommendation to intervene. However, risk scores fail to account for what would happen if a decision-maker does intervene.  This is problematic because effective decision-making requires consideration of how the intervention affects outcomes.  We propose ``triage scores,'' which are based on additive counterfactual utilities and include risk scores as special cases.  Unlike risk scores, triage scores can incorporate counterfactual outcomes under alternative decisions, enabling decision-makers to incorporate a wide range of ethical and practical factors. We illustrate the use of triage scores with an application to our own randomized controlled trial evaluating a pre-trial risk assessment instrument.  Our analysis demonstrates that triage scores are able to capture richer utility structures than risk scores and yield substantively distinct results regarding policy evaluation and learning. 

Host: Ran Chen

Kosuke Imai is a professor in the Department of Government and the Department of Statistics at Harvard University. Before moving to Harvard in 2018, Imai taught at Princeton University for 15 years, where he was the founding director of the Program in Statistics and Machine Learning.  Imai works on the development of statistical and machine learning methods for social science research.  He served as the President of the Society for Political Methodology from 2017 to 2019, and received the Guggenheim fellowship in 2024.  He has been recognized as one of the highly cited researchers by Clarivate Analytics since 2018.