Evaluating a machine learning tool for predicting hospital-acquired acute kidney injury

Hospital-acquired acute kidney injury (HA-AKI) is a common complication in hospitalized patients that can lead to chronic kidney disease and is associated with longer hospital stays, higher health care costs, and mortality. is by increase. Given these adverse outcomes, preventing HA-AKI may improve outcomes for hospitalized patients. However, the onset of HA-AKI is difficult to predict because of the many contributing factors involved.

Researchers at Mass General Brigham Digital tested a commercial machine learning tool, the Epic Risk of HA-AKI Prediction Model, and found that it was moderately successful in predicting the risk of HA-AKI in recorded patient data. Stayed. The study found performance lower than that recorded by Epic Systems Corporation’s internal validation, highlighting the importance of validating AI models prior to clinical implementation.

The Epic model works by predicting adult inpatient encounters for the risk of HA-AKI, which is marked by a predetermined increase in serum creatinine levels. After training the model using data from MGB hospitals, the researchers tested it on data from nearly 40,000 inpatient hospitals for a five-month period between August 2022 and January 2023. The data set was extensive and included many points collected on patient encounters, including information. such as patient demographics, comorbidities, principal diagnosis, serum creatinine level and length of hospital stay. Two analyzes were completed looking at encounter-level and predictor-level model performance.

The investigators observed that the instrument was more reliable when assessing patients at low risk of HA-AKI. Although this model could confidently identify which low-risk patients would not develop HA-AKI, it struggled to predict when high-risk patients might develop HA-AKI. Is. Results also vary depending on the stage at which HA-AKI is diagnosed -; Predictions were more successful for stage 1 HA-AKI than for more severe cases.

The authors concluded overall that implementation may result in high false-positive rates and called for further study of the clinical impact of this tool.

We found that the episodic prediction model was better at rejecting low-risk patients than identifying high-risk patients. Identifying the risk of HA-AKI with predictive models may aid in clinical decisions such as warning providers against ordering nephrotoxic drugs, but further study is needed before clinical implementation.”

Sion Dutta, MD, MPH, lead author of the study from the clinical informatics team at Mass General Brigham Digital, and an emergency medicine physician at Massachusetts General Hospital.