Predicting mortality

Algorithms could be used in assessing safety and quality of care.

Predicting which hospitalized patients are most likely to die soon isn't easy.

Despite a plethora of existing algorithms for assessing risks of various outcomes, hospitalists still have to rely mostly on their own judgments about which patients are sickest.

Photo by Thinkstock
Photo by Thinkstock.

This is an imperfect system, according to experts. “It's possible that we frequently are unaware of a patient's poor prognosis when they come into the hospital,” said Carl Van Walraven, MD, associate professor of epidemiology and community medicine at the University of Ottawa in Canada.

“They might be talking and eating and looking healthy, but they might actually be at higher risk of dying than you might realize,” agreed Mark E. Cowen, MD, medical director of the Quality Institute for St. Joseph Mercy Health System in Ann Arbor, Mich.

These physicians and other researchers are working on ways to remedy this problem—specifically, an algorithm to predict an inpatient's risk of death based on data available at admission. Their results are not yet ready for widespread implementation in practice, but they show promise for providing useful data for hospitalists sometime soon.

Why predict?

“There's been a huge hunger on the part of clinicians for this information,” said Dr. Cowen.

Mortality-risk prediction rules have a number of potential applications. Perhaps the simplest is retrospective risk adjustment. In health care systems where facilities or clinicians are paid or rated based on outcomes, knowing whether some patients started out sicker improves the fairness and accuracy of performance measures.

“[Risk prediction] allows proper adjustment when comparing hospitals or providers within the hospital,” said Dr. Van Walraven.

The data could also be useful for hospitals and hospitalists looking to improve their quality of care, noted Gabriel J. Escobar, MD, regional director for hospital operations research for Kaiser Permanente's Northern California Region and Division of Research in Oakland, Calif.

“It would be very productive to look at unexpected deaths in your hospital, as opposed to deaths where the patient came in with an a priori risk that was already quite high,” he said. “You really shouldn't be having deaths in patients who came in with, say, an uncomplicated appendix.”

Risk-prediction scores could identify these cases sooner than current quality assurance methods, he added. “You could provide feedback to clinicians in a day or two about all the harms that are happening in their unit. Right now, the typical lag time for reviewing bad events in the hospital is weeks to months,” Dr. Escobar said.

He and his colleagues developed one of the most well- validated algorithms for such retrospective risk prediction, the Kaiser Permanente risk adjustment methodology. The method uses patients' sex, age, admitting diagnosis (and whether it is surgical or not and emergency or not), and laboratory and comorbidity scores based on preadmission patient records.

The Kaiser Permanente team published evidence of their method's effectiveness in the March 2008 Medical Care, and Dr. Van Walraven showed that it worked in his hospital, too, in a study in the July 2010 Journal of Clinical Epidemiology.

Timing is everything

The only downside is that the methodology is not convenient for the biggest potential application of risk prediction—identifying high-risk patients when they're admitted to the hospital. “It won't be useful for front-line physicians, because the calculation would be very, very difficult,” said Dr. Van Walraven. “You could potentially use hospital data warehouses to calculate these.”

Hospitals that aren't part of closed systems may also lack the warehouses of preadmission data needed to accurately calculate this kind of score. “We don't have quite Kaiser's level of integration, so we had to come up with something else,” said Dr. Cowen.

That something was another rule for predicting hospitalized patients' mortality risk, published online by the Journal of Hospital Medicine in December 2012. Dr. Cowen and colleagues' rule is based on admission diagnoses, any available data from hospitalizations in the preceding 12 months, and the worst value of blood tests obtained in the 30 days prior to admission.

“We can identify a group of patients that we know has a high risk of dying during the next 30 days or even the next 180 days,” said Dr. Cowen. The rule was developed on one set of patients at St. Joseph Mercy Ann Arbor, then validated on another set and a set from another facility.

The time interval of a prediction rule is an important consideration, according to Dr. Escobar, who whose team recently refined his 2008 model and is now about to deploy a real-time alert system at two Kaiser Permanente hospitals. “It has to be narrow so that it's imminent, but with enough warning so that you can do something. If you have a model that says this patient's going to die in the next five minutes, that's totally unhelpful,” he said.

For these predictions to be helpful, hospitalists also have to know how to respond to them. “Let's say you have a patient who comes in, and this patient's risk of dying in the hospital is 20%. What do you do?” asked Dr. Escobar. “Predicting mortality in the hospital is not necessarily that helpful if you don't have the right protocols in place.”

Protocols needed

The Kaiser Permanente and St. Joseph Mercy teams are both currently working on protocol development. Mortality-risk scores might be used to determine what level of care patients receive. “Who should go to an intensive care unit? Who needs intermediate care? Who can go to the regular medical floor?…Here's one objective criterion,” said Dr. Cowen.

Higher-risk patients could also be given more individualized attention. “We think it has implications on physician staffing. Should all hospitalists get 15 patients each, or do we vary the workload based on the acuity and severity of illness?” Dr. Cowen said.

Scores could help multidisciplinary care teams communicate better, said Lakshmi K. Halasyamani, MD, ACP Member, a coauthor with Dr. Cowen, and a hospitalist and chief medical officer of the St. Joseph Mercy Ann Arbor and Livingston hospitals. “Nurses train with nurses, physicians train with physicians, and we think we're communicating, but we don't always have data that helps us say, ‘This is a really sick patient,’” she said.

Although the calculated probabilities are not exact enough to be conveyed directly to patients, they might spur certain conversations between hospitalists and patients. “Advanced care planning is definitely one of those issues,” Dr. Halasyamani said.

“It's not accurate enough to tell a patient, ‘You need to go into comfort care now,’” said Dr. Cowen. However, a high score could be an impetus to start a discussion with the patient regarding do-not-resuscitate orders or palliative care. “If you have to pick the ones to start having that conversation, these are the group to start with,” he said.

Creating ethical protocols will be an important next task for mortality-prediction researchers, said Dr. Escobar. “I've heard that some people have proposed using severity-of-illness scores to determine whether to discontinue life support. I would not be in favor of that,” he said. “[The protocol] has to be something that would be ethically acceptable to the patient, to you, and to the community.”

The rules' findings should also be balanced with clinical assessments, noted Dr. Van Walraven. “While I think these indices would be helpful, I think the clinical experience of the clinician will also be very important,” he said.

Another implementation challenge will be setting the appropriate trigger for action. “If you have an alarm with too many false positives, clinicians are going to burn out and stop using it,” said Dr. Escobar.

Hospitalists are already at risk of burning out on prediction rules, Dr. Cowen noted. “There are lots of prediction rules that are being generated on all sorts of subjects and all sorts of outcomes. We're now at the point where there's the danger of too many prediction rules,” he said.

Rather than adding to the confusion, a proven, effective method of predicting mortality could make many of those other rules unnecessary, he hopes. “You need something simple,” Dr. Cowen said. “Quantifying the patient mortality into a few simple categories becomes a powerful way to start organizing how the team is going to respond.”