--- title: "Predict need for palliative care in primary care with EHRs | Sutter Health" description: "Applied predictive modeling to primary care population and examine the impact of biostatistical choices on model performance--predict need for palliative care." url: "https://www.sutterhealth.org/research/publications/predict-need-for-palliative-care-in-primary-care-with-ehrs-1055312675" --- 1. [Home](https://www.sutterhealth.org/) / 2. [Research](https://www.sutterhealth.org/research) / 3. [Research publications](https://www.sutterhealth.org/research/publications) / 4. Predict need for palliative care in primary care with EHRs # Predicting need for advanced illness or palliative care in a primary care population using electronic health record data. Jung K, Sudat SEK, Kwon N, Stewart WF, & Shan NH. _J Biomed Inform. 92:103115._ 2019-04-01 ## Abstract Timely outreach to individuals in an advanced stage of illness offers opportunities to exercise decision control over health care. Predictive models built using Electronic health record (EHR) data are being explored as a way to anticipate such need with enough lead time for patient engagement. Prior studies have focused on hospitalized patients, who typically have more data available for predicting care needs. It is unclear if prediction driven outreach is feasible in the primary care setting. In this study, we apply predictive modeling to the primary care population of a large, regional health system and systematically examine the impact of technical choices, such as requiring a minimum number of health care encounters (data density requirements) and aggregating diagnosis codes using Clinical Classifications Software (CCS) groupings to reduce dimensionality, on model performance in terms of discrimination and positive predictive value. We assembled a cohort of 349,667 primary care patients between 65 and 90 years of age who sought care from Sutter Health between July 1, 2011 and June 30, 2014, of whom 2.1% died during the study period. EHR data comprising demographics, encounters, orders, and diagnoses for each patient from a 12 month observation window prior to the point when a prediction is made were extracted. L1 regularized logistic regression and gradient boosted tree models were fit to training data and tuned by cross validation. Model performance in predicting one year mortality was assessed using held-out test patients. Our experiments systematically varied three factors: model type, diagnosis coding, and data density requirements. We found substantial, consistent benefit from using gradient boosting vs logistic regression (mean AUROC over all other technical choices of 84.8% vs 80.7% respectively). There was no benefit from aggregation of ICD codes into CCS code groups (mean AUROC over all other technical choices of 82.9% vs 82.6% respectively). Likewise increasing data density requirements did not affect discrimination (mean AUROC over other technical choices ranged from 82.5% to 83%). We also examine model performance as a function of lead time, which is the interval between death and when a prediction was made. In subgroup analysis by lead time, mean AUROC over all other choices ranged from 87.9% for patients who died within 0 to 3 months to 83.6% for those who died 9 to 12 months after prediction time. [PubMed abstract](https://www.ncbi.nlm.nih.gov/pubmed/30753951) ![Sylvia Sudat, Ph.D.](https://dyn.mktgcdn.com/p/2AHRaDJwevdbwzC2ecrwY3nYIxbiqxd9NbpxBlI8Kpk/fit=crop,gravity=auto,dpr=2,width=188,height=250,quality=100) Sylvia Sudat, Ph.D. Principal investigator [View profile](https://www.sutterhealth.org/research/investigators/sylvia-sudat-phd-1049681027) ## Research topics - [Aging and Longevity](https://www.sutterhealth.org/research-search-results?query=%22%5C%22Aging+and+Longevity%5C%22%22) - [Biostatistics](https://www.sutterhealth.org/research-search-results?query=%22%5C%22Biostatistics%5C%22%22) - [Health Services](https://www.sutterhealth.org/research-search-results?query=%22%5C%22Health+Services%5C%22%22) - [Medical Informatics](https://www.sutterhealth.org/research-search-results?query=%22%5C%22Medical+Informatics%5C%22%22) ## Related publications [View all publications](https://www.sutterhealth.org/research-search-results?verticalKey=research_publication) - Publication ### [Impact of home-based, patient-centered support for people with advanced illness in an open health system: a retrospective claims analysis of health expenditures, utilization, and quality of care at end of life.](https://www.sutterhealth.org/research/publications/impact-of-palliative-care-program-cost-use-qol-1055312725) Sudat SEK, Franco A, Pressman AR, Rosenfeld K, Gornet E, Stewart W. Palliat Med. 2018 Feb;32(2):485-492. doi: 10.1177/0269216317711824. Epub 2017 Jun 7. 2018 Feb 01 - Publication ### [Predicting all-cause risk of 30-day hospital readmission using artificial neural networks.](https://www.sutterhealth.org/research/publications/predict-all-cause-risk-of-30-day-hospital-readmission-1055312686) Jamei M, Nisnevich A, Wetchler E, Sudat S, Liu E. PLoS One. 12(7):e0181173. doi: 10.1371/journal.pone.0181173. eCollection 2017. 2017 Jul 14 - Publication ### [Community-based palliative care and advance care planning documentation: evidence from a multispecialty group.](https://www.sutterhealth.org/research/publications/palliative-care-advance-care-planning-documentation-1055312325) Tai-Seale M, Yang Y, Dillon E, Tapper S, Lai S, Yu P, Allore H, Ritchie C. J Am Geriatr Soc. 2018 Feb;66(2):327-332. doi: 10.1111/jgs.15145. Epub 2017 Oct 24. 2017 Oct 24 - Publication ### [Advance care planning in older adults with multiple chronic conditions undergoing high-risk surgery.](https://www.sutterhealth.org/research/publications/advance-care-plans-with-seniors-and-high-risk-surgery-1055312419) Tang VL, Dillon EC, Yang Y, Tai-Seale M, Boscardin J, Kata A, Sudore RL. JAMA Surg. doi: 10.1001/jamasurg.2018.4647. \[Epub ahead of print\] 2018 Dec 05 - Publication ### [Patient experiences in selecting a Medicare Part D prescription drug plan.](https://www.sutterhealth.org/research/publications/pt-experiences-selecting-medicare-part-d-rx-drug-plan-1055312300) Stults CD, Baskin AS, Bundorf MK, Tai-Seale M. J Patient Experience. https://doi.org/10.1177/2374373517739413 2017 Nov 21 [View all publications](https://www.sutterhealth.org/research-search-results?verticalKey=research_publication) ```json {"@context":"https://schema.org","@type":"MedicalOrganization","name":"Sutter Health","@id":"https://www.sutterhealth.org/#medicalorganization","url":"https://www.sutterhealth.org","logo":"https://www.sutterhealth.org/static/images/sutter-health-logo.svg","address":{"@type":"PostalAddress","streetAddress":"2200 River Plaza Drive","addressLocality":"Sacramento","addressRegion":"CA","postalCode":"95833","addressCountry":"US"},"areaServed":{"@type":"AdministrativeArea","name":"California"},"sameAs":["https://www.facebook.com/SutterHealth","https://www.instagram.com/sutterhealth/","https://x.com/SutterHealth","https://www.youtube.com/sutterhealth","https://www.linkedin.com/company/sutter-health","https://en.wikipedia.org/wiki/Sutter_Health"]} ``` ```json {"@context":"https://schema.org","@type":"BreadcrumbList","@id":"https://www.sutterhealth.org/research/publications/predict-need-for-palliative-care-in-primary-care-with-ehrs-1055312675#breadcrumblist","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://www.sutterhealth.org/"},{"@type":"ListItem","position":2,"name":"Research","item":"https://www.sutterhealth.org/research"},{"@type":"ListItem","position":3,"name":"Research publications","item":"https://www.sutterhealth.org/research/publications"},{"@type":"ListItem","position":4,"name":"Predict need for palliative care in primary care with EHRs"}]} ``` ```json {"@context":"https://schema.org","@type":"MedicalScholarlyArticle","@id":"https://www.sutterhealth.org/research/publications/predict-need-for-palliative-care-in-primary-care-with-ehrs-1055312675#medicalscholarlyarticle","name":"Predicting need for advanced illness or palliative care in a primary care population using electronic health record data.","headline":"Predict need for palliative care in primary care with EHRs","abstract":"Timely outreach to individuals in an advanced stage of illness offers opportunities to exercise decision control over health care. Predictive models built using Electronic health record (EHR) data are being explored as a way to anticipate such need with enough lead time for patient engagement. Prior studies have focused on hospitalized patients, who typically have more data available for predicting care needs. It is unclear if prediction driven outreach is feasible in the primary care setting. In this study, we apply predictive modeling to the primary care population of a large, regional health system and systematically examine the impact of technical choices, such as requiring a minimum number of health care encounters (data density requirements) and aggregating diagnosis codes using Clinical Classifications Software (CCS) groupings to reduce dimensionality, on model performance in terms of discrimination and positive predictive value. We assembled a cohort of 349,667 primary care patients between 65 and 90 years of age who sought care from Sutter Health between July 1, 2011 and June 30, 2014, of whom 2.1% died during the study period. EHR data comprising demographics, encounters, orders, and diagnoses for each patient from a 12 month observation window prior to the point when a prediction is made were extracted. L1 regularized logistic regression and gradient boosted tree models were fit to training data and tuned by cross validation. Model performance in predicting one year mortality was assessed using held-out test patients. Our experiments systematically varied three factors: model type, diagnosis coding, and data density requirements. We found substantial, consistent benefit from using gradient boosting vs logistic regression (mean AUROC over all other technical choices of 84.8% vs 80.7% respectively). There was no benefit from aggregation of ICD codes into CCS code groups (mean AUROC over all other technical choices of 82.9% vs 82.6% respectively). Likewise increasing data density requirements did not affect discrimination (mean AUROC over other technical choices ranged from 82.5% to 83%). We also examine model performance as a function of lead time, which is the interval between death and when a prediction was made. In subgroup analysis by lead time, mean AUROC over all other choices ranged from 87.9% for patients who died within 0 to 3 months to 83.6% for those who died 9 to 12 months after prediction time.","description":"Applied predictive modeling to primary care population and examine the impact of biostatistical choices on model performance--predict need for palliative care.","keywords":["Aging and Longevity","Biostatistics","Health Services","Medical Informatics"],"datePublished":"2019-04-01","url":"https://www.sutterhealth.org/research/publications/predict-need-for-palliative-care-in-primary-care-with-ehrs-1055312675","inLanguage":"en-US","identifier":[{"@type":"PropertyValue","propertyID":"PMID","value":"30753951"}],"sameAs":["https://pubmed.ncbi.nlm.nih.gov/30753951/"],"isPartOf":{"@type":"Periodical","name":"J Biomed Inform"},"author":[{"@type":"Person","name":"Jung K"},{"@type":"Person","@id":"https://www.sutterhealth.org/research/investigators/sylvia-sudat-phd-1049681027","name":"Sudat SEK","url":"https://www.sutterhealth.org/research/investigators/sylvia-sudat-phd-1049681027"},{"@type":"Person","name":"Kwon N"},{"@type":"Person","name":"Stewart WF"},{"@type":"Person","name":"Shan NH"}],"sourceOrganization":{"@type":"MedicalOrganization","name":"Palo Alto Medical Foundation Research Institute"},"about":[{"@type":"DefinedTerm","name":"Aging and Longevity","url":"https://www.sutterhealth.org/research-search-results?facets=%5B%7B%22i%22%3A%22c_researchTopics.name%22%2C%22v%22%3A%5B%22Aging+and+Longevity%22%5D%7D%5D&verticalKey=research_publication"},{"@type":"DefinedTerm","name":"Biostatistics","url":"https://www.sutterhealth.org/research-search-results?facets=%5B%7B%22i%22%3A%22c_researchTopics.name%22%2C%22v%22%3A%5B%22Biostatistics%22%5D%7D%5D&verticalKey=research_publication"},{"@type":"DefinedTerm","name":"Health Services","url":"https://www.sutterhealth.org/research-search-results?facets=%5B%7B%22i%22%3A%22c_researchTopics.name%22%2C%22v%22%3A%5B%22Health+Services%22%5D%7D%5D&verticalKey=research_publication"},{"@type":"DefinedTerm","name":"Medical Informatics","url":"https://www.sutterhealth.org/research-search-results?facets=%5B%7B%22i%22%3A%22c_researchTopics.name%22%2C%22v%22%3A%5B%22Medical+Informatics%22%5D%7D%5D&verticalKey=research_publication"}],"mainEntityOfPage":{"@type":"WebPage","@id":"https://www.sutterhealth.org/research/publications/predict-need-for-palliative-care-in-primary-care-with-ehrs-1055312675","breadcrumb":{"@id":"https://www.sutterhealth.org/research/publications/predict-need-for-palliative-care-in-primary-care-with-ehrs-1055312675#breadcrumblist"}}} ```