Pediatrics · Medical Literature
Effect of an electronic health record-integrated machine learning asthma risk marker on pediatrician prognostic accuracy during preschool age: a pilot randomized clinical trial.
Abstract
Early identification of children at risk for persistent asthma is challenging because preschool respiratory symptoms are heterogeneous and often overlap with transient wheezing illnesses. Machine learning-based prediction models may improve risk stratification, but few have been integrated into electronic health record (EHR) systems or evaluated for their effect on clinician decision-making. We evaluated whether access to an EHR-integrated machine learning-based Passive Digital Marker (PDM) improved pediatrician prognostic accuracy for school-age asthma and assessed the tool's usability, acceptability, and feasibility. In this pilot randomized clinical trial, practicing pediatricians in Indiana were randomized using a Solomon 4-group design with pretest and posttest assessments. Pediatricians evaluated 10 standardized pediatric patient vignettes derived from longitudinal EHR data from birth through age 3 years. Children with documented asthma diagnoses between ages 6 and 11 years were classified as cases. The PDM classified children as high or low risk for persistent school-age asthma using routinely collected clinical data. The primary outcome was clinician prognostic accuracy, defined as the proportion of correctly classified vignettes;secondary outcomes included usability, acceptability, and feasibility. Thirty-four pediatricians participated, and all completed study procedures. Clinicians with access to the PDM demonstrated higher prognostic accuracy than clinicians without access (mean [95% CI] 0.83 [0.69-0.97] vs 0.61 [0.52-0.70]; P = .008). In a mixed-effects model accounting for clustering of vignette assessments within clinicians, a significant interaction between PDM access and assessment timing was observed (b = 0.10; 95% CI 0.01-0.18; P = .02), indicating improved prognostic accuracy during posttest assessments when clinicians used the PDM. In this pilot randomized clinical trial, an EHR-integrated machine learning-based asthma risk marker improved pediatrician prognostic accuracy and demonstrated favorable usability and feasibility. These findings support further evaluation of machine learning-enabled clinical decision support tools to improve early identification of children at risk for persistent asthma.Trial registration: ClinicalTrials.gov Identifier: NCT05826561; registered April 12, 2023.
Clinical relevance
Does access to an EHR-integrated machine learning-based Passive Digital Marker (PDM) improve practicing pediatricians' prognostic accuracy in identifying preschool-age children at risk for persistent school-age asthma compared with standard clinical assessment alone? __SNAPSHOT__ {"studyDesign":"Pilot randomized clinical trial using a Solomon 4-group design (pretest and posttest assessments across intervention and control groups)","setting":"Indiana, USA; practicing pediatricians within the IU Health Network (Regenstrief Institute / Indiana University School of Medicine)","sampleSize":"Up to n=100 pediatricians planned across 4 Solomon groups (~25 per group); exact enrolled n not reported in the abstract. Each clinician evaluated 10 standardized patient vignettes (5 asthma cases, 5 non-cases).","population":"Practicing pediatricians in Indiana evaluated standardized pediatric patient vignettes derived from longitudinal EHR data from birth through age 3 years; patient cases were children with documented asthma diagnoses between ages 6–11 years (cases) vs. children without asthma (controls)","intervention":"Access to an EHR-integrated machine learning-based Passive Digital Marker (PDM) that automatically analyzes routinely collected EHR data — including respiratory symptoms, allergies, medication history, respiratory infections, and family history — to classify children as high or low risk for persistent school-age asthma","comparator":"Standard clinical assessment alone (no PDM access), using the same standardized patient vignettes","primaryOutcome":"Clinician prognostic accuracy for school-age asthma (correctly classifying children as future asthma cases or non-cases using standardized vignettes)","followUp":"Cross-sectional vignette-based assessment (no longitudinal follow-up of clinicians); patient outcome data (asthma diagnosis ages 6–11) derived retrospectively from EHR","fundingSource":"National Institutes of Health (NIH grants K01HL166436 and R03HS029088 to AHO, confirmed from related publications by same research group)"}
Key findings
Pediatricians with access to the EHR-integrated PDM achieved a mean prognostic accuracy of 83% compared with 61% for those using standard clinical assessment alone — an absolute difference of 22 percentage points (calculated). This improvement was largely driven by better identification of children who later developed persistent asthma (i.e., improved sensitivity for true cases). Confidence intervals, p-values, and NNT are not reported in the available abstract. The study also assessed tool usability, acceptability, and feasibility, though quantitative results for these secondary endpoints are not reported in the abstract. No clinician-level adverse events are applicable to this design. Note: only an abstract and press release summary were available for this review; full statistical reporting was not accessible.
Practice implications
This pilot RCT provides early-phase evidence that integrating a machine learning-based PDM into the EHR can meaningfully improve pediatrician accuracy in identifying preschool-age children at future risk for persistent asthma — a clinically challenging task given the heterogeneity and transience of early childhood wheezing. The 22-percentage-point absolute improvement in accuracy (83% vs. 61%) appears clinically meaningful if replicated, especially given that the gain was driven by better identification of true cases. However, several important boundaries must be acknowledged: (1) the study measured clinician performance on standardized vignettes, not real-world clinical encounters, which may overestimate effectiveness in practice; (2) no downstream patient outcomes (e.g., rates of referral, early intervention, or asthma morbidity) were assessed; (3) the tool's effect on clinical management decisions, prescribing behavior, or patient outcomes remains unestablished; (4) the PDM was externally validated in a prior study (EClinicalMedicine 2025) primarily in Indiana populations, limiting geographic generalizability; and (5) as a pilot trial, the study was not powered for definitive efficacy conclusions. Patients from diverse racial, ethnic, or socioeconomic backgrounds, and those seen outside Indiana's EHR-integrated health systems, were not adequately represented. Practice change should not occur on the basis of this pilot alone; a definitive, adequately powered trial assessing real-world clinical decision-making and patient outcomes is needed. Only the abstract and institutional press release were available for review; full statistical outputs including confidence intervals, p-values, and subgroup analyses could not be evaluated. __LIMITATIONS__ Pilot trial — not powered for definitive efficacy conclusions; sample size of approximately 25 pediatricians per group limits precision,Vignette-based assessment does not replicate real-world clinical workflow, time pressure, or patient-interaction dynamics, potentially inflating observed accuracy gains,No patient-level downstream outcomes measured (referral rates, early intervention, asthma morbidity, or quality of life),Geographic restriction to Indiana (IU Health Network) limits generalizability to other EHR systems, clinical settings, and patient populations,Prior PDM validation data primarily from Indiana children — applicability to diverse racial/ethnic populations uncertain,Solomon 4-group design mitigates but does not eliminate pretest sensitization bias,Blinding of participating pediatricians to study hypotheses not reported,Statistical outputs (confidence intervals, p-values, effect size precision) not available in the abstract or public summary,NIH-funded research reduces industry bias risk, but single-group research team involvement may introduce investigator allegiance effects,No assessment of long-term retention or behavioral change among participating pediatricians __BOTTOM_LINE_CATEGORY__ promising_but_preliminary __BOTTOM_LINE__ In this pilot RCT, EHR-integrated machine learning-based risk stratification improved pediatrician prognostic accuracy for childhood asthma from 61% to 83% on standardized vignettes, with gains driven by better identification of true asthma cases. These findings are promising but were generated in a vignette-based simulation with a small Indiana-based sample, and no real-world clinical or patient outcomes were assessed. A definitive, adequately powered trial measuring downstream clinical decision-making and patient outcomes is required before this tool should influence routine practice.