Background: The transportability of models for postoperative high-volume cervical nodal disease in papillary thyroid carcinoma (PTC) and the incremental predictive value of molecular variables remain uncertain.
Methods: We developed models in the public Deyang cohort (521 patients; 90 events) and performed geographic external validation in the independent Liaocheng cohort (193 patients; 73 events). The outcome was >5 pathologically confirmed metastatic cervical lymph nodes. Five algorithms were compared using repeated stratified nested cross-validation with a prespecified 10-variable postoperative feature set. Ridge logistic regression (C = 0.1) was selected and applied without refitting. Paired comparisons of 8-variable clinicopathological and 10-variable clinicopathological-molecular models assessed the incremental value of high-risk co-mutation status and mutation count in both cohorts.
Results: Internal validation yielded an area under the receiver operating characteristic curve (AUROC) of 0.795 (95% CI, 0.745-0.842), average precision (AP) of 0.497 (0.406-0.598), and Brier score of 0.115 (0.098-0.134). External AUROC was 0.752 (0.678-0.821), AP 0.601 (0.498-0.728), and Brier score 0.217 (0.180-0.254). Observed and mean predicted risks were 37.8% and 23.9%, respectively; the observed-to-expected ratio was 1.582 (1.319-1.860) and the weak-calibration slope was 0.831 (0.542-1.191). At the prespecified threshold of 0.158, sensitivity was 0.795 and specificity 0.600. Net benefit was 0.254 for the model and 0.261 for classifying all patients as high risk (difference, -0.008; 95% CI, -0.051 to 0.033). External paired differences after adding molecular variables were +0.002 for AUROC (-0.005 to 0.010), -0.002 for AP (-0.016 to 0.010), -0.001 for Brier score (-0.002 to 0.000), and +0.009 for net benefit (-0.003 to 0.026).
Conclusion: The model retained moderate discrimination across centers but underestimated absolute risk and did not demonstrate a net benefit advantage over classifying all patients as high risk at the prespecified threshold. High-risk co-mutation status and mutation count showed no clear incremental predictive value. Further validation with standardized lymph-node assessment and patient outcomes is required before clinical application.
Rationale And Objectives: Cerebral tuberculomas (CTB) and lung cancer brain metastases (BM) frequently show overlapping appearances on conventional magnetic resonance imaging (MRI) but require entirely different treatment strategies. This study aimed to develop and validate a machine learning model integrating clinical and contrast-enhanced MRI features to noninvasively differentiate these two entities.
Materials And Methods: The internal dataset (n = 188, 122 CTB, 66 BM) was split into training/validation (n = 131) and internal test (n = 57) sets. Feature selection (univariate and stepwise logistic regression) was performed only on the training and validation set, yielding four predictors. Machine learning classifiers were trained using these predictors with repeated 5-fold cross-validation. An external test set (n = 51, 26 CTB, 25 BM) from another hospital was used for independent validation.
Results: Multivariate logistic regression retained four independent predictors: age (OR = 1.046, 95%CI: 1.003-1.091, p = 0.036), carcinoembryonic antigen (CEA) (odds ratios [OR] = 1.711, 95%confidence intervals [CI]: 1.235-2.370, p = 0.001), multiple lesions (OR = 0.080, 95%CI: 0.015-0.438, p = 0.004), and ring with mural nodule (OR = 11.289, 95%CI: 2.551-49.952, p = 0.001). The Gradient Boosting model built on these four features achieved the best performance, with an area under the receiver operating characteristic curve of 0.970 (95%CI: 0.925-0.998) on the internal test set and 0.957 (95%CI: 0.900-0.995) on the external test set. On the external test set, the model showed a sensitivity of 88.0% and a specificity of 92.3%.
Conclusion: A Gradient Boosting model integrating age, CEA, multiple lesions, and ring with mural nodule effectively distinguishes CTB from BM with robust external test performance, providing a noninvasive tool to assist clinical decision-making.
Jaeckle F, Gillett PM, Kirkwood KJ et al. · EClinicalMedicine
External validation 98%Prospective 32%
Abstract
Background: Coeliac disease diagnosis on duodenal biopsies is limited by interobserver variability. We have previously demonstrated pathologist-level performance with our artificial intelligence (AI) model for the histopathological diagnosis of adult coeliac disease, but not in paediatric practice. As paediatric coeliac disease screening programmes expand internationally, accurate and scalable diagnostic tools are needed. We investigated whether an AI model trained exclusively on adult whole-slide images (WSIs) can generalise to paediatric coeliac disease diagnosis across independent centres.
Methods: A training and validation dataset of 9958 WSIs from 8421 adult patients (961 coeliac disease; collected Jan 1, 2014, to March 31, 2025) from five centres was used to develop an ensemble of multiple-instance learning models using features from a foundation model. Testing was performed on 708 consecutive paediatric patients (86 with coeliac disease) from two centres (Edinburgh and Southampton) not included in training. Model calibration was assessed, and probability outputs were grouped into clinically interpretable categories.
Findings: In adult cross-validation, the AI model had an area under the receiver operating characteristic curve (AUC) of 99.1% (95% CI 98.9-99.4), sensitivity of 89.3% (87.2-91.1), specificity of 98.8% (98.5-99.0), and negative predictive value (NPV) of 98.6% (98.3-98.9). On testing (paediatric) datasets, performance remained high (AUC 98.8%, 95% CI 98.2-99.4; sensitivity 80.2%, 70.6-87.3; specificity 98.4%, 97.1-99.1; NPV 97.3%, 95.7-98.3). Restricting analysis to predictions outside the intermediate-probability range (predicted coeliac disease probability <10% or ≥65%; 604 [85.3%] of 708 cases) improved sensitivity to 100% (95% CI 93.7-100.0) and specificity to 98.7% (97.4-99.4). No misclassifications were observed among high-confidence predictions (<2% or ≥85%; 467 [66.0%] of 708 cases). The expected calibration error was 0.03. Performance improved significantly when biopsies from both duodenal sites (bulb [D1] and descending [D2/3]) were considered.
Interpretation: Our AI model, trained on adult biopsies, generalises to paediatric coeliac disease diagnosis across centres and scanner platforms. Well calibrated probability outputs provide clinically interpretable measures of diagnostic confidence and could support safe identification of coeliac disease-negative biopsies within defined thresholds. These findings demonstrate the feasibility of applying adult-derived AI models in paediatric populations and reinforce the importance of multi-site (D1 & D2) biopsy sampling.
Funding: The National Institute for Health and Care Research.
Background: Accurate gross tumor volume (GTV) delineation is essential for esophageal cancer radiotherapy, but CT-only automatic delineation remains difficult because of indistinct tumor boundaries.
Purpose: This study aimed to develop and validate a two-stage dual-prompt framework combining a spatial-mask prior with LLaMA3-encoded clinical-text conditioning for primary esophageal GTV delineation.
Methods: This multicenter study included 857 patients: 500 for development, 125 for internal validation, 50 for ablation analysis, and 182 for external validation. The proposed framework consists of two stages: (1) a CT-based automatic segmentation network generating an initial GTV as a spatial mask prompt, and (2) a refinement network that jointly incorporates spatial mask prompt and LLaMA3-encoded clinical text prompt derived from structured clinical information. Performance was evaluated using overlap, surface distance, and volumetric metrics, and compared with multiple state-of-the-art deep learning models. Ablation and radiomics analyses assessed component contributions and downstream consistency.
Results: In internal validation, the framework achieved a DSC of 80.32 ± 6.65%, an HD95 of 11.42 ± 8.90 mm, and an absolute volume difference of 9.99 ± 10.49 mL; all Holm-adjusted p values were <0.001 versus the comparison models. Mean DSCs were 78.60 ± 13.39% and 78.36 ± 15.54% in the external cohorts. Mean DSCs were 78.60 ± 13.39% and 78.36 ± 15.54% in the two external cohorts, respectively. In external cohort A, ContextUNETR showed a lower mean HD95 than the proposed model (10.93 ± 8.03 vs. 13.44 ± 18.90 mm), although the paired difference was not statistically significant after Holm correction (adjusted p = 0.906). Compared with the direct Stage-1 output, the complete framework consistently improved overlap, boundary-distance, and volumetric-agreement metrics across the internal and both external validation cohorts (all adjusted p < 0.001). LLaMA3 encoding improved DSC versus an MLP using the same variables (adjusted p = 0.009), whereas ASSD did not differ significantly. Automatic and manual GTVs showed high radiomic concordance and comparable prognostic performance.
Conclusion: The framework improved overlap and volumetric agreement for CT-only primary esophageal GTV delineation across multicenter validation cohorts, although boundary-distance performance varied across external cohorts.
Background: The blood urea nitrogen-to-creatinine ratio (BUN/Cr) may reflect renal perfusion and systemic stress, but its early course after acute stroke is unclear. We identified 72-h BUN/Cr trajectories and assessed their role in interpretable machine-learning evaluation of 28-day mortality.
Methods: This dual-center landmark study included 2,805 ICU patients with acute ischemic stroke, nontraumatic intracerebral hemorrhage, or nontraumatic subarachnoid hemorrhage from the Medical Information Mart for Intensive Care IV (MIMIC-IV) version 3.1. Before modeling, patients were split into a training set (n = 1,964) and an internal test set (n = 841). The trajectory and prediction models developed in the training set were applied without modification to the internal test set and the Qinghai external validation cohort (n = 387). Eight algorithms and prespecified nested models were evaluated using Cox regression, discrimination, calibration, paired comparisons, decision curves, and TreeSHAP.
Results: A quadratic three-class model identified low/stable, high U-shaped, and rising-to-plateau patterns, with mortality rates of 21.6, 35.9, and 39.7%. Adjusted hazard ratios versus Class 1 were 1.87 (95% CI 1.25-2.79) for Class 2 and 1.73 (1.31-2.29) for Class 3. The training-selected Light Gradient Boosting Machine (LightGBM) achieved an AUROC of 0.734 (95% CI 0.695-0.773), a calibration slope of 0.894, and an O/E ratio of 1.015 in the internal test set. In the Qinghai external validation cohort, the AUROC was 0.624 (95% CI 0.559-0.689), the calibration slope was 0.459, and the O/E ratio was 1.920. Adding trajectory class to a model already containing routine clinical and static renal variables changed the AUROC by 0.003 in the internal test set (p = 0.515) and -0.014 in the Qinghai external validation cohort (p = 0.130), without a clear gain in discrimination.
Conclusion: BUN/Cr trajectories were associated with post-landmark mortality and summarized the three-day renal course. Training-selected LightGBM showed moderate internal discrimination and underwent frozen external evaluation. Trajectory class added modest information beyond routine clinical and static renal variables, while the combined approach provides an interpretable framework for 72-h reassessment.
Background: Ovarian cancer (OC) remains highly lethal due to frequent late-stage diagnosis and the challenge of distinguishing malignant from benign adnexal masses (BAM) preoperatively. Reliable non-invasive diagnostic biomarkers remain an unmet clinical need. This multi-center study aimed to evaluate the diagnostic potential of plasma circulating cell-free mitochondrial DNA (ccf-mtDNA) fragmentomics for improving OC management.
Methods: We developed two machine learning models based on ccf-mtDNA features: an OC Detection (OD) model to distinguishing OC from healthy controls (HC), and a Benign vs. Malignant Differential diagnosis (BMD) model to discriminate OC from BAM.
Results: The OD model achieved an area under the curve (AUC) of 0.987 in the training cohort and maintained high performance (AUC above 0.979) in internal and external validation cohorts, with 91.67% sensitivity for Stage I OC at 95% specificity. The BMD model demonstrated strong discriminatory power (AUC = 0.982) and generalizability (AUC ranging from 0.961 to 0.980 in four validation cohorts), significantly outperforming serum biomarkers. Remarkably, both models performed robustly in the Chinese minority cohort.
Conclusions: These findings establish ccf-mtDNA fragmentomics as a powerful liquid biopsy approach for the early and accurate detection of OC, with significant potential for clinical translation in high-risk populations and differential diagnosis of benign vs. malignant adnexal masses.
Liu HY, Huang YC, Chen SW et al. · JMIR Mhealth Uhealth
Prospective 94%External validation 50%
Abstract
Background: AI-based tools for oral cancer screening have shown promising performance in curated image datasets, but their performance in real-world mobile community screening workflows remains less clear.
Objective: This study evaluated the prospective field performance of a parameter-locked, mobile AI-assisted oral lesion triage system embedded in a human-supervised community oral cancer screening workflow.
Methods: We conducted a prospective community-based field evaluation in eastern Taiwan from June 17 to December 31, 2025. Trained nonspecialist personnel acquired standard white-light intraoral images using handheld mobile devices during routine screening. The AI system incorporated on-site image quality assessment and lesion-level risk stratification into green, yellow, and red triage categories. Technically inadequate images after repeated acquisition attempts were classified as ungradable and excluded from performance analysis. Three board-certified oral and maxillofacial specialists established an operational clinical reference standard through structured image review and consensus adjudication. The primary outcome was lesion-level identification of high-risk lesions requiring specialist referral, defined as red versus nonred triage. Diagnostic performance was estimated with Wilson 95% CIs; precision-recall performance and preconsensus specialist agreement were also assessed.
Results: Among 602 screened participants, 4283 interpretable intraoral images were included in expert adjudication and lesion-level analysis. The AI system flagged 68 red lesions; 12 (17.6%) were confirmed as high risk, and 4 additional high-risk lesions were identified during expert review, yielding 16 high-risk events for the primary red-triage analysis. For high-risk lesion identification, sensitivity was 75% (12/16; 95% CI 50.5%-89.8%) and positive predictive value was 17.6% (12/68; 95% CI 10.4%-28.4%); specificity, negative predictive value, and accuracy were 98.6% (3928/3984), 99.9% (3928/3932), and 98.5% (3940/4000), respectively. Area under the receiver operating characteristic curve values were 0.957 for red, 0.931 for yellow, and 0.964 for green triage; precision-recall analysis showed area under the precision-recall curve values of 0.8235, 0.9052, and 0.9476, respectively. Preconsensus specialist agreement was high, with a Fleiss κ of 0.921 and a Gwet AC1 of 0.981.
Conclusions: This mobile AI-assisted oral lesion triage system was feasibly integrated into a real-world community screening workflow operated by trained nonspecialist personnel. In this low-prevalence field setting, red triage showed moderate sensitivity and modest positive predictive value, supporting its use as human-supervised referral-prioritization support rather than autonomous diagnosis. Further multicenter studies with blinded assessment, participant-level outcomes, workflow-efficiency measures, and longitudinal follow-up are needed before clinical impact and scalability can be determined.
Rationale And Objectives: Chest X-ray (CXR) is low-cost and widely accessible but lacks effective tools for identifying preserved ratio impaired spirometry (PRISm). This study aimed to develop and validate a CXR-based radiomics-clinical combined model for PRISm screening.
Materials And Methods: Paired CXR images and clinical variables from 9 hospitals were retrospectively collected between January 2017 and December 2023. Bilateral lung fields on CXR were segmented with a fully automated segmentation algorithm. PyRadiomics was adopted to extract radiomic features. Clinical, radiomic, and combined models were established. Model performance was assessed using receiver operating characteristic analysis with DeLong's test for intermodel area under the curve (AUC) comparison. Global and local SHapley Additive exPlanations (SHAP) analyses were performed to interpret the model.
Results: A total of 1563 participants (mean age, 63.1 ± 12.0 years; 857 males; 422 PRISm cases) were included. The combined model achieved optimal performance, with AUCs of 0.761, 0.753, and 0.759 across the training, internal, and external validation cohorts. In the external validation cohort, the combined model (AUC = 0.759, 95% confidence interval [CI]: 0.658-0.854) significantly outperformed individual radiomic (AUC = 0.618, 95% CI: 0.505-0.732) and clinical models (AUC = 0.655, 95% CI: 0.549-0.760; all p < 0.001). SHAP analysis revealed sex as the top predictor in the combined model, followed by radiomic features, body mass index, age, and smoking status. Local SHAP visualizations illustrated variable feature contributions for individual PRISm and normal spirometry cases.
Conclusions: The CXR-based combined model exhibits favorable stability and diagnostic performance for PRISm identification, supporting its utility as a primary-screening tool. The SHAP method enhanced the model's interpretability and clinical applicability.
Franceschini K, Valentini M, Leithner A et al. · JMIR Form Res
Prospective 93%External validation 40%
Abstract
Background: Delirium is a complex neuropsychiatric syndrome whose severe and lasting consequences in hospitalized patients are largely considered preventable. Prevention primarily relies on the early identification of high-risk patients prior to syndrome onset. However, this remains challenging in clinical practice due to the absence of a diagnostic gold standard and the considerable administrative burden associated with established screening tools such as the 4AT score.
Objective: The aim of this study is to compare the 4AT score with an established AI-supported prognostic tool in applied clinical practice. Our goal is to demonstrate that shifting from a purely diagnostic approach toward AI-supported delirium risk prediction enables more efficient clinical workflows, reduces misclassification rates, facilitates earlier and more targeted interventions, and ultimately improves long-term patient outcomes.
Methods: In this prospective study, a total of 2002 patients from a trauma ward were screened and assessed for delirium risk using the 4AT score and the AI-based tool at admission, on the evening of admission, and on the following day, with each assessment predicting the risk of delirium occurrence at any point during the remainder of the hospital stay. The AI tool based its predictions on clinical data automatically derived from electronic health records. The AI-based tool applied a previously developed and validated random forest model (not developed as part of this study).
Results: Of the 2002 processed patients, the AI prediction tool correctly predicted 87.3% (165/189) of patients with delirium onset as at-risk, compared to an early detection rate of 28.0% (5/189) achieved by the 4AT score. Moreover, the AI-based tool correctly classified 73.9% (1340/1813) patients without delirium as not-at-risk, compared to 96.3% (1746/1813) for the 4AT score. Therefore, the AI-based tool achieved very good delirium predictions, with an area under the receiver operating characteristic curve of 0.884 (95% CI 0.8634-0.9039).
Conclusions: The AI-based delirium prognostic tool demonstrated superior early predictive capacity for delirium risk at admission, compared with the 4AT score. The AI-based tool enables proactive risk management and targeted risk reduction.
Ignashina M, Bondaronek P, Shams P et al. · JMIR Form Res
Language models 92%
Abstract
Background: Depression prevalence is rising in adolescents and young adults (ages 10-24 years). Traditional screening methods for depression, such as the Patient Health Questionnaire (PHQ-9), are particularly challenging for adolescents. AI has potential to help with this, but the scarcity of annotated datasets highlights the need for off-the-shelf approaches. In this study, we investigate the feasibility of using state-of-the-art large language models (LLMs) to detect depressive symptoms in adolescent settings (ages 15-17 years).
Objective: This study aimed to evaluate the feasibility of using in-context learning with LLMs to identify depressive symptoms in free-text clinical notes for adolescent populations, to evaluate the feasibility of off-the-shelf LLM configurations in detecting PHQ-9 symptom-related information under zero- and few-shot settings, and to demonstrate the potential clinical utility of LLM-driven evidence extraction through an interpretable AI-based depression screening tool.
Methods: We examined free text of the electronic health records (EHRs) of adolescent patients with the diagnosis of depression or related mood disorders (age groups 15-17 years, 924 patients) from the Cincinnati Children's Hospital Medical Center. We leveraged the combination of the Beck Depression Inventory (BDI) and the PHQ-9 to develop tailored categories suited for adolescent depression symptoms. We applied 3 state-of-the-art LLMs (Llama, smaller FLAN-T5, and Phi) to identify these symptom categories in zero- and few-shot configurations, allowing us to assess their performance without additional task-specific training. A word-search model was used as the baseline comparator.
Results: We manually annotated clinical notes for 22 patients across 16 depression-related symptom categories. Our analysis shows that all LLMs substantially outperformed the word-matching baseline. Within the evaluated prompting configurations, FLAN-T5 achieved the highest precision (average F1-score=0.65, precision=0.78) and strong performance in detecting rare symptoms such as sleep problems (F1-score=0.92) and self-loathing (F1-score=0.80). Phi offered a balanced performance (precision=0.44, recall=0.60), while Llama 3 achieved the highest recall (0.90) but tended to overgeneralize, making it less suitable for precise screening. Within this pilot study, the best-performing LLM exceeded the baseline F1-score by 0.43. Finally, we demonstrate the clinical utility of FLAN-T5 annotations as features in a machine learning model, which distinguished depression cases from controls with high precision (0.78).
Conclusions: We used a range of popular LLMs to identify indicators of adolescent (ages 15-17 years) depression symptoms in mental health notes. Within the evaluated configurations, FLAN-T5 demonstrated the most favorable balance between precision and recall. These results support the feasibility of using off-the-shelf LLMs for symptom detection in adolescent clinical notes, while future work using standardized prompting strategies and task formulations will help establish more definitive comparisons between model architectures. The computational efficiency of FLAN-T5 further supports its potential for deployment in resource-limited clinical settings. Our findings require further validation for broader populations and other mental health conditions.
Artificial intelligence (AI) shows promise in identifying psychopathology through language, but replicability in AI models remains challenging. We develop an AI-based language assessment of posttraumatic-stress-disorder (PTSD) severity and introduce the sequential evaluation with model preregistration to rigorously evaluate its validity and replicability. This design includes two phases: development with preregistration and evaluation. Data included development (N = 1,437) and prospective (N = 346) samples, in which participants described their lives during automated interviews. In the prospective sample, preregistered models correlated with PTSD CheckList scores (r = .38, p < .001) and converged with PTSD diagnosis (area under the curve [AUC] = .76; outperforming demographics and trauma exposures: AUC = .61, p < .01). We found that for each standard-deviation increase, mental-health-care expenditure rose by $696.50 (p < .001). Our preregistered PTSD model assessments are replicable in prospectively collected clinical data and showed external validity against expense criteria. With further development, such models can be used to screen for PTSD or monitor treatment response, especially in telehealth or automated interviews, in which deployment can be seamless.