Project Details

Performance of machine interpretation for pediatric clinical encounters: A multidisciplinary study

Ryan Brewster, MD

Summary

BACKGROUND: Patients who speak languages other than English experience poorer quality of care and clinical outcomes. Machine translation and interpretation supported by generative artificial intelligence (AI), including large language models, may improve linguistically concordant care. GAP: The accuracy, quality, and potential applications of machine interpretation in clinical settings have not been studied nor validated. HYPOTHESIS: Machine interpretation will exhibit variable performance relative to professional interpreters, with lower accuracy and quality for digitally underrepresented language (i.e. languages without robust training and validation datasets for AI models) and high acuity clinical scenarios METHODS: We will conduct and evaluate multilingual simulated encounters for common pediatric encounters using two different interpreter modalities: ChatGPT-4o with interpretation capabilities, and a professional human audio interpreter. The accuracy and quality of ChatGPT-4o will be compared to the professional interpreters by a multidisciplinary panel of evaluators, including clinicians, interpreters, and family caregivers. RESULTS: Pending. IMPACT: This study will provide the foundational evidence characterizing the strengths and limitations of machine interpretation in clinical practice. Findings will carry immediate implications for institutional AI policy development, strategic implementation, and patient-centered technology development to advance safe and equitable care for patients who use LOE.