Submission ID 131086
| Issue/Objective | Ethiopia continues to face recurrent outbreaks of infectious diseases driven by limited community awareness, constrained healthcare access, and significant language barriers. These challenges are further compounded by limited training of community reporters, cultural misconceptions, and widespread misinformation, delaying timely case detection and response. Access to accurate, real-time health information in local languages remains critically insufficient, undermining effective primary self-care. This study aims to develop and evaluate a multilingual large language model (LLM)-powered chatbot, DiseaseTalk, to strengthen infectious disease community awareness and reporting at the community level. |
|---|---|
| Methodology/Approach | Disease-specific textual datasets were developed in English separately for measles, Mpox, and polio using peer-reviewed literature, national and international guidelines, and insights from community engagement. Each dataset was systematically translated into Amharic and Afaan Oromo and validated by domain experts to ensure clinical accuracy and cultural relevance. The datasets were then structured into a model-compatible JSONL format to support supervised fine-tuning. Model development was conducted using a cloud-based pipeline on Google Vertex AI, leveraging the Gemini 2.5 Flash architecture for efficient and scalable training. Evaluation was conducted using machine learning metrics, including accuracy, alongside assessments of semantic and clinical correctness of responses, and performance in multilingual adaptation to local languages. |
| Results | Preliminary results demonstrate strong model performance, achieving training accuracies of 0.98, 0.97, and 0.90 for measles, Mpox, and polio, respectively. Evaluation through in-house testing and human annotation using a structured rubric demonstrated robust performance across tasks and languages, while user feedback confirmed the system's usability, relevance, and contextual appropriateness. The chatbot effectively supports multilingual interaction, enabling users to access reliable, context-specific health information in their preferred language. |
| Discussion/Conclusion | This study demonstrates how LLM-powered systems can close critical communication gaps in infectious disease management by delivering culturally and linguistically tailored information. DiseaseTalk strengthens community engagement, supports early detection and reporting, and enables more responsive public health action, with strong potential for scalable, context-adaptable deployment aligned with global health priorities. |
| Presenters and Affiliations | Gelan Ayana Zewdie University of Toronto Gelane Biru Jimma University Jude Kong University of Toronto |
