Background
Despite significant progress in HIV treatment, maintaining patient engagement in care remains a critical challenge. While UNAIDS aims to achieve the “95-95-95” targets by 2030, many people living with HIV (PLHIV), particularly in sub-Saharan Africa, tend to discontinue care within two years. Current one-size-fits-all interventions often lack efficient targeting, and resources could be better allocated to those at the highest risk. The increasing availability of electronic medical records (EMRs) in sub-Saharan Africa, combined with advancements in AI and ML techniques, offers promising solutions for predicting which PLHIV may require additional support.
Methods
We developed a novel AI model by enhancing a pre-trained LLM (LLaMA 3.1, an open-source pre-trained LLM from Meta) using routinely collected EMRs from Tanzania’s National HIV Care and Treatment Program from January 1, 2018, to June 30, 2023 (4,809,765 records for 261,192 people) to identify people at risk of disengaging from HIV care or developing adverse outcomes. Outcomes included risk of ART non-adherence, non-suppressed viral load, and loss to follow-up. Models were evaluated both internally (Kagera region) and externally (Geita region), with performance compared against state-of-the-art ML models and zero-shot LLMs. Additionally, a team of HIV physicians in Tanzania assessed the LLM’s predictions, and LLM provided justifications for a subset of patient records to evaluate their clinical relevance and reasoning.
Results
The enhanced LLMs achieved higher AUC than the supervised ML model and zero-shot LLMs in internal and external validation. When focusing on the 25% of PLHIV predicted as most likely to LTFU, the model correctly identified 78% (2,515 of 3,224) of PLHIV genuinely at risk in internal validation and 73% (7,105 of 9,733) in external validation [Figure 1]. Attention score analysis indicated that the enhanced LLM focused on keywords such as gaps in follow-up care and ART adherence. The human expert evaluation showed alignment between clinician assessments and the LLM’s predictions in 65% of cases, with experts finding the model’s justifications reasonable and clinically relevant in 92.3% of aligned cases.
Conclusions
The results underscore the potential of LLMs in providing actionable public health insights that can guide targeted interventions to enhance patient retention. This study represents the first application of fine-tuned LLMs in predicting disengagement from HIV care using large-scale EMR data.
Table or Figure
