Background: Medications are essential for health care but can cause adverse drug events (ADEs), which are harmful and sometimes fatal. Detecting ADEs is a challenging task because they are often not documented in the structured data of electronic health records (EHRs) . There is a need for automatically extracting ADE-related information from clinical notes, as manual review is labor-intensive and time-consuming. Objective: This study aims to fine-tune the pre-trained clinical language model, SweDeClin-BERT, for medical named entity recognition (NER) and relation extraction (RE) tasks, and to implement an integrated NER-RE approach to more effectively identify ADEs in clinical notes from clinical units in Sweden. The performance of this approach is compared to our previous machine learning method, which utilized conditional random fields (CRFs) and Random Forest (RF). Methods: A subset of clinical notes from the Stockholm EPR (Electronic Patient Record) Corpus, dated 2009-2010, containing suspected ADEs based on ICD-10 codes in the A.1/A.2 categories was randomly sampled. These notes were annotated by a physician with ADE-related entities and relations following the ADE annotation guidelines. We fine-tuned the SweDeClin-BERT model for the NER and RE tasks and implemented an integrated NER-RE pipeline to extract entities and relationships from clinical notes. The models were evaluated using 395 clinical notes from clinical units in Sweden. The NER-RE pipeline was then applied to classify the clinical notes as containing or not containing ADEs. Additionally, we conducted an error analysis to better understand the model’s behavior and to identify potential areas for improvement. Results: In total 62% of notes contained an explicit description of an ADE, indicating that an ADE-related ICD-10 code alone does not ensure detailed event documentation. The fine-tuned SweDeClin-BERT model achieved an F1-score of 0.845 for NER and 0.81 for RE task, outperforming the baseline models (CRFs for NER and Random Forests for RE). In particular, the RE task showed a 53% improvement in macro-average F1-score compared to the baseline. The integrated NER-RE pipeline achieved an overall F1-score of 0.81. Conclusions: Utilizing a domain-specific language model like SweDeClin-BERT for detecting ADEs in clinical notes demonstrates improved classification performance (0.77 in strict and 0.81 in relaxed mode) compared to conventional machine learning models like CRFs and RF. The proposed fine-tuned ADE model requires further refinement and evaluation on annotated clinical notes from another hospital to evaluate the model’s generalizability. In addition, the annotation guidelines should be revised, as there is an overlap of words between the Finding and Disorder entity categories, which were not consistently distinguished by the annotators. Furthermore, future work should address the handling of compound words and split entities to better capture context in the Swedish language.

A large language model can now automatically identify adverse drug events from clinical notes in electronic health records, … #Adversedrugevents #Artificialintelligence #Drugsafety #Electronichealthrecords #Pharmacovigilance

Infective endocarditis (IE) is a rare but severe disease with high morbidity and mortality, increasingly affecting older adults. Data on antibiotic to…

Infective endocarditis (IE) is a rare but severe disease with high morbidity and mortality, increasingly affecting older adults. Data on antibiotic to…

Infective endocarditis (IE) is a rare but severe disease with high morbidity and mortality, increasingly affecting older adults. Data on antibiotic to…

Subscription and open access journals from Sage, the world's leading independent academic publisher.