Battelle NLP Name Entity Recognition Model Demo

---

Github Repo

This demo shows the results of a custom NER model trained on a dataset of medical doctor's notes (Harvard N2C2 ADE Dataset). The model was trained using HuggingFace Transformers and Pytorch, and visualized using the spaCy library. The model was trained to recognize the following entities:

• Adverse Drug Event
• Dosage of drug
• Drug Name
• Duration of dose
• Pill, IV, Liquid, etc.
• Frequency of dose
• Reason
• Method of administration
• Strength of drug

BILOU tagging was used to annotate the data.

• B- Beginning
• I- Inside
• L- Last
• U- Single word
• O- Outside

Training Improvements:

We wrapped the HuggingFace Trainer in a Ray Tune Trainer to optimize hyperparameters using the ASHA Hyperband algorithm. We also tried bandits but found better performance using ASHA. All training was done on A 4xA100 GPU server from Purdue's Anvil Supercomputer. We focused on maximizing the F1 score for entity classification.

Original Model Results