Continuous Pre-training and Performance Evaluation of NorDeClin-BERT for ICD-10 Coding Prediction (Preprint)

<sec> <title>BACKGROUND</title> Accurately assigning ICD-10 codes is critical for clinical documentation and epidemiological studies. Manual coding is time-consuming and prone to errors, underscoring the need for automated solutions in the Norwegian healthcare system. </sec> <sec> <title>OBJECTIVE</title> This study introduces NorDeClin-BERT, a Norwegian BERT-based model continuously pre-trained on a large corpus of Norwegian clinical text to predict ICD-10 diagnosis coding. </sec> <sec> <title>METHODS</title> NorDeClin-BERT was trained on the ClinCode Gastro Corpus, consisting of 8.8 million deidentified Norwegian clinical notes. The model’s performance was benchmarked against SweDeClin-BERT, SweClin-BERT, ScandiBERT, NorBERT3-base, and NorBERT3-large using accuracy, precision, recall, and F1-score metrics. </sec> <sec> <title>RESULTS</title> The results show that NorDeClin-BERT outperformed other similar-sized general BERT models and achieved equivalent performance with the larger NorBERT3-large model in classifying both prevalent and less common ICD-10 codes. The pretraining data, language specificity, and model architecture of NorDeClin-BERT contributed to its improved classification accuracy. </sec> <sec> <title>CONCLUSIONS</title> This study highlights the potential of NorDeClin-BERT in improving the task of ICD-10 code classification for the gastroenterology domain in Norway, ultimately streamlining clinical documentation and reporting processes. The benchmarking evaluation establishes NorDeClin-BERT as a state-of-the-art model for processing Norwegian clinical text and predicting ICD-10 coding. </sec>