Development of Large Language Model Specialized into Microbiome Datasets: an Application of Self-Evaluation and Scoring Comparison with Conventional Natural Language Processing Markers

The gut microbiome plays a fundamental role in host metabolism, immune regulation, and disease development. With the rapid accumulation of multi-omics and literature data, the microbiome field now faces the challenge of efficiently extracting scientific insights from massive, heterogeneous datasets. Artificial intelligence (AI) and large language models (LLMs) provide promising tools to address this complexity by enabling integrative analysis and knowledge synthesis across diverse biological sources. In this study, we developed METABOLISM, a microbiome-specialized LLM fine-tuned on 160,000 scientific abstracts to enhance literature-based contextual understanding of microbiome-liver interactions and related biological mechanisms. Using LoRA-based parameter-efficient training, METABOLISM was optimized for domain-specific reasoning and response generation. Model performance was evaluated through both automated Phi-4 scoring (a large language model-based evaluator for relevance, informativeness, and fluency) and structured human expert rubric assessments involving 20 domain specialists. The fine-tuned METABOLISM achieved superior relevance and clarity scores (mean > 7.5 ± 0.06) compared with general-purpose LLMs such as Gemma-3-12B-IT and ChatGPT-4o. Correlation analysis revealed weak to moderate negative relationships (R = -0.65, <i>p</i> < 0.0001) between traditional NLP metrics (BLEU, ROUGE) and human expert rubric scores, with a similar trend observed for correlations with Phi-4-based automated evaluation scores, indicating the limitations of surface-level similarity measures in biomedical contexts. Overall, our findings demonstrate that microbiome-adapted LLMs can effectively distill high-volume scientific data into biologically meaningful insights, supporting more efficient and interpretable research in microbiology and systems biology.