Dual-Level Bayesian Predictive Modeling of Biological Nitrogen Fixation: The Effect of TPE Bayesian Optimization on Stacked Model Performance

Biological Nitrogen Fixation (BNF) is a key ecological process performed by diazotrophs, whose nitrogenase activity is often strictly regulated by environmental factors such as oxygen levels, metal availability, and carbon sources. Given this physiological complexity, accurately predicting nitrogenase activity from protein sequences remains a challenging problem in computational biology. Existing approaches such as Carmma and NFEmbed rely on features derived from large protein language models, yet their performance is often limited by suboptimal hyperparameter tuning. In this work, we introduce a dual-level predictive framework that integrates Bayesian Optimization using the Tree-structured Parzen Estimator (TPE) to systematically optimize both classification and regression models. For the classification task, our TPE-optimized XGBoost model achieves the highest overall performance, with an AUC of 0.9396, F1-score of 0.8675, accuracy of 0.8182, and recall of 0.9231, outperforming Carmma and matching or exceeding NFEmbed in key metrics. For the regression task, our dual-level TPEoptimized stacked SVR attains an <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathrm{R}^{2}$</tex> of 0.6221 with reduced prediction error (MAE: 0.3044, RMSE: 0.5254), representing a substantial improvement over Carmma's SVR (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathrm{R}^{2} : 0.5572$</tex>) and offering competitive performance relative to NFEmbed. These findings demonstrate that TPE-based hyperparameter optimization—especially when applied at both the base- and meta-learner levels—significantly enhances the predictive reliability of complex machine learning architectures for BNF classification and activity estimation. This establishes systematic Bayesian optimization as a powerful strategy for advancing model accuracy, stability, and generalization in protein-sequence-based bioinformatics.