A Bayesian-weighted Adaptive Ensemble for Early Multi-disease Outbreak Classification from Multi-modal Big Data Streams
Rejoice E. Ogini
Department of Computer and Software Technology, Faculty of Computing, Delta State University, Abraka, Delta State, Nigeria.
Franklin O. Okorodudu *
Department of Computer and Software Technology, Faculty of Computing, Delta State University, Abraka, Delta State, Nigeria.
*Author to whom correspondence should be addressed.
Abstract
Background: Across much of sub-Saharan Africa, disease-surveillance systems remain largely reactive: health authorities are often notified only after an outbreak has already been under way for days or weeks.
Aim: this study develops and evaluates a Bayesian-weighted ensemble classifier that determines, from a 47-dimensional multi-modal feature vector spanning epidemiological, environmental, and digital signals, whether a weekly state–disease record represents an outbreak.
Methods: four complementary learners — Random Forest, a long short-term memory (LSTM) network, XGBoost, and a hybrid combining the autoregressive integrated moving average (ARIMA) model with a learned residual correction — are fused by a Bayesian-weighted adaptive meta-learner whose component weights are estimated by constrained cross-entropy minimisation on a validation set and whose posterior is calibrated by isotonic regression. The model was trained and evaluated on a benchmark of weekly records for five diseases endemic to Nigeria (Lassa fever, cholera, cerebrospinal meningitis, measles, and mpox) across 37 jurisdictions; the benchmark combines real NCDC and WHO AFRO surveillance statistics with a calibrated epidemiological simulation used to achieve the record volume and completeness required for controlled algorithm development.
Results: On a held-out 2022–2023 test partition (N = 279,500), the ensemble attained 96.3% accuracy (95% CI: 96.2–96.4%), an F1-score of 0.939, and an AUC-ROC of 0.963, exceeding a re-implemented single-LSTM baseline on every metric (McNemar and DeLong tests, both P < .001). Probability estimates were well calibrated (Brier score 0.048), and a leave-one-learner-out ablation confirmed that every component contributed positively, with the LSTM contributing the largest marginal gain.
Conclusion: disciplined model diversity combined with probability calibration yields well-behaved, decision-ready outbreak-risk scores; because the evaluation relies in part on simulated data, these figures characterise performance on a controlled benchmark rather than demonstrated real-world effectiveness, and prospective validation on live surveillance feeds is identified as the necessary next step.
Keywords: Bayesian fusion, probability calibration, disease outbreak detection, ensemble learning, long short-term memory, Random Forest, XGBoost