Enhancing Gamma–Hadron Separation in Imaging
Atmospheric Cherenkov Telescopes Using Attention-Guided
Deep Learning and Adaptive Balanced Greylag Goose
Optimization
Ebrahim A. Mattar1,*, S. K. Towfek2,3
1College of Engineering University of Bahrain, Bahrain
2Computer Science and Intelligent Systems Research Center, Blacksburg 24060, Virginia, USA
3Jadara Research Center, Jadara University, Irbid 21110, Jordan
Emails: ebmattar@uob.edu.bh; sktowfek@jcsis.org
Abstract
Gamma–hadron discrimination remains a fundamental challenge in very-high-energy gamma-ray
astronomy due to the strong overlap between gamma-ray–initiated and hadron-induced air showers
recorded by imaging atmospheric Cherenkov telescopes, particularly at low energies where background
contamination is severe. Traditional cut-based and non-optimized machine learning approaches often
struggle to fully exploit the nonlinear and correlated nature of Cherenkov image parameters, leading
to suboptimal background suppression and reduced telescope sensitivity. To address these limitations,
this paper proposes a unified deep learning and metaheuristic optimization framework that combines
an enhanced attention-based long short-term memory network (EALSTM) with advanced optimization
strategies. In particular, a novel Adaptive Balanced Greylag Goose Optimization algorithm (ABGGO)
is employed to jointly perform feature selection and hyperparameter optimization, enabling effective
exploration–exploitation balancing while preserving physically meaningful feature representations.
The proposed ABGGO+EALSTM framework is systematically evaluated against baseline deep
learning models, including artificial neural networks (ANN), convolutional neural networks (CNN),
and standard long short-term memory networks (LSTM), under identical experimental conditions.
Experimental results on a Monte Carlo–generated Cherenkov telescope dataset demonstrate clear
and consistent performance gains at every stage of the analysis. In the baseline evaluation stage,
EALSTM achieves an accuracy of 0.9294 and an F-score of 0.9266, outperforming ANN, CNN, and
LSTM. Following metaheuristic optimization, the proposed ABGGO+EALSTM model attains a peak
accuracy of 0.9718, sensitivity of 0.9694, specificity o f 0 .9740, a nd F-score o f 0 .9705, representing
absolute improvements exceeding 4% over the baseline EALSTM configuration and outperforming
GA+EALSTM, GWO+EALSTM, and PSO+EALSTM variants. These results demonstrate that
integrating attention-based deep learning with adaptive metaheuristic optimization significantly
enhances gamma–hadron discrimination, leading to improved background suppression and signal
retention. The proposed framework offers a scalable and robust solution for current and next-generation
Cherenkov observatories, with strong potential for real-time event filtering, multi-telescope analysis,
and future deployment on real observational data.
Keywords: Gamma–hadron discrimination; Imaging atmospheric Cherenkov telescopes;
Attention-based deep learning; Metaheuristic optimization; Greylag Goose Optimization