Experimental Verification of a Convolutional Neural Network Separation Method in TeV Gamma-Ray Observations by the Tibet ASγ Experiment

Abstract Since 1990, the Tibet AS$\gamma$ experiment has been observing gamma rays and cosmic rays with energies greater than several TeV using a surface air shower array. An underground muon detector (MD) array operating since 2014 enables us to significantly discriminate between gamma rays and cosmic rays by counting the number of muons in the air showers. However, discrimination with only the air shower array is challenging. We developed a convolutional neural network (CNN)-based method to improve the sensitivity of gamma-ray measurement data recorded by only the air shower array. The area-under-the-curve values of the CNN method for gamma rays generated by a Monte Carlo (MC) simulation assuming a gamma-ray source (Crab Nebula) were 0.75 at $\sim$10 TeV and 0.83 at $\sim$100 TeV. The detection significances of gamma rays were improved by factors of 1.232 $\pm$ 0.007 at $\sim$10 TeV and 1.557 $\pm$ 0.022 at $\sim$100 TeV. For verification, we applied the proposed method to experimental data including high-purity gamma-ray-like events in the direction of the Crab Nebula, acquired using both arrays. The distributions of gamma-ray-like properties obtained from the CNN were in good agreement with the MC Simulation, with reduced $\chi ^2$ values of 0.507–1.57, corresponding to an upper cumulative probability of 0.120–0.871.