Network traffic anomaly detection using clustering techniques and performance comparison

Real-time network traffic anomaly detection is crucial for the confidentiality, integrity, and security of network information. Machine learning approaches are widely used to distinguish traffic flow outliers based on different anomalies with unique statistical characteristics. K-means clustering and Gaussian Mixture Model (GMM) are effective clustering techniques with many variations and easy to implement. Fuzzy clustering is more flexible than hard clustering and is practical for intrusion detection because of the natural treatment of data using fuzzy clustering. Fuzzy c-means clustering (FCM) is an iteratively optimal algorithm normally based on the least square method to partition data sets, which has high computational overhead. This paper proposes modifications to the objective function and the distance function that reduce the computational complexity of FCM while keeping clustering accurate. A combination of FCM clustering GMM, and feature transformation methods are proposed and a comparison of the related testing results and clustering methods is presented.