CARD: A Congestion-Aware Request Dispatching Scheme for Replicated Metadata Server Cluster

Replicated metadata server cluster (RMSC) is highly efficient to be used in distributed filesystems while facing data-driven scenarios (e.g., massive-scale distributed machine learning tasks). Yet, when considering cost-effectiveness and system utilization, the cluster scale is commonly restricted in practice. Within this context, servers in the cluster start to suffer from load-oscillations at higher system utilization due to clients’ congestion-unaware behaviors and unintelligent selection strategies (i.e., servers in the cluster are preferred then evaded intermittently). The consequences brought by load-oscillations degrade the overall performance of the whole system to some extent. One solution to tackle this problem is having clients share a part of the responsibility and behave more wisely for stability concerns. So in this paper, we present a Congestion-Aware Request Dispatching scheme, CARD, which is mainly conducted at clients and directed by a rate control mechanism. Through extensive experiments, we verify that CARD is highly efficient in resolving load-oscillations in RMSC. Apart from this, our results show that RMSC with our congestion-aware based optimization achieves better scalability compared to previous implementations under targeted workloads, especially in heterogeneous environments.