Central Moment Discrepancy (CMD) for Domain-Invariant Representation\n Learning

The learning of domain-invariant representations in the context of domain\nadaptation with neural networks is considered. We propose a new regularization\nmethod that minimizes the discrepancy between domain-specific latent feature\nrepresentations directly in the hidden activation space. Although some standard\ndistribution matching approaches exist that can be interpreted as the matching\nof weighted sums of moments, e.g. Maximum Mean Discrepancy (MMD), an explicit\norder-wise matching of higher order moments has not been considered before. We\npropose to match the higher order central moments of probability distributions\nby means of order-wise moment differences. Our model does not require\ncomputationally expensive distance and kernel matrix computations. We utilize\nthe equivalent representation of probability distributions by moment sequences\nto define a new distance function, called Central Moment Discrepancy (CMD). We\nprove that CMD is a metric on the set of probability distributions on a compact\ninterval. We further prove that convergence of probability distributions on\ncompact intervals w.r.t. the new metric implies convergence in distribution of\nthe respective random variables. We test our approach on two different\nbenchmark data sets for object recognition (Office) and sentiment analysis of\nproduct reviews (Amazon reviews). CMD achieves a new state-of-the-art\nperformance on most domain adaptation tasks of Office and outperforms networks\ntrained with MMD, Variational Fair Autoencoders and Domain Adversarial Neural\nNetworks on Amazon reviews. In addition, a post-hoc parameter sensitivity\nanalysis shows that the new approach is stable w.r.t. parameter changes in a\ncertain interval. The source code of the experiments is publicly available.\n