Interactive Distance Dependent IBP Compound Dirichlet Process

The sparse topic model (sparseTM) is a nonparametric Bayesian model that employs a Spike and Slab prior, which decouples sparsity and smoothness in the topic mixtures. Though powerful, sparseTM considers the sparsity problem only for the topic-word distributions where sparsity-enhanced topic models emerged to extract focused topics and focused terms. However, these models assume that data are exchangeable, which often fails for real-world data where dependencies between features are expected. We present a generalization of the distance-dependent IBP, the interactive distance-dependent Indian buffet process compound Dirichlet process (idd-ICDP), for modeling non-exchangeable text data. To the best of our knowledge, idd-ICDP is the first nonparametric Bayesian model supporting non-exchangeable data applied to sparse topic models. The idd-ICDP allows an interactive framework integrating human experts’ knowledge with potentially an unbounded number of topics and vocabulary words in a corpus. We derive a Markov Chain Monte Carlo sampler combined with the Metropolis-Hastings algorithm and study its performance on benchmark corpora and sentiment analysis data. Experiments demonstrate that accounting for the non-exchangeability nature of real-world data gives better predictive performance and that the interactive strategy offers better high-quality topics.