Model order selection for the singular value decomposition and the discrete Karhunen–Loeve transform using a Bayesian approach

Bayesian model order selection is considered in relation to the singular value decomposition (SVD) and the discrete Karhunen–Loève transform (DKLT). There are many applications of the SVD and DKLT where it is necessary to discard some of the small singular values that may represent corrupted signal information. Often this task is performed heuristically or in an ad hoc manner. The Bayesian approach to model order selection involves the determination of the evidence or the conditional posterior probability of the model structure given the data; this framework allows the relative probabilities of all possible candidate models to be compared explicitly. Applied to the SVD, the evidence formulation enables the number of nonzero singular values (and hence the effective rank) of a singular or ill-conditioned matrix to be determined analytically. For the DKLT, the evidence allows the determination of the optimal number of basis vectors to choose for the signal reconstruction. In addition, the Bayesian method allows prior information such as physical smoothness constraints to be incorporated directly into the problem specification. Derivations of the evidence formulae are included along with results that illustrate the usefulness of the method.