MultiDrop: A Local Rademacher Complexity-Based Regularization for Multitask Models

Multitask learning combines both commons among a group of tasks and each task's individual characters to achieve better performance. However, different tasks may lead to contrary gradient directions or imbalance gradient values disrupting the training process. We propose a new regularization algorithm based on multitask local Rademacher complexity called MultiDrop, dropping some random tasks in optimization. A new regularization function has been derived based on the upper bound of local Rademacher complexity in multitask learning. Experiments on different image classification datasets have been implemented to demonstrate the effectiveness of MultiDrop compared with other regularization algorithms, and links between theoretical analyses and experiments.