Do explanations make VQA models more predictable to a human?

A rich line of research attempts to make deep neural networks more transparent by generating human-interpretable 'explanations' of their decision process, especially for interactive tasks like Visual Question Answering (VQA). In this work, we analyze if existing explanations indeed make a VQA model -its responses as well as failures -more predictable to a human. Surprisingly, we find that they do not. On the other hand, we find that humanin-the-loop approaches that treat the model as a black-box do.