RESEARCH

ML APPLICATIONS

IR-VIC: Unsupervised Discovery of Sub-goals for Transfer in RL

January 5, 2021

Abstract

We propose a novel framework to identify subgoals useful for exploration in sequential decision making tasks under partial observability. We utilize the variational intrinsic control framework (Gregor et.al., 2016) which maximizes empowerment – the ability to reliably reach a diverse set of states and show how to identify sub-goals as states with high necessary option information through an information theoretic regularizer. Despite being discovered without explicit goal supervision, our subgoals provide better exploration and sample complexity on challenging grid-world navigation tasks compared to supervised counterparts in prior work.

Download the Paper

AUTHORS

Written by

Nirbhay Modhe

Prithvijit Chattopadhyay

Mohit Sharma

Abhishek Das

Devi ParikhDhruv BatraRamakrishna Vedantam

Publisher

International Joint Conference on Artificial Intelligence (IJCAI)

Research Topics

Artificial Intelligence

Human and Machine Learning

Related Publications

Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQA | Facebook AI Research

Many visual scenes contain text that carries crucial information, and it is thus essential to understand text in images for downstream reasoning tasks. For example, a deep water label on a warning sign warns people about the danger in the…

Ronghang Hu, Amanpreet Singh, Trevor Darrell, Marcus Rohrbach

Decoupling Representation and Classifier for Long-Tailed Recognition | Facebook AI Research

The long-tail distribution of the visual world poses great challenges for deep learning based classification models on how to handle the class imbalance problem.…

Bingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, Yannis Kalantidis

From Paris to Berlin: Discovering Fashion Style Influences Around the World | Facebook AI Research

The evolution of clothing styles and their migration across the world is intriguing, yet difficult to describe quantitatively.

Ziad Al-Halah, Kristen Grauman

Permutation Equivariant Models for Compositional Generalization in Language | Facebook AI Research

Humans understand novel sentences by composing meanings and roles of core language components. In contrast, neural network models for natural language modeling fail when such compositional generalization is required. The main contribution of…

Jonathan Gordon, David Lopez-Paz, Marco Baroni, Diane Bouchacourt

Help Us Pioneer The Future of AI

We share our open source frameworks, tools, libraries, and models for everything from research exploration to large-scale production deployment.