Abhishek Das

Abhishek is a Research Scientist at Facebook AI Research (FAIR). His research focuses on deep learning and its applications in climate change, and in building agents that can see (computer vision), think (reasoning/interpretability), talk (language modeling), and act (reinforcement learning)
He received his Bachelor's in Electrical Engineering from the Indian Institute of Technology Roorkee and his Ph.D. in Computer Science from Georgia Tech. During his PhD, he also spent time working at Facebook AI Research, DeepMind and Tesla Autopilot, and his research was supported by fellowships from Facebook, Adobe, and Snap.

Abhishek's Work

Abhishek's Publications

September 30, 2020

RESEARCH

Embodied Question Answering

We present a new AI task – Embodied Question Answering (EmbodiedQA) – where an agent is spawned at a random location in a 3D environment and asked a question (‘What color is the car?’). In order to answer, the agent must first intelligently navigate to explore the environment, gather necessary…

Abhishek Das, Samyak Datta, Georgia Gkioxari, Stefan Lee, Devi Parikh, Dhruv Batra

September 30, 2020

September 30, 2020

RESEARCH

Neural Modular Control for Embodied Question Answering

We present a modular approach for learning policies for navigation over long planning horizons from language input. Our hierarchical policy operates at multiple timescales, where the higher-level master policy proposes subgoals to be executed by specialized sub-policies. Our choice of subgoals is compositional…

Abhishek Das, Georgia Gkioxari, Stefan Lee, Devi Parikh, Dhruv Batra

September 30, 2020

September 30, 2020

RESEARCH

COMPUTER VISION

Embodied Question Answering in Photorealistic Environments with Point Cloud Perception

To help bridge the gap between internet vision-style problems and the goal of vision for embodied perception we instantiate a large-scale navigation task – Embodied Question Answering [1] in photo-realistic environments (Matterport 3D). We thoroughly study navigation policies that utilize 3D point clouds…

Erik Wijmans, Samyak Datta, Oleksandr Maksymets, Abhishek Das, Georgia Gkioxari, Stefan Lee, Irfan Essa, Devi Parikh, Dhruv Batra

September 30, 2020

September 30, 2020

COMPUTER VISION

NLP

Large-scale Pretraining for Visual Dialog: A Simple State-of-the-Art Baseline

Query expansion is a technique widely used in image search consisting in combining highly ranked images from an original query into an expanded query that is then…

Vishvak Murahari, Dhruv Batra, Devi Parikh, Abhishek Das

September 30, 2020

September 30, 2020

RESEARCH

NLP

Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning

We introduce the first goal-driven training for visual question answering and dialog agents. Specifically, we pose a cooperative ‘image guessing’ game between two agents – Q-BOT and A-BOT– who communicate in natural language dialog so that…

Abhishek Das, Satwik Kottur, José M.F. Moura, Stefan Lee, Dhruv Batra

September 30, 2020

September 30, 2020

RESEARCH

COMPUTER VISION

Improving Generative Visual Dialog by Answering Diverse Questions

Prior work on training generative Visual Dialog models with reinforcement learning (Das et al., 2017b) has explored a Q-BOT-A-BOT image-guessing game and shown that this ‘self-talk’ approach can lead to improved performance at the downstream…

Vishvak Murahari, Prithvijit Chattopadhyay, Dhruv Batra, Devi Parikh, Abhishek Das

September 30, 2020

September 30, 2020

CONVERSATIONAL AI

NLP

Evaluating Visual Conversational Agents via Cooperative Human-AI Games

As AI continues to advance, human-AI teams are inevitable. However, progress in AI is routinely measured in isolation, without a human in the loop. It is crucial to benchmark progress in AI, not just in isolation, but also in terms of how it…

Prithvijit Chattopadhyay, Deshraj Yadav, Viraj Prabhu, Arjun Chandrasekaran, Abhishek Das, Stefan Lee, Dhruv Batra, Devi Parikh

September 30, 2020

September 30, 2020

RESEARCH

COMPUTER VISION

Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization

We propose a technique for producing ‘visual explanations’ for decisions from a large class of Convolutional Neural Network (CNN)-based models, making them more transparent and explainable. Our approach – Gradient-weighted Class Activation…

Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, Dhruv Batra

September 30, 2020

September 30, 2020

RESEARCH

ML APPLICATIONS

IR-VIC: research Discovery of Sub-goals for Transfer in RL

We propose a novel framework to identify subgoals useful for exploration in sequential decision making tasks under partial observability. We utilize the variational intrinsic control…

Nirbhay Modhe, Prithvijit Chattopadhyay, Mohit Sharma, Abhishek Das, Devi Parikh, Dhruv Batra, Ramakrishna Vedantam

September 30, 2020