March 29, 2023
We present Adaptive Skill Coordination (ASC) – an approach for accomplishing long-horizon tasks (e.g., mobile pick-and-place, consisting of navigating to an object, picking it, navigating to another location, placing it, repeating). ASC consists of three components – (1) a library of basic visuomotor skills (navigation, pick, place), (2) a skill coordination policy that chooses which skills are appropriate to use when, and (3) a corrective policy that adapts pre-trained skills when out-of-distribution states are perceived. All components of ASC rely only on onboard visual and proprioceptive sensing, without access to privileged information like pre-built maps or precise object locations, easing real-world deployment. We train ASC in simulated indoor environments, and deploy it zero-shot in two novel real-world environments on the Boston Dynamics Spot robot. ASC achieves near-perfect performance at mobile pick-and-place, succeeding in 59/60 (98%) episodes, while sequentially executing skills succeeds in only 44/60 (73%) episodes. It is robust to hand-off errors, changes in the environment layout, dynamic obstacles (e.g. people), and unexpected disturbances, making it an ideal framework for complex, long-horizon tasks. Supplementary videos available at adaptiveskillcoordination.github.io
Publisher
Meta AI papers
Research Topics
Robotics
April 02, 2024
Patrick Lancaster, Nicklas Hansen, Aravind Rajeswaran, Vikash Kumar
April 02, 2024
March 26, 2024
Prajjwal Bhargava, Rohan Chitnis, Alborz Geramifard, Shagun Sodhani, Amy Zhang
March 26, 2024
March 12, 2024
Xavi Puig, Eric Undersander, Andrew Szot, Mikael Dallaire Cote, Jimmy Yang, Ruslan Partsey, Ruta Desai, Alexander William Clegg, Tiffany Min, Vladimír Vondruš, Theo Gervet, Vincent-Pierre Berges, Oleksandr Maksymets, Zsolt Kira, Mrinal Kalakrishnan, Jitendra Malik, Devendra Singh Chaplot, Unnat Jain, Dhruv Batra, Akshara Rai, Roozbeh Mottaghi
March 12, 2024
December 10, 2023
Qiyang Li, Jason Zhang, Dibya Ghosh, Amy Zhang, Sergey Levine
December 10, 2023
Product experiences
Foundational models
Product experiences
Latest news
Foundational models