CORE MACHINE LEARNING

HUMAN & MACHINE INTELLIGENCE

Using Bifurcations for Diversity in Differentiable Games

July 24, 2021

Abstract

Ridge Rider (RR) is an algorithm for finding diverse solutions to optimization problems by following eigenvectors of the Hessian ("ridges''). RR is designed for conservative gradient systems (i.e. settings involving a single loss function), where it branches at saddles -- the only relevant bifurcation points. We generalize this idea to non-conservative, multi-agent gradient systems by identifying new types of bifurcation points and proposing a method to follow eigenvectors with complex eigenvalues. We give theoretical motivation for our method -- denoted Game Ridge Rider (GRR) -- by leveraging machinery from the field of dynamical systems. Finally, we empirically motivate our method by constructing novel toy problems where we can visualize new phenomena and by finding diverse solutions in the iterated prisoners' dilemma, where existing methods often converge to a single solution mode.

Download the Paper

AUTHORS

Written by

Jonathan Lorraine

Jack Parker-Holder

Paul Vicol

Aldo Pacchiano

Luke Metz

Tal Kachman

Jakob Foerster

Publisher

Beyond First-order methods in ML Systems

Research Topics

Core Machine Learning

Human & Machine Intelligence

Related Publications

August 15, 2019

HUMAN & MACHINE INTELLIGENCE

PHYRE: A New Benchmark for Physical Reasoning | Facebook AI Research

Understanding and reasoning about physics is an important ability of intelligent agents. We develop the PHYRE benchmark for physical reasoning that contains a set of simple classical mechanics puzzles in a 2D physical environment. The benchmark…

Anton Bakhtin, Laurens van der Maaten, Justin Johnson, Laura Gustafson, Ross Girshick

August 15, 2019

July 09, 2018

HUMAN & MACHINE INTELLIGENCE

Continuous Reasoning: Scaling the Impact of Formal Methods | Facebook AI Research

This paper describes work in continuous reasoning, where formal reasoning about a (changing) codebase is done in a fashion which mirrors the iterative, continuous model of software development that is increasingly practiced in industry. We…

Peter O'Hearn

July 09, 2018

July 12, 2018

HUMAN & MACHINE INTELLIGENCE

OASIs: oracle assessment and improvement tool

The oracle problem remains one of the key challenges in software testing, for which little automated support has been developed so far.…

Gunel Jahangirova, David Clark, Mark Harman, Paolo Tonella

July 12, 2018

April 24, 2017

HUMAN & MACHINE INTELLIGENCE

COMPUTER VISION

Episodic Exploration for Deep Deterministic Policies for StarCraft Micro-Management | Facebook AI Research

We consider scenarios from the real-time strategy game StarCraft as benchmarks for reinforcement learning algorithms. We focus on micromanagement, that is, the short-term, low-level control of team members during a battle. We propose several…

Nicolas Usunier, Gabriel Synnaeve, Zeming Lin, Soumith Chintala

April 24, 2017

Help Us Pioneer The Future of AI

We share our open source frameworks, tools, libraries, and models for everything from research exploration to large-scale production deployment.