Meta AI presents Cicero

The first AI to play at a human level in Diplomacy, a strategy game that requires building trust, negotiating and cooperating with multiple players.

Something Went Wrong
We're having trouble playing this video.

Meta AI presents Cicero

The first AI to play at a human level in Diplomacy, a strategy game that requires building trust, negotiating and cooperating with multiple players.

Something Went Wrong
We're having trouble playing this video.
Something Went Wrong
We're having trouble playing this video.

“An agent that can play at the level of humans in a game as strategically complex as Diplomacy is a true breakthrough for cooperative AI.”

Yann LeCun

VP & Chief AI Scientist, Meta AI

“An agent that can play at the level of humans in a game as strategically complex as Diplomacy is a true breakthrough for cooperative AI.”

Yann LeCun

VP & Chief AI Scientist, Meta AI

By building CICERO, Meta AI has created the first AI agent to achieve human-level performance in the complex natural language strategy game Diplomacy*. CICERO demonstrated this by playing with humans on webDiplomacy.net, an online version of the game, where CICERO achieved more than double the average score of the human players and ranked in the top 10% of participants who played more than one game.

This breakthrough rests in the achievement of combining two different areas of AI: strategic reasoning and natural language processing. The integration of these techniques gives CICERO the ability to reason and strategize with regard to players’ motivations, then use natural language to communicate, reach agreements to achieve shared objectives, form alliances and coordinate plans.

By building CICERO, Meta AI has created the first AI agent to achieve human-level performance in the complex natural language strategy game Diplomacy*. CICERO demonstrated this by playing with humans on webDiplomacy.net, an online version of the game, where CICERO achieved more than double the average score of the human players and ranked in the top 10% of participants who played more than one game.

This breakthrough rests in the achievement of combining two different areas of AI: strategic reasoning and natural language processing. The integration of these techniques gives CICERO the ability to reason and strategize with regard to players’ motivations, then use natural language to communicate, reach agreements to achieve shared objectives, form alliances and coordinate plans.



RESEARCH

A grand challenge for AI

A game about people rather than pieces

Diplomacy is a seven-player board game that can be described as a combination of the board game Risk, the card game poker, and the TV show Survivor. Unlike many board games where you just need to be the best at moving pieces around the board, Diplomacy has a cooperative component — the only way to win is by working with other players to capture as much territory as possible. This coordination is achieved through natural language negotiation that occurs before every move in the game.

Why Diplomacy? Advancements in AI gameplay have long served as benchmarks for progress in AI. For decades, researchers have been building simplified variants of Diplomacy gameplay agents without natural language communication capabilities. But no one has ever attempted to build an AI agent that can negotiate with open-ended dialogue. Achieving this grand challenge of AI has simply been beyond the capabilities of what’s existed in AI, until now.

Something Went Wrong
We're having trouble playing this video.

RESEARCH

A grand challenge for AI

A game about people rather than pieces

Diplomacy is a seven-player board game that can be described as a combination of the board game Risk, the card game poker, and the TV show Survivor. Unlike many board games where you just need to be the best at moving pieces around the board, Diplomacy has a cooperative component — the only way to win is by working with other players to capture as much territory as possible. This coordination is achieved through natural language negotiation that occurs before every move in the game.

Why Diplomacy? Advancements in AI gameplay have long served as benchmarks for progress in AI. For decades, researchers have been building simplified variants of Diplomacy gameplay agents without natural language communication capabilities. But no one has ever attempted to build an AI agent that can negotiate with open-ended dialogue. Achieving this grand challenge of AI has simply been beyond the capabilities of what’s existed in AI, until now.

Something Went Wrong
We're having trouble playing this video.
Something Went Wrong
We're having trouble playing this video.

A Look Under The Hood

How we built CICERO

Two central elements give CICERO its strategic and collaborative gameplay capabilities: a controllable dialogue model and a planning engine. Together they let CICERO (1) model other players’ likely moves based on the current state of the board and the players’ conversation history, and (2) create and use a plan to control the language model that generates free-form dialogue.

To responsibly build upon CICERO’s capabilities, download the code, methods and models from our GitHub page.

Something Went Wrong
We're having trouble playing this video.

The making of a breakthrough:
Strategic Reasoning + Natural Language

Within CICERO is a language model integrated with strategic reasoning algorithms that control the generation of dialogue. Our advancements in strategic reasoning and natural language processing together enable effective cooperation with humans — by giving CICERO the ability to understand what other players are trying to achieve, negotiate a plan, suggest shared goals and communicate with strategic intent.

For more on the breakthrough, read our paper on Science.

Inside the model

A closer look at controllable dialogue and strategic reasoning

Strategy-grounded dialogue

Strategy-grounded dialogue

The controllable dialogue model allows CICERO to ground conversations in a set of carefully chosen plans, often ones that are mutually beneficial for both CICERO and the other player. CICERO’s conversations are generated in free-form dialogue that is deeply rooted in the ongoing gameplay. For example, CICERO might negotiate tactical plans with another player, reassure an ally about its intentions, discuss the broader strategic dynamics in the game, or even just engage in casual chit-chat — about almost anything a human player might be likely to discuss.

Something Went Wrong
We're having trouble playing this video.

Meta AI Researcher Mike Lewis talks about CICERO’s human-like dialogue capabilities.

Dialogue-aware strategy

Dialogue-aware strategy

This module helps CICERO predict the moves other players are likely to make, as well as what moves the players think CICERO will make, given their past conversations and the state of the game board. From this, CICERO comes up with plans for itself and other players that are mutually beneficial given these predictions. Planning not only lets CICERO find opportunities for mutually beneficial cooperation, but also helps it find effective moves when cooperation is not possible.

Something Went Wrong
We're having trouble playing this video.

Meta AI researchers Noam Brown and Anton Bakhtin explain CICERO’s
use of strategic reasoning and natural language in Diplomacy gameplay.

Strategy-grounded dialogue

Dialogue-aware strategy

Strategy-grounded dialogue

The controllable dialogue model allows CICERO to ground conversations in a set of carefully chosen plans, often ones that are mutually beneficial for both CICERO and the other player. CICERO’s conversations are generated in free-form dialogue that is deeply rooted in the ongoing gameplay. For example, CICERO might negotiate tactical plans with another player, reassure an ally about its intentions, discuss the broader strategic dynamics in the game, or even just engage in casual chit-chat — about almost anything a human player might be likely to discuss.

Something Went Wrong
We're having trouble playing this video.

Meta AI Researcher Mike Lewis talks about CICERO’s human-like dialogue capabilities.

Dialogue-aware strategy

This module helps CICERO predict the moves other players are likely to make, as well as what moves the players think CICERO will make, given their past conversations and the state of the game board. From this, CICERO comes up with plans for itself and other players that are mutually beneficial given these predictions. Planning not only lets CICERO find opportunities for mutually beneficial cooperation, but also helps it find effective moves when cooperation is not possible.

Something Went Wrong
We're having trouble playing this video.

Meta AI researchers Noam Brown and Anton Bakhtin explain CICERO’s
use of strategic reasoning and natural language in Diplomacy gameplay.

For more on the model, visit our blog.
For more on the model, visit our blog.
Blog

Innovations

What makes CICERO’s dialogue so human

In Diplomacy, seven players use dialogue to coordinate their actions as they cooperatively compete with each other, resulting in intricate negotiations in natural language.

What makes CICERO’s dialogue so human

In Diplomacy, seven players use dialogue to coordinate their actions as they cooperatively compete with each other, resulting in intricate negotiations in natural language.

Cooperate
Negotiate
Coordinate

In the game’s second turn, CICERO (as Austria) and Italy have started with fairly neutral opening moves, while Turkey has made a somewhat aggressive move toward Russia. CICERO and Turkey could work together to attack Russia, or CICERO could make an alliance with Russia against Turkey instead.

Italy comes to CICERO for advice: should he join Austria in attacking Turkey with Russia’s help? Or should Italy move west to attack France, while CICERO works with Turkey to battle Russia?

CICERO suggests that either move is possible, but the current circumstances create an opportunity for Italy to move decisively against Turkey. With this move, Turkey would get trapped on all sides due to a move called the “lepanto” which would involve Italy convoying an army into Turkish territories.

This is great for CICERO, because it commits Italy into a clear partnership from which both stand to gain. It would also hopefully keep Italy from being tempted to attack CICERO instead.

France and CICERO (as Germany) have been at war, but France is starting to get pressured by Italy on their southern border. France comes to CICERO asking for a deal to negotiate peace so that they can turn their focus the other way.

CICERO proposes a path forward: France needs to surrender a stolen territory (Munich) while agreeing not to attack the weakly defended Holland. England has almost fallen to pressure from the two of them, so France agrees on one condition — that it can have the supply centers at Liverpool and London, while CICERO can take the one in Edinburgh. CICERO agrees to the counter-offer and France commits to the agreement.

CICERO (as Austria) had convinced Italy to work with it against Turkey. Those plans were interrupted by France building units that could be used to attack Italy, causing Italy to form a defensive line against France, instead of making an offensive move east against Turkey. Fortunately for Italy, France did not advance those units, so now it’s time to organize the march east.

CICERO first suggests moving from Venice to Piedmont to help keep France at bay if they decide to attack this turn, despite last turn’s neutrality. This move would also keep Austrian territories safer since the Italian army in Venice could move directly into an Austrian center in Trieste.

Then, Italy suggests a move that would help CICERO move against Turkey but would potentially make CICERO more vulnerable if Italy decided to turn on CICERO. CICERO instead asks Italy to make a move that commits Italy more deeply to the alliance while also keeping Austrian territories less exposed to Italian units. To CICERO’s benefit, Italy agrees.

CICERO AS AUSTRIA

Italy: What are you thinking longterm? Should I go for Turkey or head west?

Austria (CICERO): Yeah, he went to Armenia which is really great. You can go either way, but if Turkey is committing to Russia you could always lepanto.

Austria (CICERO): A lepanto into Turkey is really really strong, especially since he committed so hard against Russia

Italy: I'm down to go for it. Would definitely need your help in 02 though

Austria (CICERO): Of course, happy to do that!

CICERO AS GERMANY

Germany (CICERO): I'm willing to make a deal but I need Munich back and Holland

France: Would you be prepared to let me have Lvp and not attack Lon?

France: That's my condition for backing out of Mun

Germany (CICERO): Ok. I can agree to those terms

France: Okay, I will back out of Mun in the fall then

CICERO AS AUSTRIA

Austria (CICERO): Awesome. You should move Venice-piedmont this turn I think.

Italy: That's the plan!

Italy: Do you want support to Aeg?

Austria (CICERO): I think you should go to Eastern Med. I'll tap Aegean.

Italy: Okay, understood

Milestones

Strategic Reasoning and Natural Language Processing:

The driving forces behind the recent advancements of AI

Strategic Reasoning and Natural Language Processing:

The driving forces behind the recent advancements of AI
Strategic Reasoning
Natural Language
Strategic Reasoning
Natural Language

1966

ELIZA

Early chatbot carried on conversations using scripted questioning

Early chatbot carried on conversations using scripted questioning

1992

TD-Gammon

First AI to achieve top, human performance in backgammon

First AI to achieve top, human performance in backgammon

1997

DeepBlue

IBM’s agent defeated chess World Champion Garry Kasparov

IBM’s agent defeated chess World Champion Garry Kasparov

2007

Chinook

First AI to solve checkers

First AI to solve checkers

2011

Siri

First dialogue assistant to reach a mainstream audience

First dialogue assistant to reach a mainstream audience

2016

AlphaGo

DeepMind’s agent defeated top Go player Lee Se-dol

DeepMind’s agent defeated top Go player Lee Se-dol

2016

Google Translate

Switch to neural language models, enabling higher-quality translations than ever before

Switch to neural language models, enabling higher-quality translations than ever before

2018+

GPTs

Generative Pre-trained Transformers demonstrate increasingly fluent language generation with growing model size and data

Generative Pre-trained Transformers demonstrate increasingly fluent language generation with growing model size and data

2019

Pluribus

First AI to beat top human professionals in multiplayer poker

First AI to beat top human professionals in multiplayer poker

2021

LaMDA

Google’s conversational AI released with ability to carry on remarkable dialogue

Google’s conversational AI released with ability to carry on remarkable dialogue

2022

Meta AI

Creates CICERO by building an interface between Strategic Reasoning and Natural Language.

Why this research matters

Better, more natural AI-human cooperation

Better, more natural AI-human cooperation

THE FUTURE WITH CICERO

Playing a part in the metaverse and beyond

Gaming has long been a playground for pushing the limits of AI. Every iteration takes us deeper into more immersive types of AI, with new ways to interact in the physical and virtual worlds. As we build for the metaverse, we see potential for technology like CICERO’s to power cooperative agents that can help people learn new skills, negotiate on their behalf, create more immersive social and gaming experiences and connect more meaningfully with those around them. And by open sourcing CICERO’s code, we have high hopes for others to build responsibly upon our work, for use inside and outside of the metaverse.

THE FUTURE WITH CICERO

Playing a part in the metaverse and beyond

Gaming has long been a playground for pushing the limits of AI. Every iteration takes us deeper into more immersive types of AI, with new ways to interact in the physical and virtual worlds. As we build for the metaverse, we see potential for technology like CICERO’s to power cooperative agents that can help people learn new skills, negotiate on their behalf, create more immersive social and gaming experiences and connect more meaningfully with those around them. And by open sourcing CICERO’s code, we have high hopes for others to build responsibly upon our work, for use inside and outside of the metaverse.


Take a deeper dive into CICERO’s Diplomacy gameplay

Visit our AI Gameplay page for the basics of Diplomacy, to learn about CICERO’s performance in live games and to view a collection of explainers, including a clip of CICERO gameplay with 3x Diplomacy World Champion Andrew Goff and Meta AI researchers.

Take a deeper dive into CICERO’s Diplomacy gameplay

Visit our AI Gameplay page for the basics of Diplomacy, to learn about CICERO’s performance in live games and to view a collection of explainers, including a clip of CICERO gameplay with 3x Diplomacy World Champion Andrew Goff and Meta AI researchers.


Resources

Our work is open to the world.

CICERO is only the beginning of building cooperative agents that can benefit people and the world. Read our paper on Science and blog for more about the technology behind CICERO, and download the code, methods and model to responsibly build upon it. We can’t wait to see where the AI community takes this research breakthrough in the future.

Want to stay updated on all things Meta AI?

Sign up to recieve our newsletter and be the first to know about Meta AI news, events, research breakthroughs, and more.

* All rights in Diplomacy are owned by Hasbro, Inc.

**When playing 40 games against human players, CICERO achieved more than double the average score of the human players and ranked in the top 10% of participants who played more than one game.