A Game Theory-Based Tool Reveals Strategic Behavior of AI Models

A new study offers a way to examine the decision-making of large language models, such as ChatGPT-type systems, using game theory. The goal is to understand what kinds of strategies these models adopt in interactive situations and what intentions lie behind their responses.

The research expands on the previous FAIRGAME framework, designed for testing AI systems in so-called social dilemmas. A social dilemma refers to a situation where it would be beneficial for an individual to act selfishly in the short term, but the best outcome for everyone arises from cooperation.

Researchers introduce two new game-based experimental environments. The first is a variation of the classic prisoner's dilemma, where a player can either cooperate or betray the other. In this version of the game, the magnitude of rewards is systematically scaled to separately measure how sensitive the language model is to the strength of incentives—in other words, whether it changes its strategy as the stakes increase.

The second environment is a public goods game, where multiple AI agents decide how much of their resources to put into a common pot. The game features dynamic profit distributions and includes all players' past actions, allowing for the exploration of complex interactions between multiple agents.

Through such games, it is possible to assess not only the individual responses produced by AI but also the underlying strategic behavior, such as willingness to cooperate, tendency towards opportunism, or sensitivity to changes in the game's rules. According to the researchers, this is essential when integrating AI into social and economic systems, where safe and predictable operation is important.

Source: Understanding LLM Agent Behaviours via Game Theory: Strategy Recognition, Biases and Multi-Agent Dynamics, ArXiv (AI).