This paper introduces a framework that allows equilibrium solvers to work over the space of natural language dialogue generated by large language models (LLMs). By modeling the players, strategies, and payoffs in a "game" of dialogue, the framework creates a binding between natural language interactions and the symbolic logic of game theory. This enables existing game-theoretic algorithms to provide strategic solutions, such as what string an LLM should generate to maximize payoff in the face of strategic partners or opponents. The framework is evaluated across three domains: scheduling meetings, trading fruit, and debate. Results show that LLMs guided by game-theoretic solvers generate less exploitable dialogue and achieve higher rewards in all negotiation domains. The paper discusses the implications of this work, suggesting that game-theoretic solvers that leverage the expressivity of natural language can open up new avenues for guiding language research. The framework maps natural language dialogue tasks to the formal framing of extensive-form games, allowing off-the-shelf equilibrium solvers to find optimal actions for agents. The paper also presents experimental results showing that LLMs guided by game-theoretic solvers generate more strategic responses and achieve higher payoffs. The framework is evaluated using three dialogue domains, and the results demonstrate that game-theoretic solvers can improve LLM outputs. The paper also discusses the limitations of the framework, including the computational cost of dialogue game transitions and the assumptions made about player payoffs and action spaces. The paper concludes that the framework has the potential to guide language models with optimal strategies found by game-theoretic models, paving the way for more intelligent language model agents. The paper also discusses the ethical implications of strategic dialogue agents, noting that seemingly benign behavior can lead to poor outcomes for the group. The paper highlights the importance of forecasting the equilibria of large techno-societal changes before they happen.This paper introduces a framework that allows equilibrium solvers to work over the space of natural language dialogue generated by large language models (LLMs). By modeling the players, strategies, and payoffs in a "game" of dialogue, the framework creates a binding between natural language interactions and the symbolic logic of game theory. This enables existing game-theoretic algorithms to provide strategic solutions, such as what string an LLM should generate to maximize payoff in the face of strategic partners or opponents. The framework is evaluated across three domains: scheduling meetings, trading fruit, and debate. Results show that LLMs guided by game-theoretic solvers generate less exploitable dialogue and achieve higher rewards in all negotiation domains. The paper discusses the implications of this work, suggesting that game-theoretic solvers that leverage the expressivity of natural language can open up new avenues for guiding language research. The framework maps natural language dialogue tasks to the formal framing of extensive-form games, allowing off-the-shelf equilibrium solvers to find optimal actions for agents. The paper also presents experimental results showing that LLMs guided by game-theoretic solvers generate more strategic responses and achieve higher payoffs. The framework is evaluated using three dialogue domains, and the results demonstrate that game-theoretic solvers can improve LLM outputs. The paper also discusses the limitations of the framework, including the computational cost of dialogue game transitions and the assumptions made about player payoffs and action spaces. The paper concludes that the framework has the potential to guide language models with optimal strategies found by game-theoretic models, paving the way for more intelligent language model agents. The paper also discusses the ethical implications of strategic dialogue agents, noting that seemingly benign behavior can lead to poor outcomes for the group. The paper highlights the importance of forecasting the equilibria of large techno-societal changes before they happen.