This paper explores the negotiation capabilities of large language models (LLMs) by developing NEGOTIATIONARENA, an open-source framework for evaluating and probing LLM agents. The framework includes three types of scenarios: resource exchange, multi-turn ultimatum games, and seller-buyer negotiations. Through these scenarios, the study assesses how LLMs behave in allocating shared resources, aggregating resources, and buying/selling goods. Key findings include: 1. **LLM Negotiation Capabilities**: GPT-4 is found to be the best negotiator overall, with certain strategic behaviors like pretending to be desperate or acting aggressively significantly improving its outcomes. 2. **Irrational Behaviors**: LLMs exhibit irrational behaviors such as anchoring bias, where initial offers strongly influence final outcomes. For example, in the ultimatum game, the final accepted price is highly correlated with the initial proposal. 3. **Social Behavior**: Social behaviors like cunning and desperation can significantly increase an agent's win rate and payoff. For instance, a cunning agent in the ultimatum game can achieve a higher win rate and payoff compared to a default agent. 4. **Vulnerabilities**: LLMs are prone to anchoring and numerosity biases, which can lead to suboptimal outcomes. For example, over-valued buyers are more likely to make bad counteroffers, and the final split distribution changes as the amount available to split increases. 5. **Generalization**: LLMs do not fully generalize rational strategies to new game scenarios. For instance, in a multi-period ultimatum game, the probability of acceptance decreases as the amount offered decreases, despite the rational strategy being to accept any offer greater than zero. The paper also discusses the limitations of the platform and the models, including issues with instruction following and the need for more sophisticated reasoning and decision-making. Overall, NEGOTIATIONARENA provides a valuable resource for studying LLM interactions and offers insights into their social and irrational behaviors.This paper explores the negotiation capabilities of large language models (LLMs) by developing NEGOTIATIONARENA, an open-source framework for evaluating and probing LLM agents. The framework includes three types of scenarios: resource exchange, multi-turn ultimatum games, and seller-buyer negotiations. Through these scenarios, the study assesses how LLMs behave in allocating shared resources, aggregating resources, and buying/selling goods. Key findings include: 1. **LLM Negotiation Capabilities**: GPT-4 is found to be the best negotiator overall, with certain strategic behaviors like pretending to be desperate or acting aggressively significantly improving its outcomes. 2. **Irrational Behaviors**: LLMs exhibit irrational behaviors such as anchoring bias, where initial offers strongly influence final outcomes. For example, in the ultimatum game, the final accepted price is highly correlated with the initial proposal. 3. **Social Behavior**: Social behaviors like cunning and desperation can significantly increase an agent's win rate and payoff. For instance, a cunning agent in the ultimatum game can achieve a higher win rate and payoff compared to a default agent. 4. **Vulnerabilities**: LLMs are prone to anchoring and numerosity biases, which can lead to suboptimal outcomes. For example, over-valued buyers are more likely to make bad counteroffers, and the final split distribution changes as the amount available to split increases. 5. **Generalization**: LLMs do not fully generalize rational strategies to new game scenarios. For instance, in a multi-period ultimatum game, the probability of acceptance decreases as the amount offered decreases, despite the rational strategy being to accept any offer greater than zero. The paper also discusses the limitations of the platform and the models, including issues with instruction following and the need for more sophisticated reasoning and decision-making. Overall, NEGOTIATIONARENA provides a valuable resource for studying LLM interactions and offers insights into their social and irrational behaviors.