This paper investigates whether large language models (LLMs) exhibit no-regret behavior in online learning and game-theoretic settings. We analyze the performance of several pre-trained LLMs, including GPT-4, in benchmark decision-making scenarios, using the regret metric, which measures the difference between the accumulated loss of an agent and the best-in-hindsight loss. Our findings show that LLMs often exhibit no-regret behavior in non-stationary online learning settings and in repeated games, where equilibria emerge as long-term interactions. However, we also identify cases where advanced LLMs like GPT-4 fail to be no-regret. To promote no-regret behavior, we propose a novel unsupervised training loss called regret-loss, which does not require labels of optimal actions. We establish statistical and optimization guarantees for regret-loss minimization, showing that it can lead to known no-regret learning algorithms. Our experiments demonstrate the effectiveness of regret-loss, especially in addressing regrettable cases. We also provide theoretical insights into the no-regret behavior of LLMs, based on a hypothetical model of human decision-makers and assumptions about pre-training. Our results suggest that pre-trained LLMs may exhibit similar regret as humans, and that their no-regret behavior can be influenced by factors such as the decision error and the pre-training data distribution. We also show that LLMs may fail to be no-regret in certain adversarial settings, and that their performance can be improved through additional training. Overall, our work provides a deeper understanding of the limits and capabilities of LLMs in decision-making scenarios.This paper investigates whether large language models (LLMs) exhibit no-regret behavior in online learning and game-theoretic settings. We analyze the performance of several pre-trained LLMs, including GPT-4, in benchmark decision-making scenarios, using the regret metric, which measures the difference between the accumulated loss of an agent and the best-in-hindsight loss. Our findings show that LLMs often exhibit no-regret behavior in non-stationary online learning settings and in repeated games, where equilibria emerge as long-term interactions. However, we also identify cases where advanced LLMs like GPT-4 fail to be no-regret. To promote no-regret behavior, we propose a novel unsupervised training loss called regret-loss, which does not require labels of optimal actions. We establish statistical and optimization guarantees for regret-loss minimization, showing that it can lead to known no-regret learning algorithms. Our experiments demonstrate the effectiveness of regret-loss, especially in addressing regrettable cases. We also provide theoretical insights into the no-regret behavior of LLMs, based on a hypothetical model of human decision-makers and assumptions about pre-training. Our results suggest that pre-trained LLMs may exhibit similar regret as humans, and that their no-regret behavior can be influenced by factors such as the decision error and the pre-training data distribution. We also show that LLMs may fail to be no-regret in certain adversarial settings, and that their performance can be improved through additional training. Overall, our work provides a deeper understanding of the limits and capabilities of LLMs in decision-making scenarios.