1 May 2024 | Max Peepkorn, Tom Kouwenhoven, Dan Brown, and Anna Jordanous
Is Temperature the Creativity Parameter of Large Language Models?
Large language models (LLMs) are used for creative tasks, and their outputs vary from beautiful to pastiche. The temperature parameter controls randomness, leading to more diverse outputs, and is often claimed to be the creativity parameter. This study investigates this claim using a narrative generation task with a fixed context, model, and prompt. It analyzes LLM outputs for four creativity conditions: novelty, typicality, cohesion, and coherence. Results show a weak correlation between temperature and novelty, and a moderate negative correlation with coherence. However, temperature's influence on creativity is nuanced and weak. Overall, higher temperatures slightly increase novelty but do not significantly enhance creativity. The study suggests that temperature alone is not sufficient for controlled creativity, and more research is needed to develop methods for better creative outputs. The paper also discusses the limitations of current approaches and proposes future research directions, including benchmarking creativity, designing advanced decoding strategies, and exploring implicit information in LLMs.Is Temperature the Creativity Parameter of Large Language Models?
Large language models (LLMs) are used for creative tasks, and their outputs vary from beautiful to pastiche. The temperature parameter controls randomness, leading to more diverse outputs, and is often claimed to be the creativity parameter. This study investigates this claim using a narrative generation task with a fixed context, model, and prompt. It analyzes LLM outputs for four creativity conditions: novelty, typicality, cohesion, and coherence. Results show a weak correlation between temperature and novelty, and a moderate negative correlation with coherence. However, temperature's influence on creativity is nuanced and weak. Overall, higher temperatures slightly increase novelty but do not significantly enhance creativity. The study suggests that temperature alone is not sufficient for controlled creativity, and more research is needed to develop methods for better creative outputs. The paper also discusses the limitations of current approaches and proposes future research directions, including benchmarking creativity, designing advanced decoding strategies, and exploring implicit information in LLMs.