2024 | Marko Sarstedt, Susanne J. Adler, Lea Rau, Bernd Schmitt
This review article explores the use of large language models (LLMs) to generate silicon samples in consumer and marketing research. The authors, Marko Sarstedt, Susanne J. Adler, Lea Rau, and Bernd Schmitt, examine the challenges and opportunities of using silicon samples, which are synthetic datasets designed to mimic human respondents' behavior. They find that results vary across different domains, with LLMs performing well in tasks related to personality traits, framing effects, and political attitudes but struggling with tasks that require more nuanced human responses, such as the endowment effect or sunk cost fallacy.
The authors recommend that silicon samples be used in upstream parts of the research process, such as qualitative pretesting and pilot studies, to gather external information and safeguard follow-up design choices. They also provide guidelines for using silicon samples in main studies, including critically assessing the relevance of training data to the research question, customizing the LLM and optimizing prompts, benchmarking against human samples, justifying and adapting analytical procedures, and optimizing reproducibility and transparency.
Ethical considerations are also discussed, highlighting issues such as privacy, copyright, misinformation, bias in training data, job replacement, and malicious intent. The authors conclude by emphasizing the need for further research to understand the capabilities and limitations of LLMs in consumer and marketing research, and to develop guidelines and methods for silicon sampling. They suggest that future research should focus on identifying areas where silicon samples align well with human samples and actual behavior, and on exploring the potential of LLMs in qualitative data analysis and theory generation.This review article explores the use of large language models (LLMs) to generate silicon samples in consumer and marketing research. The authors, Marko Sarstedt, Susanne J. Adler, Lea Rau, and Bernd Schmitt, examine the challenges and opportunities of using silicon samples, which are synthetic datasets designed to mimic human respondents' behavior. They find that results vary across different domains, with LLMs performing well in tasks related to personality traits, framing effects, and political attitudes but struggling with tasks that require more nuanced human responses, such as the endowment effect or sunk cost fallacy.
The authors recommend that silicon samples be used in upstream parts of the research process, such as qualitative pretesting and pilot studies, to gather external information and safeguard follow-up design choices. They also provide guidelines for using silicon samples in main studies, including critically assessing the relevance of training data to the research question, customizing the LLM and optimizing prompts, benchmarking against human samples, justifying and adapting analytical procedures, and optimizing reproducibility and transparency.
Ethical considerations are also discussed, highlighting issues such as privacy, copyright, misinformation, bias in training data, job replacement, and malicious intent. The authors conclude by emphasizing the need for further research to understand the capabilities and limitations of LLMs in consumer and marketing research, and to develop guidelines and methods for silicon sampling. They suggest that future research should focus on identifying areas where silicon samples align well with human samples and actual behavior, and on exploring the potential of LLMs in qualitative data analysis and theory generation.