The paper introduces a novel auction mechanism for aggregating preferences over outputs from Large Language Models (LLMs) in the context of online advertising. The mechanism aims to balance the interests of advertisers and the platform while ensuring user satisfaction. Key contributions include: 1. **Mechanism Design**: The mechanism is designed to be strategyproof, meaning that truthful reporting is a dominant strategy for advertisers. It also ensures that each advertiser's utility aligns with their contribution to social welfare, promoting long-term market viability. 2. **Convergence and Efficiency**: The mechanism converges to the output of an optimally fine-tuned LLM as computational resources increase, without requiring fine-tuning or access to model weights. It uses post-processing of multiple LLM outputs, making it computationally efficient. 3. **Contextual Information**: The mechanism incorporates contextual information about advertisers, which significantly accelerates convergence and increases value for both advertisers and the platform. 4. **Experiments**: Experiments with a publicly available LLM demonstrate that the mechanism significantly boosts advertiser value and platform revenue with low computational overhead. It maintains equity while ensuring positive advertiser utility. The paper also discusses the limitations of existing auction mechanisms and how their proposed mechanism addresses these issues, making it a general-purpose solution for truthfully aggregating the preferences of self-interested agents over LLM-generated replies.The paper introduces a novel auction mechanism for aggregating preferences over outputs from Large Language Models (LLMs) in the context of online advertising. The mechanism aims to balance the interests of advertisers and the platform while ensuring user satisfaction. Key contributions include: 1. **Mechanism Design**: The mechanism is designed to be strategyproof, meaning that truthful reporting is a dominant strategy for advertisers. It also ensures that each advertiser's utility aligns with their contribution to social welfare, promoting long-term market viability. 2. **Convergence and Efficiency**: The mechanism converges to the output of an optimally fine-tuned LLM as computational resources increase, without requiring fine-tuning or access to model weights. It uses post-processing of multiple LLM outputs, making it computationally efficient. 3. **Contextual Information**: The mechanism incorporates contextual information about advertisers, which significantly accelerates convergence and increases value for both advertisers and the platform. 4. **Experiments**: Experiments with a publicly available LLM demonstrate that the mechanism significantly boosts advertiser value and platform revenue with low computational overhead. It maintains equity while ensuring positive advertiser utility. The paper also discusses the limitations of existing auction mechanisms and how their proposed mechanism addresses these issues, making it a general-purpose solution for truthfully aggregating the preferences of self-interested agents over LLM-generated replies.