This paper introduces a novel approach to probabilistic inference in language models using twisted sequential Monte Carlo (SMC). The method leverages learned twist functions to estimate the expected future value of a potential function at each time step, enabling efficient inference by focusing computation on promising partial sequences. The twist functions are learned using a contrastive method inspired by energy-based modeling and density ratio estimation. The framework also includes novel bidirectional SMC bounds on the log partition function, which can be used to estimate the KL divergence between inference and target distributions in both directions. These bounds are shown to be effective for evaluating the quality of language model inference techniques, including sampling undesirable outputs, generating reviews with varied sentiment, and performing infilling tasks. The paper also discusses connections between the proposed method and soft reinforcement learning, and presents experimental results demonstrating the effectiveness of twisted SMC for controlled generation and inference quality evaluation.This paper introduces a novel approach to probabilistic inference in language models using twisted sequential Monte Carlo (SMC). The method leverages learned twist functions to estimate the expected future value of a potential function at each time step, enabling efficient inference by focusing computation on promising partial sequences. The twist functions are learned using a contrastive method inspired by energy-based modeling and density ratio estimation. The framework also includes novel bidirectional SMC bounds on the log partition function, which can be used to estimate the KL divergence between inference and target distributions in both directions. These bounds are shown to be effective for evaluating the quality of language model inference techniques, including sampling undesirable outputs, generating reviews with varied sentiment, and performing infilling tasks. The paper also discusses connections between the proposed method and soft reinforcement learning, and presents experimental results demonstrating the effectiveness of twisted SMC for controlled generation and inference quality evaluation.