TabPFGen is a novel energy-based generative model that leverages the pre-trained TabPFN, a highly performant transformer designed for in-context discriminative tabular tasks. The model defines a class-conditional energy within the pre-trained TabPFN and employs the stochastic gradient Langevin dynamics (SGLD) algorithm for sample generation. TabPFGen inherits TabPFN's in-context learning capabilities, requiring no additional training or hyperparameter tuning. Experiments on 18 datasets from OpenML-CC18 demonstrate strong results in data augmentation, class balancing, and imputation, outperforming competitive baselines. The method shows significant improvement in downstream model performance and produces samples that closely align with the training data distribution. However, limitations such as input size constraints and focus on numerical datasets restrict its current applicability for large-scale datasets. The authors suggest that advancements in transformer architectures will gradually alleviate these limitations.TabPFGen is a novel energy-based generative model that leverages the pre-trained TabPFN, a highly performant transformer designed for in-context discriminative tabular tasks. The model defines a class-conditional energy within the pre-trained TabPFN and employs the stochastic gradient Langevin dynamics (SGLD) algorithm for sample generation. TabPFGen inherits TabPFN's in-context learning capabilities, requiring no additional training or hyperparameter tuning. Experiments on 18 datasets from OpenML-CC18 demonstrate strong results in data augmentation, class balancing, and imputation, outperforming competitive baselines. The method shows significant improvement in downstream model performance and produces samples that closely align with the training data distribution. However, limitations such as input size constraints and focus on numerical datasets restrict its current applicability for large-scale datasets. The authors suggest that advancements in transformer architectures will gradually alleviate these limitations.