**Abstract:** This paper introduces Conformal Alignment, a framework for ensuring that outputs from foundation models align with human values in high-stakes tasks. The framework guarantees that a prescribed fraction of selected outputs meet a specified alignment criterion, regardless of the model or data distribution. It leverages reference data with known alignment status to train an alignment predictor and selects new outputs whose predicted alignment scores exceed a data-dependent threshold. Applications to question answering and radiology report generation demonstrate the method's ability to accurately identify trustworthy outputs with lightweight training over moderate reference data. **Introduction:** Large-scale foundation models are powerful but prone to errors, hallucinations, and bias, raising concerns about their reliable use in critical scenarios. Conformal Prediction (CP) offers a distribution-free solution for uncertainty quantification, but its application to ensure alignment remains unexplored. This paper introduces Conformal Alignment, which certifies the alignment of model outputs, ensuring they are mostly correct. Unlike standard CP, which constructs prediction sets, Conformal Alignment selects outputs based on quantified confidence. **Problem Setup:** Given a pre-trained foundation model and a set of reference data, Conformal Alignment trains a predictor to predict alignment scores. It then selects outputs whose predicted scores exceed a data-driven threshold, ensuring a controlled False Discovery Rate (FDR) while maximizing power. **Experiments:** The paper evaluates Conformal Alignment on question answering and radiology report generation tasks. In question answering, it strictly controls FDR and outperforms heuristic baselines. In radiology report generation, it demonstrates tight FDR control and satisfactory power with a small reference dataset. **Discussion:** Conformal Alignment provides principled, distribution-free guarantees for aligning foundation model outputs with human values. It is flexible, lightweight, and effective, offering practical recommendations for sample size, data splitting, and feature engineering. Future work includes extending the framework to other applications and controlling other error notions.**Abstract:** This paper introduces Conformal Alignment, a framework for ensuring that outputs from foundation models align with human values in high-stakes tasks. The framework guarantees that a prescribed fraction of selected outputs meet a specified alignment criterion, regardless of the model or data distribution. It leverages reference data with known alignment status to train an alignment predictor and selects new outputs whose predicted alignment scores exceed a data-dependent threshold. Applications to question answering and radiology report generation demonstrate the method's ability to accurately identify trustworthy outputs with lightweight training over moderate reference data. **Introduction:** Large-scale foundation models are powerful but prone to errors, hallucinations, and bias, raising concerns about their reliable use in critical scenarios. Conformal Prediction (CP) offers a distribution-free solution for uncertainty quantification, but its application to ensure alignment remains unexplored. This paper introduces Conformal Alignment, which certifies the alignment of model outputs, ensuring they are mostly correct. Unlike standard CP, which constructs prediction sets, Conformal Alignment selects outputs based on quantified confidence. **Problem Setup:** Given a pre-trained foundation model and a set of reference data, Conformal Alignment trains a predictor to predict alignment scores. It then selects outputs whose predicted scores exceed a data-driven threshold, ensuring a controlled False Discovery Rate (FDR) while maximizing power. **Experiments:** The paper evaluates Conformal Alignment on question answering and radiology report generation tasks. In question answering, it strictly controls FDR and outperforms heuristic baselines. In radiology report generation, it demonstrates tight FDR control and satisfactory power with a small reference dataset. **Discussion:** Conformal Alignment provides principled, distribution-free guarantees for aligning foundation model outputs with human values. It is flexible, lightweight, and effective, offering practical recommendations for sample size, data splitting, and feature engineering. Future work includes extending the framework to other applications and controlling other error notions.