DeAL is a decoding-time alignment framework for large language models (LLMs) that enables users to customize alignment objectives and improve adherence to them during generation. The framework treats decoding as a heuristic-guided search process, allowing for the use of various alignment objectives, including both programmatically verifiable constraints (e.g., keyword and length constraints) and abstract objectives (e.g., harmlessness and helpfulness). DeAL improves alignment by incorporating alignment objectives into the decoding process, enabling fine-grained trade-offs and better adherence to alignment goals. It can be used in conjunction with reinforcement learning with human feedback (RLHF) and prompting techniques, although it may slow down decoding due to its generality. The paper evaluates DeAL on various tasks, including keyword/concept constrained generation, length-constrained summarization, and abstract alignment objectives. Results show that DeAL improves keyword coverage, length satisfaction, and alignment to abstract objectives like harmlessness and helpfulness. It also demonstrates effectiveness in security scenarios, where traditional prompting approaches can be easily bypassed, while DeAL provides stronger alignment enforcement. DeAL allows users to combine multiple alignment objectives and calibrate them using parametric reward models. It can be used with different reward models to achieve desired levels of harmlessness and helpfulness. The framework is flexible and can be applied to various alignment goals, making it a valuable tool for improving alignment in LLMs. However, it may not be as efficient as other methods due to the increased decoding time required for alignment. The paper also highlights the importance of using DeAL in security scenarios where alignment is critical, as it provides a more reliable solution compared to traditional prompting approaches.DeAL is a decoding-time alignment framework for large language models (LLMs) that enables users to customize alignment objectives and improve adherence to them during generation. The framework treats decoding as a heuristic-guided search process, allowing for the use of various alignment objectives, including both programmatically verifiable constraints (e.g., keyword and length constraints) and abstract objectives (e.g., harmlessness and helpfulness). DeAL improves alignment by incorporating alignment objectives into the decoding process, enabling fine-grained trade-offs and better adherence to alignment goals. It can be used in conjunction with reinforcement learning with human feedback (RLHF) and prompting techniques, although it may slow down decoding due to its generality. The paper evaluates DeAL on various tasks, including keyword/concept constrained generation, length-constrained summarization, and abstract alignment objectives. Results show that DeAL improves keyword coverage, length satisfaction, and alignment to abstract objectives like harmlessness and helpfulness. It also demonstrates effectiveness in security scenarios, where traditional prompting approaches can be easily bypassed, while DeAL provides stronger alignment enforcement. DeAL allows users to combine multiple alignment objectives and calibrate them using parametric reward models. It can be used with different reward models to achieve desired levels of harmlessness and helpfulness. The framework is flexible and can be applied to various alignment goals, making it a valuable tool for improving alignment in LLMs. However, it may not be as efficient as other methods due to the increased decoding time required for alignment. The paper also highlights the importance of using DeAL in security scenarios where alignment is critical, as it provides a more reliable solution compared to traditional prompting approaches.