This paper introduces a novel alignment pipeline called Alignment with Perceived Ambiguity (APA) to enhance the ability of large language models (LLMs) to handle ambiguous queries. The main challenge is that LLMs are not explicitly trained to manage ambiguous utterances, and the degree of ambiguity perceived by the model can vary based on its knowledge. APA addresses these issues by leveraging the model's own assessment of ambiguity, enabling it to explicitly detect and manage ambiguous queries while retaining the ability to answer clear questions. The method involves a four-stage pipeline: initial prediction assessment, perceived ambiguity detection, response construction, and supervised fine-tuning. The APA pipeline first identifies samples that the model currently fails to handle and then uses the model's intrinsic knowledge to self-disambiguate these samples. It measures the information gain from this disambiguation as an implicit measure of the model's perceived ambiguity. Based on this measure, the model generates clarification requests for ambiguous queries. The final training dataset is constructed by combining samples that the model correctly handles with those that require clarification. Experimental results on various question-answering datasets show that APA outperforms existing baselines in handling ambiguous queries while maintaining the model's ability to answer clear questions. The method also demonstrates effectiveness in out-of-distribution scenarios, where the model is not trained on the specific input. Additionally, the paper presents three new datasets to evaluate ambiguity: AmbigTriviaQA, AmbigWebQuestions, and AmbigFreebaseQA. These datasets facilitate a more extensive evaluation of models' robustness in addressing ambiguity, contributing to the further expansion of related research. The results highlight the effectiveness of leveraging perceived ambiguity for alignment, enhancing generalization and robustness.This paper introduces a novel alignment pipeline called Alignment with Perceived Ambiguity (APA) to enhance the ability of large language models (LLMs) to handle ambiguous queries. The main challenge is that LLMs are not explicitly trained to manage ambiguous utterances, and the degree of ambiguity perceived by the model can vary based on its knowledge. APA addresses these issues by leveraging the model's own assessment of ambiguity, enabling it to explicitly detect and manage ambiguous queries while retaining the ability to answer clear questions. The method involves a four-stage pipeline: initial prediction assessment, perceived ambiguity detection, response construction, and supervised fine-tuning. The APA pipeline first identifies samples that the model currently fails to handle and then uses the model's intrinsic knowledge to self-disambiguate these samples. It measures the information gain from this disambiguation as an implicit measure of the model's perceived ambiguity. Based on this measure, the model generates clarification requests for ambiguous queries. The final training dataset is constructed by combining samples that the model correctly handles with those that require clarification. Experimental results on various question-answering datasets show that APA outperforms existing baselines in handling ambiguous queries while maintaining the model's ability to answer clear questions. The method also demonstrates effectiveness in out-of-distribution scenarios, where the model is not trained on the specific input. Additionally, the paper presents three new datasets to evaluate ambiguity: AmbigTriviaQA, AmbigWebQuestions, and AmbigFreebaseQA. These datasets facilitate a more extensive evaluation of models' robustness in addressing ambiguity, contributing to the further expansion of related research. The results highlight the effectiveness of leveraging perceived ambiguity for alignment, enhancing generalization and robustness.