This paper proposes LAPDOG, a framework for personalized dialogue generation that leverages external knowledge through retrieval augmentation. The model consists of a retriever and a generator. The retriever uses persona profiles to fetch relevant stories from a corpus, which are then used to enhance the persona profile. The generator then uses both the dialogue history and the augmented persona to generate personalized responses. The model is trained using a joint framework that optimizes the retriever towards desired metrics like BLEU to improve the quality of generated responses. Experiments on the CONVAI2 dataset with ROCStory as a supplementary data source show that LAPDOG outperforms baselines, demonstrating the effectiveness of retrieval augmentation in personalized dialogue generation. The model is end-to-end trainable and uses non-differentiable metrics to guide the training process. The framework also incorporates candidate augmentation to ensure diversity in retrieval and improve the quality of generated dialogues. The results show that LAPDOG significantly enhances the performance of personalized dialogue generation, particularly in terms of coherence, context richness, and alignment with the persona. The model is open-sourced for further exploration.This paper proposes LAPDOG, a framework for personalized dialogue generation that leverages external knowledge through retrieval augmentation. The model consists of a retriever and a generator. The retriever uses persona profiles to fetch relevant stories from a corpus, which are then used to enhance the persona profile. The generator then uses both the dialogue history and the augmented persona to generate personalized responses. The model is trained using a joint framework that optimizes the retriever towards desired metrics like BLEU to improve the quality of generated responses. Experiments on the CONVAI2 dataset with ROCStory as a supplementary data source show that LAPDOG outperforms baselines, demonstrating the effectiveness of retrieval augmentation in personalized dialogue generation. The model is end-to-end trainable and uses non-differentiable metrics to guide the training process. The framework also incorporates candidate augmentation to ensure diversity in retrieval and improve the quality of generated dialogues. The results show that LAPDOG significantly enhances the performance of personalized dialogue generation, particularly in terms of coherence, context richness, and alignment with the persona. The model is open-sourced for further exploration.