The paper explores the integration of Generative Artificial Intelligence (GenAI), particularly Large Language Models (LLMs), into educational settings through conversational user interfaces (CUIs). While CUIs offer valuable opportunities for educators, they also present challenges such as the need for AI literacy, ethical concerns, and limitations in handling complex tasks. To address these issues, the authors propose a framework for pedagogically sound and ethically responsible integration of GenAI into educational tools, emphasizing human-centered design. The framework consists of two main components: application design and user interaction design. The application design component guides the selection of educational tasks, pedagogical frameworks, evaluation criteria, and GenAI models. The user interaction design component focuses on creating user interfaces and workflows that facilitate effective interaction with GenAI, including prompt preview, evaluation criteria selection, and output preview with evaluation results. The authors then demonstrate the application of this framework through the development of a tool called Feedback Copilot, which enables instructors to provide personalized qualitative feedback on students' assignments. An evaluation involving 338 students shows that the more advanced variation of Feedback Copilot, which includes additional prompting and guidance, produces feedback of higher quality compared to the base version. The study also finds a correlation between lower assignment grades and lower-quality feedback, highlighting the importance of educator oversight in feedback generation. The paper concludes by discussing the implications of the proposed framework for researchers, educators, and educational technologists, emphasizing the potential of GenAI to enhance teaching and learning experiences.The paper explores the integration of Generative Artificial Intelligence (GenAI), particularly Large Language Models (LLMs), into educational settings through conversational user interfaces (CUIs). While CUIs offer valuable opportunities for educators, they also present challenges such as the need for AI literacy, ethical concerns, and limitations in handling complex tasks. To address these issues, the authors propose a framework for pedagogically sound and ethically responsible integration of GenAI into educational tools, emphasizing human-centered design. The framework consists of two main components: application design and user interaction design. The application design component guides the selection of educational tasks, pedagogical frameworks, evaluation criteria, and GenAI models. The user interaction design component focuses on creating user interfaces and workflows that facilitate effective interaction with GenAI, including prompt preview, evaluation criteria selection, and output preview with evaluation results. The authors then demonstrate the application of this framework through the development of a tool called Feedback Copilot, which enables instructors to provide personalized qualitative feedback on students' assignments. An evaluation involving 338 students shows that the more advanced variation of Feedback Copilot, which includes additional prompting and guidance, produces feedback of higher quality compared to the base version. The study also finds a correlation between lower assignment grades and lower-quality feedback, highlighting the importance of educator oversight in feedback generation. The paper concludes by discussing the implications of the proposed framework for researchers, educators, and educational technologists, emphasizing the potential of GenAI to enhance teaching and learning experiences.