Large Language Models (LLMs) cannot plan or self-verify, but can assist in planning tasks through the LLM-Modulo framework. This framework combines LLMs with external model-based verifiers for tighter integration and better neuro-symbolic reasoning. LLMs are seen as approximate knowledge sources, not as planners or verifiers. They can generate candidate plans, translate them into formats for external verifiers, and help refine incomplete specifications. The LLM-Modulo framework allows LLMs to act as idea generators while external verifiers ensure correctness. LLMs cannot generate executable plans autonomously and are not effective at self-critiquing or improving through synthetic data. The framework leverages LLMs for knowledge acquisition and plan generation, with external verifiers ensuring correctness. The framework is designed to handle complex planning tasks, including travel planning, and has shown improved performance in benchmark tasks. The LLM-Modulo framework provides a robust approach to planning by integrating LLMs with external verifiers, ensuring correctness and efficiency. The paper argues against the over-optimism about LLMs' planning capabilities and highlights the need for careful integration with external systems to ensure reliable planning.Large Language Models (LLMs) cannot plan or self-verify, but can assist in planning tasks through the LLM-Modulo framework. This framework combines LLMs with external model-based verifiers for tighter integration and better neuro-symbolic reasoning. LLMs are seen as approximate knowledge sources, not as planners or verifiers. They can generate candidate plans, translate them into formats for external verifiers, and help refine incomplete specifications. The LLM-Modulo framework allows LLMs to act as idea generators while external verifiers ensure correctness. LLMs cannot generate executable plans autonomously and are not effective at self-critiquing or improving through synthetic data. The framework leverages LLMs for knowledge acquisition and plan generation, with external verifiers ensuring correctness. The framework is designed to handle complex planning tasks, including travel planning, and has shown improved performance in benchmark tasks. The LLM-Modulo framework provides a robust approach to planning by integrating LLMs with external verifiers, ensuring correctness and efficiency. The paper argues against the over-optimism about LLMs' planning capabilities and highlights the need for careful integration with external systems to ensure reliable planning.