The paper proposes an LLM-based Generative IoT (GloT) system designed for local network settings, addressing the limitations of commercial LLMs in terms of hardware requirements, data privacy, and performance. The system includes a Prompt Management Module and a Post-processing Module to enhance the capabilities of open-source LLMs through prompt engineering methods. A case study on Table Question Answering (Table-QA) demonstrates the effectiveness of the proposed system, showing competitive performance compared to state-of-the-art LLMs. The Table-QA task is chosen due to its complexity, involving semi-structured tables with heterogeneous data types and large sizes. The proposed three-stage prompting method—task-planning, task-conducting, and task-correction— addresses the challenges of information extraction and reasoning, reducing inference costs. Experimental results on the WikiTableQA and TabFact datasets validate the proposed system's performance and highlight the benefits of using Python code for reasoning steps. The study also discusses the impact of LLM quantization and different Python implementations, providing insights into optimizing the system's efficiency and performance.The paper proposes an LLM-based Generative IoT (GloT) system designed for local network settings, addressing the limitations of commercial LLMs in terms of hardware requirements, data privacy, and performance. The system includes a Prompt Management Module and a Post-processing Module to enhance the capabilities of open-source LLMs through prompt engineering methods. A case study on Table Question Answering (Table-QA) demonstrates the effectiveness of the proposed system, showing competitive performance compared to state-of-the-art LLMs. The Table-QA task is chosen due to its complexity, involving semi-structured tables with heterogeneous data types and large sizes. The proposed three-stage prompting method—task-planning, task-conducting, and task-correction— addresses the challenges of information extraction and reasoning, reducing inference costs. Experimental results on the WikiTableQA and TabFact datasets validate the proposed system's performance and highlight the benefits of using Python code for reasoning steps. The study also discusses the impact of LLM quantization and different Python implementations, providing insights into optimizing the system's efficiency and performance.