The paper introduces a novel generative pretrained hierarchical transformer model, named GPHT, for time series forecasting. GPHT addresses two critical drawbacks of existing methods: limited generalizability due to single dataset training and the need for customized forecasting heads in one-step generation schemes. Key contributions include: 1. **Mixed Dataset Pretraining**: GPHT constructs a mixed dataset from various data scenarios, expanding the scale of training data and enhancing the model's ability to uncover commonalities in time series data. 2. **Auto-Regressive Forecasting**: The model employs an auto-regressive approach to effectively model temporal dependencies in the output series, eliminating the need for a customized forecasting head and enabling seamless adaptation to different horizon lengths. Experiments on eight widely used datasets show that GPHT outperforms baseline models in various fine-tuning and zero/few-shot learning settings, demonstrating its superior performance and generalizability. The model's effectiveness is further validated through zero-shot and few-shot evaluations, where it consistently achieves competitive results. Ablation studies confirm the importance of the hierarchical architecture and pretraining on the mixed dataset. Overall, GPHT provides a robust and efficient solution for time series forecasting, leveraging advanced pretraining techniques and hierarchical modeling.The paper introduces a novel generative pretrained hierarchical transformer model, named GPHT, for time series forecasting. GPHT addresses two critical drawbacks of existing methods: limited generalizability due to single dataset training and the need for customized forecasting heads in one-step generation schemes. Key contributions include: 1. **Mixed Dataset Pretraining**: GPHT constructs a mixed dataset from various data scenarios, expanding the scale of training data and enhancing the model's ability to uncover commonalities in time series data. 2. **Auto-Regressive Forecasting**: The model employs an auto-regressive approach to effectively model temporal dependencies in the output series, eliminating the need for a customized forecasting head and enabling seamless adaptation to different horizon lengths. Experiments on eight widely used datasets show that GPHT outperforms baseline models in various fine-tuning and zero/few-shot learning settings, demonstrating its superior performance and generalizability. The model's effectiveness is further validated through zero-shot and few-shot evaluations, where it consistently achieves competitive results. Ablation studies confirm the importance of the hierarchical architecture and pretraining on the mixed dataset. Overall, GPHT provides a robust and efficient solution for time series forecasting, leveraging advanced pretraining techniques and hierarchical modeling.