This paper introduces a transformer-based framework for unsupervised representation learning of multivariate time series. The framework is designed to be effective for both regression and classification tasks, and it demonstrates superior performance compared to existing methods, even with limited training data. The model uses a transformer encoder, which is trained to extract dense vector representations of multivariate time series through an input denoising (autoregressive) objective. This pre-trained model can then be applied to various downstream tasks, including regression, classification, imputation, and forecasting. The framework is evaluated on several benchmark datasets for multivariate time series regression and classification, showing that it outperforms current state-of-the-art methods, even when only a small number of training samples are available. The study also demonstrates that unsupervised pre-training of the transformer models provides a significant performance advantage over fully supervised learning, even without leveraging additional unlabeled data. The model is trained efficiently, even on CPUs, and can be scaled to larger datasets with GPUs. The framework is also shown to be effective for both univariate and multivariate time series tasks, and it is the first unsupervised method to exceed the performance of supervised methods for multivariate time series regression and classification. The paper also discusses related work, including previous approaches to time series regression and classification, as well as unsupervised learning for multivariate time series. The methodology includes a detailed description of the base model, regression and classification tasks, and unsupervised pre-training. The experiments and results show that the proposed framework achieves state-of-the-art performance on multiple benchmark datasets, demonstrating its effectiveness for multivariate time series representation learning.This paper introduces a transformer-based framework for unsupervised representation learning of multivariate time series. The framework is designed to be effective for both regression and classification tasks, and it demonstrates superior performance compared to existing methods, even with limited training data. The model uses a transformer encoder, which is trained to extract dense vector representations of multivariate time series through an input denoising (autoregressive) objective. This pre-trained model can then be applied to various downstream tasks, including regression, classification, imputation, and forecasting. The framework is evaluated on several benchmark datasets for multivariate time series regression and classification, showing that it outperforms current state-of-the-art methods, even when only a small number of training samples are available. The study also demonstrates that unsupervised pre-training of the transformer models provides a significant performance advantage over fully supervised learning, even without leveraging additional unlabeled data. The model is trained efficiently, even on CPUs, and can be scaled to larger datasets with GPUs. The framework is also shown to be effective for both univariate and multivariate time series tasks, and it is the first unsupervised method to exceed the performance of supervised methods for multivariate time series regression and classification. The paper also discusses related work, including previous approaches to time series regression and classification, as well as unsupervised learning for multivariate time series. The methodology includes a detailed description of the base model, regression and classification tasks, and unsupervised pre-training. The experiments and results show that the proposed framework achieves state-of-the-art performance on multiple benchmark datasets, demonstrating its effectiveness for multivariate time series representation learning.