TOTEM: TOkized Time Series EMBeddings for General Time Series Analysis

TOTEM: TOkized Time Series EMBeddings for General Time Series Analysis

26 Feb 2024 | Sabera Talukder, Yisong Yue, Georgia Gkioxari
TOTEM is a simple tokenizer architecture for time series analysis that enables generalist training across multiple domains and tasks with minimal to no tuning. The method proposes a self-supervised, discrete tokenization approach using a VQVAE (Vector Quantized Variational Autoencoder) to learn a tokenized representation of time series data. This representation is effective for various tasks including imputation, anomaly detection, and forecasting. TOTEM's architecture consists of an encoder, quantizer, latent codebook, and decoder, which allows for generalist training on multiple domains and zero-shot testing on new domains. The model is evaluated on 17 real-world time series datasets across three tasks, showing that TOTEM matches or outperforms previous state-of-the-art methods on several benchmarks. TOTEM's tokenized representation is effective for both in-domain and zero-shot testing, and it demonstrates strong performance in forecasting, anomaly detection, and imputation. The model is trained using a self-supervised approach, and its performance is evaluated across different training and testing regimes. TOTEM's codebook is used for both in-domain and zero-shot testing, and the model is shown to outperform other methods in these settings. The method is also evaluated in a generalist setting, where a single model is trained on multiple domains, and it outperforms the leading state-of-the-art model in both in-domain and zero-shot testing regimes. TOTEM's performance is further validated through ablation studies and exploratory experiments, demonstrating the effectiveness of its tokenized representation for time series analysis.TOTEM is a simple tokenizer architecture for time series analysis that enables generalist training across multiple domains and tasks with minimal to no tuning. The method proposes a self-supervised, discrete tokenization approach using a VQVAE (Vector Quantized Variational Autoencoder) to learn a tokenized representation of time series data. This representation is effective for various tasks including imputation, anomaly detection, and forecasting. TOTEM's architecture consists of an encoder, quantizer, latent codebook, and decoder, which allows for generalist training on multiple domains and zero-shot testing on new domains. The model is evaluated on 17 real-world time series datasets across three tasks, showing that TOTEM matches or outperforms previous state-of-the-art methods on several benchmarks. TOTEM's tokenized representation is effective for both in-domain and zero-shot testing, and it demonstrates strong performance in forecasting, anomaly detection, and imputation. The model is trained using a self-supervised approach, and its performance is evaluated across different training and testing regimes. TOTEM's codebook is used for both in-domain and zero-shot testing, and the model is shown to outperform other methods in these settings. The method is also evaluated in a generalist setting, where a single model is trained on multiple domains, and it outperforms the leading state-of-the-art model in both in-domain and zero-shot testing regimes. TOTEM's performance is further validated through ablation studies and exploratory experiments, demonstrating the effectiveness of its tokenized representation for time series analysis.
Reach us at info@study.space
[slides and audio] TOTEM%3A TOkenized Time Series EMbeddings for General Time Series Analysis