17 April 2018 | Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag & Yan Liu
This paper introduces GRU-D, a novel deep learning model for multivariate time series with missing values. GRU-D is based on the Gated Recurrent Unit (GRU), a state-of-the-art recurrent neural network. It incorporates two representations of missing patterns—masking and time interval—into the GRU architecture to effectively capture long-term temporal dependencies and utilize missing patterns for better prediction. Experiments on real-world clinical datasets (MIMIC-III, PhysioNet) and synthetic datasets show that GRU-D achieves state-of-the-art performance and provides useful insights into the role of missing values in time series analysis.
Multivariate time series data are common in many practical applications, including healthcare, geoscience, and biology. These datasets often contain missing values, which can be informative, meaning they provide useful information about the target labels. The paper demonstrates that missing values are often correlated with the target labels, and that this correlation can be exploited to improve prediction performance.
Various approaches have been developed to address missing values in time series, including imputation methods such as smoothing, interpolation, and matrix factorization. However, these methods often fail to capture variable correlations and complex patterns. GRU-D addresses these limitations by incorporating missing patterns directly into the GRU architecture, allowing the model to learn decay rates that reflect the importance of missing values over time.
The GRU-D model uses two types of decay mechanisms: input decay and hidden state decay. Input decay decays the input values over time toward the empirical mean, while hidden state decay decays the hidden states of the GRU. These decay mechanisms allow the model to capture the informative missingness patterns and improve prediction performance.
The paper evaluates GRU-D on several real-world datasets, including MIMIC-III and PhysioNet, and shows that it outperforms other baselines, including non-RNN models and simple imputation methods. GRU-D is also shown to perform well in multi-task prediction tasks, such as predicting mortality and ICD-9 diagnosis categories.
The paper concludes that GRU-D is a promising approach for handling missing values in multivariate time series data. It provides a general deep learning framework for time series with missing data and offers insights into the impact of missingness on prediction tasks. The model is applicable to a wide range of applications, including healthcare, and can be used to improve the accuracy of predictions in scenarios with missing data.This paper introduces GRU-D, a novel deep learning model for multivariate time series with missing values. GRU-D is based on the Gated Recurrent Unit (GRU), a state-of-the-art recurrent neural network. It incorporates two representations of missing patterns—masking and time interval—into the GRU architecture to effectively capture long-term temporal dependencies and utilize missing patterns for better prediction. Experiments on real-world clinical datasets (MIMIC-III, PhysioNet) and synthetic datasets show that GRU-D achieves state-of-the-art performance and provides useful insights into the role of missing values in time series analysis.
Multivariate time series data are common in many practical applications, including healthcare, geoscience, and biology. These datasets often contain missing values, which can be informative, meaning they provide useful information about the target labels. The paper demonstrates that missing values are often correlated with the target labels, and that this correlation can be exploited to improve prediction performance.
Various approaches have been developed to address missing values in time series, including imputation methods such as smoothing, interpolation, and matrix factorization. However, these methods often fail to capture variable correlations and complex patterns. GRU-D addresses these limitations by incorporating missing patterns directly into the GRU architecture, allowing the model to learn decay rates that reflect the importance of missing values over time.
The GRU-D model uses two types of decay mechanisms: input decay and hidden state decay. Input decay decays the input values over time toward the empirical mean, while hidden state decay decays the hidden states of the GRU. These decay mechanisms allow the model to capture the informative missingness patterns and improve prediction performance.
The paper evaluates GRU-D on several real-world datasets, including MIMIC-III and PhysioNet, and shows that it outperforms other baselines, including non-RNN models and simple imputation methods. GRU-D is also shown to perform well in multi-task prediction tasks, such as predicting mortality and ICD-9 diagnosis categories.
The paper concludes that GRU-D is a promising approach for handling missing values in multivariate time series data. It provides a general deep learning framework for time series with missing data and offers insights into the impact of missingness on prediction tasks. The model is applicable to a wide range of applications, including healthcare, and can be used to improve the accuracy of predictions in scenarios with missing data.