Title: Devising Transformers as an Autoencoder for Unsupervised Multivariate Time Series Imputation

Abstract: Time-series data processing is essential across various fields, including healthcare, transportation, and weather forecasting. Multivariate time-series data, in particular, exhibit a correlation pattern over a common independent variable. This is illustrated by concurrent sensor readings in applications like autonomous driving or multiple channels in data collection devices used in medical diagnoses. However, the increasing incidence of data acquisition failures, including sensor malfunctions and human errors, results in gaps and substantial loss of information. We propose a novel method called Multivariate Time-Series Imputation with Transformers (MTSIT) to tackle these challenges. This method employs an unsupervised  autoencoder model with a transformer encoder to leverage unlabeled observed data for simultaneous reconstruction and imputation of multivariate time series. The MTSIT strategy presents an input sequence with gaps (missing patterns) to the transformer encoder. The final encoder block produces an output sequence that is linearly transformed into the imputed sequence. The Mean Squared Error (MSE) is subsequently computed between the missing values and their predicted imputations, guiding the network’s training toward minimizing the MSE.

Bio: Aykut Koç received the B.S. degree from the Electrical and Electronics Engineering Department, the M.S. and Ph.D. degrees in electrical  engineering from Stanford University, Stanford, CA, USA, in 2005, 2007, and 2011, respectively. He is a Faculty Member with the Electrical and Electronics Engineering, Bilkent University, Ankara, Turkey. He has authored or co-authored more than 80 research papers and one book chapter and issued five patents. His research interests are machine learning and signal processing, extending into natural language and graph signal processing. He currently serves as an Associate Editor of IEEE SIGNAL PROCESSING LETTERS, IEEE TRANSACTIONS ON SIGNAL AND INFORMATION PROCESSING OVER NETWORKS, and IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY. He is the current Chair of IEEE Signal Processing Society (SPS) Turkiye Chapter. He received the 2023 Science Academy Young Scientists Award (BAGEP).