In this paper, we present a deep learning solution to detect and correct anomalous values present in historical temperature timeseries, that are likely associated to human and weather instruments errors. Our solution consists in a joint peaks detection and end-to-end sequence prediction involving synchronous measurements of individual meteorological stations along with their neighboring peers. We designed our models in a way that the false positive rate (FPR) of the anomaly detection is minimized and the accuracy maximized, so that the historical records are corrected as less as possible. The method was applied to temperature records of 24 meteorological stations in Belgium, and allowed to automatically correct more than 80% of all errors in both max/min daily temperature records by modifying less than 15% of all the timeseries values, with an overall detection accuracy of 90%. The corrected temperature timeseries yielded a perfect match with respect to errors-free signals in several climate indicators. Our method can be potentially applied to other historical timeseries such as precipitation.
Alioscha-Perez, M, Oveneke, MC, Diaz Berenguer, A, BERTRAND, C & Sahli, H 2019, End-to-end Anomaly Detection, Correction and Prediction of Missing Values in Historical Daily Temperature Timeseries. in Proceedings of the 31st Benelux Conference on Artificial Intelligence (BNAIC2019) and the 28th Belgian Dutch Conference on Machine Learning (Benelearn2019). vol. 2491, CEUR Workshop Proceedings, CEUR Workshop Proceedings, pp. 1-2, BNAIC 2019, Brussels, Belgium, 7/11/19. <http://ceur-ws.org/Vol-2491/>
Alioscha-Perez, M., Oveneke, M. C., Diaz Berenguer, A., BERTRAND, C., & Sahli, H. (2019). End-to-end Anomaly Detection, Correction and Prediction of Missing Values in Historical Daily Temperature Timeseries. In Proceedings of the 31st Benelux Conference on Artificial Intelligence (BNAIC2019) and the 28th Belgian Dutch Conference on Machine Learning (Benelearn2019) (Vol. 2491, pp. 1-2). (CEUR Workshop Proceedings). CEUR Workshop Proceedings. http://ceur-ws.org/Vol-2491/
@inproceedings{376adda459034936a75874c1ee645e1f,
title = "End-to-end Anomaly Detection, Correction and Prediction of Missing Values in Historical Daily Temperature Timeseries",
abstract = "In this paper, we present a deep learning solution to detect and correct anomalous values present in historical temperature timeseries, that are likely associated to human and weather instruments errors. Our solution consists in a joint peaks detection and end-to-end sequence prediction involving synchronous measurements of individual meteorological stations along with their neighboring peers. We designed our models in a way that the false positive rate (FPR) of the anomaly detection is minimized and the accuracy maximized, so that the historical records are corrected as less as possible. The method was applied to temperature records of 24 meteorological stations in Belgium, and allowed to automatically correct more than 80% of all errors in both max/min daily temperature records by modifying less than 15% of all the timeseries values, with an overall detection accuracy of 90%. The corrected temperature timeseries yielded a perfect match with respect to errors-free signals in several climate indicators. Our method can be potentially applied to other historical timeseries such as precipitation.",
author = "Mitchel Alioscha-Perez and Oveneke, {Meshia C{\'e}dric} and {Diaz Berenguer}, Abel and C{\'e}dric BERTRAND and Hichem Sahli",
year = "2019",
month = nov,
day = "7",
language = "English",
volume = "2491",
series = "CEUR Workshop Proceedings",
publisher = "CEUR Workshop Proceedings",
pages = "1--2",
booktitle = "Proceedings of the 31st Benelux Conference on Artificial Intelligence (BNAIC2019) and the 28th Belgian Dutch Conference on Machine Learning (Benelearn2019)",
note = "BNAIC 2019 ; Conference date: 07-11-2019 Through 08-11-2019",
}