Articles | Volume 11, issue 2
https://doi.org/10.5194/ascmo-11-133-2025
https://doi.org/10.5194/ascmo-11-133-2025
05 Sep 2025
 | 05 Sep 2025

A new data-standardization procedure for comprehensive outlier detection in correlated meteorological sensor data

Natalie D. Benschop, Temesgen Zewotir, Rajen N. Naidoo, and Delia North

Cited articles

Almorox, J., Hontoria, C., and Benito, M.: Statistical validation of daylength definitions for estimation of global solar radiation in Toledo, Spain, Energ. Convers. Manage., 46, 1465–1471, https://doi.org/10.1016/j.enconman.2004.07.007, 2005. 
Blazquez-Garcia, A., Conde, A., Mori, U., and Lozano, J. A.: A Review on Outlier/Anomaly Detection in Time Series Data, ACM Comput. Surv., 54, 56, https://doi.org/10.1145/3444690, 2021. 
Campbell Scientific: Automatic Weather Stations (AWS) and Meteorologic Instruments: Worldwide standard for climate and boundary-layer meteorology, https://www.campbellsci.co.za/aws-meteorology (last access: 12 March 2025), 2024. 
Lewinson, E.: Outlier Detection with Hampel Filter: How to implement the Hampel filter in Python from scratch, Towards Data Science, https://towardsdatascience.com/outlier-detection-with-hampel-filter-85ddf523c73d (last access: 12 March 2025), 2019. 
Leys, C., Ley, C., Klein, O., Bernard, P., and Licata, L.: Detecting outliers: Do not use standard deviation around the mean, use absolute deviation around the median, J. Exp. Soc. Psychol., 49, 764–766, https://doi.org/10.1016/j.jesp.2013.03.013, 2013. 
Download
Short summary
Meteorological data recorded at high frequency by automatic sensors are often marred by multiple forms of error. Existing validation techniques, in isolation, are sub-optimal for such error-prone data. We propose a new data-standardization procedure for the validation of strongly correlated series which commonly arise in meteorology. We show the procedure to be more comprehensive in the simultaneous detection of solitary spikes, shifts in series means, and irregular diurnal patterns.
Share