How does AI predict ozone based on environmental features?

Scarlet Stadtler — Thu, 03 Mar 2022 08:49:04 +0000

In our previous study (Betancourt et al. 2021), we published a benchmark dataset on ozone metrics extracted from the TOAR database along with a machine learning task. We want to tackle the challenge with machine learning to predict the ozone metrics based upon geospatial features. As a baseline, we predicted, for example, the average ozone with different methods, among them a random forest and a shallow neural network.

Although many studies, like ours, reach their goal by just making the predictions – we asked more questions. How exactly do the machine learning algorithms predict average ozone? We analyzed our machine learning models’ functioning to understand their predictions and found out even more than we expected.

By focusing on inaccurate predictions and explaining why these predictions fail, we (i) identified underrepresented samples, (ii) flagged unexpected inaccurate predictions, and (iii) pointed out training samples irrelevant for predictions. We suggested locations for building new measurement stations based on the underrepresented samples. We also showed which training samples do not substantially contribute to the model performance. We can even drop these samples without performance loss! Our study demonstrates the application of explainable machine learning beyond simply explaining the trained model.

On this map, we show non-influential training stations in white, untrustworthy predictions on the test set in blue, and underrepresented test stations in plum. The marker indicates the model. The neural network’s marker is ×, the random forest’s marker is +, and where those two symbols overlap, we get something like an asterisk ∗. We indicate regions where the models recommend building new red, blue, and violet stations. The neural network recommends building new stations in red-colored areas; the random forest recommends building new stations in the blue-colored areas. Violet represents the intersection of regions recommended by both models.

Stadtler et al., Explainable Machine Learning Reveals Capabilities, Redundancy, and Limitations of a Geospatial Air Quality Benchmark Dataset, Machine Learning and Knowledge Extraction. 2022; 4(1):150-171, https://doi.org/10.3390/make4010008

A benchmark dataset for machine learning on global air quality metrics

Scarlet Stadtler — Mon, 31 Jan 2022 11:58:00 +0000

Why is the field of machine learning advancing so fast?
There are many reasons why machine learning research is flourishing. One of them is benchmark datasets. Loosely speaking, benchmark datasets combine a task with preprocessed data. The task is usually performed with a machine learning algorithm, accelerating development and performance testing. Our benchmark dataset paper (Betancourt et al. 2021) provides geospatial data paired with ozone metrics, called AQ-Bench. We tackle predicting the ozone metrics based on the geospatial data with different machine learning models. The figure below shows the concept of our study.

What is the goal of this study?
In the end, we want to predict ozone. Predicting ozone metrics, for example, related to health, supports mitigating adverse effects. Nevertheless, ozone prediction is difficult due to its atmospheric chemistry and interactions with weather patterns. Computationally expensive and sophisticated models exist but we want to use machine learning. Therefore, our goal is to compose a benchmark dataset to develop machine learning for ozone prediction.

What data is in AQ-Bench?
AQ-Bench consists of globally available geospatial data and ozone metrics based upon measurements, which are scarce and unevenly distributed worldwide. We took the ozone metrics from the TOAR database. The TOAR community put an enormous effort into collecting the data from different countries and providing it to us.

What about machine learning?
Using our AQ-Bench, we trained different machine learning models, a linear regression, a shallow neural network, and a random forest. The study shows their performance in predicting different ozone metrics. We hope other researchers can easily reproduce our work and join the ozone research using machine learning.

Clara Betancourt, Timo Stomberg, Ribana Roscher, Martin G. Schultz, and Scarlet Stadtler, AQ-Bench: a benchmark dataset for machine learning on global air quality metrics, Earth Syst. Sci. Data, 13, 3013–3033, 2021 https://doi.org/10.5194/essd-13-3013-2021

Scarlet Stadtler – IntelliAQ

How does AI predict ozone based on environmental features?

A benchmark dataset for machine learning on global air quality metrics