A bias-corrected GEMS geostationary satellite product for nitrogen dioxide using machine learning to enforce consistency with the TROPOMI satellite instrument

Abstract

The Geostationary Environment Monitoring Spectrometer (GEMS) launched in February 2020 is now providing continuous daytime hourly observations of nitrogen dioxide (NO${}_{\mathrm{2}}$) columns over eastern Asia (5° S–45° N, 75–145° E) with 3.5 $\times$ 7.7 km${}^{\mathrm{2}}$ pixel resolution. These data provide unique information to improve understanding of the sources, chemistry, and transport of nitrogen oxides (NO${}_x$) with implications for atmospheric chemistry and air quality, but opportunities for direct validation are very limited. Here we correct the operational level-2 (L2) NO${}_{\mathrm{2}}$ vertical column densities (VCDs) from GEMS with a machine learning (ML) model to match the much sparser but more mature observations from the low Earth orbit TROPOspheric Monitoring Instrument (TROPOMI), preserving the data density of GEMS but making them consistent with TROPOMI. We first reprocess the GEMS and TROPOMI operational L2 products to use common prior vertical NO${}_{\mathrm{2}}$ profiles (shape factors) from the GEOS-Chem chemical transport model. This removes a major inconsistency between the two satellite products and greatly improves their agreement with ground-based Pandora NO${}_{\mathrm{2}}$ VCD data in source regions. We then apply the ML model to correct the remaining differences, $\mathrm{\Delta }$(GEMS–TROPOMI), using the GEMS NO${}_{\mathrm{2}}$ VCDs and retrieval parameters as predictor variables. We train the ML model with colocated GEMS and TROPOMI NO${}_{\mathrm{2}}$ VCDs, taking advantage of TROPOMI off-track viewing to cover the wide range of effective zenith angles (EZAs) observed by GEMS. The two most important predictor variables for $\mathrm{\Delta }$(GEMS–TROPOMI) are GEMS NO${}_{\mathrm{2}}$ VCD and EZA. The corrected GEMS product is unbiased relative to TROPOMI and shows a diurnal variation over source regions more consistent with Pandora than the operational product.

1 Introduction

Nitrogen oxides (NO${}_x$ $\equiv$ NO $+$ NO${}_{\mathrm{2}}$) are reactive trace gases emitted from combustion, lightning, and microbial activity in soils. Emission is mainly as nitrogen oxide (NO), which cycles rapidly with nitrogen dioxide (NO${}_{\mathrm{2}}$). This cycling produces tropospheric ozone (O${}_{\mathrm{3}}$) and oxidation of NO${}_x$ produces nitrate particulate matter (PM), with implications for air quality, climate forcing, and nitrogen deposition. Satellite measurements of NO${}_{\mathrm{2}}$ vertical column densities (VCDs) by solar backscatter from polar sun-synchronous low Earth orbit (LEO) have been used extensively to monitor NO${}_x$ emissions and their trends worldwide (Martin et al., 2003; Lamsal et al., 2015; Curier et al., 2014; Duncan et al., 2016; Liu et al., 2017) and to improve understanding of NO${}_x$ oxidation chemistry (Boersma et al., 2008; Valin et al., 2013; Miyazaki et al., 2017; Beirle and Wagner, 2024).

NO${}_{\mathrm{2}}$ has been measured continuously from LEO since 1995, starting with the Global Ozone Monitoring Experiment (GOME) (Burrows et al., 1999), followed by the operational GOME-2 series (Munro et al., 2016). The Ozone Monitoring Instrument (OMI) was launched in 2004 and is providing global daily continuous data at $\mathrm{13}\times \mathrm{24}$ km${}^{\mathrm{2}}$ nadir resolution (Levelt et al., 2006). The TROPOspheric Monitoring Instrument (TROPOMI) launched on board the Sentinel-5 Precursor (S5P) satellite in 2017 improved the resolution to $\mathrm{3.5}\times \mathrm{5.5}$ km${}^{\mathrm{2}}$ (Veefkind et al., 2012).

OMI and TROPOMI retrievals of NO${}_{\mathrm{2}}$ have been extensively validated using ground-based measurements of NO${}_{\mathrm{2}}$ VCDs from sun-staring Pandora spectrometers and Multi-Axis Differential Optical Absorption Spectroscopy (MAX-DOAS) instruments and also by intercomparisons with each other (Herman et al., 2019; Pinardi et al., 2020; Wang et al., 2020; Cai et al., 2022; Gu et al., 2023). TROPOMI NO${}_{\mathrm{2}}$ is routinely validated against Pandora, MAX-DOAS, and OMI by the Royal Netherlands Meteorological Institute (KNMI), showing overall good agreement with a $-\mathrm{7}$ % mean bias (Lambert et al., 2023). It has been used to quantify NO${}_x$ emissions (Goldberg et al., 2019), infer surface NO${}_{\mathrm{2}}$ concentrations (Cooper et al., 2020), and evaluate air quality models (Douros et al., 2023).

A limitation with polar sun-synchronous LEO satellites is that they observe a given location at most once per day and at the same time of day. Geostationary satellites can observe at much higher frequency and in principle continuously, providing much denser data and a unique capability for tracking the diurnal cycle of emissions, oxidation chemistry, and pollutant transport. The Geostationary Environment Monitoring Spectrometer (GEMS) was launched onboard the Korea Aerospace Research Institute GEO-KOMPSAT2B (GK2B) satellite in February 2020 in an equatorial plane at 128.2° longitude with a continuous view of eastern Asia at $\mathrm{3.5}\times \mathrm{7.7}$ km${}^{\mathrm{2}}$ pixel resolution over Korea (Kim et al., 2020). It is now providing the first NO${}_{\mathrm{2}}$ observations from geostationary orbit. GEMS is part of a geostationary air quality constellation that includes TEMPO over North America launched in April 2023 (Zoogman et al., 2017) and Sentinel-4 over Europe to be launched in 2025 (Ingmann et al., 2012).

As with all satellite observations, GEMS retrievals take some time to mature. NO${}_{\mathrm{2}}$ VCDs (L2 products) are currently retrieved operationally with the version 2.0 algorithm (National Institute of Environmental Research, 2020a). Evaluation of the GEMS NO${}_{\mathrm{2}}$ product with four urban Pandora observations in South Korea found an underestimate and disagreements in diurnal patterns (Kim et al., 2023). However, the local Pandora data may not be representative of the $\mathrm{3.5}\times \mathrm{7.7}$ km${}^{\mathrm{2}}$ GEMS pixels, particularly in urban environments. GEMS evaluation for China found both high and low biases in comparison with ground-based observations and other satellite products (Li et al., 2023; Zhang et al., 2023). Yang et al. (2024) reprocessed the GEMS NO${}_{\mathrm{2}}$ version 2.0 data with prior NO${}_{\mathrm{2}}$ vertical profiles from the GEOS-Chem chemical transport model (CTM) on a 0.25° $\times$ 0.3125° grid and found good agreement with Pandora in Seoul and Beijing including diurnal variations.

Here we introduce a bias-corrected GEMS NO${}_{\mathrm{2}}$ product using machine learning (ML) to minimize the biases between GEMS and TROPOMI and provide in this manner a more reliable and consistent satellite product for scientific applications and for improving the GEMS retrieval. Although TROPOMI observes over a limited range of solar zenith angles (SZAs) on account of the overpass time at approximately 13:30 local time (LT), it has off-track viewing to $\pm \mathrm{50}$°, providing a range of viewing zenith angles (VZAs) to mimic the wider range of SZAs seen by GEMS. The bias-corrected GEMS product combines the high data density of GEMS with the accuracy of TROPOMI demonstrated by extensive validation and algorithm development (Gu et al., 2023; Eskes et al., 2022). Our approach is as follows. First, we adjust for biases caused by different prior information used in the operational L2 retrievals by applying common prior vertical profiles simulated by GEOS-Chem. Second, we train an ML model using GEMS and TROPOMI NO${}_{\mathrm{2}}$ VCDs to minimize differences in colocated data during 2022–2023 using GEMS retrieval parameters as explanatory variables. Third, we apply the trained ML model to the ensemble of GEMS data for July 2022–June 2023, producing a bias-corrected GEMS product that has the temporal coverage of GEMS and is consistent with TROPOMI. We use the ML model to identify the GEMS retrieval parameters associated with the largest discrepancies with TROPOMI and validate this bias-corrected GEMS product with ground-based observations from the Pandora spectrometers.

2 GEMS, TROPOMI, and Pandora operational products

GEMS is an ultraviolet–visible (UV-VIS) hyperspectral imaging spectrometer that measures solar backscatter with 0.6 nm spectral resolution over eastern Asia (5–45° latitude, 75–145° longitude) with $\mathrm{3.5}\times \mathrm{7.7}$ km${}^{\mathrm{2}}$ pixels at 37.5° latitude and hourly repeat times (08:45–17:45 LT) (Kim et al., 2020). Total NO${}_{\mathrm{2}}$ slant column densities (SCDs) along the sun–satellite light path are retrieved using differential optical absorption spectroscopy (DOAS) (Platt, 1994) with a fitting window of 435–450 nm (National Institute of Environmental Research 2020b; Kim et al., 2020). We use the operational L2 product from version 2.0 (available online: https://nesc.nier.go.kr/en/html/index.do, last access: 19 January 2024).

TROPOMI is similarly a UV-VIS hyperspectral imaging spectrometer with a full width at half maximum (FWHM) of 0.55 nm. It is a push-broom instrument with a wide swath of $\sim$ 2600 km and an off-track viewing angle of $\pm \mathrm{50}$° off-nadir, providing global daily viewing once a day at approximately 13:30 LT with $\mathrm{5.5}\times \mathrm{3.5}$ km${}^{\mathrm{2}}$ nadir spatial resolution. Total NO${}_{\mathrm{2}}$ SCDs are retrieved using DOAS in the 405–465 nm window (van Geffen et al., 2022). Here we use the offline (OFFL) L2 product produced by the KNMI NO${}_{\mathrm{2}}$ processor version 2.4.0 (available online: https://dataspace.copernicus.eu/explore-data/data-collections/sentinel-data/sentinel-5p, last access: 19 January 2024).

Ground-based NO${}_{\mathrm{2}}$ column measurements from Pandora instruments have been widely used to validate satellite retrievals (Park et al., 2022; Kim et al., 2023; Pinardi et al., 2020; Yang et al., 2024). Pandora spectrometers provide direct-sun observations to retrieve ground-based NO${}_{\mathrm{2}}$ VCDs by DOAS using a spectral fitting window of 400–470 nm (Herman et al., 2009). Here we use the L2 total NO${}_{\mathrm{2}}$ VCDs processed by the BlickP software (available online: https://www.pandonia-global-network.org, last access: 19 January 2024) at 16 Pandora stations located in northeastern Asia (China, South Korea, Japan) from the Pandonia Global Network (PGN). We select data with quality flags of 0, 1, 10, or 11 and with SZA $\le$ 70°.

3 Reprocessing of GEMS and TROPOMI retrievals to use common vertical profiles

Spectral fitting of satellite solar backscatter observations yields NO${}_{\mathrm{2}}$ SCDs, which must be converted to VCDs, the geophysical quantity more relevant for column interpretation, using air mass factors (AMF $=$ SCD/VCD) (Palmer et al., 2001). Standard retrieval algorithms separate the stratospheric and tropospheric portions of the total VCDs to account for the NO${}_{\mathrm{2}}$ background in the stratosphere from oxidation of nitrous oxide (N${}_{\mathrm{2}}$O) (Bucsela et al., 2013), but this introduces uncertainty related to the stratosphere–troposphere separation (STS) algorithm, which may differ between retrievals (Geddes et al., 2018) and leads to ambiguity in the allocation of near-tropopause NO${}_{\mathrm{2}}$ such as from lightning and aircraft (Travis et al., 2016; Dang et al., 2023). Since the stratospheric background component is readily predictable from CTMs such as GEOS-Chem (Knowland et al., 2022) and the spatial structure in the NO${}_{\mathrm{2}}$ column is mainly from the troposphere, the purpose for separating stratosphere and troposphere is in fact not clear. We focus here on total NO${}_{\mathrm{2}}$ VCDs as most useful for scientific applications and to avoid STS errors.

The AMF from the surface to the top of the atmosphere (TOA) can be decomposed into two parts, scattering weights and vertical shape factors, as follows:

1$$\mathrm{AMF}=\underset{\mathrm{0}}{\overset{\mathrm{TOA}}{\int }}w\left(z\right)S\left(z\right)\mathrm{d}z,$$ where $w\left(z\right)$ is the scattering weight at altitude $z$ measuring the sensitivity of the instrument to NO${}_{\mathrm{2}}$ at altitude $z$ as computed by a radiative transfer model (RTM) and $S\left(z\right)$ is the vertical shape factor of normalized number densities obtained from a CTM. Scattering weights for a given wavelength depend on SZA, VZA, relative azimuth angle, surface albedo, and cloud and aerosol optical depths and vertical distributions (Kwon et al., 2019; Hong et al., 2017). The operational GEMS retrieval algorithm uses a lookup table (LUT) of pre-calculated scattering weights from the VLIDORT RTM at 441 nm (Spurr and Christi, 2014). TROPOMI retrievals use a LUT from the Doubling Adding KNMI (DAK) RTM at 437.5 nm (Stammes, 2001).

The vertical shape factor $S\left(z\right)$ in Eq. (1) describes how NO${}_{\mathrm{2}}$ is distributed with altitude as determined by emissions, chemistry, and transport. It must be prescribed in the retrieval as independent information. The L2 GEMS and TROPOMI retrievals use normalized partial column densities simulated for the local scene by GEOS-Chem (Bey et al., 2001; Yang et al., 2023) at 0.25° $\times$ 0.3125° resolution and TM5-MP (Williams et al., 2017; Bucsela et al., 2013) at 1° $\times$ 1° resolution, respectively. However, the L2 version 2.0 GEMS product used incorrect GEOS-Chem vertical coordinates. To eliminate differences caused by using different vertical shape factors, we replace the profiles in the GEMS and TROPOMI L2 products with identical GEOS-Chem model profiles extending from the surface to the stratopause. Yang et al. (2023) showed that GEOS-Chem successfully reproduces the vertical profiles of NO${}_{\mathrm{2}}$ and their diurnal variations observed over South Korea in the KORUS-AQ aircraft campaign, supporting the use of GEOS-Chem profiles in a common AMF calculation for GEMS and TROPOMI satellite retrievals in eastern Asia. We use monthly mean hourly profiles from a 0.25° $\times$ 0.3125° resolution simulation with GEOS-Chem version 13.0.0 (10.5281/zenodo.4618180, The International GEOS-Chem User Community, 2021). Details on model configuration and emissions can be found in Lee et al. (2024).

Scattering weights are not provided in the GEMS L2 NO${}_{\mathrm{2}}$ product; therefore, we use scattering weights calculated by VLIDORT using GEMS geometry and atmospheric conditions at 448 nm, which are provided in the GEMS L2 glyoxal product (National Institute of Environmental Research 2020a). The GEMS glyoxal algorithm uses a spectral fitting window of 433.0–461.5 nm (Ha et al., 2024), which sufficiently overlaps with the NO${}_{\mathrm{2}}$ fitting window (435–450 nm). The AMF for the GEMS total column can then be calculated using vertical shape factors from GEOS-Chem, following Eq. (1). The TROPOMI L2 product reports averaging kernels $A\left(z\right)$, which normalize the scattering weights to the reported AMF (Eskes and Boersma, 2003). 2$$A\left(z\right)={\displaystyle \frac{w\left(z\right)}{\mathrm{AMF}}}$$ The AMF for the TROPOMI total column can then be calculated from the GEOS-Chem vertical shape factors as follows: 3$${\mathrm{AMF}}^{\prime }=\mathrm{AMF}\underset{\mathrm{0}}{\overset{\mathrm{TOA}}{\int }}A\left(z\right)S\left(z\right)\mathrm{d}z,$$ where AMF is from the L2 product and AMF${}^{\prime }$ is the reprocessed value using GEOS-Chem vertical shape factors. The TROPOMI product reports $A\left(z\right)$ for 34 layers, corresponding to the TM5-MP vertical grid, and we interpolate the values to the 47-layer vertical grid of GEOS-Chem. GEMS pixels with AlgorithmQualityFlags $>$ 112, AMFQualityFlags $>$ 64, and FinalAlgorithmFlags $>$ 1 and TROPOMI pixels with qa_value $<$ 0.75 are filtered out as per quality control recommendations. No other filtering is used. We apply area-weighted regridding to the filtered satellite products and use hourly gridded data at 0.25° $\times$ 0.3125° resolution with cloud fraction $\le$ 0.3 and SZA $\le$ 70° for the remainder of this study.

Figure 1 shows how the reprocessing of AMFs modifies the TROPOMI and GEMS NO${}_{\mathrm{2}}$ VCDs compared to the operational L2 products. In what follows, we denote the operational L2 products as “TROPOMI L2” and “GEMS L2” and the products reprocessed with GEOS-Chem vertical profiles as “reprocessed TROPOMI” and “reprocessed GEMS”. The reprocessing increases TROPOMI in the northeastern Asia source regions, including eastern China, South Korea, and Japan. GEMS decreases over eastern China and increases elsewhere.

Figure 1

NO${}_{\mathrm{2}}$ total vertical column densities (VCDs) from TROPOMI and GEMS. Values are averages for July 2022–June 2023 sampled at the overpass time of TROPOMI (13:30 LT). Panels (a)–(d) show the TROPOMI operational product (L2), our product reprocessed with GEOS-Chem NO${}_{\mathrm{2}}$ vertical profiles, and the absolute and relative differences between the two. Panels (e)–(h) show the same information for GEMS.

[Figure omitted. See PDF]

Figure 2 compares NO${}_{\mathrm{2}}$ VCDs from the L2 and reprocessed products for the ensemble of GEMS and TROPOMI daily data sampled at the overpass time of TROPOMI. For the L2 products we find a negative normalized mean bias (NMB) of $-$14 % in GEMS compared to TROPOMI due to lower background values, but source regions are higher in GEMS. Reprocessing to common prior profiles greatly reduces GEMS–TROPOMI differences, with the exception of the background, where some differences increase.

Figure 2

Scatterplot comparison of NO${}_{\mathrm{2}}$ total VCDs from TROPOMI and GEMS. Individual points are daily data for July 2022–June 2023 on the 0.25° $\times$ 0.3125° grid sampled at the overpass time of TROPOMI. Panel (a) compares the operational TROPOMI and GEMS products, and panel (b) compares our reprocessed products with GEOS-Chem NO${}_{\mathrm{2}}$ vertical profiles. Panel (c) compares the reprocessed TROPOMI product with the GEMS data corrected for residual differences with TROPOMI using machine learning (ML) (Sect. 4). The color scale shows the density of points. The dashed line indicates the 1 : 1 line. The coefficient of determination ($R^{\mathrm{2}}$) and normalized mean bias (NMB $=$ $\frac{\sum \left(\text{GEMS-TROPOMI}\right)}{\sum \mathrm{TROPOMI}}\times \mathrm{100}$ %) are given as an inset. Negative TROPOMI values reflect noise in the SCD spectral fitting. The GEMS L2 product has no negative values.

[Figure omitted. See PDF]

Figure 3a compares the mean seasonal variations in Pandora and satellite NO${}_{\mathrm{2}}$ VCDs averaged over the 16 Pandora stations in northeastern Asia. The VCDs are maximum in winter and minimum in summer, reflecting the lifetime of NO${}_x$ against photochemical oxidation. The operational TROPOMI L2 product has a $-$16 % NMB relative to the Pandora data that is reduced to 7 % when reprocessed with GEOS-Chem vertical profiles. The operational GEMS L2 product has a 23 % NMB relative to Pandora that is reduced to 7 % when reprocessed with GEOS-Chem vertical profiles. The reprocessed TROPOMI and GEMS products are in close agreement, in contrast to the large differences between the TROPOMI L2 and GEMS L2 products, showing that much of the discrepancy in the L2 products stems from different vertical shape factors. Incorrect use of GEOS-Chem vertical coordinates in the L2 version 2.0 GEMS product will be corrected in the next operational product with the GEOS-Chem profiles used here for reprocessing. In the following section, we correct the remaining discrepancies between the reprocessed TROPOMI and GEMS products using machine learning.

Figure 3b and c show the mean diurnal variations in the warm and cold seasons, comparing GEMS and Pandora. The Pandora data in the cold season increase over the course of the day due to daytime emissions, while the data in the warm season are at a minimum in the early afternoon due to chemical loss (Yang et al., 2024). The operational GEMS L2 data feature a midday maximum in the cold season that is not seen in the Pandora data. Our reprocessed product is more consistent with the diurnal variation observed by Pandora. More detailed comparisons of diurnal variations in Pandora and GEMS are presented by Yang et al. (2024) for Beijing and Seoul.

Figure 3

Seasonal and diurnal variations in NO${}_{\mathrm{2}}$ total VCDs from Pandora, TROPOMI, and GEMS averaged at 16 Pandora stations in northeastern Asia over July 2022–June 2023. TROPOMI and GEMS data are shown for the operational L2 products and the products reprocessed with GEOS-Chem NO${}_{\mathrm{2}}$ vertical profiles. Also shown is the GEMS product corrected for residual differences with TROPOMI using ML (Sect. 4). Seasonal variations in (a) are for the TROPOMI overpass time (13:30 LT). Diurnal variations in (b) and (c) are only for Pandora and GEMS and are shown for April–September and October–March, respectively. NMB relative to Pandora (NMB $=$ $\frac{\sum \left(\text{Satellite-Pandora}\right)}{\sum \mathrm{Pandora}}\times \mathrm{100}$ %) are given as an inset. The numbers of observations for each month and hour are also indicated.

[Figure omitted. See PDF]

4 Bias correction in GEMS using machine learning

Here we construct a TROPOMI-corrected GEMS product by developing an ML model that can predict the differences, $\mathrm{\Delta }$(GEMS–TROPOMI), remaining between GEMS and TROPOMI after reprocessing to common vertical profiles. TROPOMI is used as reference because of the greater maturity of its retrieval. The ML model uses as predictors the GEMS NO${}_{\mathrm{2}}$ VCD and the GEMS retrieval parameters provided in the L2 product including effective zenith angle (EZA), relative azimuth angle, aerosol optical depth, aerosol layer height, O${}_{\mathrm{3}}$ column amount, surface reflectance at 440 nm, single-scattering albedo, cloud fraction, and cloud top pressure. EZA combines the geometric effects of SZA and VZA on the AMF, as defined by the geometric AMF $=$ sec(EZA) $+$ 1 in the absence of scattering (Palmer et al., 2001):

4$$\sec \left(\mathrm{EZA}\right)=\sec \left(\mathrm{SZA}\right)+\sec \left(\mathrm{VZA}\right)-\mathrm{1}.$$ TROPOMI observations are for a single time of day but extend off the track to viewing angles $\pm \mathrm{50}$°, meaning that the colocated dataset covers EZA values ranging up to 75°. This allows us to build an ML model valid for a sufficient range of EZAs, as is necessary for application to GEMS observations over their full diurnal range.

We tested five ensemble method algorithms including both bagging (random forest, extra tree) and boosting (LightGBM, XGBoost, CatBoost) using the Fast and Lightweight AutoML Library (FLAML) (Wang et al., 2021). We separated colocated $\mathrm{\Delta }$(GEMS–TROPOMI) pairs into training (July and October 2022, January and April 2023) and test (the rest of the colocated data for July 2022–June 2023) datasets. The average NO${}_{\mathrm{2}}$ VCD from the test dataset is higher than the training data by 5 %, but the two datasets display similar spatial distributions. We trained the models to fit 7 489 498 $\mathrm{\Delta }$(GEMS–TROPOMI) pairs (training data) for 4 months as representative of the four seasons and found that the LightGBM algorithm has the best performance. We excluded $\mathrm{\Delta }$(GEMS–TROPOMI) data lying outside 6 times the interquartile range (6 $\times$ IQR; 0.1 % of the training data) to avoid contamination by outliers.

Figure 4 compares observed and predicted differences for the test data. We conducted a $z$-score transform to correct possible systematic biases associated with the LightGBM algorithm (Belitz and Stackelberg, 2021; Balasus et al., 2023). The $R^{\mathrm{2}}$ for the ML prediction is 0.51 and the root-mean-square error (RMSE) is $\mathrm{0.65}\times {\mathrm{10}}^{\mathrm{15}}$ molec. cm${}^{-\mathrm{2}}$, which lies within the estimated single-retrieval errors in GEMS and TROPOMI NO${}_{\mathrm{2}}$ of 0.15–2.47 $\times$ 10${}^{\mathrm{15}}$ molec. cm${}^{-\mathrm{2}}$ (Kim et al., 2020) and approximately 0.5 $\times$ 10${}^{\mathrm{15}}$ molec. cm${}^{-\mathrm{2}}$ (van Geffen et al., 2022), respectively.

Figure 4

Predicted versus observed $\mathrm{\Delta }$(GEMS–TROPOMI) in the test dataset. The color scale shows the density of points. The dashed line indicates the 1 : 1 line. $R^{\mathrm{2}}$ and root-mean-square error (RMSE) are given as an inset.

[Figure omitted. See PDF]

We produced a corrected GEMS product for the duration of the GEMS L2 version 2.0 record (November 2020 to present) by subtracting $\mathrm{\Delta }$(GEMS–TROPOMI) from the GEMS data previously reprocessed to the GEOS-Chem vertical profiles: 5$${\mathrm{GEMS}}_{\mathrm{corrected}}=\mathrm{GEMS}-\mathrm{\Delta }(\mathrm{GEMS}-\mathrm{TROPOMI}).$$ The ML calculation of $\mathrm{\Delta }$(GEMS–TROPOMI) only requires GEMS retrieval information and is therefore applied to all GEMS retrievals, not requiring collocation with TROPOMI.

Figure 5 compares TROPOMI, GEMS, and the corrected GEMS NO${}_{\mathrm{2}}$ VCDs for the warm and cold seasons. The corrected GEMS product increases VCDs in the remote ocean background in the southeastern part of the GEMS scan domain by up to 200 % and decreases VCDs in central Asia by up to 40 % regardless of season. However, corrections to the GEMS product in the polluted regions in northeastern Asia display different patterns during the warm and cold months. Figure 6 shows an enlarged view of the northeastern Asian domain along with observations from the Pandora stations. In the warm season, GEMS displays consistent 10 % negative biases relative to TROPOMI in eastern parts of China and South Korea (Fig. 6d), resulting in an upward correction. The GEMS bias in the cold season is much noisier and tends to be positive, resulting in downward correction (Fig. 6j).

Figure 5

Comparison of TROPOMI, GEMS, and corrected GEMS NO${}_{\mathrm{2}}$ products in the GEMS scan domain. (a–c) NO${}_{\mathrm{2}}$ total VCDs averaged for April–September at the TROPOMI overpass time. The TROPOMI and GEMS data have been reprocessed to common GEOS-Chem vertical profiles for the observation scenes. (d) $\mathrm{\Delta }$(GEMS–TROPOMI) for April–September. (e) Correction to the GEMS product for April–September. (f–j) The same as panels (a)–(e) but for October–March.

[Figure omitted. See PDF]

Figure 6

The same as Fig. 5 but in the northeastern Asian domain with Pandora observations shown as circles. The color scales are different from Fig. 5.

[Figure omitted. See PDF]

Comparison of the monthly mean variations in GEMS VCDs at the Pandora sites in northeastern Asia also shows that the ML model performs an upwards correction during May to August and a larger downwards correction during other months, resulting in an overall low bias of $\sim$ 5 % compared to TROPOMI (Fig. 3a). However, we see from Fig. 2 that the correction successfully reduces remaining residual differences between GEMS and TROPOMI (NMB $=$ 0 %) for the entire GEMS domain and increases consistency with the observed variability from TROPOMI ($R^{\mathrm{2}}=\mathrm{0.72}$).

We can determine the contribution of each predictor variable to the model prediction using the SHapley Additive exPlanations (SHAP) analysis with the TreeExplainer method (Lundberg et al., 2020), as shown in Fig. 7. The SHAP value can be interpreted as the relative importance of the predictor variable to the bias correction, where negative SHAP values indicate low biases in GEMS and vice versa. Figure 7a shows that the GEMS NO${}_{\mathrm{2}}$ VCD contributes the largest corrections, followed by the EZA, while variables related to atmospheric scattering and surface reflectance are less important. Figure 7b shows that the corrections from NO${}_{\mathrm{2}}$ VCD and EZA are strongly nonlinear. The NO${}_{\mathrm{2}}$ VCD drives the correction of the low bias in the ocean background and the high bias in polluted regions. EZA drives a correction for high biases at angles exceeding 60°. The dominant corrections from VCD and EZA might be viewed as reflecting a correction from the SCD, as we would have SCD/VCD $=$ sec(EZA) $+$ 1 in the absence of scattering. However, we found that using GEMS SCD as a predictor variable was less successful than using VCD.

Figure 7

SHAP analysis for predictors of GEMS–TROPOMI differences, $\mathrm{\Delta }$(GEMS–TROPOMI), in the ML training dataset. (a) SHAP analysis results ranking the predictor variables in order of their contributions to the fit. (b) SHAP contribution to $\mathrm{\Delta }$(GEMS–TROPOMI) from GEMS NO${}_{\mathrm{2}}$ VCD and effective zenith angle (EZA). The color scale shows the density of points.

[Figure omitted. See PDF]

The effect of the correction on the diurnal profiles observed by GEMS at the Pandora sites is shown in Fig. 3b–c. The correction in the warm season dampens the diurnal variability because EZA varies from 50° at local noon to 65° at 07:00 or 17:00 LT, a range limited by the constant VZA set by latitude. The corrected diurnal variability improves agreement with Pandora in the late afternoon while degrading it in the early morning. By contrast, the correction in the cold season decreases GEMS by similar increments for all hours of the day, resulting in no change in the diurnal profile but better agreement with Pandora. The lack of diurnal variability in the correction is because EZA varies over only a limited range. The VZA averages 44° for the Pandora sites in northeastern Asia, and the SZA ranges from 52° at local noon to 65° at 08:00 or 16:00 LT, meaning that the EZA varies only from 60 to 68°.

5 Conclusions

We have presented an improved NO${}_{\mathrm{2}}$ vertical column density (VCD) product from the Geostationary Environment Monitoring Spectrometer (GEMS) by calibrating it to TROPOspheric Monitoring Instrument (TROPOMI) with a machine learning (ML) algorithm. A first step was to reprocess both GEMS and TROPOMI datasets to adopt common NO${}_{\mathrm{2}}$ vertical profiles and resulting air mass factors (AMFs) from the GEOS-Chem model. The second step was to correct the residual difference, $\mathrm{\Delta }$(GEMS–TROPOMI), with the ML model. The corrected GEMS product preserves the data density of GEMS, provides hourly daytime data over eastern and southern Asia and neighboring oceans, and is consistent with TROPOMI. It is available for the duration of the GEMS record (November 2020 to present).

Reprocessing with a common AMF removed most of the differences between GEMS and TROPOMI operational L2 products. It also resulted in better agreement with the ground-based Pandora observations. Even after this reprocessing, GEMS displayed low biases compared to TROPOMI in polluted regions of eastern China and South Korea, as well as in the ocean background, and high biases in central Asia.

We used the LightGBM ML algorithm to correct these remaining biases in GEMS relative to TROPOMI. We trained the ML model to fit colocated $\mathrm{\Delta }$(GEMS–TROPOMI) pairs for 4 months (July and October 2022, January and April 2023) to GEMS NO${}_{\mathrm{2}}$ VCDs and GEMS retrieval parameters. This took advantage of the wide range of TROPOMI viewing angles to train the ML in a manner relevant to GEMS observations at different times of day. The ML was successful in correcting the remaining GEMS differences with TROPOMI. SHAP analysis showed that NO${}_{\mathrm{2}}$ VCD and effective zenith angle (EZA) were the predictor variables associated with largest corrections. ML correction increases VCDs in the remote ocean regions by up to 200 % and decreases VCDs in central Asia by up to 40 %. GEMS values in NO${}_x$ source regions of eastern China and South Korea increase by $\sim$ 10 % during the warm season but decrease during the cold season, resulting in better agreement with Pandora observations. The GEMS correction in these source regions is largest for the early morning and late afternoon in the warm season but is similar for all times of day in the cold season.

Our corrected GEMS NO${}_{\mathrm{2}}$ product is designed to be consistent with the TROPOMI product, supporting the combined use of both datasets for analyses of eastern Asia air quality including NO${}_x$ emissions and chemistry and their diurnal variations. Our approach of calibrating GEMS NO${}_{\mathrm{2}}$ observations to TROPOMI can be extended to other observed species (such as formaldehyde or glyoxal) and to other geostationary satellite instruments including TEMPO over North America and Sentinel-4 over Europe. This would produce consistent datasets across the geostationary air quality constellation with reference to a common TROPOMI calibration for global observing capability.

Data availability

The The GEMS L2, TROPOMI L2, and Pandora NO${}_{\mathrm{2}}$ products are available online through https://nesc.nier.go.kr/en/html/index.do (data inquiry and approval from the PI is required; NESC, 2024), https://dataspace.copernicus.eu/explore-data/data-collections/sentinel-data/sentinel-5p (Copernicus, 2024), and https://www.pandonia-global-network.org (Pandonia, 2024), respectively. The corrected GEMS product from November 2020 to present is available from Harvard Dataverse (https://doi.org/10.7910/DVN/ZQQJRO, Oak, 2024).

Author contributions

Original draft preparation, data processing, analysis, investigation, and visualization were done by YJO. DJJ and NB contributed to project conceptualization. Review and editing were done by DJJ, NB, HC, RJP, HAK, and JK. LHY, JP, HL, GTL, and ESH provided additional resources and support in analysis.

Competing interests

The contact author has declared that none of the authors has any competing interests.

Disclaimer

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. While Copernicus Publications makes every effort to include appropriate place names, the final responsibility lies with the authors.

Special issue statement

This article is part of the special issue “GEMS: first year in operation (AMT/ACP inter-journal SI)”. It is not associated with a conference.

Acknowledgements

We thank NIER for providing the GEMS data, the ESA for providing the TROPOMI data, and the PIs and support staff of the PGN.

Financial support

This research has been supported by the Samsung Advanced Institute of Technology (grant no. A41602).

Review statement

This paper was edited by Myoung-Hwan Ahn and reviewed by three anonymous referees.