XCO2 retrieval for GOSAT and GOSAT-2 based on the FOCAL algorithm

Abstract

Since 2009, the Greenhouse gases Observing SATellite (GOSAT) has performed radiance measurements in the near-infrared (NIR) and shortwave infrared (SWIR) spectral region. From February 2019 onward, data from GOSAT-2 have also been available.

We present the first results from the application of the Fast atmOspheric traCe gAs retrievaL (FOCAL) algorithm to derive column-averaged dry-air mole fractions of carbon dioxide (${\mathrm{XCO}}_{\mathrm{2}}$) from GOSAT and GOSAT-2 radiances and their validation. FOCAL was initially developed for OCO-2 ${\mathrm{XCO}}_{\mathrm{2}}$ retrievals and allows simultaneous retrievals of several gases over both land and ocean. Because FOCAL is accurate and numerically very fast, it is currently being considered as a candidate algorithm for the forthcoming European anthropogenic ${\mathrm{CO}}_{\mathrm{2}}$ Monitoring (CO2M) mission to be launched in 2025.

We present the adaptation of FOCAL to GOSAT and discuss the changes made and GOSAT specific additions. This particularly includes modifications in pre-processing (e.g. cloud detection) and post-processing (bias correction and filtering).

A feature of the new application of FOCAL to GOSAT and GOSAT-2 is the independent use of both S- and P-polarisation spectra in the retrieval. This is not possible for OCO-2, which measures only one polarisation direction. Additionally, we make use of GOSAT's wider spectral coverage compared to OCO-2 and derive not only ${\mathrm{XCO}}_{\mathrm{2}}$, water vapour (${\mathrm{H}}_{\mathrm{2}}\mathrm{O}$), and solar-induced fluorescence (SIF) but also methane (${\mathrm{XCH}}_{\mathrm{4}})$, with the potential for further atmospheric constituents and parameters like semi-heavy water vapour ($\mathrm{HDO}$). In the case of GOSAT-2, the retrieval of nitrous oxide (${\mathrm{XN}}_{\mathrm{2}}\mathrm{O}$) and carbon monoxide ($\mathrm{CO}$) may also be possible.

Here, we concentrate on the new FOCAL ${\mathrm{XCO}}_{\mathrm{2}}$ data products. We describe the generation of the products as well as applied filtering and bias correction procedures. GOSAT-FOCAL ${\mathrm{XCO}}_{\mathrm{2}}$ data have been produced for the time interval 2009 to 2019. Comparisons with other independent GOSAT data sets reveal agreement of long-term temporal variations within about 1 $\mathrm{ppm}$ over 1 decade; differences in seasonal variations of about 0.5 $\mathrm{ppm}$ are observed. Furthermore, we obtain a station-to-station bias of the new GOSAT-FOCAL product to the ground-based Total Carbon Column Observing Network (TCCON) of 0.56 $\mathrm{ppm}$ with a mean scatter of 1.89 $\mathrm{ppm}$.

The GOSAT-2-FOCAL ${\mathrm{XCO}}_{\mathrm{2}}$ product is generated in a similar way as the GOSAT-FOCAL product, but with adapted settings. All GOSAT-2 data until the end of 2019 have been processed. Because of this limited time interval, the GOSAT-2 results are considered to be preliminary only, but first comparisons show that these data compare well with the GOSAT-FOCAL results and also TCCON.

1 Introduction

Carbon dioxide (${\mathrm{CO}}_{\mathrm{2}}$) is the most important greenhouse gas in the context of global warming e.g.. The amount of ${\mathrm{CO}}_{\mathrm{2}}$ in the atmosphere is primarily determined by natural and anthropogenic sources and sinks, but our current understanding of these sources and sinks has significant gaps e.g.. Retrievals of column-averaged carbon dioxide (${\mathrm{XCO}}_{\mathrm{2}}$) from the satellite sensors SCIAMACHY (ENVISAT) and TANSO FTS (GOSAT) , as well as from the Orbiting Carbon Observatory-2 (OCO-2) satellite have been used for over a decade to obtain information on natural ${\mathrm{CO}}_{\mathrm{2}}$ sources and sinks e.g. and on anthropogenic ${\mathrm{CO}}_{\mathrm{2}}$ emissions e.g..

The first satellite measurements of ${\mathrm{XCO}}_{\mathrm{2}}$ were performed by the Scanning Imaging Absorption Spectrometer for Atmospheric CHartographY (SCIAMACHY) instrument on the European environmental satellite ENVISAT launched in 2002 and operating until April 2012.

Whereas greenhouse gases were only one field of application among others of SCIAMACHY, later satellite missions focused explicitly on these. In 2009, the Greenhouse gases Observing SATellite GOSAT; was launched, followed by the Orbiting Carbon Observatory-2 OCO-2; in 2014. Furthermore, in 2016 the Chinese TanSat mission was launched; the first results were presented by . Follow-on instruments to GOSAT and OCO-2 GOSAT-2; OCO-3; have been in orbit since 2018 and 2019, respectively. The TanSat, GOSAT, and OCO-2 and OCO-3 instruments are still operating, and several different retrieval algorithms have been developed to derive ${\mathrm{XCO}}_{\mathrm{2}}$ from their near-infrared (NIR) and shortwave infrared (SWIR) spectra.

The main challenge for spaceborne ${\mathrm{XCO}}_{\mathrm{2}}$ measurements is the required accuracy of the resulting data products as the atmospheric background of ${\mathrm{XCO}}_{\mathrm{2}}$ is high compared to the variability, which is typically less than a few percent (about 2 % seasonal cycle variations in the Northern Hemisphere in addition to an annual increase of about 0.5 % yr${}^{-\mathrm{1}}$; see e.g. ). Depending on the application, even higher accuracies are needed. An accurate ${\mathrm{XCO}}_{\mathrm{2}}$ retrieval usually requires a complex retrieval method and large computational effort. This is no major problem for the number of measurements provided by the GOSAT instruments, but even current OCO-2 retrievals require significantly larger computational effort. However, new missions with much higher spatial resolution and coverage are currently in preparation to address the challenging questions on ${\mathrm{CO}}_{\mathrm{2}}$ local and global sources and sinks in a changing climate; one amongst them is the forthcoming European anthropogenic ${\mathrm{CO}}_{\mathrm{2}}$ Monitoring (CO2M) mission , dramatically increasing the computational power needed for retrievals.

About 3 years ago, developed the Fast atmOspheric traCe gAs retrievaL (FOCAL) and applied it to OCO-2 data. To show the applicability of the FOCAL method not only to OCO-2 but also to other satellite sensors, we present in this study a new application of FOCAL to GOSAT and also some first results from an application to GOSAT-2. GOSAT-FOCAL has several advantages over GOSAT BESD , the currently used IUP GOSAT ${\mathrm{XCO}}_{\mathrm{2}}$ retrieval product , which provides only ${\mathrm{XCO}}_{\mathrm{2}}$ data over land. However, FOCAL is able to retrieve not only ${\mathrm{XCO}}_{\mathrm{2}}$ but – depending on the used spectral ranges – also other atmospheric parameters like ${\mathrm{XCH}}_{\mathrm{4}}$, ${\mathrm{H}}_{\mathrm{2}}\mathrm{O}$, $\mathrm{HDO}$, $\mathrm{CO}$, and ${\mathrm{N}}_{\mathrm{2}}\mathrm{O}$. In the present study we concentrate on ${\mathrm{XCO}}_{\mathrm{2}}$, as this is the most important (and because of its high requirements for accuracy possibly the most challenging) anthropogenic greenhouse gas.

The paper is organised as follows: in Sect.  we list all data sets used in this study. The retrieval algorithm is described in Sect. . Sections  and  then show the results of the retrieval and the validation. Finally, the conclusions are given in Sect. .

2 Data sets used

2.1 GOSAT and GOSAT-2

The Greenhouse gases Observing SATellite GOSAT; was launched in January 2009 and is still in operation. The Thermal And Near infrared Sensor for carbon Observation (TANSO) on-board GOSAT consists of a Cloud and Aerosol Imager (TANSO CAI) and a Fourier Transform Spectrometer (TANSO FTS), which measures radiances in the NIR and SWIR spectral region with S and P polarisation and in the thermal infrared spectral region without polarisation with a spectral resolution of 0.2 ${\mathrm{cm}}^{-\mathrm{1}}$. The FOCAL retrieval uses as the main input calibrated GOSAT L1B V220.220 spectra from the three NIR–SWIR bands (around 0.76, 1.6, and 2.0 $\mathrm{\mu }\mathrm{m}$) of TANSO FTS.

GOSAT-2 was launched in October 2018 and comprises similar instrumentation as GOSAT. The GOSAT-2 FTS has the same spectral resolution but an extended spectral range for $\mathrm{SIF}$ and $\mathrm{CO}$ retrievals.

We use calibrated GOSAT-2 L1B NIR–SWIR data V101.101.

Both GOSAT and GOSAT-2 perform point measurements with a spatial resolution (footprint diameter) of about 10 $\mathrm{km}$. For both instruments, we use a tabulated instrumental line shape (ILS) with a kernel width of 15 ${\mathrm{cm}}^{-\mathrm{1}}$. For GOSAT this has been generated by a theoretical formula parameterising a “real-world” FTS instrument see e.g. formula 5.21 in, which depends on the maximum optical path difference (MOPD, $\pm \mathrm{2.5}$ $\mathrm{cm}$ for GOSAT) and the size of the instantaneous field of view (IFOV, 15.8 $\mathrm{mrad}$ for GOSAT). The same formula has been used by . This ILS is symmetric and the same for S and P polarisation.

For GOSAT-2, we use a preliminary tabulated ILS provided by JAXA generated on 16 January 2020, which is different for S and P polarisation and asymmetric, especially in the NIR band. Meanwhile, this ILS has been officially released and is available via the NIES web site.

2.2 Reference spectra and external databases

For the retrieval several reference spectra and databases are used.

The solar spectrum used in the forward model is based on a high-resolution solar transmittance spectrum in combination with an ISS solar reference spectrum . For the SIF retrieval we used a chlorophyll fluorescence spectrum by , which has been scaled to 1.0 $\mathrm{mW}{\mathrm{m}}^{-\mathrm{2}}{\mathrm{sr}}^{-\mathrm{1}}{\mathrm{nm}}^{-\mathrm{1}}$ at 760 $\mathrm{nm}$.

We use tabulated cross sections at a 0.001 ${\mathrm{cm}}^{-\mathrm{1}}$ sampling based on HITRAN2016 and the absorption cross section database ABSCO v5.0 from the NASA (National Aeronautics and Space Administration) ACOS OCO-2 project.

Surface elevation, surface roughness, and surface type are derived from the Global Multi-resolution Terrain Elevation Data GMTED2010; of the US Geological Survey (USGS) and the National Geospatial-Intelligence Agency (NGA) at a spatial resolution of 0.025${}^{\circ }$. Meteorological information (pressure, temperature, water vapour profiles) is obtained from the ECMWF (European Centre for Medium-Range Weather Forecasts) ERA 5 model data , which are available every 1 h on a 0.25${}^{\circ }$ horizontal grid and on 137 altitude layers.

We use ${\mathrm{XCO}}_{\mathrm{2}}$ data from the CarbonTracker (CT) model CT2019 and CT-NRT v2020-1 and data from the Total Carbon Column Observing Network TCCON; see e.g. in the context of the bias correction reference database (see Sect. ). TCCON data are also used for validation (see Sect. ). Table  lists the TCCON stations which provided data for the present study.

Table 1

TCCON stations used in this study.

Site	Long. (deg)	Lat. (deg)	Elev. ($\mathrm{km}$)	Reference(s)
Ascension Island (SH)	$-\mathrm{14.33}$	$-\mathrm{7.92}$	0.01
Bialystok (PL)	23.03	53.23	0.18
Bremen (DE)	8.85	53.10	0.04
Burgos (PH)	120.65	18.53	0.04
Darwin (AU)	130.89	$\mathrm{12.42}$	0.03
Edwards (US)	$-\mathrm{117.88}$	34.96	0.70
East Trout Lake (CA)	$-\mathrm{104.99}$	54.35	0.50
Eureka (CA)	$-\mathrm{86.42}$	80.05	0.61
Four Corners (US)	$-\mathrm{108.48}$	36.80	1.64
Garmisch-Partenkirchen (DE)	11.06	47.48	0.74
Hefei (CN)	117.17	31.90	0.04
Indianapolis (US)	$-\mathrm{86.00}$	39.86	0.27
Izaña (ES)	$-\mathrm{16.50}$	28.30	2.37
Karlsruhe (DE)	8.43	49.10	0.11
Lamont (US)	$-\mathrm{97.49}$	36.60	0.32
Lauder (NZ)	169.68	$-\mathrm{45.04}$	0.37	,
Ny Ålesund (NO)	11.90	78.90	0.02
Orleans (FR)	2.11	47.97	0.13
Paris (FR)	2.36	48.85	0.06
Park Falls (US)	$-\mathrm{90.27}$	45.95	0.44
Pasadena (US)	$-\mathrm{118.13}$	34.13	0.21
Réunion Island (FR)	55.49	$-\mathrm{20.90}$	0.09
Rikubetsu (JP)	143.77	43.46	0.36
Saga (JP)	130.29	33.24	0.01
Sodankylä (FI)	26.63	67.37	0.18
Tsukuba (JP)	140.12	36.05	0.03
Wollongong (AU)	150.88	$-\mathrm{34.41}$	0.03
Zugspitze (DE)	10.98	47.42	2.96

${\mathrm{XCO}}_{\mathrm{2}}$ a priori profiles are derived using the 2018 version of the simple empirical ${\mathrm{CO}}_{\mathrm{2}}$ model SECM . In the context of validation, we use the 2020 version of SECM. ${\mathrm{XCH}}_{\mathrm{4}}$ a priori data are from the simple ${\mathrm{CH}}_{\mathrm{4}}$ climatological model SC4C2018 developed and used by and briefly described by .

For ${\mathrm{CO}}_{\mathrm{2}}$ we use the “synth” a priori error covariance matrix described by . For ${\mathrm{H}}_{\mathrm{2}}\mathrm{O}$, we use the same error covariance matrix as , but scaled by a factor of 5 to reduce the dependencies of the retrieval results on the a priori. For ${\mathrm{CH}}_{\mathrm{4}}$, for convenience, we scale the ${\mathrm{CO}}_{\mathrm{2}}$ matrix to result in an ${\mathrm{XCH}}_{\mathrm{4}}$ uncertainty of 45 $\mathrm{ppb}$, which is considered to be a reasonable estimate. Note that only the matrices are scaled, not the a priori values.

2.3 The reference database

Quality filtering and bias correction usually require knowledge of a “true” (in this case ${\mathrm{XCO}}_{\mathrm{2}}$) value. For this, we do not simply use model data as truth, as one aim of ${\mathrm{XCO}}_{\mathrm{2}}$ products is to improve models. Another method is to take ground-based TCCON measurements as a basis for bias correction. However, although TCCON measurements are very accurate estimated 1$\mathit{\sigma }$ precision is 0.4 $\mathrm{ppm}$; see, they are only available at certain locations and are therefore more suited for validation.

Our choice is therefore to use a database generated from a combination of TCCON measurements and CarbonTracker (CT) model data for a reference year (2018 for GOSAT, 2019 for GOSAT-2), which should on average reproduce large-scale features correctly.

This database is produced in the following way: as a first step, we determine from the CT data global daily 3D maps close to 13:00 local time (i.e. GOSAT and GOSAT-2 Equator crossing time). We reduce the altitude grid to five layers with the same dry-air sub-columns, i.e. the same quantity of particles, and interpolate the data from the native CT horizontal resolution of $\mathrm{3}{}^{\circ }\times \mathrm{2}{}^{\circ }$ to $\mathrm{0.5}{}^{\circ }\times \mathrm{0.5}{}^{\circ }$. Then we determine from the TCCON data for each day mean values (${\mathrm{XCO}}_{\mathrm{2}}^{\mathrm{TCCON}}$) for 13:00$\pm \mathrm{2}\mathrm{h}$ local time. Next, we select collocated CT data and correct them for the TCCON averaging kernels, resulting in a TCCON-corrected CT value at the TCCON location (${\mathrm{XCO}}_{\mathrm{2}}^{\mathrm{CT}}$). The application of the averaging kernels corrects for different vertical resolutions and/or sensitivities see e.g.. We look for contiguous regions where CT ${\mathrm{XCO}}_{\mathrm{2}}$ data differ by less than 0.75 $\mathrm{ppm}$ from the CT value at the TCCON location; these data are then used for the reference database. The reference database therefore does not contain any TCCON data – it only contains CT data which were confirmed by TCCON, but individual values may differ by up to 1.5 ppm. The choice of the 0.75 $\mathrm{ppm}$ ranges is based on a trade-off between accuracy (agreement with TCCON) and spatial coverage of the database.

The results are daily maps containing ${\mathrm{CO}}_{\mathrm{2}}$ data for five vertical sub-layer altitudes. The spatial coverage is usually not global and varies from day to day. There are typically more data in the Southern Hemisphere during the second half of the year compared to the first half of the year (see Fig. ). When comparing with GOSAT or GOSAT-2 measurement results, the true ${\mathrm{XCO}}_{\mathrm{2}}$ is then computed from the ${\mathrm{CO}}_{\mathrm{2}}$ layers of the reference database considering the retrieval's averaging kernels.

Figure 1

${\mathrm{XCO}}_{\mathrm{2}}$ from the GOSAT-FOCAL reference database. The black markers show the locations of TCCON stations. (a) 1 April 2018. (b) 1 October 2018.

[Figure omitted. See PDF]

Specifically, for GOSAT we use CT2019 data in combination with TCCON GGG2014 (see Table ) for 2018 here. For GOSAT-2 we also use TCCON GGG2014 data, but need to rely on CT-NRT v2020-1 for 2019. Because the CT NRT data are not yet available for the whole year of 2019, the GOSAT-2 reference database does not cover the whole year; there are essentially no data after August 2019. This is a limiting factor for GOSAT-2, especially because this also means that data in the Southern Hemisphere are less present in the 2019 database.

2.4 GOSAT Level 2 products

To assess the quality of the newly created GOSAT-FOCAL ${\mathrm{XCO}}_{\mathrm{2}}$ products, they have been compared with several other well-established GOSAT Level 2 data sets (see Sect. ). The GOSAT BESD v01.04 product from IUP is a near-real-time product generated in the context of the Copernicus Atmospheric Monitoring Service (CAMS, https://atmosphere.copernicus.eu/, last access: 30 July 2020) project. It is available from 2014 onward. The GOSAT RemoTeC v2.3.8 product from SRON and the full-physics GOSAT product from the University of Leicester v7.3 were generated in the context of the Copernicus Climate Change Service (C3S, https://climate.copernicus.eu/, last access: 30 July 2020) and cover the GOSAT time series from 2009 until the end of 2019. The recently released NASA GOSAT ACOS v9r product is also available for the years 2009 to 2019. The operational GOSAT ${\mathrm{XCO}}_{\mathrm{2}}$ product v02.95 (bias corrected) from NIES currently ends in August 2020. The BESD product contains only ${\mathrm{XCO}}_{\mathrm{2}}$ data over land; all other products are available for water and land surfaces.

3 Retrieval algorithm

The retrieval is performed in three main steps: pre-processing, processing, and post-processing. These are described in the following subsections.

3.1 Pre-processing

During pre-processing all required input data for the main processing step are collected. Furthermore, a first filtering of data is performed to reduce processing time.

The pre-processing procedure is largely based on the pre-processing as present in the BESD GOSAT product . The sequence of pre-processing activities is as follows.

• Extract measured spectra, geolocation, and information on quality and measurement modes (e.g. gain, scan direction) from the GOSAT L1B product.

• Estimate instrument noise and cloud parameters.

• Filter for data quality, latitudes, solar zenith angle, signal-to-noise ratio, and clouds (see Table  for settings).

• Extract surface type, elevation, and roughness derived from the surface database for each measurement.

• Add corresponding meteorological information (pressure, temperature, dry-air column, and water vapour profiles) for the time and place of the measurements. This includes a correction for surface elevation; i.e. model profiles are extended or cut according to the value from the surface database.

• Add a priori gas profiles for each measurement (${\mathrm{CO}}_{\mathrm{2}}$ from SECM, ${\mathrm{CH}}_{\mathrm{4}}$ from SC4C, ${\mathrm{H}}_{\mathrm{2}}\mathrm{O}$ from meteorology). For GOSAT-2, a priori profiles for $\mathrm{CO}$ and ${\mathrm{N}}_{\mathrm{2}}\mathrm{O}$ are also added. The latter do not depend on geolocation; they are based on the tropical reference atmosphere from , scaled to column average values of $\mathrm{XCO}$ $=\mathrm{0.1}$ $\mathrm{ppm}$ and ${\mathrm{XN}}_{\mathrm{2}}\mathrm{O}$ $=\mathrm{330}$ $\mathrm{ppb}$.

Table 2

Pre-processing filter limits.

Filter	Value
GOSAT quality flag	good or poor
GOSAT-2 quality flag	good, fair, or poor
Maximum solar zenith angle	70${}^{\circ }$
Maximum latitude	70${}^{\circ }$
Minimum SNR	10
Maximum effective albedo	0.7
Maximum water vapour filter	4.0

Because FOCAL is a fast algorithm and the number of GOSAT and GOSAT-2 measurements is much lower than for OCO-2, we chose to set the pre-processing filters to be relatively relaxed and to apply the quality filtering mostly in the post-processing. As can be seen from Fig.  about two-thirds of the measurements are filtered out during pre-processing.

Figure 2

Example for GOSAT data filtering during the different processing steps (April 2019). Filters are listed in sequential order from top to bottom on the vertical axis. Numbers in the horizontal bars denote the percentage of remaining data after this filter was applied. Orange: total number of measurements before filtering. Yellow: pre-processing filters. Blue: step 1 post-processing filters (convergence and noise). Green: random forest post-processing filter. Light blue: additional post-processing filters.

[Figure omitted. See PDF]

Figure 3

Same as Fig. , but for GOSAT-2.

[Figure omitted. See PDF]

3.1.1 Noise estimate

Similar to the spectral noise is initially assumed to be independent from the wavenumber for each band. It is estimated from the standard deviation of the real part of the “dark” off-band signal (i.e. the first 500 spectral points in each band). In a later step (see Sect. ) this noise will be modified to account for additional forward model errors and overall scaling.

3.1.2 Cloud filter

The cloud filtering is based on two physical properties of clouds: clouds are (usually) bright and clouds are high (higher than the surface) so that little water vapour is above them. In the pre-processing these properties are described by two quantities: effective albedo and water vapour filter. These are derived for each spectrum as described in . The effective albedo for each band is estimated from the mean reflectance $L$ within a spectral range outside the absorption. $L$ is determined from the mean radiance $I$, the mean irradiance $I_{\mathrm{0}}$, and the solar zenith angle $\mathit{\alpha }$ via

1$$L={\displaystyle \frac{\mathit{\pi }I}{I_{\mathrm{0}}\cos \mathit{\alpha }}}.$$ The specific wavenumber ranges and irradiance values used for filtering are given in Table .

Table 3

Parameters for cloud filtering.

Effective albedo
GOSAT band	Waveno. range (${\mathrm{cm}}^{-\mathrm{1}}$)	Irradiance ($\mathrm{W}{\mathrm{cm}}^{\mathrm{-}\mathrm{2}}\mathrm{cm}$)
SWIR-1	13 190–13 210	$\mathrm{7.4}\times {\mathrm{10}}^{-\mathrm{6}}$
SWIR-2	6267–6279	$\mathrm{6.0}\times {\mathrm{10}}^{-\mathrm{6}}$
SWIR-3	4800–4810	$\mathrm{4.3}\times {\mathrm{10}}^{-\mathrm{6}}$
Water vapour filter
GOSAT band	Waveno. range (${\mathrm{cm}}^{-\mathrm{1}}$)
SWIR-3	5176–5193

The water vapour filter is determined from a spectral region with strong water vapour absorption in the SWIR-3 band (see Table ). This filter is defined as the ratio between the median radiance and the median of the estimated noise in this spectral range.

A ground pixel is assumed to be cloudy if either the effective albedo in one of the bands or the water vapour filter exceeds the thresholds given in Table .

3.2 Processing

The processing is based on the Fast atmOspheric traCe gAs retrievaL (FOCAL) algorithm, which is described in detail in . A first successful application of this algorithm to OCO-2 data is given in . Therefore, we only summarise the main features of the algorithm here and point out the differences to the OCO-2 application.

FOCAL approximates modifications of the direct light path due to scattering in the atmosphere by a single scattering layer, which is characterised by its height (pressure level), its optical thickness, and an Ångström parameter that describes the wavenumber dependence of scattering. The layer height is normalised to the surface pressure. Furthermore, Lambertian scattering on the surface is considered. For atmospheric scattering processes an isotropic phase function is assumed. With this approximation, the FOCAL forward model is essentially an analytical formula; it uses pre-calculated and tabulated cross sections such that calculations can be performed considerably fast. The inversion of the forward model is based on optimal estimation and uses the Levenberg–Marquardt–Fletcher method to minimise the cost function.

The OCO-2 retrieval of uses four fit windows in the NIR (near-infrared) and SWIR spectral range to derive the atmospheric parameters ${\mathrm{XCO}}_{\mathrm{2}}$, water vapour, and SIF. In contrast to OCO-2, GOSAT and GOSAT-2 cover a wider spectral range and provide spectra in two polarisation directions referred to as S and P. Therefore, we treat both polarisation directions as independent spectra in our retrieval as opposed to the average of both as usually used in other GOSAT retrievals see e.g.. However, recently presented a methane retrieval for GOSAT based on an algorithm from , which also makes use of both polarisation directions.

We use both polarisation corrections mainly for the following reasons.

• In principle, information is lost when averaging S and P spectra.

• In general, the sensitivity of the instruments and therefore the calibration of the measured spectra is different for S and P. For example, the measured ILS is given independently for S and P.

• S and P include different information on scattering, which can also be used in filtering and/or bias correction.

Furthermore, the FOCAL fitting windows (see Table ) have been adapted to the specific GOSAT(-2) spectral bands such that, in addition, other atmospheric constituents and parameters like $\mathrm{HDO}$ can be derived. In the case of GOSAT-2, ${\mathrm{XN}}_{\mathrm{2}}\mathrm{O}$ and $\mathrm{CO}$ total columns can also possibly be retrieved. This results in six fitting windows for GOSAT and eight windows for GOSAT-2 for each polarisation. The retrieval is performed on a wavenumber axis.

Table 4

Definition of GOSAT and GOSAT-2 spectral fit windows (same for S and P). Windows 7 and 8 are only available for GOSAT-2.

No.	Primary target	Waveno. range (${\mathrm{cm}}^{-\mathrm{1}}$)	Considered gases
1	SIF	13 170–13 220	${\mathrm{O}}_{\mathrm{2}}$
2	${\mathrm{O}}_{\mathrm{2}}$	12 930–13 170	${\mathrm{O}}_{\mathrm{2}}$
3	$\mathrm{HDO}$	6337–6410	${\mathrm{CO}}_{\mathrm{2}}$, ${\mathrm{H}}_{\mathrm{2}}\mathrm{O}$, $\mathrm{HDO}$, ${\mathrm{CH}}_{\mathrm{4}}$
4	${\mathrm{CO}}_{\mathrm{2}}$	6161–6297	${\mathrm{CO}}_{\mathrm{2}}$, ${\mathrm{H}}_{\mathrm{2}}\mathrm{O}$, $\mathrm{HDO}$, ${\mathrm{CH}}_{\mathrm{4}}$
5	${\mathrm{CH}}_{\mathrm{4}}$	5945–6135	${\mathrm{CO}}_{\mathrm{2}}$, ${\mathrm{H}}_{\mathrm{2}}\mathrm{O}$, $\mathrm{HDO}$, ${\mathrm{CH}}_{\mathrm{4}}$
6	${\mathrm{CO}}_{\mathrm{2}}$	4801–4907	${\mathrm{CO}}_{\mathrm{2}}$, ${\mathrm{H}}_{\mathrm{2}}\mathrm{O}$, $\mathrm{HDO}$
7	${\mathrm{N}}_{\mathrm{2}}\mathrm{O}$	4364–4449	${\mathrm{N}}_{\mathrm{2}}\mathrm{O}$, ${\mathrm{H}}_{\mathrm{2}}\mathrm{O}$, $\mathrm{HDO}$, ${\mathrm{CH}}_{\mathrm{4}}$
8	$\mathrm{CO}$	4228–4328	$\mathrm{CO}$, ${\mathrm{H}}_{\mathrm{2}}\mathrm{O}$, $\mathrm{HDO}$, ${\mathrm{CH}}_{\mathrm{4}}$

Because of the large number of target gases and spectral bands the retrieval requires various state vector elements. These are listed together with the fit windows, from which they are determined, and their a priori values and uncertainty ranges in Table  for GOSAT and GOSAT-2.

Table 5

State vector elements and related retrieval settings. A priori values are also used as a first guess. “Fit windows” lists the spectral windows (see Table ) from which the element is determined. “All” means that an element is determined from all fit windows of the specified polarisation. “Each” means that a corresponding element is fitted in each fit window. A priori values labelled “PP” are taken from pre-processing; “est.” denotes that they have been estimated from the background signal.

Element	Fit windows	A priori	A priori uncertainty	Comment
Gases
co2_lay	3, 4 , 5, 6 (S and P)	PP	10.0	${\mathrm{CO}}_{\mathrm{2}}$ profile (five layers; ppm)
ch4_lay	3, 4, 5 (S and P)	PP	0.045	${\mathrm{CH}}_{\mathrm{4}}$ profile (five layers; ppm)
h2o_lay	3, 4, 5, 6 (S and P)	PP	5.0	${\mathrm{H}}_{\mathrm{2}}\mathrm{O}$ profile (five layers; ppm)
sif_fac	1 (S and P)	0.0	5.0	SIF spectrum scaling factor
delta_d	3, 4, 5, 6 (S and P)	$-\mathrm{200.0}$	1000.0	$\mathit{\delta }D$ profile scaling factor
n2o_scl	7 (S and P)	1.0	0.1	${\mathrm{N}}_{\mathrm{2}}\mathrm{O}$ profile scaling factor, only GOSAT-2
co_scl	8 (S and P)	1.0	1.0	$\mathrm{CO}$ profile scaling factor, only GOSAT-2
Scattering parameters
pre_sca_s	all S	0.2	1.0	Layer height (pressure), S
tau_sca_0_s	all S	0.01	0.1	Optical depth, S
ang_sca_s	all S	4.0	1.0	Ångström coefficient, S
pre_sca_p	all P	0.2	1.0	Layer height (pressure), P
tau_sca_0_p	all P	0.01	0.1	Optical depth, P
ang_sca_p	all P	4.0	1.0	Ångström coefficient, P
Polynomial coefficients (surface albedo)
poly0	each	est.	0.1	Estimated surface albedo
poly1	each	0.0	0.01
poly2	each	0.0	0.01	Not in SIF window (1)
Spectral corrections
wav_shi	each	0.0	0.1	Wavenumber shift
wav_squ	each	0.0	0.001	Wavenumber squeeze

Note: for GOSAT-2 the polynomial degree in fit window 1P is (accidentally) set to 2 and in fit window 7S to 1.

For GOSAT, the retrieval determines ${\mathrm{CO}}_{\mathrm{2}}$, ${\mathrm{CH}}_{\mathrm{4}}$, and ${\mathrm{H}}_{\mathrm{2}}\mathrm{O}$ on five layers with same number of air particles, from which the column average values ${\mathrm{XCO}}_{\mathrm{2}}$, ${\mathrm{XCH}}_{\mathrm{4}}$, and ${\mathrm{XH}}_{\mathrm{2}}\mathrm{O}$ are then calculated. Furthermore, solar-induced fluorescence (SIF) is determined by scaling a corresponding reference spectrum.

Instead of the $\mathrm{HDO}$ column, we fit a scaling factor for the relative abundance of $\mathrm{HDO}$ compared to ${\mathrm{H}}_{\mathrm{2}}\mathrm{O}$, $\mathit{\delta }D$, which is defined as 2$$\mathit{\delta }D={\displaystyle \frac{R_{\mathrm{meas}}}{R_{\mathrm{VSMOW}}}}-\mathrm{1},$$ where $R_{\mathrm{meas}}$ is the ratio of the measured $\mathrm{HDO}$ and ${\mathrm{H}}_{\mathrm{2}}\mathrm{O}$ columns, and $R_{\mathrm{VSMOW}}$ ($\mathrm{3.1152}\times {\mathrm{10}}^{-\mathrm{4}}$) is the corresponding value for Vienna Standard Mean Ocean Water (VSMOW). $\mathit{\delta }D$ is usually given in units of per mill.

$\mathit{\delta }D=\mathrm{0}$ ‰ corresponds to $\mathrm{HDO}$ concentrations as in VSMOW and $\mathit{\delta }D=-\mathrm{1000}$ ‰ to no $\mathrm{HDO}$. We assume the same profile shape for $\mathrm{HDO}$ as for ${\mathrm{H}}_{\mathrm{2}}\mathrm{O}$. For GOSAT-2, we also fit scaling factors to (fixed) $\mathrm{CO}$ and ${\mathrm{N}}_{\mathrm{2}}\mathrm{O}$ profiles.

As mentioned above, atmospheric scattering is considered in FOCAL by a single scattering layer, which is described by three parameters (height, optical depth, and Ångström coefficient). As scattering is different for S- and P-polarised light, we fit two independent layers for S and P.

In addition, we determine in each fit window (independently for S and P) a polynomial background function describing the surface albedo. For this we use second-order polynomials except for the small SIF windows (no. 1) for which a linear function is sufficient. Note that in the current version of the GOSAT-2 product the polynomial degree in fit window 1P is accidentally set to 2 and in fit window 7S to 1. This has no major impact on the retrieved ${\mathrm{XCO}}_{\mathrm{2}}$ and will be corrected in a future product version.

The GOSAT data files only contain a fixed spectral axis. As described in e.g. , the spectral calibration of GOSAT changes with time, especially at the beginning of the mission. This change can be corrected by a spectral scaling factor. We determine this overall scaling factor by a spectral fit in the SIF window before the retrieval. So far, this spectral pre-fitting seems to be unnecessary for GOSAT-2. In the retrieval, we then additionally consider for both GOSAT and GOSAT-2 possible additional spectral shifts and squeezes in each fit window by corresponding state vector elements, but the influences of these spectral changes on the results is rather small.

Noise model

The noise $N$ derived from the off-band signal is only an estimate. It does not consider a possible wavenumber dependence of the noise within one spectral band. Furthermore, a potential error of the forward model needs to be considered. In the optimal estimation method this can be achieved by including the forward model error in the measurement error covariance. For this, we define a scaling factor $s$ for the estimated noise and the quantity $\mathit{\delta }F$, which denotes the relative error of the forward model. The forward model error is proportional to the continuum radiance outside the absorption $I$, which is estimated from the 0.99 percentile of the measured radiance at the edge of each fit window. The quantities $\mathit{\delta }F$ and $s$ are determined using the approach described in , i.e. by running the retrieval for a representative subset of data and then fitting the function 3$$\text{RSR(NSR)}=\sqrt{(s\text{NSR}{)}^{\mathrm{2}}+\mathit{\delta }{F}^{\mathrm{2}}}$$ to binned values of the residual-to-signal ratio (RSR) as a function of the noise-to-signal ratio (NSR). RSR is defined as the standard deviation of the retrieved spectral residual in each fit window divided by the continuum signal $I$; NSR is the standard deviation of the noise divided by $I$.

With the method described in it is also possible to define a 2$\mathit{\sigma }$ outlier limit based on NSR and RSR data, which will be used to filter out data that are too noisy during post-processing (see Sect. ). This is parameterised by a second-order polynomial as a function of the uncorrected NSR: 4$$f_N\left(\text{NSR}\right)=a_{\mathrm{0}}+a_{\mathrm{1}}\text{NSR}+a_{\mathrm{2}}{\text{NSR}}^{\mathrm{2}},$$ which is added to the RSR function of Eq. (). The coefficients $a_i$ are determined via a fit. To avoid extrapolation, $f_N$ is set to the edge values outside the fitting range.

In order to cover the varying signal over the year, we base the noise model fits on data from 1 d per month for one reference year. For GOSAT, we take data from December 2017 to November 2018 (as there are few GOSAT data available in December 2018). For GOSAT-2 we use data from February 2019 to December 2019. In the case of GOSAT-2 we further restrict the input data for the noise model parameter fit to data over land because some of the data over water show an unexpected behaviour (low RSR in the case of large NSR), which needs further investigation. In this sense, the current GOSAT-2 noise model is considered to be preliminary and may need some refinement in the future.

Figure 4

Example results from the noise model. (a) GOSAT fit window 2P. (b) GOSAT 2 fit window 2P. (c) GOSAT fit window 6P. (d) GOSAT 2 fit window 6P.

[Figure omitted. See PDF]

Figure  shows as an example the noise model results for GOSAT and GOSAT-2 in the fit windows 2 (${\mathrm{O}}_{\mathrm{2}}\left(\mathrm{A}\right)$ band) and 6 (strong ${\mathrm{CO}}_{\mathrm{2}}$ absorption) for P polarisation. The orange line gives the fitted RSR function and the red line the outlier limit. The derived values from the noise model for all fitting windows and polarisations are given in Tables  and  for GOSAT and GOSAT-2. The forward model errors $\mathit{\delta }F$ are on average slightly larger for GOSAT-2 than for GOSAT. In the SWIR, values similar to OCO-2 are obtained, but in the NIR the OCO-2 $\mathit{\delta }F$ is typically smaller (about 0.003). This indicates that for GOSAT and GOSAT-2 instrumental and/or calibration effects seem to impact the radiance errors more in the NIR than in the SWIR.

Table 6

Parameters of the GOSAT noise model.

Fit window	$s$	$\mathit{\delta }F$	$a_{\mathrm{0}}$	$a_{\mathrm{1}}$	$a_{\mathrm{2}}$	NSR range
S polarisation
1	1.12	$\mathrm{2.17}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{9.369}\times {\mathrm{10}}^{-\mathrm{5}}$	$\mathrm{1.613}\times {\mathrm{10}}^{-\mathrm{1}}$	$-\mathrm{9.185}\times {\mathrm{10}}^{-\mathrm{1}}$	0.003–0.049
2	1.07	$\mathrm{5.50}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{1.183}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{2.557}\times {\mathrm{10}}^{-\mathrm{2}}$	$\mathrm{1.107}\times {\mathrm{10}}^{\mathrm{0}}$	0.003–0.047
3	1.14	$\mathrm{3.61}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{5.241}\times {\mathrm{10}}^{-\mathrm{4}}$	$\mathrm{1.251}\times {\mathrm{10}}^{-\mathrm{1}}$	$-\mathrm{6.776}\times {\mathrm{10}}^{-\mathrm{1}}$	0.003–0.043
4	1.07	$\mathrm{3.37}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{5.480}\times {\mathrm{10}}^{-\mathrm{4}}$	$\mathrm{7.250}\times {\mathrm{10}}^{-\mathrm{2}}$	$\mathrm{3.716}\times {\mathrm{10}}^{-\mathrm{2}}$	0.003–0.041
5	1.07	$\mathrm{3.58}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{8.836}\times {\mathrm{10}}^{-\mathrm{4}}$	$\mathrm{3.433}\times {\mathrm{10}}^{-\mathrm{2}}$	$\mathrm{8.853}\times {\mathrm{10}}^{-\mathrm{1}}$	0.003–0.047
6	1.00	$\mathrm{7.12}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{1.680}\times {\mathrm{10}}^{-\mathrm{3}}$	$-\mathrm{9.060}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{1.190}\times {\mathrm{10}}^{\mathrm{0}}$	0.001–0.047
P polarisation
1	1.13	$\mathrm{2.42}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{5.961}\times {\mathrm{10}}^{-\mathrm{4}}$	$\mathrm{8.736}\times {\mathrm{10}}^{-\mathrm{2}}$	$\mathrm{6.867}\times {\mathrm{10}}^{-\mathrm{1}}$	0.003–0.049
2	1.05	$\mathrm{7.50}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{3.177}\times {\mathrm{10}}^{-\mathrm{3}}$	$-\mathrm{1.109}\times {\mathrm{10}}^{-\mathrm{1}}$	$\mathrm{2.711}\times {\mathrm{10}}^{\mathrm{0}}$	0.003–0.049
3	1.21	$\mathrm{3.17}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{4.909}\times {\mathrm{10}}^{-\mathrm{4}}$	$\mathrm{1.226}\times {\mathrm{10}}^{-\mathrm{1}}$	$\mathrm{5.539}\times {\mathrm{10}}^{-\mathrm{2}}$	0.003–0.037
4	1.09	$\mathrm{3.33}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{6.725}\times {\mathrm{10}}^{-\mathrm{4}}$	$\mathrm{4.661}\times {\mathrm{10}}^{-\mathrm{2}}$	$\mathrm{1.002}\times {\mathrm{10}}^{\mathrm{0}}$	0.003–0.035
5	1.04	$\mathrm{3.58}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{6.457}\times {\mathrm{10}}^{-\mathrm{4}}$	$\mathrm{5.710}\times {\mathrm{10}}^{-\mathrm{2}}$	$\mathrm{2.663}\times {\mathrm{10}}^{-\mathrm{1}}$	0.003–0.039
6	1.00	$\mathrm{6.96}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{1.488}\times {\mathrm{10}}^{-\mathrm{3}}$	$-\mathrm{9.996}\times {\mathrm{10}}^{-\mathrm{4}}$	$\mathrm{1.262}\times {\mathrm{10}}^{\mathrm{0}}$	0.003–0.049

Table 7

Parameters of the GOSAT-2 noise model.

Fit window	$s$	$\mathit{\delta }F$	$a_{\mathrm{0}}$	$a_{\mathrm{1}}$	$a_{\mathrm{2}}$	NSR range
S polarisation
1	0.90	$\mathrm{3.65}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{5.895}\times {\mathrm{10}}^{-\mathrm{4}}$	$\mathrm{2.314}\times {\mathrm{10}}^{-\mathrm{1}}$	$-\mathrm{1.524}\times {\mathrm{10}}^{\mathrm{1}}$	0.003–0.009
2	0.94	$\mathrm{5.21}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{1.143}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{1.352}\times {\mathrm{10}}^{-\mathrm{2}}$	$\mathrm{4.058}\times {\mathrm{10}}^{\mathrm{0}}$	0.003–0.009
3	1.20	$\mathrm{3.14}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{3.972}\times {\mathrm{10}}^{-\mathrm{4}}$	$\mathrm{2.756}\times {\mathrm{10}}^{-\mathrm{1}}$	$\mathrm{7.753}\times {\mathrm{10}}^{\mathrm{0}}$	0.001–0.009
4	1.08	$\mathrm{5.25}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{2.491}\times {\mathrm{10}}^{-\mathrm{4}}$	$\mathrm{2.683}\times {\mathrm{10}}^{-\mathrm{1}}$	$-\mathrm{4.440}\times {\mathrm{10}}^{-\mathrm{1}}$	0.001–0.007
5	1.06	$\mathrm{4.04}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{2.184}\times {\mathrm{10}}^{-\mathrm{4}}$	$\mathrm{4.112}\times {\mathrm{10}}^{-\mathrm{1}}$	$-\mathrm{2.730}\times {\mathrm{10}}^{\mathrm{1}}$	0.001–0.007
6	1.01	$\mathrm{4.73}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{6.001}\times {\mathrm{10}}^{-\mathrm{4}}$	$\mathrm{4.469}\times {\mathrm{10}}^{-\mathrm{1}}$	$-\mathrm{2.822}\times {\mathrm{10}}^{\mathrm{1}}$	0.001–0.011
7	1.16	$\mathrm{7.53}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{1.289}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{3.984}\times {\mathrm{10}}^{-\mathrm{1}}$	$-\mathrm{1.377}\times {\mathrm{10}}^{\mathrm{1}}$	0.001–0.009
8	1.11	$\mathrm{9.34}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{9.305}\times {\mathrm{10}}^{-\mathrm{4}}$	$\mathrm{5.126}\times {\mathrm{10}}^{-\mathrm{1}}$	$-\mathrm{2.263}\times {\mathrm{10}}^{\mathrm{1}}$	0.003–0.017
P polarisation
1	0.96	$\mathrm{2.83}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{8.010}\times {\mathrm{10}}^{-\mathrm{5}}$	$\mathrm{2.856}\times {\mathrm{10}}^{-\mathrm{1}}$	$-\mathrm{9.105}\times {\mathrm{10}}^{\mathrm{0}}$	0.003–0.017
2	0.97	$\mathrm{6.08}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{2.258}\times {\mathrm{10}}^{-\mathrm{3}}$	$-\mathrm{1.191}\times {\mathrm{10}}^{-\mathrm{1}}$	$\mathrm{6.365}\times {\mathrm{10}}^{\mathrm{0}}$	0.003–0.015
3	1.19	$\mathrm{3.20}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{6.571}\times {\mathrm{10}}^{-\mathrm{4}}$	$\mathrm{9.767}\times {\mathrm{10}}^{-\mathrm{2}}$	$\mathrm{1.049}\times {\mathrm{10}}^{\mathrm{1}}$	0.001–0.011
4	1.10	$\mathrm{5.25}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{3.868}\times {\mathrm{10}}^{-\mathrm{4}}$	$\mathrm{2.064}\times {\mathrm{10}}^{-\mathrm{1}}$	$-\mathrm{1.886}\times {\mathrm{10}}^{\mathrm{0}}$	0.001–0.009
5	1.08	$\mathrm{4.17}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{6.688}\times {\mathrm{10}}^{-\mathrm{5}}$	$\mathrm{4.935}\times {\mathrm{10}}^{-\mathrm{1}}$	$-\mathrm{3.445}\times {\mathrm{10}}^{\mathrm{1}}$	0.001–0.009
6	1.02	$\mathrm{4.68}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{1.181}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{1.123}\times {\mathrm{10}}^{-\mathrm{1}}$	$\mathrm{1.245}\times {\mathrm{10}}^{\mathrm{0}}$	0.001–0.015
7	1.01	$\mathrm{7.11}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{1.907}\times {\mathrm{10}}^{-\mathrm{3}}$	$-\mathrm{3.405}\times {\mathrm{10}}^{-\mathrm{2}}$	$\mathrm{1.012}\times {\mathrm{10}}^{\mathrm{1}}$	0.003–0.015
8	1.10	$\mathrm{9.52}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{2.093}\times {\mathrm{10}}^{-\mathrm{3}}$	$\mathrm{1.632}\times {\mathrm{10}}^{-\mathrm{1}}$	$-\mathrm{2.418}\times {\mathrm{10}}^{\mathrm{0}}$	0.003–0.021

3.3 Post-processing

The purpose of post-processing is to filter out invalid data and to perform a bias correction for the products. The current post-processing focuses on ${\mathrm{XCO}}_{\mathrm{2}}$. The post-processing is performed in several steps, namely the following:

• basic filtering based on physical knowledge;

• filtering out low-quality data using parameters and limits determined with a random forest classifier;

• application of a bias correction using a random forest regressor; and

• additional filtering-out of data with a bias correction that is too large.

These steps are described in the following subsections.

3.3.1 Basic filter

The basic filtering removes measurements for which the retrieval does not converge or for which the quality of the fit results is too low. We consider this to be the case if the ${\mathit{\chi }}^{\mathrm{2}}$ calculated over all fit windows is larger than 2 or if for at least one of the fit windows the RSR outlier limits (see Sect. ) are exceeded. Furthermore, we apply some initial filters for nonphysical values to the derived scattering parameters (i.e. layer height outside the atmosphere, Ångström coefficient not within [1,5]). Note that all retrieved scattering parameters such as the Ångström exponent can be considered “effective” parameters as they have to account for not only cloud–aerosol scattering but also Rayleigh scattering (which has an Ångström coefficient of 4). Because Rayleigh scattering is always present and we filter out cloudy scenes, we usually get higher effective Ångström coefficients than those expected from clouds or aerosols only.

We also limit the maximum allowed optical depth of the scattering layer to 0.02 to filter out clouds or aerosol amounts that are too thick and use a maximum allowed ${\mathrm{XCO}}_{\mathrm{2}}$ posteriori uncertainty of 2 $\mathrm{ppm}$. As described by , FOCAL simulates scattering only for an isotropic phase function. The prominent forward peak, usually existing for Mie scattering phase functions of cloud and aerosol particles, basically does not modify the light path. As FOCAL's optical depths of the scattering layer do not include this forward peak, these optical depths are much smaller than optical depths including a strong forward peak while having a similar influence on the light path modification (see discussion in the publication of ). The maximum value of 0.02 for the layer optical depth should therefore not be interpreted as e.g. an aerosol optical depth.

The limits for the optical depth of the scattering layer and the ${\mathrm{XCO}}_{\mathrm{2}}$ error are somewhat arbitrary and actually result from visual inspection of the retrieval results. However, they are only intended as a first rough quality filter to facilitate later filter and bias correction methods, which will partly use the same parameters (see below). The detailed choice of these limits is therefore considered uncritical for the final results.

The above-mentioned filter parameters and limits (see Table ) are applied to both land and water surfaces and are the same for GOSAT and GOSAT-2 except for the RSR outlier limit, which has been determined individually for each instrument. Figures  and  show how many data points are typically filtered out in this step. The different performance of the RSR filters for both instruments indicates that the filtering for GOSAT-2 needs some further optimisation, which is planned for the next version of the products.

Table 8

Basic filter parameters.

Filter	Range for valid data
Good convergence	${\mathit{\chi }}^{\mathrm{2}}\le \mathrm{2}$
RSR (each fit window, S and P)	below outlier limit
Scattering layer height (S and P)	0 to 1
Ångström coefficient (S and P)	1 to 5
Scattering layer optical depth (S and P)	0 to 0.02
${\mathrm{XCO}}_{\mathrm{2}}$ error	0 to 2.0 $\mathrm{ppm}$

3.3.2 Random forest filter

In the next step, data are filtered out based on their expected ${\mathrm{XCO}}_{\mathrm{2}}$ bias, i.e. the difference to a true ${\mathrm{XCO}}_{\mathrm{2}}$. Of course, this true ${\mathrm{XCO}}_{\mathrm{2}}$ value is normally not known. We therefore use the ${\mathrm{XCO}}_{\mathrm{2}}$ reference database (as described in Sect. ) to train a random forest classifier to identify those variables which would remove – in combination with a corresponding random forest database – a pre-described percentage $p$ of data based on their ${\mathrm{XCO}}_{\mathrm{2}}$ bias. This is done independently for data over land and water. Note that we are only interested here in the ${\mathrm{XCO}}_{\mathrm{2}}$ bias on top of an overall global bias as the latter will be handled via the bias correction.

This method is similar to the random forest filter used by in the context of ${\mathrm{CH}}_{\mathrm{4}}$ and $\mathrm{CO}$ retrieval. Other approaches for data filtering used e.g. by and identify the best variables through the minimisation of local ${\mathrm{XCO}}_{\mathrm{2}}$ variability in the retrieved products. However, all methods essentially serve the same purpose, i.e. to derive a reproducible filtering procedure which does not solely rely on expert knowledge.

We determine the list of relevant variables and the random forest database for the filtering in the following way: we use the (uncorrected) results of the retrieval for the reference and apply the basic filtering as described in Sect. . Then, the subset of these data is selected which has a corresponding true value in the reference database. For these data we determine the ${\mathrm{XCO}}_{\mathrm{2}}$ bias (measurement – reference ${\mathrm{XCO}}_{\mathrm{2}}$) and subtract the global median for land and water of this bias. We then sort the data according to this bias and flag those $p$ percent of data with the highest absolute bias values as “bad”.

For GOSAT, this results in a total of 54 317 data points over land and 109 414 over water. These numbers are smaller for GOSAT-2 (land: 10 625, water: 40 459). The random forest classifier is then trained by randomly using 90 % of these data as input. The training is done in two iterations: first with a complete set of possible input variables (“features”) and output variables (“estimators”) and then using only a reduced set consisting of the 10 best features and/or estimators, i.e. those with the highest random forest score (relevance) in the first run. This relevance is a quantity which describes the relative importance of each feature for the filtering. Relevances are normalised such that the sum of all relevances is 1. The random forest classifier then decides for each measurement based on these 10 variables if it is filtered out or not.

The initial list of possible features and estimators essentially includes all quantities available after the retrieval, including viewing angles, surface properties, and the continuum signal for each fit window. Furthermore, the retrieved values of the state vector elements and their errors are included in this list as are averaging kernels for the profiles. We explicitly exclude the geolocation of the measurement (latitude, longitude) and the retrieved values (but not the errors) for the data products we are interested in, i.e. the gases and SIF. This is to avoid e.g. the filtering-out of certain geographical regions or removing all points with high ${\mathrm{XCO}}_{\mathrm{2}}$ values.

Note that in glint mode over ocean, the inclusion of information about the viewing geometry as possible features bears the risk that the random forest procedure may infer the geolocation from a combination of these.

However, we include as a possible filter variable the gradient of the retrieved ${\mathrm{CO}}_{\mathrm{2}}$ profile (i.e. the difference between the two lowermost layers) as this has shown to be a suitable quantity.

The original number of candidate variables presented to the random forest classifier is quite high (193 for GOSAT and 246 for GOSAT-2) as can be seen from Figs.  and  (top left plots), but there are few with high relevance.

Figure 5

Random forest results for GOSAT. (a–c) Results from the random forest filter. (d–f) Results from random forest bias correction. (a, d) Normalised relevance (score) of all filter variables. (b, e, c, f) Selected variables and their relevance for the land and water surfaces.

[Figure omitted. See PDF]

Figure 6

As Fig. , but for GOSAT-2.

[Figure omitted. See PDF]

The 10 best variables selected partly differ for land and water surface (as shown in the middle and left top panels), but they usually comprise scattering parameters, polynomial coefficients, spectral corrections, and some ${\mathrm{XCO}}_{\mathrm{2}}$-related parameters.

The other 10 % of the input data are used to test the performance of the classifier. The results from this test and other cross-validation activities indicate that the random forest classification is – depending on surface – only accurate in about two-thirds of the cases. This accuracy is defined as the fraction of correctly classified samples. This means that the filtering also removes possibly valid data points and does not remove all possibly bad ones. However, we do not expect a perfect classification because it is not possible to describe all inter-dependencies via the set of input features. Note that the performance of the filter is similar for both the training and the test data sets. This indicates that there are no problems with over-fitting.

To obtain high quality of the remaining ${\mathrm{XCO}}_{\mathrm{2}}$ data, we therefore need to filter out quite a large percentage of data (and perform an additional filtering at a later time; see below). For future data products further investigations are planned to improve the performance of the classifier, e.g. by providing additional features from the combination of existing ones (like the already used ${\mathrm{CO}}_{\mathrm{2}}$ gradient). The percentage $p$ of data to be filtered out is usually a trade-off between data quality and the remaining amount of data. In the present case a 50 % limit has been selected. Actually, as can be seen from Figs.  and , the relative amount of data filtered out via the random forest classifier is not exactly 50 % of the data remaining after the previous filters.

3.3.3 Bias correction and filtering

${\mathrm{XCO}}_{\mathrm{2}}$ retrieval methods usually require a bias correction to be applied to the data. This correction is often based on multi-linear regressions using parameters identified from correlation analyses of differences to an assumed true ${\mathrm{XCO}}_{\mathrm{2}}$ data set. Different data products use different methods to define this “truth”. In many cases, ground-based TCCON measurements are taken as a reference for the true ${\mathrm{XCO}}_{\mathrm{2}}$, like in the GOSAT BESD product , the SRON products , the product from the University of Leicester , and also the operational GOSAT product from NIES . The bias correction of the NASA ACOS OCO-2 product is quite complex ; it uses as a reference a modification of the Southern Hemisphere approximation introduced by . The ACOS product takes multi-model mean data as a reference and derives corrections from data in the Southern Hemisphere below 20${}^{\circ }$ S where ${\mathrm{CO}}_{\mathrm{2}}$ is assumed to be quite uniform in small areas. A similar small area approximation is also used by to correct FOCAL OCO-2 data. This correction method is not possible for GOSAT and GOSAT-2 data because of their sparse sampling. We therefore follow a different approach here.

For the bias correction we use as input the same data set as for the random forest filter, but with this random forest filter applied; 50 % of the resulting data set is then used to train a random forest regressor see also, which aims to minimise the true ${\mathrm{XCO}}_{\mathrm{2}}$ bias, i.e. the difference to the reference database value without the global median subtracted, as a function of the specified features. To create the bias correction database and the corresponding list of best features we again run the training twice, first with the full list (the same as for the filter) and then with the top 10 features. Again, we use different corrections for land and water. The resulting parameters and their performance are shown in the bottom panels of Figs.  and . The bias correction selects similar best features as the filter, but not exactly the same quantities in the same sequence.

The actual number of features and variables to be used for both filtering and bias correction is a trade-off between many variables (explaining all relations with a risk of over-fitting and high computational effort) and few variables (no over-fitting, but maybe missing some relations). Considering 10 features for the bias correction is more than other algorithms typically use, but this is appropriate in our case because we take into account the different relevance of these parameters, which drops off rapidly within the first 10 variables (see top panels of Figs.  and ).

The validity of this choice is confirmed by the application of the bias correction to the training data set and the other 50 % of the input data, which shows a comparable reduction of the ${\mathrm{XCO}}_{\mathrm{2}}$ scatter. This is an indication for a good performance (e.g. no over-fitting) of the regressor.

During application of the bias correction, the random forest regressor estimates the ${\mathrm{XCO}}_{\mathrm{2}}$ bias based on the values of the input variables. This bias is then subtracted from the retrieved value.

Currently, there is only a bias correction for ${\mathrm{XCO}}_{\mathrm{2}}$, but in principle this method is also applicable to other quantities depending on the availability of a corresponding reference database.

After the bias correction there are still a few outliers left in the ${\mathrm{XCO}}_{\mathrm{2}}$ data. These are filtered out by an additional filter on the ${\mathrm{XCO}}_{\mathrm{2}}$ bias derived via the random forest classifier. The limits for this filter are the global median bias for the test data set plus and minus $\mathrm{2}$ $\mathrm{ppm}$. The median bias is different for land and water surfaces and also for GOSAT and GOSAT-2. The actual limits are given in Table . The value 2 $\mathrm{ppm}$ is estimated from visual inspection of the data. Figures  and  show that typically less than 1 %–2 % of the remaining data (less than 0.1 % of all) are affected by this last filter.

The spatial variability of the finally derived bias correction (see Fig. ) is typically below 1 $\mathrm{ppm}$, but as mentioned above there is a systematic difference between land and water data of about of 1–1.5 $\mathrm{ppm}$ (see Table ).

Figure 7

Gridded GOSAT (a) and GOSAT-2 (b) ${\mathrm{XCO}}_{\mathrm{2}}$ bias correction for 2019.

[Figure omitted. See PDF]

Table 9

Bias filter limits.

Filter	Range for valid data
GOSAT land	$-\mathrm{6.9}$ to $-\mathrm{2.9}$ $\mathrm{ppm}$
GOSAT water	$-\mathrm{8.1}$ to $-\mathrm{4.1}$ $\mathrm{ppm}$
GOSAT-2 land	$-\mathrm{4.0}$ to $\mathrm{0.0}$ $\mathrm{ppm}$
GOSAT-2 water	$-\mathrm{5.5}$ to $-\mathrm{1.5}$ $\mathrm{ppm}$

4 Results

The FOCAL retrieval has been applied to all GOSAT and GOSAT-2 measurements until the end of 2019. On average, FOCAL needs 22 $\mathrm{s}$ with six iterations for the processing of one GOSAT ground pixel. For GOSAT-2 the numbers are slightly higher (28 $\mathrm{s}$ for seven iterations) because of the additional fit windows and state vector elements. All performance values are given for a single core of an Intel Xeon E5-2667v3 CPU (3.2 $\mathrm{GHz}$). These numbers are actually about 1 order of magnitude larger than the ones given in for the FOCAL application to OCO-2. This is because we use separate S- and P-polarisation spectra and more retrieved variables for GOSAT(-2), which requires more and larger fit windows. For each of these fit windows and parameters, weighting functions have to be calculated, which involves a convolution with the ILS. This convolution is the most time-consuming part of the FOCAL retrieval. This is even more relevant for GOSAT(-2) because the FTS ILS is in principle sinc-shaped; i.e. it has a sharp peak in the centre but wide wings, which requires a large kernel width (in our case 15 ${\mathrm{cm}}^{-\mathrm{1}}$) for the convolution.

The left plots in Figs.  and  show examples for measured and fitted nadir-mode radiance spectra for GOSAT and GOSAT-2 over land in the different fitting windows for P polarisation. Since the difference between measured and modelled spectra is small and thus hard to see, we show in these figures on the right side the corresponding residuals and the estimated noise. The results for S polarisation look similar and are therefore not shown here. The residuals are on the order of magnitude of the noise, which is slightly higher for P polarisation than for S polarisation. Some small spectral structures are visible in the residuals; they appear more clearly in the smoothed residuals (convoluted with a 21-pixel boxcar), e.g. for GOSAT and GOSAT-2 in the ${\mathrm{O}}_{\mathrm{2}}$(A) band (window 2), and some broadband oscillations in window 4 and 5 for GOSAT-2. These features are present in both S and P polarisations and also occur in other measurements, so they seem systematic. A reduction of these features could further improve future products.

Figure 8

Example for a single GOSAT measurement (P polarisation): (a, c, e, g, i, k) measured (red) and retrieved (green) spectra in the different fit windows; because of the good agreement the red curve is essentially barely visible below the green curve. (b, d, f, h, j, l) Corresponding residuals (measurement – fit). Light blue: unsmoothed. Blue: smoothed with a boxcar width of 21 spectral pixels ($=\mathrm{4.2}$ ${\mathrm{cm}}^{-\mathrm{1}}$). Red: estimated noise error range. The radiance unit is W cm${}^{\mathrm{-}\mathrm{2}}$ cm sr${}^{-\mathrm{1}}$.

[Figure omitted. See PDF]

Figure 9

Same as Fig. , but for GOSAT-2.

[Figure omitted. See PDF]

In Fig.  some statistical information about the GOSAT-FOCAL data products is given. A time series for the number of valid data is given in the top plot. In recent years, about 5 %–6 % of the available measurements could be transferred to valid ${\mathrm{XCO}}_{\mathrm{2}}$ data. The number of valid data points increases from 2009 to 2019. This is mainly due to an increase in the data over water, which is related to optimisations in GOSAT operations (better use of glint geometry) over water. As expected, the mean global ${\mathrm{XCO}}_{\mathrm{2}}$ shown in the middle plot increases with time. Global mean values over water are typically slightly higher than over land. The observed ${\mathrm{XCO}}_{\mathrm{2}}$ variability (standard deviation, bottom plot) is larger over land. Part of this variability is attributed to influences of surface elevation and to different scattering pathways between the land and water measurements. For GOSAT-2, only retrieved data from 2019 are available so far. The total number of available measurements is about 2.8 million compared to about 3.5 million GOSAT measurements in 2019. Only about 3 % of the GOSAT-2 data remain after all filtering and post-processing, which is roughly half of the corresponding number for GOSAT (but similar to the first year of GOSAT). As can be seen from Fig.  more GOSAT-2 data are filtered out due to failed or bad convergence and by the RSR outlier limits than for GOSAT (Fig. ). Future improvements of the GOSAT-2 calibration or the noise model are considered to help here.

Figure 10

Statistics for valid GOSAT measurements (after pre- and post-processing filtering) for each year. Blue: measurements over water. Green: measurements over land. Red: all data. (a) Number of valid measurements, including the percentage of all originally available measurements. (b) Global mean ${\mathrm{XCO}}_{\mathrm{2}}$. (c) Corresponding standard deviation.

[Figure omitted. See PDF]

For further analyses, we have generated monthly maps on a $\mathrm{5}{}^{\circ }\times \mathrm{5}{}^{\circ }$ grid. Example plots for the months April and August 2019 (the beginning and end of the growing season) are shown in Fig.  for GOSAT and in Fig.  for GOSAT-2. The data are not filtered for low numbers of input data in the grid points, which explains some individual outliers in the plots. Overall, the spatial patterns observed by GOSAT and GOSAT-2 look reasonable. The north–south gradient in ${\mathrm{XCO}}_{\mathrm{2}}$ is visible with a different sign in April and August for both instruments. The spatial coverage of the GOSAT-2 data is lower than for GOSAT because more data are filtered out (see above). This especially affects regions like the northern part of Africa.

Figure 11

Example for gridded GOSAT ${\mathrm{XCO}}_{\mathrm{2}}$ data. (a) April 2019. (b) August 2019.

[Figure omitted. See PDF]

Figure 12

Example for gridded GOSAT-2 ${\mathrm{XCO}}_{\mathrm{2}}$ data. (a) April 2019. (b) August 2019.

[Figure omitted. See PDF]

5 Verification and validation

For the verification and validation of the GOSAT- and GOSAT-2-FOCAL products we perform a comparison with various reference data sets (see Sect. ), namely the following.

• The GOSAT BESD v01.04 product from IUP (referred to as BESD later).

• The GOSAT ACOS v9r product from NASA (referred to as ACOS later).

• The GOSAT UoL_FP v7.3 product from the University of Leicester (referred to as UoL later).

• The GOSAT RemoTeC v2.3.8 product from SRON (referred to as SRON later).

• The GOSAT operational product v02.95 (bias corrected) from NIES (referred to as NIES later).

• Collocated TCCON GGG2014 data (referred to as TCCON later).

For the comparisons, all data have been adjusted using the same a priori (SECM2020).

Since all GOSAT products use different retrieval and filter methods, they do not contain the same number of data (see Fig. ). Currently, the NASA ACOS product has the largest number of valid data points, followed by the new GOSAT-FOCAL product with about 20 % less data.

Figure 13

Number of valid ${\mathrm{XCO}}_{\mathrm{2}}$ data points in the different GOSAT products from 2009 to 2019.

[Figure omitted. See PDF]

5.1 Direct comparisons

There are enough common measurement points included in the different GOSAT products to perform a direct comparison. Figure  shows as an example a comparison between the GOSAT-FOCAL data for the year 2018 with the corresponding ACOS, BESD, SRON, NIES, and UoL products. For each plot we only use data for which both data sets have a valid ${\mathrm{XCO}}_{\mathrm{2}}$ value. For these data we performed a linear regression using the orthogonal distance regression (ODR) method see e.g.. Unlike common linear regression, ODR considers uncertainties for both axes (data sets) by minimising the orthogonal distances between the model curve and the data points. The ODR results are shown by the red line and its label. The number of collocations, the median and mean, and the standard deviations of the differences are given in the titles.

Figure 14

Comparison of GOSAT-FOCAL data ($x$ axis) from 2018 with other GOSAT data ($y$ axis). The colour of the data points corresponds to the density of data points at that location (normalised to a maximum value of 1). The dashed line corresponds to perfect agreement. The red line shows the result of a linear fit using the orthogonal distance regression (ODR) method. The total number of collocated data as well as the median, mean, and standard deviation of the ${\mathrm{XCO}}_{\mathrm{2}}$ differences between the two data sets are given in the title of the sub-plots. (a) FOCAL vs. ACOS. (b) FOCAL vs. BESD. (c) FOCAL vs. NIES. (d) FOCAL vs. SRON. (e) FOCAL vs. UoL.

[Figure omitted. See PDF]

Overall, the data scatter around the $\mathrm{1}:\mathrm{1}$ line in a similar way for all comparisons. ODR slopes vary between the data sets from 0.84 (for FOCAL vs. ACOS) up to 1.08 (for FOCAL vs. BESD). Most collocations are available for the ACOS data set because this has the largest number of valid data. Mean and median differences are quite similar and reach from $-\mathrm{0.17}$ $\mathrm{ppm}$ (comparison to BESD) to 0.67 $\mathrm{ppm}$ (comparison with UoL). The scatter (standard deviation of the differences) reaches from 1.4 $\mathrm{ppm}$ (ACOS, NIES) to 1.8 $\mathrm{ppm}$ (BESD).

5.2 TCCON comparisons

TCCON provides high-quality ${\mathrm{XCO}}_{\mathrm{2}}$ (and other) data, which are currently considered to be the main reference for greenhouse gas data obtained from satellite measurements. Therefore, we compared the different GOSAT data sets with collocated TCCON measurements from 2009 to the end of 2018. BESD data are not included because they do not cover the complete time interval. Collocation criteria are the following:

• maximum time difference of 2 $\mathrm{h}$;

• maximum spatial distance of satellite measurement from TCCON station of 500 $\mathrm{km}$; and

• maximum surface elevation difference between satellite measurement and TCCON station of 250 $\mathrm{m}$.

In addition to these criteria we also consider in the validation only stations and TCCON data sets which have at least 50 collocations for all algorithms. This improves the comparability of regional and seasonal biases. As a consequence, not all stations listed in Table  contribute to the validation.

The comparison procedure is the same as used by and described by . In summary, for each TCCON site, the time series of satellite minus TCCON differences are computed under consideration of the averaging kernels, i.e. different vertical sensitivities. The resulting time series are fitted with a trend model, which includes an offset term, a slope term, and a sine term for seasonal fluctuations. The offset term is considered the station bias, and the station scatter is computed from the standard deviation of the fit residual. Results for the GOSAT time series at the TCCON stations are shown in Fig. . Overall, the temporal variations of ${\mathrm{XCO}}_{\mathrm{2}}$ are well reproduced by all data.

Figure 15

Time series of collocated GOSAT data at various TCCON sites for different data products including the new GOSAT-FOCAL data set.

[Figure omitted. See PDF]

Figure  shows as a summary of the GOSAT TCCON comparisons the derived bias and scatter for the different stations and products. The new GOSAT-FOCAL product compares well with the other data sets. Its differences to TCCON have a station-to-station bias (the standard deviation of the station bias) of 0.56 $\mathrm{ppm}$ and a mean scatter (root mean square, or rms, scatter per station) of 1.89 $\mathrm{ppm}$. The seasonal component of the bias has a station-to-station average standard deviation of 0.37 $\mathrm{ppm}$. Overall, the ACOS product performs best in this comparison.

Figure 16

Overview of TCCON validation results for GOSAT.

[Figure omitted. See PDF]

Note that the biases shown in Fig.  essentially correspond to a bias anomaly since the mean bias over all stations was removed from all products. This is because for most applications this mean bias is not relevant since most information is contained in gradients. Subtracting a mean bias also facilitates a comparison of different bias patterns between the algorithms. The subtracted mean station bias is actually small; for GOSAT it varies between 0.17 $\mathrm{ppm}$ (for ACOS) and 0.64 $\mathrm{ppm}$ (for NIES). Because of the subtracted mean bias different signs of biases for different products could be coincidental. However, the biases of FOCAL and ACOS are consistent with the biases found by .

In Figs.  and  we show results from a preliminary comparison of the new GOSAT-2-FOCAL data with TCCON. These results are considered less reliable because they are based on a data set which covers less than a year. There are sufficient collocations for only seven TCCON stations, and some of them comprise only a few months of data, which limits the fit of our trend model. The mean scatter of the GOSAT-2 data is 1.86 $\mathrm{ppm}$ and therefore similar to the one for GOSAT. The derived station-to-station bias for GOSAT-2-FOCAL is 1.14 $\mathrm{ppm}$. This high value is mainly due to the biases derived for the stations Orleans and Réunion Island (where few data are available) as well as the station Pasadena, which also showed larger discrepancies to the GOSAT products (see Fig. ). We are confident that the station-to-station bias will improve when more and improved GOSAT-2 data are available.

Figure 17

Time series of collocated FOCAL GOSAT-2 data at various TCCON sites.

[Figure omitted. See PDF]

Figure 18

Overview of TCCON validation results for the GOSAT-2-FOCAL product.

[Figure omitted. See PDF]

Via the TCCON comparison it is also possible to validate the reported precision of the FOCAL data products (i.e. the specified ${\mathrm{XCO}}_{\mathrm{2}}$ error). The basic idea is to estimate the true precision from the variability of the ${\mathrm{XCO}}_{\mathrm{2}}$ bias relative to trend-corrected, collocated TCCON data. For this purpose, we define 20 bins with increasing reported ${\mathrm{XCO}}_{\mathrm{2}}$ uncertainty and compute the corresponding true precision from the scatter relative to TCCON (i.e. the fit residual mentioned above).

The corresponding scatter plot is shown in Fig. . We use the fitted linear curve to correct the reported uncertainty of the GOSAT-FOCAL data. After the correction, all data scatter around the $\mathrm{1}:\mathrm{1}$ line (dashed).

Figure 19

Comparisons of the (binned) original ${\mathrm{XCO}}_{\mathrm{2}}$ errors (reported precision without correction) of the GOSAT-FOCAL product with estimated errors based on collocated TCCON data (true precision).

[Figure omitted. See PDF]

A similar correction will be performed for the GOSAT-2-FOCAL product as soon as sufficient data (GOSAT-2 and TCCON) are available, which is currently not the case.

5.3 Time series

To investigate the temporal behaviour of the FOCAL ${\mathrm{XCO}}_{\mathrm{2}}$ data sets, we performed comparisons based on monthly data from 2009 to 2019, which were spatially gridded to $\mathrm{5}{}^{\circ }\times \mathrm{5}{}^{\circ }$ (examples are shown in Figs.  and ). Similar data sets have been generated for the SRON, UoL, ACOS, and NIES GOSAT products. We also produced a corresponding gridded GOSAT BESD data set; since these are near-real-time (NRT) data only, there are no GOSAT BESD data before 2014 available (when the NRT processing started). GOSAT-2 data start in February 2019.

We then selected grid points for which the standard error of the mean is less than 1.6 $\mathrm{ppm}$ for each combination of GOSAT-FOCAL ${\mathrm{XCO}}_{\mathrm{2}}$ and a correlative data set as a basic quality filter, similar as done by. These data were then averaged over different latitudinal ranges, namely

• global (90${}^{\circ }$ S–90${}^{\circ }$ N),

• the Northern Hemisphere (25–90${}^{\circ }$ N),

• the tropics (25${}^{\circ }$ S–25${}^{\circ }$ N), and

• the Southern Hemisphere (25–90${}^{\circ }$ S).

Figure  shows the results of these comparisons. The left plots display time series of the different data sets and the right plots the difference between the GOSAT-FOCAL ${\mathrm{XCO}}_{\mathrm{2}}$ and the reference data. All data products reproduce the overall increase in ${\mathrm{XCO}}_{\mathrm{2}}$ with time and the seasonal variations. On average, FOCAL data are typically about 0.5 $\mathrm{ppm}$ higher than the other data sets (except for BESD). This is related to the choice of the true ${\mathrm{XCO}}_{\mathrm{2}}$ for the bias correction. There are long-term changes of the order of 1 $\mathrm{ppm}$ over the complete time series, which differ for each data set. For example, the GOSAT-FOCAL data show in the tropics relative to SRON a higher value at the start of the time series, but both data sets agree at the end. On the other hand, the average difference to the UoL data in the Northern Hemisphere is negative during the first years, but it increases to an almost constant small positive offset below about 0.5 $\mathrm{ppm}$. There is not much difference in the temporal behaviour between the GOSAT-FOCAL and the ACOS and NIES time series. The seasonal shapes also differ slightly, with amplitudes of about 0.5 $\mathrm{ppm}$ and somewhat larger differences in the Southern Hemisphere where seasonal variations are generally smaller.

Figure 20

Gridded monthly mean time series of different GOSAT ${\mathrm{XCO}}_{\mathrm{2}}$ products. (a, c, e, g) Time series of mean ${\mathrm{XCO}}_{\mathrm{2}}$ for four different regions (from top to bottom: global, Northern Hemisphere, tropics, Southern Hemisphere). (b, d, f, h) Corresponding differences to the GOSAT-FOCAL product.

[Figure omitted. See PDF]

Overall, the agreement within the GOSAT data sets is broadly consistent with the systematic regional and seasonal biases derived from the TCCON validation, especially considering that all gridded data sets are based on a different spatial and temporal sampling. Also, the FOCAL products for GOSAT and GOSAT-2 seem to agree quite well, but more GOSAT-2 data are needed to confirm this.

6 Conclusions

Based on the FOCAL retrieval method a new ${\mathrm{XCO}}_{\mathrm{2}}$ data set for GOSAT and a first ${\mathrm{XCO}}_{\mathrm{2}}$ data set for GOSAT-2 have been generated, making use of both measured polarisation directions. The GOSAT-FOCAL data set compares well with corresponding data from other currently available GOSAT retrieval algorithms, i.e. the RemoTeC product from SRON, the UoL FP product, the NASA ACOS product, the NIES product, and the BESD product from IUP. All data sets use different filtering and bias correction schemes and therefore also comprise a different number and sampling of data. The GOSAT-FOCAL product performs well in this context and has almost as many valid data as the ACOS product. Based on gridded data, differences in long-term variations of all data sets of the order of 1 $\mathrm{ppm}$ per decade are observed. Also, seasonal variations differ by about 0.5 $\mathrm{ppm}$.

Comparisons with ground-based TCCON data reveal for the GOSAT-FOCAL product an overall station-to-station bias of 0.56 $\mathrm{ppm}$ and a mean scatter of 1.89 $\mathrm{ppm}$. These values are comparable to and in some cases even better than those of the already existing GOSAT products, some of which have less valid data.

The first GOSAT-2 results using the FOCAL method are also quite promising, but further investigations, longer time series, and more correlative data sets are required for a quantitative assessment of the GOSAT-2-FOCAL data quality.

Overall, the FOCAL method has proven to be computationally fast and to produce ${\mathrm{XCO}}_{\mathrm{2}}$ results with similar accuracy as other, typically more time-consuming, retrieval algorithms. This is the case not only when applied to OCO-2 but also for GOSAT and GOSAT-2. FOCAL is therefore considered to be a good candidate algorithm for future satellite sensors producing large amounts of data, like the forthcoming European anthropogenic ${\mathrm{CO}}_{\mathrm{2}}$ Monitoring (CO2M) mission.

Data availability

The GOSAT-FOCAL V1.0 data set and the preliminary GOSAT-2-FOCAL data are available on request from the authors.

Author contributions

SN adapted the FOCAL method to GOSAT and GOSAT-2, generated the FOCAL data products, and performed the validation. MR developed the FOCAL method and provided the ${\mathrm{XCO}}_{\mathrm{2}}$ reference databases and the TCCON validation tools. JB provided the Python implementation used for the SC4C methane climatology from Schneising et al. (2019). MH provided the original Python implementation of FOCAL (OCO-2 version). ADN and YY provided the UoL and NIES GOSAT data products. HS provided information on GOSAT and GOSAT-2, especially the preliminary GOSAT-2 ILS.

The following co-authors provided TCCON data: MB, NMD, DGF, DWTG, FH, RK, IM, JN, HO, CP, JRP, DFP, MKS, KS, RS, YT, VAV, and TW.

All authors provided support in writing the paper.

Competing interests

The authors declare that they have no conflict of interest.

Acknowledgements

GOSAT and GOSAT-2 spectral data have been provided by JAXA and NIES. CarbonTracker CT2019 and CT-NRT.v2020-1 results were provided by NOAA ESRL (Boulder, Colorado, USA) from the website at http://carbontracker.noaa.gov (last access: 8 May 2021). ABSCO cross sections for ${\mathrm{CO}}_{\mathrm{2}}$ were provided by NASA and the ACOS/OCO-2 team. GMTED2010 topography data were provided by the US Geological Survey (USGS) and the National Geospatial-Intelligence Agency (NGA). We thank the European Centre for Medium-Range Weather Forecasts (ECMWF) for providing us with analysed meteorological fields (ERA5 data).

We thank the OCO-2 Science Team for the GOSAT ACOS Level 2 ${\mathrm{XCO}}_{\mathrm{2}}$ product obtained from https://oco2.gesdisc.eosdis.nasa.gov/data/GOSAT_TANSO_Level2/ACOS_L2_Lite_FP.9r/ (10.5067/VWSABTO7ZII4; last access: 16 October 2020). The SRON GOSAT ${\mathrm{XCO}}_{\mathrm{2}}$ data product has been obtained from the Copernicus Climate Data Store (https://cds.climate.copernicus.eu/, last assess: 15 October 2020).

We used TCCON GGG2014 ground-based validation data (see Table ). TCCON data from the Eureka and Izaña stations were provided by Kimberly Strong and Omaira E. Garcia, respectively. The Ascension Island TCCON station has been supported by the European Space Agency (ESA) under grant 4000120088/17/I-EF and by the German Bundesministerium für Wirtschaft und Energie (BMWi) under grants 50EE1711C and 50EE1711E. We thank the ESA Ariane tracking station at North East Bay, Ascension Island, for hosting and local support. The TCCON site at Réunion Island has been operated by the Royal Belgian Institute for Space Aeronomy with financial support since 2014 by the EU project ICOS-Inwire and the ministerial decree for ICOS (FR/35/IC1 to FR/35/C5), with local activities supported by LACy/UMR8105 – Université de La Réunion. The Paris TCCON site has received funding from Sorbonne Université, the French research centre CNRS, the French space agency CNES, and Région Île-de-France. The TCCON stations at Rikubetsu, Tsukuba, and Burgos are supported in part by the GOSAT series project. Local support for Burgos is provided by the Energy Development Corporation (EDC, Philippines). Nicholas M. Deutscher is funded by ARC Future Fellowship FT180100327. The Darwin and Wollongong TCCON stations are supported by ARC grants DP160100598, LE0668470, DP140101552, DP110103118, and DP0879468; Darwin receives additional support from NASA grants NAG5-12247 and NNG05-GD07G as well as technical assistance from the Australian Bureau of Meteorology. The TCCON stations Garmisch and Zugspitze have been supported by the European Space Agency (ESA) under grant 4000120088/17/I-EF and by the German Bundesministerium für Wirtschaft und Energie (BMWi) via the DLR under grant 50EE1711D as well as by the Helmholtz Society via the research programme ATMO.

Financial support

This work has received funding from JAXA (GOSAT and GOSAT-2 support, contracts 19RT000692 and JX-PSPC-527269), EUMETSAT (FOCAL-CO2M study, contract EUM/CO/19/4600002372/RL), the ESA (GHG-CCI+ project, contract 4000126450/19/I-NB), and the State and the University of Bremen. The article processing charges for this open-access publication were covered by the University of Bremen.

Review statement

This paper was edited by Joanna Joiner and reviewed by two anonymous referees.