Uncertainty analysis of total ozone derived from direct solar irradiance spectra in the presence of unknown spectral deviations

Abstract

We demonstrate the use of a Monte Carlo model to estimate the uncertainties in total ozone column (TOC) derived from ground-based direct solar spectral irradiance measurements. The model estimates the effects of possible systematic spectral deviations in the solar irradiance spectra on the uncertainties in retrieved TOC. The model is tested with spectral data measured with three different spectroradiometers at an intercomparison campaign of the research project “Traceability for atmospheric total column ozone” at Izaña, Tenerife on 17 September 2016. The TOC values derived at local noon have expanded uncertainties of 1.3 % (3.6 DU) for a high-end scanning spectroradiometer, 1.5 % (4.4 DU) for a high-end array spectroradiometer, and 4.7 % (13.3 DU) for a roughly adopted instrument based on commercially available components and an array spectroradiometer when correlations are taken into account. When neglecting the effects of systematic spectral deviations, the uncertainties reduce by a factor of 3. The TOC results of all devices have good agreement with each other, within the uncertainties, and with the reference values of the order of 282 DU during the analysed day, measured with Brewer spectrophotometer #183.

Introduction

Atmospheric ozone has been defined as an essential climate variable in the global climate observing system () of the World Meteorological Organization (WMO). Its long-term monitoring is necessary to document the expected recovery of the ozone layer due to the implementation of the Montreal Protocol () and its amendments on the protection of the ozone layer. Atmospheric ozone, first discovered by , protects humans, the biosphere, and infrastructure from adverse effects of ultraviolet (UV) radiation by shielding the Earth's surface from excessive radiation levels (). Since the 1970s, it is known that human-produced chlorofluorocarbons (CFCs) destroy atmospheric ozone () and lead to recurring massive losses of total ozone in the Antarctic in the form of the ozone hole (). Unprecedented ozone depletion has also been recently observed in the Arctic (), while in the midlatitudes, moderate ozone depletion has been observed (). The Montreal Protocol and its amendments have been successful in reducing the emission of ozone-depleting substances (). Nevertheless, recent studies give conflicting results with respect to the observation of the recovery of the ozone layer, and model projections have shown the recovery to not occur before the middle of the 21st century (). Therefore, careful monitoring of the thickness of the ozone layer with uncertainties of 1 % or less is crucial for verifying the successful implementation of the Montreal Protocol and the eventual recovery of the ozone layer to pre-1970s levels.

“Traceability for atmospheric total column ozone” (ATMOZ) was a 3-year project funded partly by the European Metrology Research Programme (EMRP) and the European Union (). The goal of this project was to produce traceable measurements of total ozone column (TOC) with uncertainties down to 1 % through a systematic investigation of the radiometric and spectroscopic aspects of the methodologies used in retrieval. Another objective of the project was to provide a comprehensive treatment of uncertainties in all parameters affecting the TOC retrievals using spectrophotometers. This paper presents an outcome of the work on studying the uncertainty in TOC obtained from spectral direct solar irradiance measurements, taking unknown spectral errors explicitly into account.

TOC can be determined from spectral measurements of direct solar UV irradiance (). We have developed a Monte Carlo (MC)-based model to estimate the uncertainties in the derived TOC values. One frequently overlooked problem with uncertainty evaluation is that the spectral data may hide systematic wavelength-dependent errors due to unknown correlations (). Omitting possible correlations may lead to underestimated uncertainties for derived quantities, since spectrally varying systematic errors typically produce larger deviations than uncorrelated noise-like variations that traditional uncertainty estimations predict. Complete uncertainty budgets for quantities measured are necessary to understand long-term environmental trends, such as changes in the stratospheric ozone concentration (e.g. ) and solar UV radiation (e.g. ).

Physically, spectral correlations may originate from lamps or other light sources used in calibrations. If their temperatures change, e.g. due to ageing or current setting, a spectral change in the form of Planck's radiation law is introduced. Non-linearity in the responsivity of a detector causes systematic differences between high and low measured values. The introduced spectrally systematic but unknown changes in irradiance may change the derived TOC values significantly, exceeding the uncertainties calculated, assuming that the uncertainty in irradiance behaves like noise. The presence of correlations in measurements can be seen in many ways. For example, problems have occurred when new ozone absorption cross sections have been used (). Derived ozone values may change significantly because different systematic errors are included in the different cross sections. Also, TOC estimated from a measured spectrum often depends on the wavelength region chosen, although the measurement region should not affect the result much.

In this paper, we introduce a new method for dealing with possible correlations in spectral irradiance data and analyse uncertainties in ozone retrievals for three different spectroradiometers used in a recent ATMOZ intercomparison campaign at Izaña, Tenerife, to demonstrate how it can be used in practice. One of the instruments is QASUME (), which is the reference UV spectroradiometer at the World Radiation Center (PMOD/WRC). The second one is an array-based high-quality spectroradiometer BTS2048-UV-S-WP (BTS) from Gigahertz-Optik (), operated by Physikalisch-Technische Bundesanstalt (PTB). The third one is an array-based spectroradiometer AvaSpec-ULS2048LTEC (AVODOR) from Avantes, operated by PMOD/WRC. The field of view of the spectroradiometers has been limited so that they measure the direct spectral irradiance of the Sun, excluding most of the indirect radiation from the remainder of the sky.

ATMOZ field measurement campaign and instrument description

The ATMOZ project arranged a field measurement campaign () that took place 12–30 September 2016 at the Izaña Atmospheric Observatory, a high-mountain Global Atmosphere Watch (GAW) station located at an altitude of 2.36 km above the sea level in Tenerife, Canary Islands, Spain ($\mathrm{28.3090}{}^{\circ }$ N, $\mathrm{16.4990}{}^{\circ }$ W). The measurement campaign was organized by the Spanish State Meteorological Agency (AEMET) and the PMOD/WRC for the intercomparison of TOC measured with different participating instruments, including Dobson and Brewer spectrophotometers and various spectroradiometers.

The focus of this paper is to study uncertainties in the TOC values derived from direct solar UV irradiance spectra. Total ozone column TOC is the vertical ozone profile ${\mathit{\rho }}_{{\mathrm{O}}_{\mathrm{3}}}\left(z\right)$ integrated over altitude as $$\mathrm{TOC}=\underset{z_{\mathrm{0}}}{\overset{z_{\mathrm{1}}}{\int }}{\mathit{\rho }}_{{\mathrm{O}}_{\mathrm{3}}}\left(z\right)\mathrm{d}z$$ from the station altitude $z_{\mathrm{0}}$ to the top-of-the-atmosphere altitude $z_{\mathrm{1}}$. We study the data measured during the day of 17 September 2016 with three different spectroradiometers using the ozone retrieval algorithm introduced in Sect. . Station pressure was monitored during the campaign and determined to be 772.8 hPa with a standard uncertainty of 1.3 hPa. The ozone and temperature profiles were measured with a sonde during the campaign and examples of them are shown in Fig. .

Temperature and ozone profiles as a function of the altitude measured with a sonde during the ATMOZ field measurement campaign.

[Figure omitted. See PDF]

Our ozone retrieval method uses one atmospheric layer to reduce computational complexity. With the single-layer model, the ozone absorption cross section is a function of the effective temperature, and the relative air mass is a function of the effective altitude of the ozone layer. Using the vertical ozone profile ${\mathit{\rho }}_{{\mathrm{O}}_{\mathrm{3}}}\left(z\right)$, the effective altitude $h_{\mathrm{eff}}=\mathrm{26}$ km $\pm$ 0.5 km of the ozone layer was estimated by integration over altitude: $$h_{\mathrm{eff}}={\displaystyle \frac{{\int }_{z_{\mathrm{0}}}^{z_{\mathrm{1}}}z{\mathit{\rho }}_{{\mathrm{O}}_{\mathrm{3}}}\left(z\right)\mathrm{d}z}{{\int }_{z_{\mathrm{0}}}^{z_{\mathrm{1}}}{\mathit{\rho }}_{{\mathrm{O}}_{\mathrm{3}}}\left(z\right)\mathrm{d}z}},$$ from the station altitude $z_{\mathrm{0}}$ to the top-of-the-atmosphere altitude $z_{\mathrm{1}}$. The corresponding effective temperature $T_{\mathrm{eff}}=\mathrm{228}$ K $\pm$ 1 K was estimated () as $$T_{\mathrm{eff}}={\displaystyle \frac{{\int }_{z_{\mathrm{0}}}^{z_{\mathrm{1}}}T\left(z\right){\mathit{\rho }}_{{\mathrm{O}}_{\mathrm{3}}}\left(z\right)\mathrm{d}z}{{\int }_{z_{\mathrm{0}}}^{z_{\mathrm{1}}}{\mathit{\rho }}_{{\mathrm{O}}_{\mathrm{3}}}\left(z\right)\mathrm{d}z}}.$$ The uncertainties stated for $h_{\mathrm{eff}}=\mathrm{26}$ km $\pm$ 0.5 km and $T_{\mathrm{eff}}=\mathrm{228}$ K $\pm$ 1 K are standard deviations. The values and uncertainties are estimated from the measured vertical profiles (Fig. ). The profiles are extended to 80 km using climatology.

The data sets measured with three different spectroradiometers were studied in this work. These spectroradiometers use different techniques to measure the spectral distribution of radiation. Monochromator-based spectroradiometers, such as QASUME, measure one nearly monochromatic wavelength band at a time, and thus measuring the full spectrum is relatively slow. On the other hand, they usually have significantly better stray light properties than array-based spectroradiometers, such as BTS and AVODOR, which image the full spectrum at once by dispersing the incoming radiation towards a photodiode array.

The QASUME spectroradiometer collects and guides the incoming radiation with input optics and a quartz fibre bundle to the entrance slit of a Bentham DM150 double monochromator (). One wavelength at a time is selected by rotating the two gratings of the double monochromator. Then, the monochromatic signal is measured with a photomultiplier tube. QASUME is usually operated in global spectral irradiance mode (), but during the campaign it was equipped with a collimator tube with a full opening angle of 2.5${}^{\circ }$, allowing the measurement of direct solar spectral irradiance (). The measurement range of QASUME during the campaign was limited to 250–500 nm with a step interval of 0.25 nm, so that one spectrum was measured every 15 min. To ensure stable outdoor measurements, the double monochromator of QASUME was mounted inside a temperature-controlled weatherproof box ().

The BTS spectroradiometer utilizes a stationary grating and a back-thinned cooled CCD array detector, mounted in a Czerny–Turner configuration (). To measure direct solar spectrum, BTS was equipped with a collimator tube with a full opening angle of 2.8${}^{\circ }$ designed by PTB, and it uses an internal filter wheel system with eight filter positions together with a specific measurement routine to reduce stray light. BTS was mounted on a solar tracker, EKO STR-32G by EKO Instruments Co., Ltd., with pointing accuracy better than 0.01${}^{\circ }$. A weatherproof housing with temperature control allows BTS to operate at ambient temperatures from $-$25 to 50 ${}^{\circ }$C. During the ATMOZ campaign, the housing temperature of BTS was measured to be stable within 0.1 ${}^{\circ }$C (). The measurement range of BTS was 200–430 nm with a step size of 0.2 nm during the campaign. One spectrum was measured every 45 s.

AVODOR spectroradiometer has a stationary grating and a back-thinned cooled CCD array detector in a Czerny–Turner configuration. AVODOR measures the spectrum from 200 to 540 nm with a step size of 0.14 nm in the UV region. During the ATMOZ campaign, the field of view of AVODOR was limited to 1.5${}^{\circ }$ by a commercial collimator tube used, J1004-SMA by CMS Ing.Dr.Schreder GmbH. The spectral range of AVODOR was limited between 295 and 345 nm by a combination of two solar blind filters to reduce stray light from the visible and infrared parts of the solar spectrum. The solar blind filters were mounted between the collimator tube and the fibre entrance of the spectroradiometer. One spectrum was measured every 30 s.

The slit functions of the three spectroradiometers shown in Fig.  were measured with lasers before the measurement campaign. They are needed when fitting the modelled spectra at the Earth's surface to the measured spectra. In addition, it is important to note the different wavelength steps of the data: 0.25 nm for QASUME, 0.2 nm for BTS, and 0.14 nm for AVODOR. The wavelength steps in the spectral data affect the magnitude of the uncertainties in TOC created by spectrally random components. In our full-spectrum TOC retrieval, the number of data points $n$, which is smaller with a larger wavelength step interval, affects uncertainties with a factor of $\mathrm{1}/\sqrt{n}$, as demonstrated in .

Slit functions measured with narrowband lasers for the spectroradiometers used in the ATMOZ field measurement campaign. The laser wavelengths are stated in the legend.

[Figure omitted. See PDF]

Brewer MkIII spectrophotometers used as reference devices for ozone measurements established by the International Meteorological Organization, the Commission for Instruments and Methods of Observation (CIMO) (), also measure the spectral irradiance at UV region, but using four narrow wavelength bands at 310.1, 313.5, 316.8, and 320.0 nm (). The Brewer MkIII instruments solve absolute TOC values by comparing the logarithms of ratios of count rates between four wavelength channels, i.e. using the double ratio technique. Determining the TOC using the double ratio method is invariant for spectral deviations which have the same relative magnitude at all wavelengths, i.e spectrally constant deviations.

Our full-spectrum retrieval method performs averaging in the wavelength domain, whereas the Brewer spectrophotometer does it in the time domain. A Brewer can measure up to tens of seconds to get millions of photons, so that the photon noise reduces to a level of 0.1 %. At this low noise level, it is not critical that only four wavelengths are used. As Brewer instruments are well-known and widely used, we also compare TOC results obtained using our full-spectrum retrieval method and the spectroradiometers to those measured with Brewer #183 during the same day.

Atmospheric model

In this study, we use an atmospheric ozone retrieval algorithm that is in many aspects similar to that in the article by . The relationship between the spectral irradiance $E_{\mathrm{s}}\left(\mathit{\lambda }\right)$ at the Earth's surface and the extraterrestrial solar spectrum $E_{\mathrm{ext}}\left(\mathit{\lambda }\right)$ is based on Beer–Lambert–Bouguer absorption law () as $$\begin{array}{cc}{\displaystyle }{E}_{\mathrm{s}}\left(\mathit{\lambda }\right)=& {\displaystyle }c\cdot E_{\mathrm{ext}}\left(\mathit{\lambda }\right)\cdot \exp \left[-{\mathit{\alpha }}_{{\mathrm{O}}_{\mathrm{3}}}\left(\mathit{\lambda },T_{\mathrm{eff}}\right)\cdot \mathrm{TOC}\cdot m_{\mathrm{TOC}}\right.\\ & {\displaystyle }& {\displaystyle }\left.-{\mathit{\tau }}_{\mathrm{R}}\left(\mathit{\lambda },P_{\mathrm{0}},z_{\mathrm{0}},\mathit{\varphi }\right)\cdot m_{\mathrm{R}}-{\mathit{\tau }}_{\mathrm{AOD}}\left(\mathit{\lambda }\right)\cdot m_{\mathrm{AOD}}\right],\end{array}$$ where ${\mathit{\alpha }}_{{\mathrm{O}}_{\mathrm{3}}}\left(\mathit{\lambda },T_{\mathrm{eff}}\right)$ is the ozone absorption cross section at the effective ozone temperature $T_{\mathrm{eff}}$, ${\mathit{\tau }}_{\mathrm{R}}\left(\mathit{\lambda },P_{\mathrm{0}},z_{\mathrm{0}},\mathit{\varphi }\right)$ is the Rayleigh scattering optical depth, which depends on the station pressure $P_{\mathrm{0}}$, the station altitude $z_{\mathrm{0}}$, and the geographic latitude $\mathit{\varphi }$. The QASUME-FTS (Fourier transform spectroscopy) data set by was used as the extraterrestrial spectrum $E_{\mathrm{ext}}\left(\mathit{\lambda }\right)$. Parameter $c$ is a scaling factor set as a free parameter to compensate for spectrally constant deviations.

The relative air mass of the ozone layer with the Earth's curvature taken into account can be expressed as $$m_{\mathrm{TOC}}={\displaystyle \frac{\mathrm{1}}{\cos \left[\arcsin \left(\frac{R}{R+h_{\mathrm{eff}}}\cdot \sin \mathit{\theta }\right)\right]}},$$ where $\mathit{\theta }$ is the incident solar zenith angle at the observing site that is a function of the local time, date, and geographic coordinates (). Parameter $h_{\mathrm{eff}}$ is the altitude of the ozone layer from the ground, and $R$ is the radius of the Earth. As the ozone and other molecules creating scattering are distributed at different altitudes, we calculate the relative air mass factor $m_{\mathrm{R}}$ for Rayleigh scattering at the altitude of 5 km () and approximate the relative air mass factor of aerosols so that $m_{\mathrm{AOD}}\approx m_{\mathrm{R}}$ (). The temperature dependence of ${\mathit{\alpha }}_{{\mathrm{O}}_{\mathrm{3}}}\left(\mathit{\lambda },T_{\mathrm{eff}}\right)$ between 203 and 253 K () was interpolated by a second-degree polynomial at each wavelength (): $${\mathit{\alpha }}_{{\mathrm{O}}_{\mathrm{3}}}\left(\mathit{\lambda },T_{\mathrm{eff}}\right)=a_{\mathrm{1}}\left(\mathit{\lambda }\right)T_{\mathrm{eff}}^{\mathrm{2}}+a_{\mathrm{2}}\left(\mathit{\lambda }\right)T_{\mathrm{eff}}+a_{\mathrm{3}}\left(\mathit{\lambda }\right).$$

We take the Rayleigh scattering optical depth into account using the state-of-the-art model by . The aerosol optical depth (AOD) is approximated from the Ångström AOD model () as $${\mathit{\tau }}_{\mathrm{AOD}}\left(\mathit{\lambda }\right)=\mathit{\beta }\cdot {\left({\displaystyle \frac{\mathit{\lambda }}{\mathrm{1}\mathrm{\mu }\mathrm{m}}}\right)}^{-\mathit{\alpha }},$$ where constant $\mathit{\alpha }\approx \mathrm{1.4}$ is the Ångström coefficient at typical atmospheric conditions and $\mathit{\beta }\ge \mathrm{0}$ is the Ångström turbidity coefficient.

The model spectrum $E_{\mathrm{s}}\left(\mathit{\lambda }\right)$ at the Earth's surface, convolved by the slit function of the spectroradiometer as shown in Fig. , is fitted with parameters TOC, $\mathit{\beta }$, and $c$ to the measured ground-based spectrum $E\left(\mathit{\lambda }\right)$. The absolute TOC level obtained depends on the fitting method used. We used a least squares fitting method () with trust-region optimization with the Matlab function “lsqnonlin”: $$S=\sum _{i=\mathrm{1}}^{n}w\left({\mathit{\lambda }}_i\right)\cdot {\left[E_{\mathrm{s}}\left({\mathit{\lambda }}_i\right)-E\left({\mathit{\lambda }}_i\right)\right]}^{\mathrm{2}},$$ where $S$ is the sum of the squared residuals to be minimized, and $w\left({\mathit{\lambda }}_i\right)$ is the weight for each point measured. Index $i=\mathrm{1},\mathrm{2},\mathrm{\dots },n$ runs over the wavelengths of the spectral measurements.

Figure  presents examples of measured and modelled spectra for the spectroradiometers used in this work. As can be seen, the signal-to-noise ratios and stray light properties of the devices differ significantly among different spectroradiometers. All spectra measured with QASUME are practically noiseless above ${\mathrm{10}}^{-\mathrm{6}}$ W m${}^{-\mathrm{2}}$ nm${}^{-\mathrm{1}}$, resulting in a dynamic range of approximately 4 orders of magnitude. The dynamic range for BTS is approximately 2 orders of magnitude and less than 2 orders of magnitude for AVODOR.

For QASUME, we use the relative least squares fitting method (RLS) minimization with $w\left(\mathit{\lambda }\right)=E{\left(\mathit{\lambda }\right)}^{-\mathrm{2}}$, as QASUME does not suffer from stray light and RLS fitting has been used in the past for monochromator-based spectroradiometers, e.g. by  . These least squares fitting selections are discussed in more detail in Appendix .

To minimize the effect of stray light, we use the absolute least squares fitting method, also known as the ordinary least squares fitting method (OLS), with $w\left(\mathit{\lambda }\right)=\mathrm{1}$ for BTS and AVODOR, as this selection is less affected by the lowest irradiance levels where the stray light and noise are dominant. As shown in Appendix A, using OLS introduces an offset, approximately 1 % in these measurements, to the retrieved TOC values. We take this into account by analysing the results at noon that are also less influenced by stray light when using RLS, and make a correction to the OLS results: $${\mathrm{TOC}}_{\mathrm{UTC},\mathrm{OLS},\mathrm{c}}={\displaystyle \frac{{\mathrm{TOC}}_{\mathrm{noon},\mathrm{RLS}}}{{\mathrm{TOC}}_{\mathrm{noon},\mathrm{OLS}}}}\cdot {\mathrm{TOC}}_{\mathrm{UTC},\mathrm{OLS}}.$$ After this correction, results of all instruments, QASUME, BTS, and AVODOR are comparable.

The shortest fitting wavelength for the spectroradiometers in this work was selected to be 300 nm, since the typical stray light compensation methods are not effective below 300 nm (). The upper fitting wavelength limit was set to 340 nm with all three spectroradiometers, as the ozone absorption is not effective above that wavelength. Due to the relatively large bandwidths of the spectroradiometers (Fig. ), calculations before the convolution and the convolution itself were carried out over a wider range 295–345 nm.

To see whether a global optimum is achieved with our atmospheric ozone retrieval method, we varied the initial guess values of TOC from 10 to 700 DU, $\mathit{\beta }$ from 0 to 0.5, and $c$ from 0 to 100. Within the ranges stated, the free parameters always converged to the same final values regardless of the initial guess values.

Examples of fitting the atmospheric model to the direct ground-based solar UV spectra between 300 and 340 nm for QASUME (a–b), BTS (c–d), and AVODOR (e–f). In figures on the left-hand side, the coloured symbols indicate measured spectra, and the black solid curves indicate modelled spectra. Figures on the right-hand side show the relative spectral residuals of the fits. In (a) the abbreviation DR refers to the dynamic range of QASUME data used in the least squares fitting.

[Figure omitted. See PDF]

Uncertainty estimation

Uncertainty model

In uncertainty analysis, the combined uncertainties are calculated with the square sum of the standard deviations of the components; i.e. their variances are summed up. If correlations of uncertainties are known, they should be taken into account. This can be carried out with the methods defined in the Guide to the Expression of Uncertainty in Measurement (). In this paper, we do this for all uncertainty components, where the mechanism of contributing to the uncertainty in TOC is known. However, with some of the components, we do not exactly know the mechanisms leading to correlations. With these uncertainties, we estimate the effects that possible correlations might have using a newly developed MC model described in .

In our MC model, possible systematic deviations within uncertainties are reproduced using a cumulative Fourier series $$\mathit{\delta }\left(\mathit{\lambda }\right)=\sum _{i=\mathrm{0}}^N{\mathit{\gamma }}_i{f}_i\left(\mathit{\lambda }\right),$$ with sinusoidal base functions, shown in Fig.  as $$f_i\left(\mathit{\lambda }\right)=\sqrt{\mathrm{2}}\cdot \sin \left[i\left(\mathrm{2}\mathit{\pi }{\displaystyle \frac{\mathit{\lambda }-{\mathit{\lambda }}_{\mathrm{1}}}{{\mathit{\lambda }}_{\mathrm{2}}-{\mathit{\lambda }}_{\mathrm{1}}}}\right)+{\mathit{\varphi }}_i\right],$$ where index $i=\mathrm{1},\mathrm{2},\mathrm{\dots },N$ depicts the order of complexity of the deviation (); ${\mathit{\lambda }}_{\mathrm{1}}$ and ${\mathit{\lambda }}_{\mathrm{2}}$ limit the wavelength range of the analysis. For calculations before the convolution due to slit function, this range was set to 295–345 nm. Otherwise, the results at the ends of the range 300–340 nm would be distorted due to incomplete convolution. This concerns the uncertainty in the extraterrestrial spectrum and the ozone absorption cross section. After the convolution, actual fitting of the modelled spectra to the measured spectra was carried out at 300–340 nm, and this range was also used in the uncertainty analysis of the components related to the measured spectra. The phase ${\mathit{\varphi }}_i$ of the base function is an equally distributed MC variable between 0 and $\mathrm{2}\mathit{\pi }$. In addition, $f_{\mathrm{0}}\left(\mathit{\lambda }\right)=\mathrm{1}$ is used to account for constant offset. The weights ${\mathit{\gamma }}_i$ for the base functions are selected in an $N$-dimensional spherical coordinate system () in such a way that the variance of the final deviation function always equals unity. In practice, this means that the weights ${\mathit{\gamma }}_i$ are generated by scaling the random variables $Y_i\sim \mathcal{N}\left(\mathrm{0},\mathrm{1}\right)$ as $${\mathit{\gamma }}_i={\displaystyle \frac{Y_i}{\sqrt{Y_{\mathrm{0}}^{\mathrm{2}}+Y_{\mathrm{1}}^{\mathrm{2}}+\mathrm{\dots }+Y_N^{\mathrm{2}}}}}.$$

The complete $N$-dimensional system requires orthogonal base functions, such as full periods of sine functions, to allow an arbitrary shape of deviation function $\mathit{\delta }\left(\mathit{\lambda }\right)$ with unity variance. It is possible to use other orthogonal sets of functions, such as Chebyshev polynomials instead of sinusoidal base functions, but that would involve more complicated mathematics. This is discussed in more detail in Appendix .

First three base functions with unity variance, $f_{\mathrm{1}}$ and $f_{\mathrm{2}}$ plotted with the phases ${\mathit{\varphi }}_i=\mathit{\pi }$ and ${\mathit{\varphi }}_i=\mathit{\pi }/\mathrm{2}$.

[Figure omitted. See PDF]

The deviation functions obtained with the cumulative Fourier series are used to distort the measured spectra $E\left(\mathit{\lambda }\right)$ as $$E_{\mathrm{e}}\left(\mathit{\lambda }\right)=\left[\mathrm{1}+\mathit{\delta }\left(\mathit{\lambda }\right)u\left(\mathit{\lambda }\right)\right]E\left(\mathit{\lambda }\right),$$ where $u\left(\mathit{\lambda }\right)$ is the relative standard uncertainty in the spectrum. The corresponding spectral deviation is applied separately to other factors of Eq. (), i.e. the extraterrestrial solar spectrum, ozone absorption cross section, and Rayleigh scattering.

Variances of the TOC values obtained by varying the weights ${\mathit{\gamma }}_i$ and the phase terms ${\mathit{\varphi }}_i$ give the desired uncertainties. Figure a presents how the uncertainty induced by deviation in spectral irradiance $E\left(\mathit{\lambda }\right)$ (circles) changes with increasing $N$. Each standard uncertainty in TOC in Fig. a was estimated from the MC results obtained by running the TOC retrieval 1000 times so that the phases ${\mathit{\varphi }}_i$ and the weights ${\mathit{\gamma }}_i$ of the base functions were independent at each round. Retrieved TOC deviations resemble the Gaussian distribution when the order of complexity of the deviation function is $N\ge \mathrm{2}$ as illustrated in Fig. b and c. As we can see, full correlation with the base function $f_{\mathrm{0}}\left(\mathit{\lambda }\right)$ at $N=\mathrm{0}$ causes a negligible uncertainty to TOC. The maximum at $N=\mathrm{1}$ gives uncertainty for an unfavourable case of correlations with base functions $f_{\mathrm{0}}\left(\mathit{\lambda }\right)$ and $f_{\mathrm{1}}\left(\mathit{\lambda }\right)$. Cases $N=\mathrm{80}$ for QASUME, $N=\mathrm{100}$ for BTS, or $N=\mathrm{125}$ for AVODOR correspond to the Nyquist criterion ($N=n/\mathrm{2}$) with base functions and give the uncertainty with no spectral correlations (only random noise). The obtained TOC value is affected most by spectral distortion that mimics the spectral shape of the ozone absorption. The first combination of constant offset and one sinusoidal function with two sign changes within the region of interest is closest to this extreme.

The ozone absorption cross section ${\mathit{\alpha }}_{{\mathrm{O}}_{\mathrm{3}}}\left(\mathit{\lambda },T_{\mathrm{eff}}\right)$ is a direct multiplier of TOC, and thus the uncertainty in TOC is directly proportional to the deviations in the ozone absorption cross section. The uncertainty in TOC arising from the spectral deviation in ${\mathit{\alpha }}_{{\mathrm{O}}_{\mathrm{3}}}\left(\mathit{\lambda },T_{\mathrm{eff}}\right)$ is plotted as crosses in Fig. a as a function of increasing $N$. The ozone absorption cross section, from the Serdyuchenko–Gorshelev data set (), has a wavelength step size of 0.01 nm, and thus the standard uncertainty of 0.05 % at the Nyquist criterion $N=\mathrm{2500}$ is outside the range displayed in Fig. a. Unlike the negligible effect of full spectral correlation in the spectral irradiance $E\left(\mathit{\lambda }\right)$ in TOC, full correlation ($N=\mathrm{0}$) in the ozone absorption cross section produces approximately the same uncertainty as the unfavourable correlation ($N=\mathrm{1}$). Generally, these results cannot be known before the analysis is carried out, using a method that does not have any internal limitation to the shape of the deviation function $\mathit{\delta }\left(\mathit{\lambda }\right)$. In some other cases, the uncertainty extreme appears at higher $N$ values, e.g. $N=\mathrm{3}$, noted for correlated colour temperature by .

One major problem in uncertainty estimation is that typically many of the correlations in spectral irradiance data are unknown. Figure a can be used to find limits for the uncertainties assuming different correlation scenarios. In the analysis carried out in this paper, we estimate for each uncertainty component which kind of correlation it most likely has. For this, we divide the correlation into three categories, full, unfavourable, and random and estimate fractions on the assortment of these correlations. “Full” indicates that relative deviation is wavelength independent, such as with distance setting in spectral irradiance measurements. “Random” indicates no correlation between spectral values. As can be seen in Fig. a, the uncertainty caused by noise depends on the Nyquist criterion, which increases with a smaller number of base functions. “Unfavourable” indicates an unknown deviation with systematic spectral structure that produces a large deviation in TOC.

(a) Standard uncertainties in TOC at local noon as a function of the order of complexity $N$ for QASUME spectroradiometer with 1 % deviation in spectral irradiance $E\left(\mathit{\lambda }\right)$ plotted as circles and 1 % deviation in ozone absorption cross section ${\mathit{\alpha }}_{{\mathrm{O}}_{\mathrm{3}}}\left(\mathit{\lambda },T_{\mathrm{eff}}\right)$ plotted as crosses. The distributions of TOC values arising from 1 % deviation in $E\left(\mathit{\lambda }\right)$ with the order of complexity of $N=\mathrm{2}$ in (b) and $N=\mathrm{80}$ in (c). The black solid curve denotes a Gaussian distribution.

[Figure omitted. See PDF]

Uncertainty budgets for spectroradiometers

Uncertainty budgets of the direct solar spectral irradiance measurements for QASUME, BTS, and AVODOR are presented in Tables –. The tables also state fractions that we estimate for the three correlation types introduced for each component. The uncertainties due to radiometric calibration include factors such as the uncertainty in the standard lamp used and the additional uncertainty due to noise and alignment. QASUME has been validated using various methods; thus the uncertainty due to calibration is low, at 0.55 % (). For QASUME and BTS, we assume the correlations to be equally distributed between full correlation, unfavourable correlation, and random correlation (). Spectra measured with AVODOR are significantly noisier; thus half of the calibration uncertainty is associated with the random component. Values for the instability of the calibration lamp are based on long-term monitoring. The lamp irradiances have been noted to gradually drop, with no significant wavelength structure within the wavelength region concerned. Non-linearity values are estimations of the operators of the devices. Non-linearity is typically manifested so that the responsivity of the device changes gradually from high readings to low readings. This can cause significant changes to the TOC values; thus we assume the correlation type to be unfavourable. Uncertainties due to device stability and temperature dependence are based on long-term monitoring. The changes have been found to be wavelength independent in the region concerned; thus full correlation is assumed. Noise is the average standard deviation of typical measurements at noon over the wavelength region concerned. The wavelength scales of the devices have been checked using emission lines of gas discharge lamps. The uncertainty values given are the estimated standard deviations of the possible remaining errors after corrections. Wavelength error can introduce a significant change in TOC, because it introduces an error in the form of the derivative of the spectral irradiance. Thus, unfavourable correlation is assumed. Most of the uncertainty components are slightly wavelength dependent, but to simplify simulations, average uncertainty values are used over the wavelength range between 300 and 340 nm.

Uncertainties in the measurement for the QASUME spectroradiometer.

QASUME	Standard	Correlation fraction
Source of uncertainty	uncertainty	Full	Unfavourable	Random
in measured $E\left(\mathit{\lambda }\right)$	%
Radiometric calibration	0.55	$\mathrm{1}/\sqrt{\mathrm{3}}$	$\mathrm{1}/\sqrt{\mathrm{3}}$	$\mathrm{1}/\sqrt{\mathrm{3}}$
250 W lamp stability	0.14	1	0	0
Non-linearity	0.25	0	1	0
Stability	0.60	1	0	0
Temperature dependence	0.20	1	0	0
Measurement noise	0.20	0	0	1
Wavelength shift	0.10	0	1	0
Combined uncertainty ($k=\mathrm{1}$)	0.91 %	0.72 %	0.42 %	0.38 %

Uncertainties in the measurement for the BTS spectroradiometer ().

BTS	Standard	Correlation fraction
Source of uncertainty	uncertainty	Full	Unfavourable	Random
in measured $E\left(\mathit{\lambda }\right)$	%
Radiometric calibration	0.80	$\mathrm{1}/\sqrt{\mathrm{3}}$	$\mathrm{1}/\sqrt{\mathrm{3}}$	$\mathrm{1}/\sqrt{\mathrm{3}}$
250 W lamp stability	0.20	1	0	0
Non-linearity	0.40	0	1	0
Stability	0.80	1	0	0
Temperature dependence	0.10	1	0	0
Measurement noise	0.20	0	0	1
Wavelength shift	0.10	0	1	0
Combined uncertainty ($k=\mathrm{1}$)	1.24 %	0.95 %	0.62 %	0.50 %

Uncertainties in the measurement for AVODOR spectroradiometer.

AVODOR	Standard	Correlation fraction
Source of uncertainty	uncertainty	Full	Unfavourable	Random
in measured $E\left(\mathit{\lambda }\right)$	%
Radiometric calibration	2.50	1/2	1/2	$\mathrm{1}/\sqrt{\mathrm{2}}$
250 W lamp stability	0.14	1	0	0
Non-linearity	0.50	0	1	0
Stability	0.60	1	0	0
Temperature dependence	0.20	1	0	0
Measurement noise	1.30	0	0	1
Wavelength shift	0.10	0	1	0
Combined uncertainty ($k=\mathrm{1}$)	2.94 %	1.41 %	1.35 %	2.19 %

Uncertainty budget for total ozone column

Table  presents an uncertainty budget for TOC that would be obtained with the high-accuracy QASUME spectroradiometer at local noon. All major uncertainty components are listed and estimated. The uncertainty components divided into the three correlation types have been analysed with the new model. The other components in Table  have been solved using traditional MC modelling because the mechanism for the uncertainty propagating to TOC is known.

An example uncertainty budget for QASUME spectroradiometer measured at local noon on the clear day of 17 September 2016. The last column states the standard deviations $u\left(\mathrm{TOC}\right)$ corresponding to each individual uncertainty component for TOC $=$ 284 DU retrieved from the QASUME spectrum using the spectral range of 300–340 nm at the solar zenith angle of 26.35${}^{\circ }$. The stated expanded uncertainty, $U\left(\mathrm{TOC}\right)=\mathrm{3}$.6 DU, was obtained by multiplying the combined standard uncertainty with a coverage factor $k=\mathrm{2}$.

	Standard uncertainty		Correlation fraction
Source of uncertainty	in $E\left(\mathit{\lambda }\right)$	in exponent	Full	Unfavourable	Random	$u\left(\mathrm{TOC}\right)$
	%	%				DU
Measurement
Radiometric calibration	0.55		$\mathrm{1}/\sqrt{\mathrm{3}}$	$\mathrm{1}/\sqrt{\mathrm{3}}$	$\mathrm{1}/\sqrt{\mathrm{3}}$	0.44
250 W lamp stability (1 year)	0.14		1	0	0	0.00
Non-linearity	0.25		0	1	0	0.33
Stability	0.60		1	0	0	0.00
Temperature dependence	0.20		1	0	0	0.00
Measurement noise	0.20		0	0	1	0.06
Wavelength shift	0.10		0	1	0	0.13
Uncertainties related to $E\left(\mathit{\lambda }\right)$
Extraterrestrial spectrum ()	1.00		$\mathrm{1}/\sqrt{\mathrm{3}}$	$\mathrm{1}/\sqrt{\mathrm{3}}$	$\mathrm{1}/\sqrt{\mathrm{3}}$	0.95
Uncertainties related to exponent of Eq. ()
${\mathrm{O}}_{\mathrm{3}}$ cross section ()		1.5	0.23	0.23	0.95	1.41
Rayleigh scattering ()		0.1	$\mathrm{1}/\sqrt{\mathrm{3}}$	$\mathrm{1}/\sqrt{\mathrm{3}}$	$\mathrm{1}/\sqrt{\mathrm{3}}$	0.09
${\mathrm{O}}_{\mathrm{3}}$ layer altitude of 26 km, $u=\mathrm{0.5}$ km		${}^{\mathrm{a}}$				0.01
Rayleigh layer altitude of 5 km, $u=\mathrm{0.5}$ km		${}^{\mathrm{b}}$				0.00
Temperature of ${\mathrm{O}}_{\mathrm{3}}$ cross section at 228 K, $u=\mathrm{1}$ K		${}^{\mathrm{c}}$				0.28
Station pressure of 772.8 hPa, $u=\mathrm{1.3}$ hPa		${}^{\mathrm{d}}$				0.05
					$U\left(\mathrm{TOC}\right)$	3.6

${}^{\mathrm{a}}$ Air mass $m_{\mathrm{TOC}}$ varies as a function of the altitude of the ${\mathrm{O}}_{\mathrm{3}}$ layer. ${}^{\mathrm{b}}$ Air mass $m_{\mathrm{R}}$ varies as a function of the altitude of the Rayleigh scattering layer. ${}^{\mathrm{c}}$ ${\mathrm{O}}_{\mathrm{3}}$ cross section varies as a function of temperature. ${}^{\mathrm{d}}$ Rayleigh scattering depends on the station pressure.

The uncertainties produced in TOC were obtained separately for all components by setting other uncertainties to zero. Division of the correlation to the three categories introduced are stated for each row as fractions $r_{\mathrm{full}}$, $r_{\mathrm{unfav}}$, and $r_{\mathrm{random}}$. For example, the ground-based spectrum $E\left(\mathit{\lambda }\right)$ is deviated with the three correlation components as $$\begin{array}{cc}{\displaystyle }{E}_{\mathrm{e}}\left(\mathit{\lambda }\right)=& {\displaystyle }\left(\mathrm{1}+u{r}_{\mathrm{full}}{f}_{\mathrm{0}}\left(\mathit{\lambda }\right)\right)\left(\mathrm{1}+u{r}_{\mathrm{unfav}}\sum _{i=\mathrm{0}}^{\mathrm{1}}{\mathit{\gamma }}_i^{\prime }{f}_i\left(\mathit{\lambda }\right)\right)\\ & {\displaystyle }& {\displaystyle }\cdot \left(\mathrm{1}+u{r}_{\mathrm{random}}\sum _{j=\mathrm{0}}^N{\mathit{\gamma }}_j^{\prime \prime }{f}_j\left(\mathit{\lambda }\right)\right)E\left(\mathit{\lambda }\right),\end{array}$$ where ${\mathit{\gamma }}_i^{\prime }$ and ${\mathit{\gamma }}_j^{\prime \prime }$ are independent MC variables generated using Eq. ().

It is worth noting that not all uncertainty components affect the spectrum $E\left(\mathit{\lambda }\right)$ directly but via the exponent of Eq. (). Corresponding formulas are used to evaluate the effect of uncertainties in extraterrestrial solar spectrum, ozone absorption cross section, and Rayleigh scattering. The last column in Table  states the standard uncertainties produced by each uncertainty component with the assumed fractions, calculated with an irradiance spectrum measured at local noon with QASUME (). The expanded uncertainty in the TOC, obtained as the square sum of the individual components and multiplied with coverage factor $k=\mathrm{2}$, for this spectral measurement, is 3.6 DU (1.3 %).

The QASUME spectroradiometer has a combined measurement standard uncertainty of 0.91 % () arising from the uncertainty components explained in Sect. . The uncertainty components stated are typical in solar UV spectral irradiance measurements (). Division of the radiometric calibration uncertainty to equal fractions of $\mathrm{1}/\sqrt{\mathrm{3}}$ is based on typical spectral correlations noted in intercomparisons (). The lamp data obtained from national standard laboratories are highly accurate but also typically spectrally correlated. Due to very low noise, elements such as interpolation functions, offsets, and slopes are present in the data. When the calibration is transferred further, uncertainty increases due to noise, and correlations reduce. We thus assume that in this high accuracy calibration, there are equal amounts of fully correlated, unfavourably correlated, and uncorrelated uncertainties.

For $E_{\mathrm{ext}}\left(\mathit{\lambda }\right)$, we use QASUME-FTS (). We assume the correlation to be similar to a standard lamp, thus containing equal fractions of full, unfavourable, and random correlations. The QASUME-FTS is provided in air wavelengths with a step size of 0.01 nm. Otherwise, the wavelength shift due to the vacuum–air interface should be corrected from the extraterrestrial spectrum.

As the reference ozone absorption cross section, the Serdyuchenko–Gorshelev data set given in air wavelengths with a step size of 0.01 nm was used with 1.5 % standard uncertainty (). The systematic and random uncertainties in the Serdyuchenko–Gorshelev data set are given separately (). We further estimate that the systematic uncertainty may include equal fractions of fully correlated and unfavourably correlated uncertainty. Thus, according to that, we use the fractions of 0.23 for full, 0.23 for unfavourable, and 0.95 for random correlations. Full correlation with a fraction of 0.23 produces a standard uncertainty of 0.96 DU, unfavourable correlation with a fraction of 0.23 produces a standard uncertainty of 1.01 DU, and random correlation with a fraction of 0.95 produces a standard uncertainty of 0.22 DU. Altogether, the ozone cross section causes an uncertainty of 1.41 DU to TOC and is thus the dominating component in the uncertainty. If the fractions of correlations are not equally distributed between full and unfavourable, uncertainty in TOC does not change significantly from 1.41 DU. Fractions of 0.31 for full, 0 for unfavourable (or fractions of 0 for full, 0.31 for unfavourable), and 0.95 for random correlations cause an uncertainty of 1.33 DU (or 1.43 DU) in TOC.

For components a–d in Table , the mechanism of contributing to the uncertainty in TOC is known. We know the standard uncertainty in the ${\mathrm{O}}_{\mathrm{3}}$ layer altitude of 26 km to be $u=\mathrm{0.5}$ km, so we vary the altitude accordingly and note the variance of the resulting TOC. Rayleigh scattering and aerosols are set at the altitude of 5 km $\pm \mathrm{0.5}$ km, which influences the relative air mass $m_{\mathrm{R}}\approx m_{\mathrm{AOD}}$ (). This component causes negligible uncertainty to TOC. To calculate ${\mathit{\tau }}_{\mathrm{R}}\left(\mathit{\lambda },P_{\mathrm{0}},z_{\mathrm{0}},\mathit{\varphi }\right)$, we use a model by () with 0.1 % uncertainty with equal estimated fractions of correlation types. The correlation has been obtained by studying how this data deviates from the model by . The ozone and temperature profiles were measured with a sonde during the campaign and based on the profiles, the effective altitude of the ozone layer was at 26 km $\pm \mathrm{0.5}$ km at the effective temperature of 228 K $\pm \mathrm{1}$ K. The effect on TOC was obtained by randomly varying the altitude using a Gaussian uncertainty distribution. The same applies to air pressure, which was 772.8 hPa with a standard uncertainty of 1.3 hPa. The effect of temperature on TOC was obtained by randomly varying the temperature with its standard uncertainty of 1 K. Varying the temperature systematically changes the spectral ozone absorption cross section according to Eq. ().

Stray light that affects TOC results at large solar zenith angles (Appendix ) has not been accounted for in the uncertainty analysis due to a lack of information. Proper correction and estimation of the uncertainty due to stray light would require the stray light correction matrix to be measured. The effect of stray light, typical for measurements with array spectroradiometers, was reduced from the TOC results by fitting BTS and AVODOR spectra with the OLS method. Then, these TOC values were corrected to be compatible with the RLS results by Eq. (). The correction factor involves a standard uncertainty, which is 0.1 % (0.28 DU) for BTS and 1.1 % (3.10 DU) for AVODOR.

Results and discussion

The calculated TOC values obtained with the three different spectroradiometers on 17 September 2016 are presented in Fig. . Expanded uncertainties in the TOC values calculated are stated in DU as error bars. Measurement results of Brewer spectrophotometer #183 used as a reference in the intercomparison have been included in Fig.  as well. Looking at the absolute values of TOC in Fig. , we may conclude that the results of QASUME and Brewer #183 are in excellent agreement. Also, the TOC values estimated for BTS and AVODOR are in agreement with Brewer #183 within uncertainties.

Total ozone columns (TOC) derived from the solar UV spectra from 300 nm to 340 nm with expanded uncertainty bars ($k=\mathrm{2}$) for QASUME indicated as blue circles, BTS indicated as green squares, and AVODOR indicated as magenta triangles. The TOC measured with Brewer #183 is plotted as black crosses with grey uncertainty bars ($k=\mathrm{2}$).

[Figure omitted. See PDF]

The expanded uncertainties in the TOC values obtained with the three instruments are shown in Fig. . The expanded uncertainties in the TOC data sets at local noon are 3.6 DU (1.3 %) for the QASUME spectroradiometer, 4.4 DU (1.5 %) for the BTS spectroradiometer, and 13.3 DU (4.7 %) for the AVODOR spectroradiometer. The expanded uncertainties stated include the standard deviations of 0.28 DU for BTS and 3.10 DU for AVODOR, arising from the fitting method.

It is of interest to compare the obtained uncertainties with values assuming no correlations. If we neglect correlations (i.e. we assume the fractions in Table  to be 0 for the full and unfavourable correlations and 1 for the random correlation, and run the simulations with the spectrum measured at local noon), we obtain the expanded uncertainty $U_{\mathrm{QASUME}}\left(\mathrm{TOC}\right)=\mathrm{0.9}$ DU (0.3 %), $U_{\mathrm{BTS}}\left(\mathrm{TOC}\right)=\mathrm{1.1}$ DU (0.4 %), and $U_{\mathrm{AVODOR}}\left(\mathrm{TOC}\right)=\mathrm{7.7}$ DU (2.7 %). These values are on average a factor of 3 lower than the uncertainties accounting for correlations. This analysis assumes random noise only.

Typical practice in an analysis like this is to add a component introduced by the standard deviation of the fit to the uncertainty. The standard uncertainty to be added to $u\left(\mathrm{TOC}\right)$ because of the standard deviation of the fit is 0.2 DU with QASUME, 0.7 DU with BTS, and 3.4 DU with AVODOR, raising the corresponding total expanded uncertainties to 1.0 DU (0.3 %), 1.8 DU (0.6 %), and 10.3 DU (3.6 %). The results are generally in agreement within these lower uncertainties as well. However, comparing differences in the TOC results of the different spectroradiometers does not represent the uncertainty in the absolute TOC scale, since the ozone retrieval algorithm uses the same extraterrestrial spectrum and ozone absorption cross section for all the instruments. Changing the extraterrestrial spectrum or the ozone absorption cross section to another data set may shift all the TOC values of the instruments beyond the latter low uncertainties.

Expanded uncertainties in the total ozone columns derived from QASUME, BTS, and AVODOR spectra obtained by the relative least squares fitting method.

[Figure omitted. See PDF]

Figure  shows that the effect of stray light can be effectively reduced from the TOC results by using the OLS fitting described in Appendix A with the correction in Eq. (). For example, BTS and QASUME results are in good agreement even at the largest zenith angles. TOC results of AVODOR are in agreement at noon, but the results before 09:00 and after 17:00 deviate from the other instruments by 10 DU.

Conclusions

In this work, we introduced one possible way to take into account spectral correlations in the uncertainties in the atmospheric ozone retrieval and estimated the TOC uncertainties obtained from the spectral data of three different spectroradiometers, measured at the ATMOZ field measurement campaign at the Izaña Atmospheric Observatory. It should be noted that the method proposed has a drawback that the unknown correlations have to be approximated based on experience. However, the method has merits in estimating the order of magnitude of possible uncertainties accounting for correlations. The typical assumption made, that uncertainties are spectrally uncorrelated, is just an assumption as well and in many cases, it is not valid. The uncertainty values obtained with the new model are higher than the uncertainties obtained with the traditional method, which neglects correlations because some of the major uncertainty components may contain systematic spectral deviations. These results demonstrate the importance of accounting for correlations. If their origins and magnitudes are known, they can be accounted for precisely using methods of .

The new model uses a similar approach to our previously developed MC uncertainty model for correlated colour temperature (CCT) (). In the article, we demonstrated the use of the model for calculating the CCT of a source resembling standard illuminant A. For standard illuminant A, the case representing uncertainty with unfavourable correlations in CCT was found at $N=\mathrm{3}$. On the contrary, for the ozone retrieval the deviation at $N=\mathrm{1}$ produces the largest uncertainty, which is in a way trivial compared with CCT. The use of a set of sine functions as base functions was originally developed for the more complicated situation of CCT, where it was not known where the unfavourable uncertainty would be. When we now have analysed the situation, an uncertainty arising from unfavourable correlations in the ozone retrieval could be modelled as well, e.g. using a combination of full spectral deviation, a simple slope, and a parabola as the deviation function mimicking unfavourable correlations. This is discussed in more detail in Appendix B.

The new MC method for estimating uncertainties in TOC in the presence of systematic spectral deviations provides more complete estimations of the uncertainty budget compared with the traditionally used methods. The TOC values retrieved from different instrument data were well in agreement within the uncertainties estimated with the new MC method. Although the TOC results obtained using different instruments have good agreement, these differences do not represent the uncertainty in the absolute TOC scale. The TOC uncertainties we have estimated cover possible offsets in the absolute TOC scale, arising from the uncertainties in the ozone absorption cross section and extraterrestrial spectrum that are the dominating components in the uncertainty budget.

The data sets of Figs. 3 and 6 (for 17 September 2016) are parts of a larger data set containing the results on all days 12–30 September 2016 of the comparison campaign. Because more publications will be written on the results of the comparison campaign, the data cannot yet be uploaded to a public repository. Data used in this article can be provided upon request by email to Anna Vaskuri ([email protected]).

Selecting the least squares fitting method

Two of the instruments being compared suffer from stray light and noise that distort the TOC results. In earlier studies, e.g. by , stray light in array spectroradiometers has been noted to decrease TOC values at large solar zenith angles, resulting in an inverted U-shape dependence of the diurnal TOC variation. The effect of stray light can be compensated by reducing the effect of the short-wavelength tail, either by limiting the wavelength range or the dynamic range used, or by weighting the results. We studied various weightings and selection methods for data in order to find an objective way to perform the ozone retrieval method and to analyse the results.

Figure  shows the TOC results as a function of time analysed with three different weighting methods for least squares minimization. The methods include a relative least squares fitting (RLS) in Eq. () with $w\left(\mathit{\lambda }\right)=E{\left(\mathit{\lambda }\right)}^{-\mathrm{2}}$, RLS fitting with the dynamic range (DR) limited to avoid stray light, and an ordinary least squares fitting method (OLS) with $w\left(\mathit{\lambda }\right)=\mathrm{1}$.

TOC values estimated for QASUME in Fig. a have no significant solar zenith angle dependence, and limiting the dynamic range of RLS fitting only affects individual TOC values in the early morning and late afternoon compared with the RLS fitting over the complete spectral range. Using the OLS fitting method, the diurnal variation of the TOC remains, but the values are underestimated by a constant factor of 1.013.

TOC values estimated for BTS in Fig. b and AVODOR in (c) have severe dependence on the solar zenith angle when using the RLS fitting method. The solar zenith angle dependence decreases to approximately half when limiting the dynamic range to exclude the baseline noise from the fitting. The best results are obtained by using the OLS minimization, which practically removes the solar zenith angle dependence for both BTS and AVODOR but introduces an offset to TOC results. An almost similar offset in TOC results was noted for QASUME. The OLS method violates the heteroscedasticity of our data sets, since we know that the absolute spectral uncertainties are not constant at the studied wavelength range. The only reason to use the OLS method for array spectroradiometers is to reduce the effect of stray light from TOC results. Hence, we correct the TOC results obtained with the OLS method with correction factors estimated from the ratios of TOC values determined from the local noon spectra using both RLS and OLS methods. The correction factors were averaged over 10 samples around noon, being 1.006 for BTS and 1.013 for AVODOR, with standard deviations of 0.1 % (0.28 DU) and 1.1 % (3.10 DU), respectively. This correction makes the TOC results comparable with devices analysed using the RLS method.

For QASUME, we use RLS minimization with the dynamic range limited to 4 orders of magnitude, as it uses most of the useful undistorted data. This is also consistent with the earlier methods used with monochromator-based spectroradiometers, e.g. by .

TOC values during the day estimated using different weightings in the least squares minimization for QASUME (a), BTS (b) and AVODOR (c). TOC values for Brewer #183 are plotted as black crosses for comparison. The colour codes and the associated figure legends denote the weighting and the dynamic range (DR) used.

[Figure omitted. See PDF]

TOC uncertainties obtained using Chebyshev polynomials

In the MC uncertainty analysis, it is possible to use other orthogonal sets of functions instead of sinusoidal functions, such as Chebyshev polynomials shown in Fig. . A Chebyshev polynomial of the first kind $T_j\left(\mathit{\lambda }\right)$ of order $j$ (, p. 209) is defined as $$T_j\left(\mathit{\lambda }\right)=\cos \left[j\arccos \left({\displaystyle \frac{\mathrm{2}\mathit{\lambda }-{\mathit{\lambda }}_{\mathrm{1}}-{\mathit{\lambda }}_{\mathrm{2}}}{{\mathit{\lambda }}_{\mathrm{2}}-{\mathit{\lambda }}_{\mathrm{1}}}}\right)\right],$$ where ${\mathit{\lambda }}_{\mathrm{1}}$ is the short-wavelength limit and ${\mathit{\lambda }}_{\mathrm{2}}$ is the long-wavelength limit for the spectra measured. To create arbitrary spectral deviations with unity variance, each Chebyshev polynomial, except for $g_{\mathrm{0}}\left(\mathit{\lambda }\right)=T_{\mathrm{0}}\left(\mathit{\lambda }\right)=\mathrm{1}$, needs to be normalized to unity variance as $$g_j\left(\mathit{\lambda }\right)={\displaystyle \frac{T_j\left(\mathit{\lambda }\right)}{{\mathit{\sigma }}_j}},$$ where ${\mathit{\sigma }}_j$ is the standard deviation of $T_j\left(\mathit{\lambda }\right)$. In practice, Chebyshev polynomials in Fig.  can be generated fast using recurrence (e.g. ) as $$T_j\left(\mathit{\lambda }\right)=\mathrm{2}\left({\displaystyle \frac{\mathrm{2}\mathit{\lambda }-{\mathit{\lambda }}_{\mathrm{1}}-{\mathit{\lambda }}_{\mathrm{2}}}{{\mathit{\lambda }}_{\mathrm{2}}-{\mathit{\lambda }}_{\mathrm{1}}}}\right)T_{j-\mathrm{1}}\left(\mathit{\lambda }\right)-T_{j-\mathrm{2}}\left(\mathit{\lambda }\right),$$ where $T_{\mathrm{0}}\left(\mathit{\lambda }\right)=\mathrm{1}$, and $T_{\mathrm{1}}\left(\mathit{\lambda }\right)=\left(\mathrm{2}\mathit{\lambda }-{\mathit{\lambda }}_{\mathrm{1}}-{\mathit{\lambda }}_{\mathrm{2}}\right)/\left({\mathit{\lambda }}_{\mathrm{2}}-{\mathit{\lambda }}_{\mathrm{1}}\right)$ is a straight line. According to Eq. () the scaling with the standard deviation is performed after generating the Chebyshev polynomials.

Each base function of the cumulative Fourier series in Eq. () was formed with sine (odd) and cosine (even) terms (, p. 628), $$\begin{array}{cc}{\displaystyle }\sin & {\displaystyle }\left[i\left(\mathrm{2}\mathit{\pi }{\displaystyle \frac{\mathit{\lambda }-{\mathit{\lambda }}_{\mathrm{1}}}{{\mathit{\lambda }}_{\mathrm{2}}-{\mathit{\lambda }}_{\mathrm{1}}}}\right)+{\mathit{\varphi }}_i\right]=\cos \left({\mathit{\varphi }}_i\right)\cdot \sin \left(\mathrm{2}\mathit{\pi }i{\displaystyle \frac{\mathit{\lambda }-{\mathit{\lambda }}_{\mathrm{1}}}{{\mathit{\lambda }}_{\mathrm{2}}-{\mathit{\lambda }}_{\mathrm{1}}}}\right)\\ & {\displaystyle }& {\displaystyle }+\sin \left({\mathit{\varphi }}_i\right)\cdot \cos \left(\mathrm{2}\mathit{\pi }i{\displaystyle \frac{\mathit{\lambda }-{\mathit{\lambda }}_{\mathrm{1}}}{{\mathit{\lambda }}_{\mathrm{2}}-{\mathit{\lambda }}_{\mathrm{1}}}}\right),\end{array}$$ where the phase ${\mathit{\varphi }}_i$ is an equally distributed MC variable between 0 and $\mathrm{2}\mathit{\pi }$. Hence, the analysis based on Chebyshev polynomials to be compatible with the sinusoidal approach, each base function $f_i\left(\mathit{\lambda }\right)$ with index $i\ge \mathrm{1}$ is formed by the combination of odd ($\mathrm{2}i-\mathrm{1}$) and even ($\mathrm{2}i$) terms as $$f_i\left(\mathit{\lambda }\right)=\cos \left({\mathit{\varphi }}_i\right)\cdot g_{\mathrm{2}i-\mathrm{1}}\left(\mathit{\lambda }\right)+\sin \left({\mathit{\varphi }}_i\right)\cdot g_{\mathrm{2}i}\left(\mathit{\lambda }\right),$$ where the weights $\cos \left({\mathit{\varphi }}_i\right)$ and $\sin \left({\mathit{\varphi }}_i\right)$ set the variance of $f_i\left(\mathit{\lambda }\right)$ to unity. These base functions $f_i\left(\mathit{\lambda }\right)$ can be used with Eqs. () and ().

First five Chebyshev polynomials with unity variance corresponding to Fig.  with sinusoidal base functions.

[Figure omitted. See PDF]

Figure  compares standard uncertainties in TOC obtained by generating arbitrary spectral deviations using Chebyshev polynomials as base functions (circles and triangles), formed using Eq. (), with those obtained by using sinusoidal base functions (crosses). The uncertainties change slightly at the lowest complexity orders of deviations when sinusoidal base functions are replaced with Chebyshev polynomials but essentially the results are similar.

Standard uncertainties in TOC simulated using a local noon spectrum of QASUME with Chebyshev polynomials as base functions. The TOC uncertainties with the input standard deviation of 1 % in the spectral irradiance values are shown as green circles and the TOC uncertainties with the input standard deviation of 1 % in the ozone absorption cross section are shown as magenta triangles. Standard uncertainties simulated with sinusoidal base functions (black crosses) from Fig. a are plotted for comparison.

[Figure omitted. See PDF]

The authors declare that they have no conflict of interest.

Acknowledgements

Peter Sperfeld from PTB is acknowledged for measuring and providing the BTS data set. Alberto Redondas and all personnel from Izaña Atmospheric Research Center, AEMET, Tenerife, Canary Islands, Spain, are acknowledged for measuring and providing the Brewer #183 data set and the environmental parameters such as the sonde data. Anna Vaskuri is grateful for the grant by the Emil Aaltonen Foundation, Finland. This work has been supported by the European Metrology Research Programme (EMRP) within the joint research project ENV59 “Traceability for atmospheric total column ozone” (ATMOZ). The EMRP is jointly funded by the EMRP participating countries within EURAMET and the European Union.Edited by: Lok Lamsal Reviewed by: four anonymous referees