DL-RMD: a geophysically constrained electromagnetic resistivity model database (RMD) for deep learning (DL) applications

Abstract

Deep learning (DL) algorithms have shown incredible potential in many applications. The success of these data-hungry methods is largely associated with the availability of large-scale datasets, as millions of observations are often required to achieve acceptable performance levels. Recently, there has been an increased interest in applying deep learning methods to geophysical applications where electromagnetic methods are used to map the subsurface geology by observing variations in the electrical resistivity of the subsurface materials. To date, there are no standardized datasets for electromagnetic methods, which hinders the progress, evaluation, benchmarking, and evolution of deep learning algorithms due to data inconsistency. Therefore, we present a large-scale electrical resistivity model database (RMD) with a wide variety of geologically plausible and geophysically resolvable subsurface structures for the commonly deployed ground-based and airborne electromagnetic systems. Potentially, the presented database can be used to build surrogate models of well-known processes and to aid in labour-intensive tasks. The geophysically constrained property of this database will not only achieve enhanced performance and improved generalization but, more importantly, incorporate consistency and credibility into deep learning models. We show the effectiveness of the presented database by surrogating the forward-modelling process, and we urge the geophysical community interested in deep learning for electromagnetic methods to utilize the presented database. The dataset is publicly available at 10.5281/zenodo.7260886 (Asif et al., 2022a).

1 Introduction

Recent years have witnessed the success of many deep learning (DL) applications. Although DL emerged in 1982 in the form of neural networks (Hopfield, 1982), it started to gain attention in 2012 due to its notable performance for image classification tasks (Krizhevsky et al., 2017, 2012). Since then, it has been applied successfully to many applications including object detection (Asif et al., 2019; Redmon et al., 2016; Ren et al., 2015), image super-resolution (Dong et al., 2016; Zhang et al., 2018), speech recognition (Zhang et al., 2017), and stock market predictions (Pang et al., 2020). The revival of DL was mainly influenced by the availability of cheap computing resources, deeper network architectures, and large-scale publicly available datasets. Deeper network architectures and an increased number of samples in the training datasets are key factors for improved performance and better generalization of DL models (Wang et al., 2016).

Geophysics is a branch of earth sciences, and geophysical methods are often used to infer information about the subsurface geology by mapping physical properties. The integration of neural networks in geophysics started several decades ago and has covered many domains of geophysics (Baan and Jutten, 2000; Dramsch, 2020), including seismic (Röth and Tarantola, 1994; Zhang et al., 2020), magneto-telluric (Conway et al., 2019; Liu et al., 2020; Zhang and Paulson, 1997), geo-mechanical (Feng and Seto, 1998; Khatibi and Aghajanpour, 2020), and electromagnetic domains (Birken and Poulton, 1999; Birken et al., 1999; Bording et al., 2021; Kwan et al., 2015; Poulton et al., 1992; Zhu et al., 2012). Interestingly, the last few years have seen a significant increase in interest in applying DL to electromagnetic (EM) methods (see Table 1), where the artificially generated EM fields are used to map variations in the electrical resistivity properties of the subsurface. For more details regarding the EM methods, readers are referred to the literature (e.g. Kirsch, 2006). The increasing interest in applying DL to EM methods is mainly influenced by the increased ability of the EM methods to collect huge datasets in short amounts of time, which make the subsequent processes extremely laborious and time consuming. Therefore, a DL method could be beneficial in surrogating well-known EM processes, e.g. forward modelling where the propagation of the EM fields is simulated, resulting in the forward responses (Xue et al., 2020), and inverse modelling (inversion) where the electrical resistivity properties of the subsurface are deduced from observed EM data (Zhdanov, 2015). DL methods can also assist with manual tasks, which may require considerable time when performed manually, such as anomaly detection in EM data. Further opportunities may lie in other tasks, e.g. data de-noising.

To apply a DL algorithm to EM methods for various applications, subsurface resistivity models and/or the corresponding EM responses are often required. To achieve optimal performance, a DL method should be trained on a large number of geologically realistic subsurface models. Evident from Table 1, the recently developed DL methods either use subsurface resistivity models acquired from field data or generate the models randomly or in a pseudorandom manner for training. However, a method trained on random models, where the resistivity of each geological layer is chosen from a probability distribution, would not result in optimal performance, as many of the training samples would be geologically unrealistic. A good solution is to use either resistivity models inverted from field data or pseudorandom resistivity models where the resistivity of the training models is based on some prior geological information to reflect various characteristics of field data (Bai et al., 2020). However, a DL method trained on such training samples would only be effective for specific geological conditions and would result in an unsatisfactory performance for significantly different geological settings (Bording et al., 2021), as bias in the training data can affect generalizability substantially. Additionally, the unavailability of a standard benchmark database hinders the progress, evaluation, benchmarking, and evolution of DL algorithms due to data inconsistency (Bergen et al., 2019; Reichstein et al., 2019).

Table 1

Recent publications (2019–2021) of DL applications in EM which show the number of training samples and type of training dataset (random, pseudorandom, or field data).

Reference	No. of samples	Training observation type	Application
	in training set
Wu et al. (2021a)	80 000	Pseudorandom resistivity models and forward responses	Inversion
Colombo et al. (2021a)	5000	Pseudorandom resistivity models and forward responses	Inversion
Colombo et al. (2021b)	20 000	Random resistivity models and forward responses	Inversion
Wu et al. (2021b)	16 800	Forward responses of random resistivity models	De-noising
Bording et al. (2021)	93 500	Field data and inversion models	Forward modelling
Puzyrev and Swidinsky (2021)	512 000	Random resistivity models and forward responses	Inversion
Asif et al. (2021a)	100 000	Field data and inversion models	Forward modelling
Moghadas et al. (2020)	20 000	Random resistivity models and forward responses	Forward modelling
Bai et al. (2020)	12 000	Pseudorandom resistivity models and forward responses	Inversion
Li et al. (2020)	1 000 000	Pseudorandom resistivity models and forward responses	Inversion
Bang et al. (2021)	25 173	Pseudorandom resistivity models and forward responses	Inversion
Noh et al. (2020)	20 000	Random resistivity models and forward responses	Inversion
Moghadas (2020)	20 000	Random resistivity models and forward responses	Inversion
Colombo et al. (2020a)	235 620	Pseudorandom resistivity models and forward responses	Inversion
Colombo et al. (2020b)	88	Pseudorandom resistivity models and forward responses	Inversion
Lin et al. (2019)	2400	Field data and inverted model forward responses	De-noising
Guo et al. (2019)	10 000	Pseudorandom resistivity models and forward responses	Inversion
Puzyrev (2019)	20 000	Pseudorandom resistivity models and forward responses	Inversion
Qin et al. (2019)	50 000	Random resistivity models and forward responses	Inversion

To have an inclusive DL solution for various applications in EM, we present a physics-driven large-scale model database ($\sim$ 1 million models) of geologically plausible and EM-resolvable 1-D subsurface resistivity models spanning the resistivity range from 1 to 2000 $\mathrm{\Omega }$m and to a depth of 500 m. This model database is suitable for ground-based and airborne EM systems in a DL context. We use broad-banded von Kármán covariance functions to generate geologically constrained resistivity models. Geophysical constraints are imposed by calculating the EM forward data of the initial resistivity models followed by inversion of the EM forward data to obtain the final resistivity models. This allows us to create a comprehensive resistivity model database (RMD) that may not only improve performance and generalization but also incorporate consistency and reliability into the DL models. We believe that the presented RMD will be a valuable resource to accelerate the inter- and trans-disciplinary research of earth and data sciences. The presented DL-RMD will also provide uniformity in training and benchmarking for DL methods in EM. Therefore, we urge the geophysical community interested in DL for EM methods to use the DL-RMD.

The rest of this paper is organized as follows. Section 2 describes the general methodology of generating the subsurface resistivity models, while specific settings for the DL-RMD for the three EM system categories are specified in Sect. 3. Section 4 provides details for training a DL method to surrogate the forward-modelling problem and shows the effectiveness of the DL-RMD. Discussion, code and data availability, and concluding remarks are given in Sects. 5, 6, and 7, respectively.

2 Methodology

Geological processes do not result in random structures, nor are the subsurface resistivity structures random, as some spatial correlation is generally present (Tacher et al., 2006). Therefore, it is reasonable that the training of a DL method is based on subsurface structures that are geologically plausible and, in an EM context, overall resolvable by the EM method. Additionally, the scale of the resistivity structure in the models should reflect the resolution capability of the EM methods, as training a DL method to resolve structures that are not evident in the input data is not possible. EM methods are diffusive methods with significantly decreasing resolution with depth, and the electrical conductivity contrast plays an important role for the resolution capability; hence, a metric number for a given EM method's resolution capability and the depth of investigation cannot be given.

To obtain geologically realistic models, we use the broad-banded von Kármán covariance functions (Møller et al., 2001) to generate geologically plausible models (von Kármán models). The suite of von Kármán models consists of fine geological structures and contain some resistivity variations and patterns that are unlikely to be resolved, due to the resolution limitation of the EM method. To replicate the resolution capability of the EM method, we generate EM forward responses of the initially over-detailed von Kármán models and invert these forward responses to obtain the final resistivity models. Since we aim at generating 1-D resistivity models, we are only concerned about the resistivity ($\mathit{\rho }$) variations in the vertical direction ($z$) from surface to some depth in our model generation.

Initially, we base the spatial variation character of ($z$, ${\log }_{\mathrm{10}}\left(\mathit{\rho }\right)$) for our von Kármán models on the broad-banded von Kármán covariance functions (Christiansen and Auken, 2003; Møller et al., 2001).

1$$C\left(z,A,v\right)=A^{\mathrm{2}}{C}_{\mathrm{0}}{\left({\displaystyle \frac{z}{L}}\right)}^{\mathit{\nu }}{K}_{\mathit{\nu }}\left({\displaystyle \frac{z}{L}}\right),$$ where $A$ becomes the amplitude of the logarithmic resistivity, $C_{\mathrm{0}}$ is a scaling constant, $z$ is the spatial (vertical) distance, $L$ characterizes the maximum correlation length accounted for, and $K_{\mathit{\nu }}$ is the modified Bessel function of the second kind and order $\mathit{\nu }$. In the model generation, $L$ is fixed to a high number (1800 m) which gives us strong correlation for $z\ll L$ (Maurer et al., 1998). By using combinations of $\mathit{\nu }$, $C_{\mathrm{0}}$, and resistivity and compiling several realizations of the stochastic von Kármán process, we generate a variety of resistivity models on multiple scales. Table 2 summarizes the $L$, $\mathit{\nu }$, $C_{\mathrm{0}}$, and resistivity values used.

Table 2

Parameters used in all combinations to generate the initial von Kármán resistivity models.

Parameter	Values
Resistivity	1 to 2000 $\mathrm{\Omega }$m, log-spaced,20 values per decade
$L$	Fixed: 1800 m
$v$	[0.6, 0.7, 0.8, 0.9, 1.0]
$C_{\mathrm{0}}$	[0.5, 1, 2, 4]
No. of sharp boundaries	[1, 2, 3, 4, 5]

Examples of this are shown in Fig. 1a–c where the von Kármán models (in black curves) are generated with a combination of the extreme values of $\mathit{\nu }$ and $C_{\mathrm{0}}$ for an initial resistivity value of 30 $\mathrm{\Omega }$m. Low $\mathit{\nu }$ and high $C_{\mathrm{0}}$ produce models with fine- and large-scale variations (Fig. 1a), while high $\mathit{\nu }$ and high $C_{\mathrm{0}}$ values produce a relatively smooth model (Fig. 1b) but still with resistivity variations spanning 2–3 decades of resistivity. The combination of low $\mathit{\nu }$ and $C_{\mathrm{0}}$ values ensures that the simple and close-to-half-space models are also represented (Fig. 1c).

Sharp layering in the subsurface is plausible, and large resistivity amplitudes and short correlation lengths in the von Kármán functions will form layering in the models. To include more models with a sharp layering, we stitch 2–6 randomly selected depth intervals of the initially generated von Kármán models from a uniform distribution. An example of a stitched model is shown in Fig. 1d. These stitched models also ensure that different combinations of $\mathit{\nu }$ and C${}_{\mathrm{0}}$ are represented within one model.

Figure 1

Examples of von Kármán models and the result after the forward and inversion process, where black curves show von Kármán models (re-discretized to 90 layers), and the red curve shows the final model. Panels (a) to (c) are for the combination of $\mathit{\nu }$ and $C_{\mathrm{0}}$ stated in the title; panel (d) is for a stitched, layered model (green arrows mark the imposed sharp layer boundaries). The red curves show the obtained model from inversion of the forward response of the black model.

[Figure omitted. See PDF]

Prior to the EM forward calculation, the von Kármán models are re-discretized to 90 layers for faster forward computation and easier handling. The top-layer thickness and depth to the last layer boundary for the re-discretized layers are detailed in Table 3 for three generic EM systems with different depths of investigations (see Sect. 3 for further details). For the forward calculation, the geometric mean of the last 5 m of the re-discretized von Kármán models is assigned to the last model layer that continues to infinite depth. In order to avoid making assumptions on the acquisition conditions, on the specific instrument setups, etc., the calculated forward data are pragmatically assigned a uniform uncertainty of 5 % to take noise into account and are inverted with a 30-layer model with a minimum structure (smooth) regularization scheme (Viezzoli et al., 2008). The layer thicknesses for the 30-layer models are fixed, and they are listed in Table 3. The red model curves in Fig. 1 represent the resistivity models after the forward and inversion process and represent the models that enter the DL-RMD. As seen from Fig. 1, the von Kármán models hold structures that are not resolved by the inverted resistivity models, so the models obtained after the forward and inversion process result in structures resolvable by the EM method. A total of $\sim$ 95 % of the inverted resistivity models explain (fit) the forward data within the assumed data uncertainty. In other words, the inverted models are explaining the more complex von Kármán models to a very high degree.

The forward and inverse modelling is carried out for three different generic time-domain EM (TEM) systems spanning different depth ranges using the AarhusInv modelling code (Auken et al., 2015). The specific DL-RMD settings for different TEM systems are summarized in Sect. 3.

3 Deep learning resistivity model database (DL-RMD)

EM systems for subsurface exploration have existed since the 1950s, and nowadays a large variety of airborne and ground-based time-domain electromagnetic (TEM) and frequency-domain electromagnetic (FEM) systems exist. Both TEM and FEM methods map the electrical resistivity of the subsurface by inducing EM fields. TEM methods record the decay of the secondary EM field in the absence of the transmitted EM field in the time domain, while FEM methods record the secondary EM field in the frequency domain in the presence of the transmitted EM field (Christiansen et al., 2006). TEM and FEM methods also differ in resolution and depth of investigation, depending on the TEM system configuration, e.g. transmitter turn-off time, transmitter moment, and airborne or ground-based. For the DL-RMD to be compatible for different TEM systems, we have compiled three model databases with $\sim$ 1 million models in each for three generic TEM systems with different depths of investigation as their primary differences. We refer to the three DL-RMDs as shallow, intermediate, and deep, with the initialisms S-RMD, I-RMD, and D-RMD, respectively. S-RMD mimics a shallow-focusing ground-based TEM system, initiated by a short transmitter turn-off time. For S-RMD, the models are discretized down to 125 m with a top-layer thickness of 0.5 m. I-RMD and D-RMD mimic airborne TEM systems with different depths of investigation and are hence discretized down to depths of 350 and 500 m and top-layer thicknesses of 3 and 5 m, respectively. The calculation of depth of investigation follows Christiansen and Auken (2012).

Table 3

Model discretization and key specifications of the generic TEM systems for three resistivity model databases. The generic TEM systems are all central loop configurations.

Type	Parameter	S-RMD	I-RMD	D-RMD
Von Kármán models	Max depth (m)	125	355	505 m
	Discretization (m)	0.1	0.1	0.1
	Re-discretization (m)	0.2–120 m, 90-layer log-spaced	1–350 m, 90-layer log-spaced	2–500 m, 90-layer log-spaced
Database resistivitymodels	Model discretization	0.5–120 m, 30-layer log-spaced	3–350 m, 30-layer log-spaced	5–500 m, 30-layer log-spaced
Generic TEM	Turn-off time ($\mathrm{\mu }$s)	4	12	40
configuration	${}^{*}$Gate time start ($\mathrm{\mu }$s)	5	13	50
	${}^{*}$Gate time end (ms)	1	10	32
	Modelling height (m)	0 – ground-based	40 – airborne	40 – airborne

${}^{*}$ Gate start/end times have zero-time reference at the beginning of turn-off time.

Figure 2

Statistical insights into the DL-RMD. (a–c) Resistivity distributions of the S-RMD, I-RMD, and D-RMD, respectively. (d–f) Distributions of depth of investigation of models in the S-RMD, I-RMD, and D-RMD, plotted as a cumulative sum.

[Figure omitted. See PDF]

The model discretization details for the three DL-RMDs for the initial von Kármán models and for the final resistivity models entering the RMD are summarized in Table 3. Table 3 also holds the key specifications of the three generic TEM systems. The settings for the generation of the von Kármán models are specified in Table 2 and are common for the three DL-RMDs. Each of the three DL-RMDs holds $\sim$ 1 million models spanning the resistivity interval 1–2000 $\mathrm{\Omega }$m, where $\mathrm{1}/\mathrm{6}$ of the models originate from the initially generated von Kármán models and where $\mathrm{5}/\mathrm{6}$ of the models come from the stitched, layered von Kármán models.

Some insights into the three DL-RMDs are given in Fig. 2, where Fig. 2a–c show the layer resistivity distribution of the three DL-RMDs. The resistivity distributions of the von Kármán models were generated uniformly, but the forward and inversion process makes the resistivity distribution slightly skewed towards the lower-resistivity end, due to the lower sensitivity/resolution in the high-resistivity end for the EM method (Christiansen et al., 2006; Jørgensen et al., 2005). The larger start and end bins compared to the neighbouring bins in Fig. 2a–c are due to the 1 and 2000 $\mathrm{\Omega }$m resistivity truncation. The estimated depths of investigation for the three DL-RMDs are shown in Fig. 2d–f. We observe that approximately 70 % of the models have depths of investigation that are less than the depth to last layer boundary of the given DL-RMD. Notably, a thick conductive layer near the surface will significantly limit the depth of investigation for a given TEM configuration. The uneven and in some cases limited depth of investigation does not pose a problem for a deep learning algorithm, as the EM method will compromise a similar depth of investigation limitation for the given resistivity model (see the Discussion section for more details).

4 Example of an EM application using the DL-RMD

EM methods can benefit from the presented DL-RMD in many ways. For example, the DL-RMD can be used to surrogate the computationally expensive numerical forward modelling by using a computationally efficient DL method, which would speed up the whole inversion process. It can also be used to develop a DL algorithm to replace the calculation of the partial derivatives in deterministic inversion methods, where the subsurface resistivity model is updated iteratively by using the partial derivatives of the model parameters. Detecting anomalies in the EM data by using a DL approach using the DL-RMD can significantly speed up the EM data processing and limit the involvement of human-centric manual workflows. Additionally, EM data de-noising also becomes plausible.

As an example in this paper, we use the DL-RMD to surrogate the forward modelling problem for a ground-based TEM system using a fast DL method, since a significant number of forward calculations are required during the inversion process, when either deterministic or stochastic inversion methods are used. By replacing the computationally expensive numerical forward modelling approach, the whole inversion process may be accelerated without further modification to a standard inversion workflow (Asif et al., 2021b). However, it is crucial that the performance of the DL method balances the numerical precision and increased speed of computation. If the prediction accuracy is not sufficiently high, the application in an inversion framework may result in spurious subsurface features and erroneous geological interpretations of the geophysical EM mapping results.

4.1 Deep learning (DL) setup

We design the surrogate model for the tTEM system (Auken et al., 2018). The tTEM system is a ground-based towed TEM system with a maximum depth of investigation of 120 m based on the data time interval from $\sim$ 5 $\mathrm{\mu }$s to $\sim$ 1 ms, which matches the specification of S-RMD; therefore, we use it to train our DL method.

The input to the DL algorithm becomes the 30-layer resistivity model $\mathit{m}$ in S-RMD, where the layer thickness of each resistivity layer is fixed. The target outputs are the numerical TEM forward responses, i.e. $\mathrm{d}\mathbf{B}/\mathrm{d}t$, for the corresponding inputs. A standard EM modelling code (Auken et al., 2015) is used to generate the TEM forward responses for the resistivity models $\mathit{m}$ with fixed layer thicknesses. We generate the responses from $\sim$ 1 ns to $\sim$ 10 ms by exponentially increasing gate widths sampled at 14 gates per decade.

Prior to the training of a DL method, inputs and the corresponding target outputs are normalized. Each resistivity model $\mathit{m}$ is normalized, where the logarithmic variations in the model parameters can take both positive and negative values.

2$${\mathit{m}}_n={\log }_{\mathrm{10}}\left(\mathit{m}\right)-{\displaystyle \frac{\mathit{\mu }\left[{\log }_{\mathrm{10}}\left(m_{\max }\right)+{\log }_{\mathrm{10}}\left(m_{\min }\right)\right]}{\mathrm{2}}},$$ where $m_{\min }$ and $m_{\max }$ are the minimum and maximum resistivity values in the training dataset of S-RMD, and $\mathit{\mu }$ is the mean.

The target outputs, i.e. $\mathrm{d}\mathbf{B}/\mathrm{d}t$, are normalized by 3$${\displaystyle \frac{\mathrm{d}{\mathbf{B}}_n}{\mathrm{d}t}}={\displaystyle \frac{\mathrm{d}\mathbf{B}/\mathrm{d}t-\mathit{\mu }\left[\mathrm{d}\mathbf{B}/\mathrm{d}t\right])}{\mathit{\sigma }\left[\mathrm{d}\mathbf{B}/\mathrm{d}t\right]}},$$ where $\mathit{\mu }$ is the mean, and $\mathit{\sigma }$ is the standard deviation of each data point in the training dataset.

We use a simple DL method where a fully connected feed-forward neural network is utilized with two hidden layers, each having 384 neurons. The hyperbolic tangent function is used as an activation function between the hidden layers, and the full-batch scaled conjugate algorithm is used for backpropagation. The loss function for training is the sum of squared errors with a regularization term consisting of the mean of sum of squares of the network weights and biases. The network configuration used here is based on our previous results (Asif et al., 2021b, 2022b). We also apply an early-stopping criterion to ensure that the training stops when the validation loss starts to increase. The validation set for the early-stopping criterion comprises of 70 000 models from S-RMD, which are excluded from the training set. Once the network is trained, it can be used for evaluation purposes. The evaluation metric for our baselines is the percentage relative error, RL${}_{\mathrm{P}}$, defined in Eq. (4), which effectively deals with the large dynamic range and patterns of TEM data. 4$${\mathrm{RL}}_{\mathrm{P}}={\displaystyle \frac{{\left(\mathrm{d}\mathbf{B}/\mathrm{d}t\right)}_{\mathrm{DL}}-{\left(\mathrm{d}\mathbf{B}/\mathrm{d}t\right)}_N}{{\left(\mathrm{d}\mathbf{B}/\mathrm{d}t\right)}_N}}\times \mathrm{100}\mathit{\%},$$ where ${\left(\mathrm{d}\mathbf{B}/\mathrm{d}t\right)}_{\mathrm{DL}}$ is the output of the DL method, and ${\left(\mathrm{d}\mathbf{B}/\mathrm{d}t\right)}_N$ is the numerically computed forward response.

Figure 3

Performance of the networks trained on S-RMD, von Kármán models, and random resistivity models. (a) RL${}_{\mathrm{P}}$ distribution. (b) Cumulative distribution of RL${}_{\mathrm{P}}$.

[Figure omitted. See PDF]

4.2 Surrogate forward-modelling results

To test the performance of our DL method trained on S-RMD, we use 697 resistivity models inverted from field data from a survey conducted in Søften, a region in Denmark. The data processing and inversion step of the field data follows the method developed by Auken et al. (2018), which covers averaging, anomaly detection, manual inspection, etc. on the data. The minimum and maximum resistivity values in the test dataset are 3.9 and 127.1 $\mathrm{\Omega }$m, respectively. The forward responses of the field-inverted resistivity models are calculated numerically to compare them with the output of our DL method. Since the output of our DL algorithm is the normalized forward response, it is de-normalized to raw data values by manipulating Eq. (3). For a relative comparison, we train another DL network with the same configuration using the initial von Kármán resistivity models. The comparison to the initial von Kármán resistivity models also allows us to examine the effect of the forward/inversion process, as described in Sect. 2, in the generation of the DL-RMD. We also train an additional network using the random resistivity models, similarly to several DL studies (Colombo et al., 2021b; Moghadas, 2020; Moghadas et al., 2020; Noh et al., 2020; Puzyrev and Swidinsky, 2021; Qin et al., 2019; Wu et al., 2021b) as mentioned in Table 1. To have the same level of complexity, the number of layers, depth discretization, and the number of random resistivity models are kept the same as used to train the other two networks for a fair comparison, and the resistivity of each layer is chosen randomly from a log-uniform distribution to take into account the non-linearity of the forward responses with the resistivity values. As such, a resistivity change from 1 to 10 $\mathrm{\Omega }$m would affect the forward data more than a change from 100 to 110 $\mathrm{\Omega }$m (Asif et al., 2021a).

Figure 3 shows the performance comparison of the trained networks based on the evaluation metric in Eq. (3) against the forward responses of 697 resistivity models from the Søften survey. Figure 3a shows the distribution of RL${}_{\mathrm{P}}$ of the DL network trained on S-RMD. We also show the accuracy performance of the DL networks trained on von Kármán and the random resistivity models. It is evident that the network trained on S-RMD results in lower errors as compared to the network trained on von Kármán resistivity models. On the other hand, the network trained on random resistivity models results in a poor accuracy performance. In quantitative terms, 71 % of the data points are evaluated to be within half a percent relative error for the network trained on S-RMD. In comparison to S-RMD, the network trained on von Kármán resistivity models results in 65 % of data points within half a percent relative error. The network trained on random resistivity models performs the worst, and only 34 % of the data points are calculated to be within half a percent relative error.

We also show the cumulative distribution of RL${}_{\mathrm{P}}$ for the networks trained on S-RMD, von Kármán models, and random models in Fig. 3b. A maximum of 9 % improvement in accuracy is achieved for the network trained on the S-RMD as compared to the von Kármán models. In comparison to the network trained on random resistivity models, an improvement of 43 % is achieved when S-RMD is used for training. The increase in accuracy is achieved only by using an appropriate dataset for training. The prediction accuracy can be improved with different data pre-processing, network configurations, loss functions, etc. while using the same training dataset to allow for consistency in benchmarking of DL algorithms. It is also important that a balance between the prediction performance and computational efficiency is maintained. As such, the computational time for the forward pass of the proposed network configuration can serve as a baseline for time comparison.

Figure 4 shows a visual comparison of a numerical forward response against the forward response from the trained networks for one of the resistivity models from the Søften survey. It is evident from Fig. 4 that the forward response from the network trained on S-RMD is the most accurate and has a maximum relative error of 1.4 % for the data point at $\sim$ 72 $\mathrm{\mu }$s (see Fig. 4a). The highest error for the forward response from the network trained on von Kármán models is observed to be 2.5 % for the data point at $\sim$ 160 $\mathrm{\mu }$s as shown in Fig. 4b. The forward response from the network trained on random models results in the worst accuracy performance and results in a maximum error of 22.3 % for the data point at 100 $\mathrm{\mu }$s (see Fig. 4c).

Figure 4

Comparison of performance of the networks trained on S-RMD, von Kármán models, and random resistivity models with a numerical forward response from the test set. The forward responses are shown only within the time range of tTEM data, and the inset shows the forward response from 16 to 20 $\mathrm{\mu }$s (a) Numerical forward response vs. the forward response from the network trained on S-RMD. (b) Numerical forward response vs. the forward response from the network trained on von Kármán models. (c) Numerical forward response vs. the forward response from the network trained on random resistivity models.

[Figure omitted. See PDF]

5 Discussion

The network trained on random resistivity models results in a poor accuracy performance as many of the resistivity models in the training dataset are geologically unrealistic. The complex, unrealistic resistivity structures in the randomly generated training models would result in forward responses similar to the ones obtained from simpler resistivity models, which further decreases the quality of the training dataset. The von Kármán models may be considered pseudorandom resistivity models where the resistivity structure of the models has a geologically realistic nature, as it considers multiple correlation lengths with a stochastic nature resembling geological processes. Due to the geological nature of the von Kármán models, the network trained on such models results in a decent performance accuracy. However, the network trained on von Kármán models has a lower accuracy performance as compared to the network trained on S-RMD, where the resolution capability of the EM method has been taken into account, resulting in resistivity structures resolvable by the EM method.

The resolution capability and the depth of investigation for a given TEM system strongly depend on the underlying resistivity model. Therefore, stating a single depth of investigation value for a given TEM system is not appropriate. A single exploration depth, depth of investigation, or a similar value stated by the instrument manufacturers will often be an optimistic one. For TEM systems with short transmitter current turn-off, the early data points provide the near-surface resolution, while the late data points strongly control the depth of investigation for a given resistivity model. The transmitter moment and the background noise level also influence the depth of investigation, but these factors are not considered in our case, since we have assumed a uniform data uncertainty in the forward and inversion process. The three DL-RMDs span different TEM systems and resolutions. Therefore, for a particular TEM system, one should pick the DL-RMD that has a similar resolution as the underlying generic TEM system. This is best evaluated by matching the time interval of the data for the particular TEM system to the data time interval (data time start/end in Table 3) for the generic TEM system.

In Table 4, we list some examples of the compatibility of our DL-RMD with some well-known TEM systems. Despite I-RMD and D-RMD being compiled for a generic airborne system, I-RMD and D-RMD are also appropriate for ground-based TEM systems since the simulated flight altitude of 40 m does not lead to a drastic change in the vertical resolution.

Table 4

Examples of DL-RMD compatibility for some TEM systems.

System	Resistivity model database
	S-RMD	I-RMD	D-RMD
EQUATOR (Karshakov et al., 2017)	$✓$
tTEM (Auken et al., 2018)	$✓$
MEGATEM (Smith et al., 2003)		$✓$
AEROTEM (Balch et al., 2003)		$✓$
SkyTEM (Sørensen and Auken, 2004)		$✓$	$✓$
GEOTEM (Smith, 2010)		$✓$	$✓$
SPECTREM${}^{\mathrm{PLUS}}$ (Leggatt et al., 2000)			$✓$

Since FEM and TEM systems follow the same laws of physics, the DL-RMD is also applicable for many FEM systems, despite the generic EM system in the forward/inversion process mimicking the TEM systems. In general, the FEM systems have a shallower depth of investigation than that of the TEM systems, hence, the S-RMD is best suited for FEM systems. An alternative to the DL-RMD is to generate the resistivity model realizations by following the described methodology for the specific EM system by using the von Kármán models provided (Asif et al., 2022a). This will ensure a 100 % match between resolution, depth of investigation, etc. in the model domain compared to sensitivity in the EM data domain.

Despite the initial von Kármán models with superimposed layering, the resistivity models in the DL-RMD have a pronounced vertical smooth behaviour due to the minimum structure (smooth) regularization scheme (Viezzoli et al., 2008) used in the inversion phase. When applying another regularization scheme in the inversion phase, e.g. the minimum support norm (Vignoli et al., 2015), or when using a few-layer model discretization with no vertical regularization, one could compile a resistivity model database with different appearances. For our DL-RMD, we chose the minimum structure regularization scheme, since it is commonly used for inverting airborne and ground-based EM data. It is important to point out that a TEM data curve itself does not hold information about whether subsurface boundaries are smooth or sharp. As such, both smooth and sharp-layered models will explain the recorded data equally well in most cases. With our approach of compiling resistivity models, we have tried to avoid the inclusion of models with different smooth/sharp behaviours that result in identical or close to identical forward data responses (equivalent models).

The DL-RMD is generated in the resistivity range of 1–2000 $\mathrm{\Omega }$m, which covers most of the geological settings, taking into account the EM mapping capability in the high-resistivity range. The resistivity limit of 2000 $\mathrm{\Omega }$m was chosen since EM methods have no or very low sensitivity in the high-resistivity range, since high-resistivity materials (granite, basalt, glacier ice, etc.) produce an EM signal below the detection level. Despite the 2000 $\mathrm{\Omega }$m limit, the resistivity distribution of the models in the DL-RMD is slightly skewed towards lower resistivities due to the limited sensitivity of the EM method to high-resistivity values. A slight bias towards lower-resistivity values may affect the performance of a DL method for highly resistive models. However, even if an actual subsurface model is represented by a highly resistive model, it is expected that any TEM method would have difficulty in resolving such a model. The RMD also has a limitation in the low-resistivity end, e.g. in settings with seawater and saltwater intrusion, which may result in subsurface materials with resistivity values below 1 $\mathrm{\Omega }$m.

Since the 1-D models of the DL-RMD hold resistivity variations in one dimension (vertical) only, they cannot be used for calculating 2-D or 3-D EM responses. Examples of geological settings where a 1-D approach would be inappropriate include steep-dipping geological structures, thin sheet mineralization, mapping close to or on the shoreline, or areas with strong topographical variations. However, one could apply the same methodology to compile a 2-D or 3-D resistivity database. In this case, one would generate the initial von Kármán models as a 2-D section or 3-D volumes and use a 2-D or 3-D forward and inversion process, which of course would be much more computationally expensive compared to the 1-D case. However, the DL-RMD provided in this study opens up the possibility of exploring more deep learning frameworks, which have reliability and consistency in performance comparisons for 1-D models.

6 Code and data availability

The DL-RMD is freely available at 10.5281/zenodo.7260886 (Asif et al., 2022a), and a ready-to-run demo code in Python Jupyter Notebook that uses the network trained on S-RMD and reproduces the results of this paper is available at https://github.com/rizwanasif/DL-RMD (last access: 17 March 2023) (DOI: 10.5281/zenodo.7740243, Asif, 2023).

The EM modelling code “AarhusInv” used to generate EM forward responses in this study is freely available to researchers for non-commercial activities. The details are available at https://hgg.au.dk/software/aarhusinv (Auken et al., 2015).

7 Conclusion

We have presented a methodology for compiling a geophysically constrained subsurface resistivity model database for applications related to electromagnetic data. We generated three 1-D resistivity databases, discretized to depths of 120, 350, and 500 m in the resistivity range of 1–2000 $\mathrm{\Omega }$m, hence covering various ground-based and airborne frequency-domain and time-domain electromagnetic systems and most of the geological settings. The upper resistivity limit of the model database is satisfactory as the electromagnetic methods have limitations for high resistivity; however, the model database has limitations in the low resistivity limit for subsurface materials below 1 $\mathrm{\Omega }$m that may occur in some cases. Additionally, the database holds 1-D models and therefore inherits the limitations of 1-D electromagnetic modelling.

An example is included using the proposed resistivity model database and deep learning for surrogating TEM forward modelling, showing that high accuracy can be obtained with our resistivity model database. Furthermore, the example shows that the forward/inversion steps in the generation of the database lead to a significantly increased performance in the forward modelling.

Despite some limitations, the generated resistivity model database is a well-organized database, which empowers the geoscience community to have consistency and credibility in the development of deep learning methods for many tasks including surrogating forward modelling, inverse modelling, data de-noising, automatic data processing, etc. Therefore, we urge the geophysical community to utilize the presented database to develop and investigate different network configurations, data pre-processing strategies, loss functions, etc. while using the presented model database to allow for consistency in benchmarking deep learning algorithms. The resistivity model database has already proven valuable in significantly improving the accuracy of neural networks for the forward modelling of electromagnetic data.

Author contributions

Conceptualization: MRA and TB. Data curation, software, and visualization: MRA. Formal analysis, methodology, and investigation: MRA, NF, TB, and AVC. Funding acquisition, project administration, resources, and supervision: JJL and AVC. Validation: NF and MRA. Writing; original draft preparation, review and editing: MRA, NF, TB, JJL, and AVC.

Competing interests

The contact author has declared that none of the authors has any competing interests.

Disclaimer

Publisher’s note: Copernicus Publications remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Special issue statement

This article is part of the special issue “Benchmark datasets and machine learning algorithms for Earth system science data (ESSD/GMD inter-journal SI)”. It is not associated with a conference.

Acknowledgements

The authors would like to thank the handling chief editor Kirsten Elger, topical editor Martin Schultz, and the two anonymous reviewers for their comments and feedback on this paper.

Financial support

This work has been supported by the Innovation Fund Denmark (IFD) under the projects “MapField” (grant no. 8055-00025B) and “SuperTEM” (grant no. 0177-00085B).

Review statement

This paper was edited by Martin Schultz and reviewed by two anonymous referees.