Introduction
Numerical modeling of the coastal ocean is important for many environmental and industrial applications. Typical scenarios include modeling circulation at regional scales, coupled river–estuary–plume systems, river networks, lagoons, and harbors. Length scales range from some tens of meters in rivers and embayments to tens of kilometers in the coastal ocean; water depth ranges from less than a meter to kilometer scale at the shelf break. The timescales of the relevant processes range from minutes to hours, yet typical simulations span weeks or even decades. The dynamics are highly non-linear, characterized by local small-scale features such as fronts and density gradients, internal waves, and baroclinic eddies. These physical characteristics imply that coastal ocean modeling is intrinsically multi-scale, which imposes several technical challenges.
Most coastal ocean models solve the hydrostatic Navier–Stokes equations under the Boussinesq approximation – a valid approximation for mesoscale and submesoscale (1 km) processes. Small-scale processes ( m) are, however, inherently three-dimensional where non-hydrostatic effects can be important, especially in areas with pronounced density structure and stratification . Non-hydrostatic modeling requires very high horizontal mesh resolution, which is currently only feasible in relatively small subregions (e.g., at the mouth of an estuary; ) due to its high computational cost.
Historically, regional ocean models have used structured, (deformed)
rectilinear lattice grids. Although structured grids offer better
computational performance , unstructured grids
are generally preferred in coastal domains as they can better represent the
complex coastal topography and local features
. Due to the large
geometrical aspect ratio of the oceans (length versus depth), most models
utilize computational grids that are layered in the vertical direction.
Typical approaches include the terrain-following sigma levels
, equipotential levels ,
isopycnal coordinates , and their generalizations
In this article, we focus on solving the hydrostatic equations on an unstructured grid. While many unstructured grid models exist, their drawbacks tend to be excessive numerical diffusion that smooths out important physical features and/or high computational cost. To address these issues, we propose a novel finite element solver for the hydrostatic equations, based on discontinuous Galerkin discretization methods.
Maintaining high numerical accuracy is crucial in ocean applications. The ocean is a forced dissipative system where the mixing of water masses only takes place at the molecular level . In practice, however, the finite grid resolution and numerical schemes used by the model introduce mixing rates of tracers and momentum that can be orders of magnitude larger than physical mixing . Such spurious, numerical mixing is often dominated by the discretization of advection , but it can arise from other components as well, such as (implicit) time integration methods or various filters introduced to improve numerical stability . In addition, wetting and drying schemes may introduce additional dissipation in order to stabilize the barotropic equation in the drying regime. We reserve consideration of this important latter topic for a future publication.
In global circulation models, numerical mixing is a major bottleneck as (diapycnal) diffusion is very low in the deep ocean basins and water masses can remain largely unchanged for hundreds of years . Numerical mixing can, however, be a major issue in coastal domains as well: coastal oceans are characterized by strong density gradients, fronts between water masses (e.g., in river plumes), small-scale dynamics (e.g., internal waves and hydraulic jumps), and baroclinic eddies. An overly diffusive model can, therefore, fail to capture many essential physical features of these domains: it can smear out fronts, underestimate the intrusion of saline waters into embayments , or misrepresent mixing in river plumes.
The most common spatial discretization scheme is the finite volume (FV) method, used in the MITgcm , GETM , ROMS , MPAS-Ocean , UnTRIM , FVCOM , SUNTANS , FESOM2 , and others. The FV method is well suited for advection-dominated problems, provides strict conservation of volume and mass, and yields good computational performance. FV methods are nominally only first-order accurate, but higher-order approximations can be introduced by increasing the size of the numerical stencil (e.g., in high-order advection schemes; ).
Some unstructured grid models are based on the continuous Galerkin finite
element (FE) method or hybrid FE–FV formulations. Such models include ADCIRC
, SELFE , and SCHISM ,
and the earlier version of FESOM . The continuous FE method
is ideal for solving elliptic equations but requires stabilization for
advection
In recent years, discontinuous Galerkin (DG) methods have gained attention in geophysical modeling . DG discretization resembles the FV method because it is local (i.e., elements are only connected by inter-element fluxes), fully conservative, and well-suited for advective problems, yet it offers higher-order accuracy. This article presents a DG discretization for the hydrostatic equations. Our goal is to design an efficient unstructured grid solver where numerical accuracy is not compromised. Specifically, we aim to meet the following design criteria:
-
a vertically extruded, layered mesh;
-
accurate representation of free surface dynamics;
-
a second-order accurate, monotone tracer advection scheme;
-
explicit time integration of 3-D variables (except for vertical diffusion); and
-
low numerical mixing.
All numerical ocean models include some form of friction, either in the form
of a numerical closure or a physical parameterization .
Numerical closure involves adding a sufficient amount of dissipation to
maintain numerical stability. There is a wealth of literature about stable
finite volume
In this article, we present an efficient DG implementation of the
three-dimensional hydrostatic equations. The model is implemented in the
Thetis project – an open-source coastal ocean circulation model
freely available online (see
Thetis is implemented using the Firedrake finite element modeling platform
The governing equations are presented in Sect. , followed by their DG finite element discretization in Sect. . The second-order coupled time integration scheme is described in Sect. . Numerical tests are presented in Sect. .
Governing equations
Let be the three-dimensional domain that spans from the sea floor , to the free surface , ; the bottom and top surfaces are denoted by and , respectively. Total water column depth is thus . The two-dimensional horizontal domain is denoted by .
The horizontal momentum equation reads where , and denote the horizontal and vertical velocity, respectively; is the horizontal gradient operator; denotes the cross product operator; is the Coriolis parameter; is the vertical unit vector; is the pressure; and and are the horizontal and vertical diffusivity, respectively. Water density is defined as , , , where and stand for temperature and salinity, respectively, and is a constant reference density.
Under the hydrostatic assumption, the horizontal pressure gradient can be written as a combination of external, internal, and atmospheric pressure gradients: where is the atmospheric pressure acting on the sea surface, and is the baroclinic head. For brevity, the internal pressure gradient field is denoted as .
Neglecting atmospheric pressure, the full horizontal momentum equation reads Vertical velocity is diagnosed from the continuity equation: Water temperature and salinity are modeled with an advection–diffusion equation of the form where and stand for the horizontal and vertical (eddy) diffusivity, respectively.
At the bottom boundary, we impose quadratic bottom stress:
where is the drag coefficient, and is the velocity in
the middle of the bottommost element. , , is the
outward normal vector, and , , 0) its horizontal
projection. The bottom boundary condition is treated implicitly;
Eq. () is linearized by keeping the magnitude
fixed at the “old” value while solving for
(and ). Typically, is computed from the logarithmic law
of the wall
Mode splitting
Following , we split the horizontal velocity field into depth-averaged and deviation components. The depth-averaged momentum equation is then defined as where is a forcing term used to couple the 2-D and 3-D modes. This equation is complemented with the depth-averaged continuity (free surface) equation: The 2-D system (Eqs. –) contains the fast-propagating, rotational surface gravity waves. The corresponding equation for is obtained by subtracting Eq. () from Eq. () : Note that the advection and viscosity terms are included in Eq. () without splitting, based on the assumption that these processes are slow enough to be captured with long time steps. The Coriolis term, on the other hand, only contains the slow modes. The vertical velocity only appears in the advection term, which is not split, and thus there is no need to split .
Coupling 2-D and 3-D modes
The 2-D and 3-D modes are coupled using the additional term . First, the 3-D momentum equation (Eq. ) is solved with , resulting in a velocity field that has a non-zero depth average, generated by the advection and viscosity terms (that depend on ). We then compute the depth-averaged and apply a correction: to enforce zero depth average. By definition, the field is a constant over the vertical, and it will be used as a forcing term in the 2-D momentum equation (Eq. ) in the subsequent solve. This procedure ensures that Eqs. () and () sum up to Eq. () and .
Equation of state
In this paper, a linear equation of state is used: where and are the thermal expansion and saline contraction coefficients, respectively, and and are reference temperature and salinity. In all the test cases presented herein, salinity does not contribute to water density (). Thetis also implements a full non-linear equation of state .
Viscosity and turbulence closure
Baroclinic flows require some form of viscosity to filter out grid-scale noise. In this paper, we only consider Laplacian horizontal viscosity, set to a constant Re corresponding to the velocity scale , horizontal mesh resolution , and the desired grid Reynolds number Re. Here, the velocity scale is taken as a global constant specific to each test case. Unless otherwise specified, the horizontal diffusivity of tracers is zero.
In most test cases, vertical viscosity is set to a constant. In certain cases, we use the gradient Richardson number dependent parameterization by : where is the gradient Richardson number, is the buoyancy frequency, and is the vertical shear frequency. The background values are set to m s, while maximum viscosity is set to m s; the dimensionless parameters are and . More sophisticated turbulence closures will be addressed in future work.
Finite element discretization
This section describes the spatial discretization of the governing equations. In Sect. , we define the finite element function spaces, followed by the weak forms of the underlying equations.
Prognostic and diagnostic variables and their function spaces.
Field | Symbol | Equation | Function space |
---|---|---|---|
Prognostic variables | |||
Water elevation | () | ||
Depth av. velocity | () | ||
Horizontal velocity | () | ||
Water temperature | () | ||
Water salinity | () | ||
Diagnostic variables | |||
Vertical velocity | () | ||
Water density | () | ||
Baroclinic head | () | ||
Int. pressure grad. | () |
Function spaces
The prognostic variables of the coupled 2-D–3-D system (Eqs. , , , ) are , , , , and . Diagnostic variables include the vertical velocity , water density , baroclinic head , and internal pressure gradient . The choice of function spaces where these variables reside is crucial for numerical stability and accuracy.
Our discretization is based on the linear discontinuous Galerkin function space, . The 2-D system is discretized with a velocity–pressure finite element pair: water elevation and both components of the depth-averaged velocity are approximated in space, i.e., , . When embedded with appropriate Riemann fluxes at element interfaces, the element pair is well suited for rotational shallow water problems .
Achieving an accurate and monotone 3-D tracer advection scheme is one of our main design criteria. The tracers, therefore, are also considered within a discontinuous function space, , (here, the operator stands for the Cartesian product of function spaces in the extruded mesh: horizontal vertical function space). Tracer consistency (sometimes called local tracer conservation) is a necessary condition for monotonicity; it ensures that a constant tracer field does not exhibit spurious local extrema. In practice, it implies that the discrete tracer equation must reduce to the discrete continuity equation for a constant tracer. In this work, we satisfy this property by requiring the vertical velocity to belong to the tracer space . In addition, compatibility between the 2-D and 3-D momentum equations requires that the 3-D horizontal velocity must be in the horizontal direction. We therefore set as well. The used function spaces are summarized in Table .
Note that this choice of function spaces is not mimetic : the discrete system does not preserve all the properties of the continuous equations; for example, enstrophy is not conserved exactly. As the coastal ocean is generally very dissipative, maintaining mimetic properties is, however, not crucial. It is possible to define a mimetic discretization as well, for example, using Raviart–Thomas elements for the velocity, i.e., element pair RT . Our preliminary experiments, however, indicate that this choice significantly increases the computational cost of the system, without a corresponding improvement in accuracy. Formal assessment of the performance of mimetic discretizations in coastal ocean applications will be investigated in the future.
In the weak forms, we use the following notation for volume and interface integrals: In interface terms, we additionally use the average and jump operators for scalar and vector fields: where the superscripts “” and “” arbitrarily label the values on either side of the interface, and is the outward unit normal vector of each element on the interface.
2-D system
Let stand for the triangulation of the 2-D domain . The set of element interfaces is denoted by , and , the outward unit normal vector of an interface . For brevity, boundary conditions are omitted from the weak forms.
Let and be test functions in the 2-D function spaces. The weak formulation of the 2-D system then reads: find , such that Here, the divergence and external gradient terms have been integrated by parts. The resulting interface terms are defined on the element edges where the state variables , are not uniquely defined. The values , are obtained from an approximate Riemann solver; here, we use the linear Roe solution and .
Momentum equation
Let denote the set of prisms of the 3-D domain ,
obtained from a vertical extrusion of . The set of horizontal and
vertical interfaces is denoted by and ,
respectively. Let be a test function. The
weak formulation of the 3-D momentum equation then reads: find such that
Here, the advection and viscosity terms have been integrated by parts
Tracer equation
The weak formulation of the tracer equations is derived analogously: find such that Note that we do not employ the Lax–Friedrichs flux in the tracer equation.
Symmetric interior penalty stabilization
The presented discretization is unstable for elliptic operators, and the diffusion operators require additional stabilization. Here, we use the symmetric interior penalty Galerkin (SIPG) method . The SIPG formulation of the tracer diffusion operators read For the viscosity terms, we get The penalty factor is defined as , where is the degree of the basis functions, is a factor depending on mesh quality, and is the local element length scale in the normal direction of the interface. Let and denote the horizontal and vertical element sizes, and , , . We then define . In this paper, we use .
Continuity equation
The vertical velocity is computed diagnostically from the continuity equation (Eq. ) by solving the weak form: find such that where both the left- and right-hand sides have been integrated by parts. Note that the terms on the bottom surface vanish due to the impermeability constraint .
Computing the internal pressure gradient
The water density is computed diagnostically using the equation of state. We use the same function space for tracers and water density. In this work, we use a linear equation of state (Eq. ), and consequently density can be computed locally (at each node of the tracer field). In general, however, the equation of state is non-linear, and the density is projected on the field.
The baroclinic head is computed from Eq. () by integrating over the vertical. In practice, we solve equation weakly with the appropriate boundary conditions: Here, the left-hand side has been integrated by parts, and denotes the value on the prism above the interface. Note that the free surface terms vanish because on by definition. We use function space for to alleviate internal pressure gradient errors .
Finally, taking a test function , we compute the internal pressure gradient with the weak form where the right-hand side has been integrated by parts. Usually, belongs to the same space as the horizontal velocity, i.e., . However, to reduce bathymetry induced internal pressure gradient errors, it is possible to use a quadratic horizontal space, i.e., and . In this paper, we use a linear field unless otherwise specified.
Slope limiters
We use vertex-based slope limiters for three-dimensional variables to ensure positivity. The limiter is applied to both tracer and horizontal velocity fields after each update of the advection operator as discussed in the next section.
Time integration
The coupled 2-D–3-D system is advanced in time with a two-stage ALE time integration scheme. In this section, we present the ALE formulation and summarize the final time integration scheme.
ALE mesh formulation
To accurately account for the free surface movement, one must move the mesh in the vertical direction. In this work, we adopt the ALE method . Here, we describe a mesh update procedure that stretches (or compresses) the mesh uniformly over the vertical direction. The ALE formulation, however, allows more complex mesh-moving methods as well, such as the (approximate) tracking of isopycnals .
In three dimensions, an ALE update consists of solving an advection–diffusion equation between two domains, and . Here, the domain is uniquely defined by the surface elevation field, such that for any time level the surface matches . Due to the chosen discretization, the elevation field is discontinuous, yet we wish to maintain a conforming mesh, i.e., a continuous coordinate field . This is achieved by projecting the elevation field to a continuous space and updating the geometry with the continuous field . The projection induces a small discrepancy between the elevation field and the 3-D domain, but its effect remains negligible in practical applications because jumps in the elevation field are typically small.
Let be the reference domain corresponding to unperturbed elevation field , and , 0] its vertical coordinate. Applying a uniform mesh stretching, the time-dependent mesh coordinates can then be written as The mesh velocity is obtained as . In practice, the consecutive fields and are known so we can evaluate Given the mesh velocity, a conservative ALE update can be written as for a generic tracer equation , , .
Coupled time integration scheme
The coupled 2-D–3-D system is advanced in time with a two-stage ALE time integration scheme. For convenience, we rewrite the 3-D momentum and tracer equations as where and denote all the terms that are treated explicitly, while and contain all the implicit terms. In this work, only vertical diffusion (Eq. ), vertical viscosity (Eq. ), and bottom friction terms are treated implicitly.
The explicit 3-D equations are advanced in time with a second-order SSP Runge–Kutta scheme, SSPRK(2,2) . For a generic problem (), the scheme reads
When applied to the explicit 3-D momentum and tracer equations, Eqs. () and (), both of these stages are ALE updates where the mesh is updated from geometry to and then . The ALE formulation of the explicit 3-D tracer equation can then be written as where the vertical velocity is adjusted by the mesh velocity .
After the SSPRK update, the implicit terms are advanced with the backward Euler method. This step is computed in domain : The 3-D momentum equation is treated analogously.
The 2-D equations are advanced in time with an implicit scheme to circumvent the strict time step constraint imposed by surface gravity waves. To ensure consistency between the movement of the 3-D mesh and the 2-D mode, the 2-D time integration scheme must be compatible with the aforementioned SSPRK(2,2) method. Here, we use a combination of a forward Euler and trapezoidal steps: Denoting the tendencies of the 2-D system (Eqs. –) by and , respectively, we can write the 2-D solver as The second implicit stage is linearized by treating the total depth explicitly in Eq. ().
The 2-D system is solved first, resulting in an updated elevation field ( and for the two stages, respectively) and consequently mesh geometry ( and ). Once the mesh geometry is known, it is straightforward to compute the corresponding mesh velocity and perform a 3-D ALE update.
The time integration method is second order for all the terms. The whole algorithm is summarized in Algorithm 1.
Choosing the time step
The maximal admissible time step is constrained by the stability of the coupled time integrator. The presented SSPRK(2,2) scheme has a CFL (Courant–Friedrichs–Lewy) factor 1. The 2-D scheme (Eq. ) and the implicit vertical solver (Eq. ), on the other hand, are unconditionally stable. This implies that the coupled system is stable under the same conditions as the explicit SSP scheme on its own.
[Figure omitted. See PDF]
The horizontal advection term imposes a constraint: where is the horizontal element size, is the maximal horizontal velocity scale, and is a length scaling factor. For the presented discretization, we take as the square root of the triangle area. For rectangular elements and second-order RK schemes, the scaling factor is approximately . In this work, we use for all the diagnostic test cases. In strongly stratified flows, internal waves may impose a stricter constraint: where is the speed of the internal waves.
Analogously, the time step constraint for vertical advection is where is the element height, is the vertical velocity scale, and is the scaling factor.
Given a horizontal viscosity scale , the explicit viscosity operator imposes a constraint: which may become stringent for small elements and large viscosity values. The scaling factor depends on the used stabilization scheme; here, a value of is used. The constraint for horizontal diffusivity is analogous.
In the simulations presented herein, the minimal admissible time step is evaluated on the mesh based on constant a-priori velocity and viscosity scales. The time step is kept constant throughout the simulation.
Test cases
We demonstrate the performance of the proposed discretization with a suite of test cases of increasing complexity. We first evaluate the conservation and convergence of the solver in a barotropic standing wave test case. The convergence of baroclinic terms is then examined in a specific steady-state test case. The baroclinic solver and its numerical mixing are then evaluated with a non-rotating lock exchange test case and a rotating baroclinic eddy test, followed by the Dynamics of Overflow Mixing and Entrainment (DOME) overflow test.
Standing wave
We first evaluate the performance of the solver in a barotropic standing wave test case. The domain is a km long rectangular channel, 625 m wide, and 100 m deep. All lateral boundaries are closed. Initially, the water is at rest. A 10 m tall sinusoidal elevation perturbation is prescribed along the channel (, m), leading to a non-linear wave as the simulation progresses.
The simulation is run for two full wave periods, approximately 3831.31 s. To investigate tracer conservation and consistency properties, two passive tracers are included: salinity is set to a constant 4.5 psu, while temperature varies between 5.0 and 15.0 C along the channel ( C).
The domain was discretized with a split-quad mesh using 40 elements along the channel (1500 m edge length) and four vertical layers. The time step is s, chosen to meet the horizontal advection condition.
During the simulation, the volume of the 3-D domain was conserved to accuracy . The “2-D volume”, i.e., the integral of the elevation field, was conserved to accuracy . Salinity remained at constant 4.5 psu with a small deviation. The total mass of salinity and temperature were both conserved to accuracy . Over- and undershoots in the temperature field were negligible due to the slope limiters. Without the limiter, temperature overshoots were . These results show that the model indeed fully conserves volume and tracers and does not exhibit overshoots. Moreover, the tracer consistency property is satisfied, verifying the integrity of the ALE formulation.
To investigate the order of convergence of the solver, we used a smaller initial elevation perturbation ( cm). In this case, the resulting standing wave is close to linear. At the end of the simulation, the solution was compared to the analytical solution of the linear wave equation (which coincides with the initial condition) by computing the error, .
Convergence of the error in the standing wave test case. Tested element sizes were 3000, 1500, 1000, 750, 500, 375, and 300 m. The number indicates the slope of the least-squares best-fit line (dashed line).
[Figure omitted. See PDF]
We ran the simulation, varying the horizontal mesh resolution between 3 km and 300 m; the number of vertical levels varied between 2 and 20. In each case, the channel was made one element wide, and the time step was chosen to meet the CFL criterion for horizontal advection. At the end of the simulation, the error was computed for water elevation and velocity (see Fig. ). The velocity field shows the expected second-order convergence, whereas elevation converges at a rate of 3.2. It is known that shallow water equations models may exhibit superconvergence properties, especially for the elevation field . Here, our results verify that the solver behaves as expected and yields second-order accuracy under barotropic forcing.
Baroclinic MMS test
Verifying model accuracy under baroclinic forcing is more challenging as no
analytical solutions are available. Here, we use the method of manufactured
solutions
Salinity is set to a constant 35 psu. We use the linear equation of state (Eq. ) with kg m, kg m C, and C. For the sake of simplicity, bathymetry is constant and elevation is set to zero initially. Coriolis frequency was set to a constant s. Bottom friction, viscosity, and diffusivity are omitted.
Without any additional forcing, the initial conditions lead to a time-dependent solution. Following the MMS strategy, we add analytical source terms in the dynamic equations that cancel all the active terms in the equations, leading to a steady-state solution. The remaining error is purely the discretization error of the advection, pressure gradient, and Coriolis operators. The source terms are derived analytically and projected to the corresponding function space. The analytical formulae are given in Appendix .
The coarsest mesh contains four elements both in and directions and two vertical levels. We refine the mesh up to 10 times (40 elements and 20 vertical levels) and compute the error of the prognostic fields against the exact solutions. In each case, the model is run for 50 iterations with a time step chosen to meet the CFL condition.
The variation of the errors with resolution is shown in Fig. . All the prognostic variables exhibit the correct second-order convergence rate. The diagnostic vertical velocity field (which depends on the divergence of ) converges linearly as expected. Therefore, we conclude that advection, pressure gradient, and Coriolis terms are discretized correctly. We have also developed similar MMS tests for the diffusivity/viscosity operators and the bottom friction term, all of which show second-order convergence as well (not shown).
Convergence of the error in the baroclinic MMS test case. The mesh was refined 1, 2, 4, 6, 8, and 10 times, resulting in resolutions of 2500, 1250, 625, 416.67, 312.5, and 250 m (shortest edge of the triangle). The time steps were 25.0, 12.5, 6.25, 4.167, 3.125, and 2.5 s, respectively. The number indicates the slope of the least-squares best-fit line (dashed line).
[Figure omitted. See PDF]
Lock exchange
The validity of the baroclinic solver and its level of spurious mixing is investigated with the standard lock exchange test case . Here, we follow the setup of and : The domain is a 64 km long and 1 km wide rectangular channel. Water depth is 20 m. Initially, the left-hand side of the domain is filled with dense water mass ( C) compared to the water on the right ( C). Salinity is kept at constant 35 psu. We use the same linear equation of state as in Sect. , resulting in a density difference of kg m. The domain is discretized with a regular split-quad mesh. The triangle edge length is 500 m and 20 equidistant levels are used in the vertical direction.
Stabilizing the internal pressure gradient requires some form of friction. To this end, we apply a constant Laplacian horizontal viscosity, using values , 10.0, 100.0, and 200.0 m s. These values correspond to grid Reynolds numbers Re, 25.0, 2.5, and 1.25, respectively, where the characteristic velocity scale of the exchange flow is m s. Vertical viscosity is set to a constant 10 m s. Bottom friction is omitted.
Figure shows the initial density field and
solution after 17 h of simulation for the three cases. Higher background
viscosity leads to a less noisy velocity field and therefore sharper density
front. The sharpness and shape of the fronts are similar to results presented
in the literature
Assuming that, in the absence of bottom friction, all available potential energy is transformed into kinetic energy, the maximum front propagation speed can be estimated as . Figure a shows the propagation of the front location at the bottom of the domain (the front at the surface behaves comparably). All three simulations are in good agreement with the theoretical propagation speed. The simulated front propagation speed is underestimated by roughly 5 %, indicating loss of energy due to mixing. These results are similar to results reported in the literature; e.g., show similar performance for ROMS, MITgcm, and MOM.
Figure b shows the maximum over- and undershoots in the temperature field during the simulation. Even in the low viscosity case (Re), the overshoots are of order 10 C, indicating that the tracer advection scheme is indeed close to monotone, due to the SSP time integration method and slope limiters. Note that, if the slope limiter is omitted, the overshoots can reach 30 C.
Water density in the lock exchange test case in the center of the domain ( km). (a) Initial condition. Solution after 17 h of simulation with Re (b) 1.25, (c) 2.5, (d) 25.0, and (e) 250.0.
[Figure omitted. See PDF]
Diagnostics of the lock exchange test. (a) Location of the density front at the bottom of the domain, (b) over- and undershoots in the temperature field (with regard to to 30.0 and 5.0 C, respectively), and (c) normalized reference potential energy (RPE) versus simulation time.
[Figure omitted. See PDF]
To diagnose the role of spurious mixing, we use the reference potential energy (RPE; ). RPE is computed as the vertical center of mass of a sorted density field : RPE . The field is defined as the unique, stratified density field where the densest water parcels are distributed over the bottom, and density increases monotonically over the water column. As such, is the steady-state density distribution, and RPE represents the portion of potential energy that cannot be transformed into kinetic energy. Mixing the two water masses increases RPE (the center of mass), and thus the amount of unavailable potential energy increases. Figure c shows the evolution of normalized RPE, , during the simulation. At h, the values are 0.612, 1.13, 2.35, for the four simulations. These results are in good agreement with those reported with MPAS-Ocean model : with the same mesh resolution, MPAS-Ocean shows slightly larger normalized RPE, for example, at h in the case of Re. The difference is likely due to the different spatial discretization ( instead of finite volumes) or differences in the numerical viscosity operator. Applying slope limiters to the velocity field is not necessary for numerical stability, but it reduces high-frequency noise in the velocity field and hence results in lower RPE values.
In order to investigate the role of the Lax–Friedrichs flux on numerical mixing, we ran the lock exchange test case with zero viscosity. After 17 h of simulation, the RPE value was approximately . When the Lax–Friedrichs flux was omitted, a similar RPE value was obtained with viscosity m s. Therefore, in this particular test case, the Lax–Friedrichs flux introduces mixing that is roughly equivalent to 3 m s viscosity, corresponding to Re. When viscosity was non-zero, it was evident from the numerical simulations that the Lax–Friedrichs flux has a negligible impact on numerical mixing if Re 10 (not shown).
Baroclinic eddies
We investigate the model's ability to generate baroclinic eddies with the eddying channel test case of . This test case is an idealization of the Antarctic Circumpolar Current, the domain spanning 500 and 160 km in the meridional and zonal directions, respectively. The domain is 1000 m deep. At the zonal boundaries, periodic boundary conditions are applied; northern and southern boundaries are closed. The Coriolis parameter is taken as a constant s.
Initially, the domain is linearly stratified with warmer water at the surface. In addition, the northern half of the domain is warmer, with a narrow sinusoidal transition band separating the warm (northern) and cold (southern) water masses (Fig. ; see for the definition of the initial temperature field). Water temperature ranges between 10 and 20 C. A linear equation of state is used with kg m, kg m C and C. Salinity is kept at constant 35 psu and it does not affect density (). Bottom friction is parameterized by a constant drag coefficient of .
The baroclinic Rossby radius of deformation is 20 km . Horizontal mesh resolution is constant in space. We use a regular split-quad mesh with two different mesh resolutions: eddy-permitting 10 km and a finer 4 km resolution. In the vertical direction, 26 and 40 equidistant sigma levels are used in the two cases, respectively. Simulations are carried out with different values of horizontal viscosity, with the grid Reynolds number ranging from 2 to 100. The different setups are summarized in Table . Vertical viscosity is set to a constant 10 m s.
As the simulation progresses, baroclinic eddies develop at the center of the domain, quickly propagating elsewhere. This is a spin-down experiment, i.e., the domain is a closed system with no forcing at the boundaries. Therefore, all the energy in the system originates from the initial potential energy, which is being dissipated during the simulation; again, the RPE is used as a metric for the energy transfer or the loss of energy due to mixing.
Experimental setup for baroclinic eddy test case. Listed are the horizontal mesh resolution (min. triangle edge length), number of vertical levels, time step, horizontal viscosity, and the approximate grid Reynolds number.
Re | |||||
---|---|---|---|---|---|
(km) | (s) | (m s) | |||
10 | 26 | 348.39 | 10.0 | 100 | |
10 | 26 | 348.39 | 20.0 | 50 | |
10 | 26 | 348.39 | 50.0 | 20 | |
10 | 26 | 348.39 | 125.0 | 8 | |
10 | 26 | 348.39 | 200.0 | 5 | |
10 | 26 | 348.39 | 500.0 | 2 | |
4 | 40 | 140.26 | 4.0 | 100 | |
4 | 40 | 140.26 | 8.0 | 50 | |
4 | 40 | 140.26 | 20.0 | 20 | |
4 | 40 | 140.26 | 50.0 | 8 | |
4 | 40 | 140.26 | 200.0 | 2 |
Figure shows the surface temperature fields at various time intervals up to 200 days after the initialization for different values of horizontal viscosity. As expected, the model captures more mesoscale features as viscosity is decreased. Qualitatively, the results are in agreement with ROMS and MITgcm results , as well as MPAS-Ocean , all of which use a comparable Laplacian scheme for horizontal viscosity.
The evolution of the normalized RPE during the simulation is shown in
Fig. a for the 4 km mesh resolution. The amount
of mixing clearly depends on the grid Reynolds number, with RPE being roughly
twice as high for Re compared to Re. The average
rate of change of RPE, averaged over days 3 to 319, is shown in
Fig. b for all the simulations. As expected, the rate of
change increases with larger grid Reynolds number and with a coarser mesh.
These RPE metrics are in good agreement with results in the literature. At
Re Thetis , values are and
W m, for the 10 and 4 km resolutions.
The corresponding values for MITgcm, Modular Ocean Model (MOM), and Parallel Ocean Program (POP) (averaged over days 3 to 319) are larger: at least and W m,
respectively
Sea surface temperature fields for the eddying channel test case at 4 km horizontal mesh resolution. Horizontal viscosity is 200 (a), 50 (b), and 20 m s (c). These values correspond to mesh Reynolds numbers 2, 8, and 20, respectively.
[Figure omitted. See PDF]
The test cases were run on a Linux cluster with 16-core Intel Xeon E5620 processors and Mellanox Infiniband interconnect. The 320-day simulation took roughly 42 h to run on 96 cores with the 4 km resolution mesh and 140.26 s time step. It should be noted, however, that the time step employed here is smaller than the maximal allowed time step. We also carried out a strong scaling test with the 4 km mesh. In the scaling test, the simulation was run for 40 time steps, recording the total elapsed wall-clock time and time spent in different parts of the solver. Figure a shows the overall speed-up up to 96 processors. The scaling efficiency drops to roughly 50 % at 96 cores, when the local degree of freedom count for the tracer field is 25 000 (see black line in Fig. b). This scaling efficiency is close to typical Firedrake performance .
Diagnostics of the eddying channel test case. (a) Evolution of normalized RPE over time in the eddying channel test case for 4 km mesh resolution. (b) Rate of change of RPE for different grid resolutions and grid Reynolds numbers. The rate of change was evaluated by computing the average RPE change from days 3 to 319.
[Figure omitted. See PDF]
Strong parallel scaling for the baroclinic eddies test case on a 4 km mesh ( m s): (a) speed-up in wall-clock time versus number of processes; (b) parallel efficiency versus the local number of degrees of freedom (DOFs) in the 3-D tracer field (top axis) and the 2-D (, ) mixed system (bottom axis). The black line is the wall-clock time; colored lines stand for the time spent in different implicit or explicit solvers. The vertical dashed lines indicate 20 000 DOFs per process for the 2-D and 3-D problems, respectively. The mesh consisted of 10 000 triangles, 40 vertical levels, and 400 000 prisms.
[Figure omitted. See PDF]
The scaling efficiency of the separate solvers is plotted with colored lines in Fig. b. The implicit vertical diffusion/viscosity solvers perform best due to the fact that the problem is purely local without any horizontal dependencies. The explicit momentum solver scales almost as well, whereas the explicit tracer solver scales worse. The implicit 2-D solver (assembly and linear solve) scales the poorest because the problem is relatively small; at 96 cores, there are only around 940 degrees of freedom in the (, ) system per core. We have also experimented with explicit 2-D solvers, but they do not scale significantly better compared to the two-stage implicit scheme used herein.
To further assess the CPU cost, we compared Thetis timing against the SLIM 3-D model which uses a similar DG formulation but is implemented in C/C. The wall-clock time, and parallel efficiency used by both Thetis and SLIM 3-D are presented in Appendix . The setup, mesh, and time step were identical for the two models. On a single core, Thetis runs 3.3 times faster. On 24 cores, the ratio is 4.0, and on 144 cores Thetis is still 2.2 times faster than SLIM 3-D. This highlights the fact that Firedrake can deliver good parallel performance compared to models written in lower level languages.
It should be noted, however, that Thetis performance is currently not fully optimized. We expect that the performance can be significantly improved both in terms of serial and strong scaling performance. These will be addressed in future work.
Horizontal mesh and bathymetry for the DOME test case. The domain is extended 120 km further to the west to avoid boundary effects (shaded region). Horizontal element size ranges from 6 to 22 km. There are triangles in the horizontal mesh and 24 uniformly distributed vertical levels resulting in prisms and tracer degrees of freedom.
[Figure omitted. See PDF]
DOME
Next, we investigate the model's ability to simulate density-driven overflows with the DOME benchmark . The domain is a 1100 by 600 km large basin, whose depth varies linearly from 600 m at the northern boundary to 3600 m in the middle of the domain (see Fig. ). To avoid boundary condition issues, we have extended the domain to the west by 120 km. At the northern boundary, there is a 100 km wide and 200 km long inlet. Initially, the basin is stably stratified with a linear temperature variation from 10 C in the deepest part of the basin to 20 C at the surface. We use the linear equation of state with kg m, kg m C, and C, resulting in a kg m density difference.
At the inlet, a dense inflow (temperature 10 C) is prescribed in the bottom layer, with the surface layer being at 20 C. The inflow is in geostrophic balance, the thickness of the bottom layer being roughly 300 m on the eastern end of the boundary diminishing exponentially westward . The total inflow in the bottom layer is 5 Sv ( m s), the surface layer being static. During the simulation, the fate of the inflowing waters is tracked with a passive tracer that is initially zero in the basin and unity at the inlet. Initially, the tracer field is set to the inflow conditions in the northern part of the basin ( km). Velocity is set to zero everywhere. The eastern and southern boundaries of the basin are closed. The western boundary is open with radiation boundary conditions and a 100 km wide band where the temperature is relaxed to the initial condition.
The domain is discretized with an unstructured grid (Fig. ). Horizontal mesh resolution is 6 km near the northern boundary, increasing southward. Overall, 24 vertical sigma levels are used. Over the slope, the mesh resolution was designed to result in a hydrostatic consistency metric () . Horizontal viscosity is set to a constant 50 m s, which corresponds to Re at the inlet. Horizontal diffusivity is constant at 10 m s. Vertical viscosity and diffusivity are parameterized by the Pacanowski–Philander scheme as described in Sect. . Bottom friction is parameterized with a quadratic drag coefficient . A quadratic function space is used for the baroclinic head and internal pressure gradient as discussed in Sect. .
As the inflowing current reaches the basin, it turns to the west and forms a
coastal plume that is approximately 150 km wide
(Fig. ). The plume detaches from the lateral boundary
as it flows westward and along the bottom slope. As the dense water mass
meets the stratified ocean, the plume becomes unstable and starts to generate
eddies and internal waves. The most vigorous eddies are found in the first
300 km after the inlet (–800 km), after which
the plume is more mixed and quiescent. Overall, the plume is shallow; most of
the passive tracer is concentrated within 200 m of the bottom.
Qualitatively, the extent and propagation of the plume, and its eddy
structure are in good agreement with the literature
Figure shows the distribution of the inflowing tracer concentration as a function of water density and the axis. The inflowing waters are initially very dense but get mixed to lower density as the plume advances along the coast. The histogram shows that the plume volume is low in the first 150 km after the inlet (–800 km) where the plume accelerates. After km, the plume slows down and starts to accumulate in volume. The density of the main plume occupies ranges from 0.8 to 1.5 kg m, the peak being around 1.28 kg m. The rate of entrainment can be used as a metric for mixing. Results herein are similar to those presented in literature: present a mean density anomaly of 1.5 kg m for their terrain-following FESOM model configuration.
The 47-day simulation took roughly 42 h to run on 90 cores with a 39.65 s time step on the same Linux cluster.
Bottom tracer concentration in the DOME test case after 10 (a), 20 (b), and 40 days (c).
[Figure omitted. See PDF]
Histogram of tracer in the DOME test case versus the coordinate and density class. At the mouth of the inlet ( km), the inflowing waters are dense; they get entrained higher up in the density spectrum as they are being transported downstream. The data are averaged over 1 week after day 40.
[Figure omitted. See PDF]
Conclusions
This paper describes a DG implementation of an eddy-permitting, unstructured
grid coastal ocean model. The solver is second-order accurate in space and
time. We have demonstrated that the formulation is fully conservative and
preserves monotonicity. The test cases indicate that the model is capable of
reproducing the expected physical behavior, including baroclinic eddies.
Moreover, numerical mixing is well-controlled and comparable to other
established structured grid models, such as MITgcm and ROMS, and the
large-scale finite volume model MPAS-Ocean. Finding an accurate formulation
is important, as commonly used unstructured grid models tend to be overly
diffusive, preventing accurate modeling of certain coastal domains
Future work will include solving the equations on a sphere, DG implementation of a biharmonic viscosity operator, two-equation turbulence closure models, wetting–drying treatment, and development of an adjoint solver, as well as improving the computational efficiency and parallel scaling of the solver.
All code used to perform the experiments in this papers is
publicly available. Firedrake, and its components, may be obtained from
For reproducibility, we also cite archives of the exact software versions
used to produce the results in this paper. All major Firedrake components
have been archived on Zenodo . This record collates
DOIs for the components and can be installed following the instructions at
No external data were used in this paper.
Source terms for the baroclinic MMS test
Using the analytical velocity and temperature fields, we can derive the steady-state solution for the remaining fields:
Now, we can evaluate the different terms that appear in the momentum and tracer equations: These terms are added as source terms to the right-hand side of Eqs. (), (), (), and (). In the weak form, this corresponds to multiplying the analytical function by the test function and integrating over the domain. The solutions were derived using the SymPy symbolic mathematics Python library .
CPU cost comparison against SLIM
A strong scaling test was carried out with both Thetis and the SLIM 3-D model using the baroclinic eddies test case. These tests were carried out on a Linux cluster with 16-core Intel Xeon E5620 processors and Mellanox Infiniband interconnect. The total time spent to run 40 time steps is presented in Table . The table also lists the speed-up , where stands for the wall-clock time for cores, and the parallel efficiency . For an ideal model, the parallel efficiency remains at unity. The results show that on a single core Thetis runs approximately 3.3 times faster than SLIM. On 24 cores, the ratio is 4.0, and on 144 cores, Thetis is still 2.2 times faster.
CPU time in the baroclinic eddies test case for Thetis and the SLIM 3-D model. Both models ran on identical triangular mesh (4 km resolution, 40 vertical levels) using m s and a 140 s time step. The wall-clock time was recorded over 40 iterations.
No. of | Wall-clock time (s) | Ratio | Speed-up | Efficiency | ||||
---|---|---|---|---|---|---|---|---|
cores | Thetis | SLIM | Thetis | SLIM | Thetis | SLIM | ||
1 | 1778.71 | 5928.32 | 3.33 | 1.00 | 1.00 | 1.00 | 1.00 | |
2 | 1034.64 | 4802.34 | 4.64 | 1.72 | 1.23 | 0.86 | 0.62 | |
4 | 500.11 | 2380.74 | 4.76 | 3.56 | 2.49 | 0.89 | 0.62 | |
8 | 290.61 | 1284.08 | 4.42 | 6.12 | 4.62 | 0.77 | 0.58 | |
16 | 206.97 | 675.14 | 3.26 | 8.59 | 8.78 | 0.54 | 0.55 | |
20 | 141.17 | 524.61 | 3.72 | 12.60 | 11.30 | 0.63 | 0.57 | |
24 | 110.83 | 440.09 | 3.97 | 16.05 | 13.47 | 0.67 | 0.56 | |
32 | 88.03 | 330.00 | 3.75 | 20.21 | 17.96 | 0.63 | 0.56 | |
40 | 73.17 | 260.47 | 3.56 | 24.31 | 22.76 | 0.61 | 0.57 | |
48 | 64.16 | 222.79 | 3.47 | 27.72 | 26.61 | 0.58 | 0.55 | |
64 | 56.62 | 158.31 | 2.80 | 31.41 | 37.45 | 0.49 | 0.59 | |
80 | 49.48 | 127.95 | 2.59 | 35.95 | 46.33 | 0.45 | 0.58 | |
96 | 43.64 | 109.10 | 2.50 | 40.76 | 54.34 | 0.42 | 0.57 | |
112 | 39.68 | 95.24 | 2.40 | 44.83 | 62.25 | 0.40 | 0.56 | |
128 | 36.91 | 83.37 | 2.26 | 48.19 | 71.11 | 0.38 | 0.56 | |
144 | 35.76 | 78.05 | 2.18 | 49.74 | 75.96 | 0.35 | 0.53 |
TK designed and implemented most of the solver and carried out the numerical simulations. SK and LM contributed to the design and implementation of the model. AB, DH, and MP supervised the work and guided the implementation of the model and the manuscript.
The authors declare that they have no conflict of interest.
Acknowledgements
The National Science Foundation partially supported this research through cooperative agreement OCE-0424602. The National Oceanic and Atmospheric Administration (NA11NOS0120036 and AB-133F-12-SE-2046), Bonneville Power Administration (00062251), and Corps of Engineers (W9127N-12-2-007 and G13PX01212) provided partial motivation and additional support. This work was supported by the UK's Engineering and Physical Science Research Council (grant numbers EP/M011054/1, EP/L000407/1) and the Natural Environment Research Council (grant number NE/K008951/1). This work used the Extreme Science and Engineering Discovery Environment (XSEDE), which is supported by National Science Foundation grant number ACI-1053575. The authors acknowledge the Texas Advanced Computing Center (TACC) at the University of Texas at Austin for providing HPC resources that have contributed to the research results reported within this paper. Edited by: James Annan Reviewed by: James Annan and one anonymous referee
You have requested "on-the-fly" machine translation of selected content from our databases. This functionality is provided solely for your convenience and is in no way intended to replace human translation. Show full disclaimer
Neither ProQuest nor its licensors make any representations or warranties with respect to the translations. The translations are automatically generated "AS IS" and "AS AVAILABLE" and are not retained in our systems. PROQUEST AND ITS LICENSORS SPECIFICALLY DISCLAIM ANY AND ALL EXPRESS OR IMPLIED WARRANTIES, INCLUDING WITHOUT LIMITATION, ANY WARRANTIES FOR AVAILABILITY, ACCURACY, TIMELINESS, COMPLETENESS, NON-INFRINGMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Your use of the translations is subject to all use restrictions contained in your Electronic Products License Agreement and by using the translation functionality you agree to forgo any and all claims against ProQuest or its licensors for your use of the translation functionality and any output derived there from. Hide full disclaimer
© 2018. This work is published under https://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Abstract
Unstructured grid ocean models are advantageous for simulating the coastal ocean and river–estuary–plume systems. However, unstructured grid models tend to be diffusive and/or computationally expensive, which limits their applicability to real-life problems. In this paper, we describe a novel discontinuous Galerkin (DG) finite element discretization for the hydrostatic equations. The formulation is fully conservative and second-order accurate in space and time. Monotonicity of the advection scheme is ensured by using a strong stability-preserving time integration method and slope limiters. Compared to previous DG models, advantages include a more accurate mode splitting method, revised viscosity formulation, and new second-order time integration scheme. We demonstrate that the model is capable of simulating baroclinic flows in the eddying regime with a suite of test cases. Numerical dissipation is well-controlled, being comparable or lower than in existing state-of-the-art structured grid models.
You have requested "on-the-fly" machine translation of selected content from our databases. This functionality is provided solely for your convenience and is in no way intended to replace human translation. Show full disclaimer
Neither ProQuest nor its licensors make any representations or warranties with respect to the translations. The translations are automatically generated "AS IS" and "AS AVAILABLE" and are not retained in our systems. PROQUEST AND ITS LICENSORS SPECIFICALLY DISCLAIM ANY AND ALL EXPRESS OR IMPLIED WARRANTIES, INCLUDING WITHOUT LIMITATION, ANY WARRANTIES FOR AVAILABILITY, ACCURACY, TIMELINESS, COMPLETENESS, NON-INFRINGMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Your use of the translations is subject to all use restrictions contained in your Electronic Products License Agreement and by using the translation functionality you agree to forgo any and all claims against ProQuest or its licensors for your use of the translation functionality and any output derived there from. Hide full disclaimer
Details




1 Center for Coastal Margin Observation & Prediction, Oregon Health & Science University, Portland, OR, USA; present address: Finnish Meteorological Institute, Helsinki, Finland
2 Department of Earth Science and Engineering, Imperial College London, London, UK
3 Department of Mathematics, Imperial College London, London, UK; Department of Computing, Imperial College London, London, UK; present address: Department of Computer Science, Durham University, Durham, UK
4 Department of Mathematics, Imperial College London, London, UK
5 Center for Coastal Margin Observation & Prediction, Oregon Health & Science University, Portland, OR, USA