OpenHydro Bench
Exact answer known1D · free surfaceCandidate

Still lake around an island

Does still water stay still over uneven, partly dry ground?

Test A1F-043
Delestre (2013)
Version 1.0.0

Hidden at first because they are far off the exact answer: Hydra, Landlab. Click a name to show it.

What happens

A lake 10 cm deep surrounds a 20 cm hump whose top stands out of the water. Nothing should move, ever.

Why it matters

Real terrain is uneven and partly dry. A model that cannot hold still water still invents currents everywhere, and they swamp the real ones.

A good model must

  • keep every velocity under 1 mm/s
  • keep the water where it is (average depth change under 0.5 mm)

How models did

In rank order: by each model's percent difference from the right answer, the largest over the test's accuracy checks (lower is better; results within 10 % of each other share a rank). Mass error, timing and NSE are shown beside it, not folded in. Each model runs the same inputs on 3 grids, from 0.2 m down to 0.05 m cells; the checks are scored on the 0.05 m grid, and each card lists the error on every grid. Compare them in the results viewer →

RankModelDifferenceLargest difference inRun time
1=LISFLOOD-FP 8.3 · ACC<0.01 %Average depth error0.33 s
1=LISFLOOD-FP 8.3 · FV1<0.01 %Average depth error0.49 s
1=Basilisk Saint-Venant<0.01 %Worst velocity error0.17 s
1=TELEMAC-2D openTELEMAC 9.1<0.01 %Average depth error17.0 s
1=GeoClaw Clawpack 5.9<0.01 %Average depth error5.5 s
1=OpenSWMM 6 finite volume<0.01 %Average depth error9.8 s
1=LISFLOOD-FP 8.3 · DG2<0.01 %Worst velocity error2.3 s
8=OpenSWMM 6 dynamic wave0.047 %Worst velocity error8.5 s
8=SWMM 5.1.15 previous release0.047 %Worst velocity error8.3 s
8=EPA SWMM 5.2.4 dynamic wave0.047 %Worst velocity error8.1 s
11ANUGA 4.0.10.078 %Average depth error–
12Hydra urban drainage12 %Worst velocity error4.1 s
13Landlab OverlandFlow13 %Worst velocity error1.1 s

Every code that ran keeps the lake still except Hydra (currents up to 0.12 m/s). The SWMM engines needed a routing step matched to the grid: SWMM's variable step follows the flow velocity, so in still water it stays at the routing step, and with a 1 s step on 20 cm conduits a lake on a 0.4 % slope starts sloshing at 1.3 m/s.

LISFLOOD-FP ACC
LISFLOOD-FP 8.3, the local-inertia solver it is best known for; it leaves out the advection term
#1= under 0.01 % off

Run time, finest grid: 0.33 s · AMD EPYC 7763 64-Core Processor, 2 cores, 15.6 GB, Linux 6.17.0-1022-azure (GitHub Actions (github-hosted, ubuntu24))

Settings: lisflood (acceleration solver, defaults; CPU, double precision)

Worst velocity error
• 0 m/s off
<0.01 % of the 0.99 m/s shallow-water wave speed in 0.1 m of water
Average depth error
• < 0.0001 mm off
<0.01 % of the lake's mean depth, 86 mm
Worst depth error
· < 0.0001 mm off
reported, not counted · <0.01 % of the 0.1 m deepest water

Scored with method 3.0 · Score card: LISFLOOD-FP 8.3 ACC

LISFLOOD-FP FV1
LISFLOOD-FP 8.3, first-order finite-volume solver (full shallow-water equations)
#1= under 0.01 % off

Run time, finest grid: 0.49 s · AMD EPYC 7763 64-Core Processor, 2 cores, 15.6 GB, Linux 6.17.0-1022-azure (GitHub Actions (github-hosted, ubuntu24))

Settings: lisflood (fv1 solver, defaults; CPU, double precision)

Worst velocity error
• 0 m/s off
<0.01 % of the 0.99 m/s shallow-water wave speed in 0.1 m of water
Average depth error
• < 0.0001 mm off
<0.01 % of the lake's mean depth, 86 mm
Worst depth error
· < 0.0001 mm off
reported, not counted · <0.01 % of the 0.1 m deepest water

Scored with method 3.0 · Score card: LISFLOOD-FP 8.3 FV1

Basilisk
Basilisk (Popinet, basilisk.fr), its second-order well-balanced Saint-Venant solver on a uniform Cartesian grid with its defaults; Manning friction added as the usual semi-implicit term (it has none built in)
#1= under 0.01 % off

Run time, finest grid: 0.17 s · Intel(R) Xeon(R) 6973P-C, 2 cores, 15.6 GB, Linux 6.17.0-1022-azure (GitHub Actions (github-hosted, ubuntu24))

Settings: basilisk saint-venant.h (uniform Cartesian grid, defaults; a C program per test, generated by scripts/run_basilisk.py)

Worst velocity error
• < 0.0001 m/s off
<0.01 % of the 0.99 m/s shallow-water wave speed in 0.1 m of water
Average depth error
• < 0.0001 mm off
<0.01 % of the lake's mean depth, 86 mm
Worst depth error
· < 0.0001 mm off
reported, not counted · <0.01 % of the 0.1 m deepest water

Scored with method 3.0 · Score card: Basilisk Saint-Venant (snapshot de851a91)

TELEMAC-2D
openTELEMAC-2D 9.1.1, run with its default finite-volume settings
#1= under 0.01 % off

Run time, finest grid: 17.0 s · AMD EPYC 7763 64-Core Processor, 2 cores, 15.6 GB, Linux 6.17.0-1022-azure (GitHub Actions (github-hosted, ubuntu24))

Settings: telemac2d (Selafin, centreline nodes)

Worst velocity error
• 0 m/s off
<0.01 % of the 0.99 m/s shallow-water wave speed in 0.1 m of water
Average depth error
• < 0.0001 mm off
<0.01 % of the lake's mean depth, 86 mm
Worst depth error
· < 0.0001 mm off
reported, not counted · <0.01 % of the 0.1 m deepest water

Scored with method 3.0 · Score card: openTELEMAC-2D v9.1.1

GeoClaw
GeoClaw 5.9.2 (Clawpack), with its stock example's settings on a uniform grid
#1= under 0.01 % off

Run time, finest grid: 5.5 s · AMD EPYC 9V74 80-Core Processor, 2 cores, 15.6 GB, Linux 6.17.0-1022-azure (GitHub Actions (github-hosted, ubuntu24))

Settings: geoclaw (uniform grid, stock example settings)

Worst velocity error
• < 0.0001 m/s off
<0.01 % of the 0.99 m/s shallow-water wave speed in 0.1 m of water
Average depth error
• 0.00075 mm off
<0.01 % of the lake's mean depth, 86 mm
Worst depth error
· 0.016 mm off
reported, not counted · 0.016 % of the 0.1 m deepest water

Scored with method 3.0 · Score card: GeoClaw 5.9.2

OpenSWMM 6
OpenSWMM Engine v6 with its new explicit finite-volume routing (FLOW_ROUTING FV): one cell per conduit on the channel tests, its default cells per pipe on the drainage networks
#1= under 0.01 % off

Run time, finest grid: 9.8 s · INTEL(R) XEON(R) PLATINUM 8573C, 2 cores, 15.6 GB, Linux 6.17.0-1022-azure (GitHub Actions (github-hosted, ubuntu24))

Settings: swmm (FV routing, conduit per grid cell)

Worst velocity error
• < 0.0001 m/s off
<0.01 % of the 0.99 m/s shallow-water wave speed in 0.1 m of water
Average depth error
• 0.0041 mm off
<0.01 % of the lake's mean depth, 86 mm
Worst depth error
· 0.036 mm off
reported, not counted · 0.036 % of the 0.1 m deepest water

Scored with method 3.0 · Score card: OpenSWMM Engine v6 finite volume (6.0.0a4.dev1)

LISFLOOD-FP DG2
LISFLOOD-FP 8.3, second-order discontinuous Galerkin solver
#1= under 0.01 % off

Run time, finest grid: 2.3 s · AMD EPYC 7763 64-Core Processor, 2 cores, 15.6 GB, Linux 6.17.0-1022-azure (GitHub Actions (github-hosted, ubuntu24))

Settings: lisflood (dg2 solver, defaults; CPU, double precision)

Worst velocity error
• < 0.0001 m/s off
<0.01 % of the 0.99 m/s shallow-water wave speed in 0.1 m of water
Average depth error
• 0.0055 mm off
<0.01 % of the lake's mean depth, 86 mm
Worst depth error
· 0.034 mm off
reported, not counted · 0.034 % of the 0.1 m deepest water

Scored with method 3.0 · Score card: LISFLOOD-FP 8.3 DG2

OpenSWMM 6
OpenSWMM Engine v6 (HydroCouple's SWMM 6, pre-release) with dynamic-wave routing
#8= 0.047 % off

Run time, finest grid: 8.5 s · INTEL(R) XEON(R) PLATINUM 8573C, 2 cores, 15.6 GB, Linux 6.17.0-1022-azure (GitHub Actions (github-hosted, ubuntu24))

Settings: swmm (DYNWAVE routing, conduit per grid cell)

Worst velocity error
• 0.00047 m/s off
0.047 % of the 0.99 m/s shallow-water wave speed in 0.1 m of water
Average depth error
• 0.012 mm off
0.014 % of the lake's mean depth, 86 mm
Worst depth error
· 1.1 mm off
reported, not counted · 1.1 % of the 0.1 m deepest water

Scored with method 3.0 · Score card: OpenSWMM Engine v6 dynamic wave (6.0.0a4.dev1)

SWMM 5.1.15
EPA SWMM 5.1.15, the release before 5.2: on the parity tests compared with SWMM 5.2.4, on the exact tests scored like any other engine
#8= 0.047 % off

Run time, finest grid: 8.3 s · INTEL(R) XEON(R) PLATINUM 8573C, 2 cores, 15.6 GB, Linux 6.17.0-1022-azure (GitHub Actions (github-hosted, ubuntu24))

Settings: swmm (DYNWAVE routing, conduit per grid cell)

Worst velocity error
• 0.00047 m/s off
0.047 % of the 0.99 m/s shallow-water wave speed in 0.1 m of water
Average depth error
• 0.012 mm off
0.014 % of the lake's mean depth, 86 mm
Worst depth error
· 1.1 mm off
reported, not counted · 1.1 % of the 0.1 m deepest water

Scored with method 3.0 · Score card: EPA SWMM 5.1.15

EPA SWMM 5.2.4
EPA SWMM 5.2.4 with dynamic-wave routing; on the channel tests, the channel is written as one conduit per grid cell
#8= 0.047 % off

Run time, finest grid: 8.1 s · INTEL(R) XEON(R) PLATINUM 8573C, 2 cores, 15.6 GB, Linux 6.17.0-1022-azure (GitHub Actions (github-hosted, ubuntu24))

Settings: swmm (DYNWAVE routing, conduit per grid cell)

Worst velocity error
• 0.00047 m/s off
0.047 % of the 0.99 m/s shallow-water wave speed in 0.1 m of water
Average depth error
• 0.012 mm off
0.014 % of the lake's mean depth, 86 mm
Worst depth error
· 1.1 mm off
reported, not counted · 1.1 % of the 0.1 m deepest water

Scored with method 3.0 · Score card: EPA SWMM 5.2.4

ANUGA
ANUGA 4.0.1 (Geoscience Australia / ANU), default settings on a triangular mesh
#11 0.078 % off

Settings: anuga (rectangular_cross mesh, defaults, sampled at cell centres)

Worst velocity error
• < 0.0001 m/s off
<0.01 % of the 0.99 m/s shallow-water wave speed in 0.1 m of water
Average depth error
• 0.067 mm off
0.078 % of the lake's mean depth, 86 mm
Worst depth error
· 1.7 mm off
reported, not counted · 1.7 % of the 0.1 m deepest water

Nothing moves (velocities 1e-14 m/s). The worst depth difference, 1.7 mm, is in the cells cut by the shoreline, where a cell-mean depth on the sloping bump differs from the exact depth at the cell centre; that check is informational.

Scored with method 3.0 · Score card: ANUGA 4.0.1

Hydra
Hydra 17 (NEER AI, Rust): its urban-drainage engine on the SWMM data model, run through its CLI with default settings
#12 12 % off

Run time, finest grid: 4.1 s · INTEL(R) XEON(R) PLATINUM 8573C, 2 cores, 15.6 GB, Linux 6.17.0-1022-azure (GitHub Actions (github-hosted, ubuntu24))

Settings: swmm (DYNWAVE routing, conduit per grid cell)

Worst velocity error
• 0.117 m/s off
12 % of the 0.99 m/s shallow-water wave speed in 0.1 m of water
Average depth error
• 0.4 mm off
0.47 % of the lake's mean depth, 86 mm
Worst depth error
· 1.5 mm off
reported, not counted · 1.5 % of the 0.1 m deepest water

Scored with method 3.0 · Score card: Hydra urban drainage (17.0.0/4.1.1)

Landlab
Landlab 2.11 (CSDMS), its OverlandFlow component: the local-inertial 2D flood scheme of de Almeida et al. (2012), written independently in NumPy, with its defaults
#13 13 % off

Run time, finest grid: 1.1 s · AMD EPYC 7763 64-Core Processor, 2 cores, 15.6 GB, Linux 6.17.0-1022-azure (GitHub Actions (github-hosted, ubuntu24))

Settings: landlab.components.OverlandFlow (local inertial, defaults; a node per cell centre)

Worst velocity error
• 0.132 m/s off
13 % of the 0.99 m/s shallow-water wave speed in 0.1 m of water
Average depth error
• 3.1 mm off
3.5 % of the lake's mean depth, 86 mm
Worst depth error
· 4.4 mm off
reported, not counted · 4.4 % of the 0.1 m deepest water

Scored with method 3.0 · Score card: Landlab 2.11.0 OverlandFlow

Not run yet: HEC-RAS 7.0, TUFLOW, MIKE 21, Iber, BASEMENT, Delft3D FM
Run it and add yours →

Status of this test

Candidate. To become frozen it needs: maintainer review recorded.

Technical details for reviewers Checks, reference, inputs, sources

Checks and why each limit was chosen

CheckDifference measured as a percentage ofLatestDifferenceWhy this check
Linf_u
Linf of u · in the score
of the 0.99 m/s shallow-water wave speed in 0.1 m of water–Nothing may move: worst velocity under 1 mm/s after 100 s. A scheme that does not balance the bed slope against the pressure gradient, or leaks at the dry shoreline on the bump, sets up currents of centimetres per second.
L1_h
L1 of h · in the score
of the lake's mean depth, 86 mm–Average depth change under 0.5 mm: the lake must keep its water where it was.
Linf_h
Linf of h · reported, not counted
of the 0.1 m deepest water–Informational: the worst depth difference sits in the cells cut by the shoreline, where a cell-mean depth over a sloping bed differs from the exact depth at the cell centre by up to a few millimetres on these grids even when nothing moves.
convergence
not applicable
––The exact state is the initial state; a well-balanced scheme keeps it to rounding error on any grid, so there is no error to converge.

Reference

Exact solution from references.swashes.lake_at_rest.

Promotion checklist

Only frozen tests have fixed inputs and answers. This list is computed from the manifest and the published score cards; the rules are in the method.

  • ✓Data licence verifiedfor candidateanalytic-public
  • ✓All inputs availablefor candidateall committed or fetched by SHA-256
  • ✓Reference (truth) availablefor candidateanalytic generator
  • ✓Every tolerance justifiedfor candidateeach gating criterion has a rationale
  • ✓A published run passesfor frozen11 passing score card(s)
  • ✓Order of accuracy shownfor frozennot applicable: The exact state is the initial state; a well-balanced scheme keeps it to rounding error on any grid, so there is no error to converge.
  • ✓Two independent codes passfor frozenpassing codes: ANUGA 4.0.1, Basilisk Saint-Venant, EPA SWMM 5.1.15, EPA SWMM 5.2.4, GeoClaw 5.9.2, LISFLOOD-FP 8.3 ACC, LISFLOOD-FP 8.3 DG2, LISFLOOD-FP 8.3 FV1, OpenSWMM Engine v6 dynamic wave, OpenSWMM Engine v6 finite volume, openTELEMAC-2D v9.1.1
  • ✗Maintainer review recordedfor frozenno maintainer review recorded

Inputs

FileRoleFormatSHA-256Get it
inputs/params.jsonparametersjsoneb5589de2de4…Download

About this test

Physics

Still water at level 0.1 m over the same bump, whose top (0.2 m) stands out of the water between \(x = 10 - \sqrt{2} = 8.586\) and \(10 + \sqrt{2} = 11.414\) m. The exact solution is the initial state: \(h = \max(0.1 - z, 0)\), \(u = 0\), for all time.

What it checks

Well-balancing (the discrete bed-slope source must cancel the discrete pressure gradient exactly in still water) and the treatment of a dry shoreline on a slope. Velocities must stay under 1 mm/s and the average depth change under 0.5 mm. The worst depth difference is reported but does not decide the result: in the cells cut by the shoreline, a cell-mean depth over the sloping bump differs from the exact depth at the cell centre even when nothing moves.

Citation

Delestre et al. (2013), SWASHES, doi:10.1002/fld.3741.

Sources

  • Delestre, O., Lucas, C., Ksinant, P.-A., Darboux, F., Laguerre, C., Vo, T. N. T., James, F., Cordier, S. (2013). SWASHES: a compilation of shallow water analytic solutions for hydraulic and environmental studies. doi:10.1002/fld.3741
  • Goutal, N., Maurel, F. (1997). Proceedings of the 2nd workshop on dam-break wave simulation (EDF HE-43/97/016/B). www.irit.fr/swashes

Test A1F-043_swashes_lake_at_rest_emerged_bump · version 1.0.0 · data licence analytic-public