Track B corpus

Does each model run on the reference engine?

A model joins a tier only if it runs cleanly and SWMM's own continuity error stays within ±5 %: a reference that loses or creates water cannot anchor parity. 297 models stop with input or engine errors (mostly older-version syntax: ERROR 205, 207, 209, 211, 235, and 32 missing rainfall files, ERROR 317), 157 exceed the 60 s census limit, and 98 run but fail the continuity rule. All stay in the index so upstream maintainers can see them.

Tiers

Does another engine match SWMM 5.2.4?

Any SWMM-compatible engine can be scored against the reference across a whole tier with scripts/corpus_parity.py: both engines run every model, and each model gets the worst node (relative peak depth) and worst link (relative volume of |Q|) — the same metrics as the catalog's Track B cases. Two candidates are scored here: the previous release, EPA SWMM 5.1.15 (OWA build in swmm-toolkit 0.10.0), HydroCouple's rewritten OpenSWMM Engine v6 (pre-release openswmm 6.0.0a4 on PyPI, run with its new engine, both with each model's own routing and with its new finite-volume routing, FLOW_ROUTING FV, which changes the numerical method and so is expected to differ), and NEER AI's Hydra 17 urban-drainage engine (CLI 4.1.1). Each week the same comparison runs on Bellinge (Odense, Denmark; Pedersen et al. 2021, CC BY 4.0), a real combined sewer system of about 1,000 nodes and 1,000 links with pumps, weirs and control rules, on a two-day storm.

• Identical
• Within 1 %
• 1–5 %
• Over 5 %
• Candidate could not run it

EPA SWMM 5.1.15, pr tier: largest differences

5 model(s) the candidate could not run: lew-chi-swmm5-2/control-rules-test, lew-chi-swmm5-2/example7-inlets, lew-chi-swmm5-2/onsag-curb-inlet-12-inflow, lew-chi-swmm5-2/onsag-grate-inlet-11-inflow, swmm5-ncimm/lid1d. Most use input syntax added after 5.1 (inlets, newer options), which an older engine rejects with ERROR 200.

Hydra 17 (urban drainage), nightly tier: largest differences

20 model(s) the candidate could not run: epa/3-pumps-kw-ideal-pumps, epa/linear-links, epa/pressure-net, epa/small-pump-fm-model, hydrology/25-subcatchments-cn-infiltration, hydrology/feng-scs-test-model, hydrology/feng-scs-test-model-v13, hydrology/horton-continuous-model …. Hydra refuses these with an explicit validation message (for example incomplete subcatchment data or ground below the initial water table), or took longer than 600 s; see docs/hydra.md.

OpenSWMM Engine v6, finite-volume routing, nightly tier: largest differences

21 model(s) the candidate could not run: epa/3-pumps-kw-ideal-pumps, epa/extran1-dummy, hydraulics/extran1-dummy, hydrology/ejemplo-epa, hydrology/greenroofs, owa-update-v5111/porous-pavement, swmm5-ncimm/1765-nodes, swmm5-ncimm/1943-h-h-elements-si-units ….

OpenSWMM Engine v6, nightly tier: largest differences

1 model(s) the candidate could not run: swmm5-ncimm/3525-nodes. The v6 pre-release stops at initialisation with an unexplained parse error; see docs/openswmm.md.

EPA SWMM 5.1.15, nightly tier: largest differences

26 model(s) the candidate could not run: epa/master-extran-rtc, lew-chi-swmm5-2/control-rules-test, lew-chi-swmm5-2/drop-grate-inlet-17, lew-chi-swmm5-2/drop-grate-inlet-18, lew-chi-swmm5-2/example7-inlets, lew-chi-swmm5-2/onsag-curb-inlet-12, lew-chi-swmm5-2/onsag-curb-inlet-12-backwater, lew-chi-swmm5-2/onsag-curb-inlet-12-inflow …. Most use input syntax added after 5.1 (inlets, newer options), which an older engine rejects with ERROR 200.

Hydra 17 (urban drainage), bellinge network (1 real system): largest differences

OpenSWMM Engine v6, finite-volume routing, bellinge network (1 real system): largest differences

OpenSWMM Engine v6, bellinge network (1 real system): largest differences

The self-check (5.2.4 scored against itself) must give exactly 0 on every PR-tier model; CI runs it on every pull request.

Parity over time

One row per nightly run of each candidate (newest first). The result collector appends a row after every nightly run and opens an issue when five or more models move in or out of "within 1 %", when the number of models a candidate cannot run changes, or when its version changes.

history.csv

Does another engine match EPANET 2.2?

The same comparison for water-distribution engines, on the six example networks bundled with WNTR 1.5.0 (EPANET's Net1, Net2, Net3 and Net6 and two Kentucky utility networks), so the set is independent of any candidate's own test fixtures. Each cell gives the worst node's relative peak pressure and the worst pipe's relative volume of |Q| against EPANET 2.2, from scripts/epanet_parity.py.

Runtime

What the tiers cover

• PR
• Nightly
• Release only
• Not in a tier

Largest routing continuity errors on SWMM 5.2.4

Continuity error is SWMM's own mass-balance check. Models beyond ±5 % are excluded from every tier: their reference results are not physically meaningful, so agreeing with them proves nothing.

Median absolute routing continuity error across models that ran: 0.156 %.

Data

Download the full index as CSV or Parquet. Fetch models with python scripts/fetch_track_b_models.py --tier nightly; every file is checked against its hash.