qMRLab cross-version benchmark

9 qMRI models fitted against identical, checksum-pinned data across 17 targets from 4 software families (qmrlab, qmrust, quit, sct). Reference target: qmrlab v3.0.0.

Did every fit this benchmark declares actually run? The grid below is one target per row, one model per column, and one fit per cell.

15 × 9 targets × models93 produced maps10 failed32 not applicable

Every cell that is not a tick prints its reason in What did not run, under the grid, rather than in a tooltip — read that list first.

The most common misreading is the muted not applicable cells. A target declares the models it supports in its target.yml, and omission is the only way this repo declares a gap, so those cells are a scope decision and not a fit that broke.

The tiles that follow count the whole run; the grid under them is version history only, which is why it holds fewer targets. The note between them names what that leaves out.

17
targets
4
software families
9
models
100
successful fits
10
failed fits

Version history: qMRLab releases and qmrust. quit v3.4, sct 7.3 are on the Cross-software tab, and so is anything of theirs that failed.

Every declared target × model in this view. A status colour never carries meaning alone — each cell has an icon, and hovering gives the reason a cell failed.
b1_afib1_daminversion_recoverymono_t2mt_ratiomt_satqmt_spgrqmt_spgr_ramanivfa_t1
qmrlab V1·······
qmrlab v2.0.0···
qmrlab v2.0.1···
qmrlab v2.0.2···
qmrlab v2.0.3···
qmrlab v2.0.5···
qmrlab v2.1.0···
qmrlab v2.2.0···
qmrlab v2.3.0··
qmrlab v2.4.0·
qmrlab v2.4.1·
qmrlab v2.5.0
qmrlab v3.0.0
qmrlab master
qmrust main
ok failed missing not applicable

What did not run