Challenges in diffusion MRI tractography – Lessons learned from international benchmark competitions

Summary

This review synthesizes a decade of tractography benchmark competitions. Physical phantoms, simulated data, and anatomical references reveal recurring tradeoffs between finding true pathways and avoiding false connections, while showing how acquisition, reconstruction, seeding, and evaluation choices shape apparent performance.

Scope

International tractography challenges provide shared datasets and reference standards for comparing algorithms under common conditions. This paper reviews lessons from a decade of such competitions, spanning local orientation reconstruction and whole-pathway connectivity. Across benchmarks, plausible-looking bundles can coexist with incomplete coverage, incorrect branching, and false connections. Denoising, sharper orientation estimates, and spatial constraints can help, but seeding and tracking choices change the balance between sensitivity and specificity. Deterministic and probabilistic methods often favor different aspects of that balance. Because each benchmark captures only part of tissue complexity and has its own reference limitations, the paper argues for interpreting competition results in context rather than treating one ranking as universal.

Topics covered

• Benchmark studies repeatedly revealed incomplete bundle coverage and false-positive connections.
• Seeding, noise, local orientation estimation, and tracking strategy materially affected results.
• Different reference datasets probe complementary limitations and cannot establish one universally optimal algorithm.