The Paper has a GitHub, the GitHub has a README, the README has Nothing: Reproducibility Signals for Review Support

Abstract

Reproducibility policies promise "checkable" medical-imaging science, yet many submissions still ship unverifiable artifacts. An analysis of 3722 MICCAI papers shows code-linking rising from 51.8% to 72.5%, but ~13% of linked repositories are inaccessible or empty. We present paper-snitch, a reviewer-facing decision-support tool that turns these signals into an evidence-grounded report. paper-snitch parses PDFs, resolves and sanity-checks repositories, and applies policy-aware checklists aligned with MICCAI expectations, producing a review-time verifiability score decomposed into interpretable sub-scores plus criterion-linked excerpts and artifacts reviewers can inspect. It never executes untrusted code or attempts GPU-heavy reproduction, focusing instead on bounded, verifiable checks. We compare paper-snitch on 100 randomly sampled MICCAI 2025 papers with human annotators using shared evaluation criteria, indicating that automated, bounded checks can scale reproducibility screening while keeping final decisions with reviewers.

Ähnliche Arbeiten