SciCheck: A Framework for Verifying the Integrity of Deep Learning Training
- Published
- Source
- arXiv
- Paper number
- 1485
- Field
- Research
- arXiv ID
- 2611.09889
Key points
- The verifying scientist judges the run in two layers: a statistical judge model that compares per-step gradient statistics against the verifier’s own reference run, and a spot-check that re-executes randomly chosen training segments whose checkpoints are committed to the ledger one epoch at a time.
- On an MLP, a CNN, and an autoencoder at 100k–500k parameters, under gradient sign flipping, label poisoning, and fabricated runs, SciCheck reaches F1 of 0.966, 0.906, and 0.976 while rejecting 0–4.4% of honest runs.
- For the attacks we test, the spot-check accounts for most of this accuracy, and we characterize what each layer can detect.
- The effective-signal statistic falls as (1 −2s)2 under sign flipping, so statistics catch strong enough sign flips; fabricated runs are invisible to every snapshot detector we test
Paper links
External research summaries. These are not HDATF publications or measured product results.