SciCheck: A Framework for Verifying the Integrity of Deep Learning Training

Published
Source
arXiv
Paper number
1485
Field
Research
arXiv ID
2611.09889

Key points

  • The verifying scientist judges the run in two layers: a statistical judge model that compares per-step gradient statistics against the verifier’s own reference run, and a spot-check that re-executes randomly chosen training segments whose checkpoints are committed to the ledger one epoch at a time.
  • On an MLP, a CNN, and an autoencoder at 100k–500k parameters, under gradient sign flipping, label poisoning, and fabricated runs, SciCheck reaches F1 of 0.966, 0.906, and 0.976 while rejecting 0–4.4% of honest runs.
  • For the attacks we test, the spot-check accounts for most of this accuracy, and we characterize what each layer can detect.
  • The effective-signal statistic falls as (1 −2s)2 under sign flipping, so statistics catch strong enough sign flips; fabricated runs are invisible to every snapshot detector we test

Paper links

External research summaries. These are not HDATF publications or measured product results.

Read original (opens in a new tab)