Severity & thresholds

How MotionPR grades differences, the default rules and their rationale, and how to change them.

On this page
  1. Default rules and rationale
  2. Timing thresholds
  3. Low-confidence matches
  4. Overrides
  5. Scroll and timeline rules

Every difference has one severity:

SeverityMeaning
criticalMotion the design relied on is gone or broken
warningThe interface moves noticeably differently; review it
infoA small, measurable adjustment
uncertainThe finding rests on a weak element match or incomplete data

comparison.failOn decides the exit code: with the default "critical", motionpr run exits 1 only for critical differences. uncertain never fails a run unless failOnUncertain: true.

Default rules and rationale

RuleDefaultWhy
scenario-failedcriticalThe interaction no longer works, so its motion cannot happen
scenario-removedcriticalA tracked scenario vanished
scenario-addedinfoNew coverage, nothing to compare yet
checkpoint-removedcriticalThe flow no longer reaches a recorded state
checkpoint-addedwarningThe flow changed shape
animation-removedcriticalFeedback users saw is gone (e.g. a reveal that now pops)
animation-addedwarningNew motion deserves a look
duration, delaygradedSee timing thresholds below
end-delayinfoRarely visible
startgraded, at most warningStart offsets are frame-quantized estimates
easingwarning ≥ 5% max deviation (or overshoot appears/disappears), else infoThe feel of motion
iterationswarning; critical when finite ↔ infiniteA loop that starts or stops looping is a different behaviour
direction, playback-ratewarning
fillinfoAffects end state, which visual comparison also covers
property-added, property-removedwarningThe animation now moves something else
keyframesgradedDeclared start/end values of non-transform properties (colour, clip-path, sizes)
trajectorygraded by pxTranslate/rotate start and end values from sampled computed styles
opacity, sizegradedSampled opacity, scale and layout size
staggergraded, at most warningInterval between consecutive siblings changed
visualwarningPixels depend on fonts and platforms; never critical by default

Timing thresholds

For a duration (relative to the baseline duration) or a delay/start (relative to the animation's duration):

  • below timing.toleranceMs (1 ms; 34 ms for start times): no difference;
  • critical when the relative change reaches criticalRelative (1.0, i.e. doubled or halved) and the absolute change reaches warnAbsoluteMs (50 ms);
  • warning when the relative change reaches warnRelative (15%) and the absolute change reaches warnAbsoluteMs;
  • info otherwise.

Examples: 450 → 700 ms is a warning (+56%); 400 → 800 ms is critical; a delay from 0 to 120 ms on a 380 ms animation is a warning (32%); 1000 → 1100 ms is info.

Try it: the controls below build two animation records and grade them with MotionPR's comparison engine (compareMatchedAnimation from @motionpr/comparison, with the default thresholds) in your browser.

Interactive playground: needs JavaScript.

Low-confidence matches

When either side's element is identified only by DOM position, every change on that animation becomes uncertain (and removal/addition too, since the element may simply have moved). Add a data-motionpr-id to fix this, or set comparison.uncertainOnLowConfidence: false.

Overrides

TypeScript
comparison: {
  failOn: "warning",
  severity: {
    easing: "critical",   // treat any easing change beyond tolerance as critical
    visual: "off",        // ignore pixel differences entirely
    "animation-added": "info",
  },
}

An override replaces the computed severity for every difference produced by that rule; "off" drops them. Overrides also bypass the low-confidence rule. Each difference records its rule, so you can see which override applies.

Scroll and timeline rules

RuleDefaultApplies to
scroll-removedcritical (info if the scroller did not move)A scroller recorded in the baseline is missing
scroll-addedwarning (info if it did not move)A new scroller moved
scroll-positionwarning ≥ warnPositionPx, else infoStart, end or target position
scroll-durationwarning above both duration thresholds, else infoTime from first movement to settling
scroll-curvewarning ≥ warnCurveDeviation, else infoNormalized progress curve, scroll-linked progress at the same position, engine and option changes
scroll-velocitywarning above 2 × velocityTolerance, else infoPeak velocity, smooth-scroll lag
scroll-overshootwarning when overshoot appears or disappearsTravel past the final position
timelinewarning; info for added labels/timelines and child countsTimeline duration, repeat, speed, labels and their order

Scroll comparisons whose data is incomplete (a scroller still moving at capture) are reported as uncertain.