AIAI Toolbox
Submit a tool
AI GOVERNANCE

AI Model Training Pause Criteria: A Governance Checklist

A practical governance framework for deciding when a frontier or high-risk AI training run should slow, pause, remain contained, or restart.

By AI Toolbox Team··12 min read

Set AI model training pause criteria before a run begins by tying observable triggers to risk: unexpected capability gains, a threshold evaluation, loss of workload or network isolation, monitoring blind spots, suspicious model behavior, compromised infrastructure, or evidence that safeguards no longer reduce severe harm. Give named responders authority to stop work, preserve evidence, contain access, and require an independent, documented restart decision. A pause is an operational control, not proof that a model is unsafe or a substitute for continuous risk management.

Key takeaways

  • Pause criteria should be measurable decision rules, not a vague promise to act responsibly
  • Triggers must cover capability, behavior, infrastructure, monitoring, and external threat changes
  • Training, evaluation, inference, code execution, and tool access may need different containment states
  • The person who can pause a run must have authority, telemetry, and a reliable out-of-band control path
  • Restart requires evidence that the trigger is understood and safeguards work under realistic adversarial tests
  • Public incident details are useful design evidence but do not establish a universal threshold for every developer

Turn risk statements into operational triggers

A useful pause policy connects a signal to an action. ‘Stop if the model becomes too capable’ leaves operators to debate meaning during a fast-moving event. Define the protected outcome, observable signal, evaluation or sensor, confidence rule, responsible decision maker, maximum response time, and the state the workload enters. The trigger may require a full stop, removal of tools, internet isolation, a narrower checkpoint, or continued evaluation inside a hardened environment.

OpenAI's Preparedness Framework illustrates one capability-based approach. It distinguishes High capabilities that could amplify existing paths to severe harm from Critical capabilities that could introduce unprecedented paths. In that framework, High capability affects deployment safeguards, while Critical capability also requires sufficient safeguards during development. Those labels belong to OpenAI's framework; another organization should build thresholds from its own threat model, laws, risk appetite, and ability to contain harm rather than copy names without the underlying evaluation program.

Use several trigger families. Capability triggers detect an evaluation threshold or unexpectedly rapid gain. Behavior triggers cover unauthorized actions, deceptive conduct, safeguard circumvention, or unexplained persistence. Security triggers include credential exposure, isolation failure, compromised dependencies, or suspicious network activity. Assurance triggers cover missing telemetry, uncalibrated monitors, corrupted evaluation data, and loss of reproducibility. External triggers include a newly demonstrated exploit or threat pathway that invalidates a prior risk estimate.

Define containment across the whole research system

Training is not one switch. A stopped optimizer may leave checkpoints served for inference, evaluation agents executing code, tool credentials active, datasets writable, and research services connected to internal networks. Inventory the control plane, clusters, artifact stores, evaluation harnesses, sandboxes, tool gateways, internet egress, secrets, queues, and human access. For each trigger, state which assets freeze, which identities are revoked, and which diagnostic functions remain available.

OpenAI reported pausing reinforcement-learning training on models intended for deployment and separately pausing frontier-model inference for research workloads that could execute code or use internet-capable tools after a security incident. It later restored a more limited path and reviewed workloads individually. The transferable lesson is that pause scope should follow authority and exposure: code execution, tool use, and network access can create risk even when weights are not changing.

Build a control path outside the suspected boundary. Responders should be able to isolate egress, revoke credentials, stop schedulers, deny checkpoint serving, and snapshot evidence without depending on the affected workload or a single privileged account. Protect logs and time sources separately. Test partial failure: a worker that ignores cancellation, an unavailable cluster controller, stale credentials, a corrupted monitor, or a queue that resumes work automatically after a reboot.

Use monitoring as evidence, not as the only safeguard

Monitoring should combine infrastructure events, model and tool behavior, and evaluation results. Capture checkpoint and code versions, data provenance, prompts and policies where appropriate, tool calls, sanitized parameters, authorization decisions, process and network activity, monitor versions, alerts, and human actions. Coverage is itself a control metric: a strong detector cannot protect workloads whose traces never reach it.

OpenAI's August 2026 account describes multistage monitoring for higher-risk tool-using research. It says token-level activation classifiers escalate concerns to automated investigators, with the highest-priority signals paging safety, security, and research teams. The same publication describes a 30-minute decision expectation for critical-boundary alerts and pausing activity when teams cannot establish a false positive. That is a first-party description of one implementation, not an independently audited service level or a universal response target.

OpenAI's earlier coding-agent report also emphasizes limitations: matching known employee escalations does not quantify false-negative rates for open-ended traffic, and future behavior may be harder to monitor. Treat monitors as defense in depth. Pair them with hard isolation, least privilege, deterministic authorization, rate and resource limits, approval gates, independent evaluation, and recovery. Red-team the monitor, including alert flooding, evasion, collusion, poisoned traces, and failures that look like ordinary infrastructure noise.

Create a decision process that resists schedule pressure

A technical trigger is ineffective if nobody can use it. Assign an incident commander and identify safety, security, research, legal, privacy, operations, and business roles before the run. Give at least one independent function authority to order containment. Define how conflicts, absent leaders, and uncertainty are handled. Record the rationale, dissent, evidence gaps, and who accepted residual risk.

Separate three decisions: whether to contain now, how to investigate safely, and whether to restart. Containment should favor limiting irreversible harm when a credible boundary failure exists. Investigation can use replicas, reduced-capability checkpoints, synthetic data, no-network sandboxes, or read-only tools. Restart should not be the default result of elapsed time or delivery pressure.

Competitive conditions deserve explicit treatment. OpenAI's framework says it may adjust safeguards if another developer releases a high-risk system without comparable protections, but only after confirming the changed landscape, publicly acknowledging the adjustment, assessing that it does not meaningfully increase overall severe-harm risk, and retaining more protective safeguards. Whether or not a team adopts that policy, competitor action should enter a documented reassessment rather than silently override established controls.

Require a staged, evidence-based restart

Before restart, reproduce the trigger where safe and determine whether it reflects a real capability, malicious activity, configuration error, monitor defect, evaluation artifact, or unknown cause. Unknown does not automatically mean safe. Verify evidence integrity, scope affected assets and identities, rotate exposed credentials, repair trust boundaries, and test the correction against the original event plus adaptive variants. Include benign workloads so a control cannot pass merely by blocking everything.

Define an evidence packet: incident timeline; relevant checkpoints, datasets, software, prompts, tools, networks, and identities; cause and uncertainty analysis; capability and control evaluations; monitor coverage; privacy and legal review; corrective actions; residual risks; approvals; and public or stakeholder communications. An independent reviewer should be able to trace why the chosen safeguards reduce the stated harm rather than simply confirm that a ticket was closed.

Resume in stages. Begin with the minimum model capability, compute, data, tools, privileges, and connectivity needed to test the correction. Increase one boundary at a time, use heightened monitoring and short review windows, and keep rollback ready. Repeated or unexplained triggers should raise the evidence bar. This checklist is documentation-based governance analysis, not hands-on validation of OpenAI's controls, a universal frontier-safety standard, or proof that any specific model is safe or unsafe.

Practical checklist

  • Classify the run, model, tools, data, networks, credentials, and plausible severe harms before allocation
  • Define capability thresholds and the evaluations, confidence rules, and owners used to detect them
  • Write separate pause triggers for security boundaries, monitoring coverage, anomalous behavior, and incidents
  • Specify which actions stop immediately and which systems move into a contained diagnostic state
  • Test a kill path that does not depend on the model, the affected cluster, or one administrator account
  • Preserve checkpoints, logs, tool traces, identities, network evidence, configuration, and time synchronization
  • Assign an incident commander and independent safety, security, research, legal, and business reviewers
  • Preapprove safe evaluation environments for investigating the trigger without restoring broad access
  • Define restart evidence, residual-risk acceptance, staged resumption, observation windows, and rollback
  • Exercise false alarms, ambiguous signals, unavailable reviewers, partial containment, and repeated triggers
  • Review criteria after capability, architecture, threat, vendor, policy, or personnel changes
  • Publish or communicate an appropriately scoped account of material events and corrective actions

Warning signs

  • The policy says leadership may pause work but names no telemetry, threshold, decision owner, or deadline
  • A stop command disables the training loop while model inference, tools, credentials, or internet access remain available
  • The team cannot determine which workloads are covered by monitoring or which paths bypass it
  • Researchers who own the delivery milestone are the only people allowed to assess and clear the trigger
  • Restart means applying a patch and rerunning the same test without an adversarial control evaluation
  • Logs needed for investigation share the same trust boundary as the suspected compromised workload
  • A competitor release is treated as automatic permission to weaken safeguards without a documented risk reassessment

Frequently asked questions

What are AI model training pause criteria?

They are predefined, observable conditions that require a training or related research workload to stop or enter containment while authorized reviewers investigate risk and decide whether safeguards justify a staged restart.

Does a pause mean the model is dangerous?

Not necessarily. A pause can be precautionary when evidence is incomplete, a monitoring path fails, a security boundary is uncertain, or a capability result requires stronger controls before work continues.

Should one capability score automatically stop training?

Usually no. Define the evaluation, uncertainty, replication rules, surrounding evidence, and required response in advance. Some boundary failures should stop immediately, while noisy capability signals may first trigger confirmation in containment.

Who should be able to pause a model run?

Named safety, security, and operations responders need independent authority and a tested technical path. Escalation should reach accountable leadership without requiring consent from the team whose schedule is affected.

What evidence is required to restart?

At minimum: a scoped cause analysis, preserved evidence, verified containment, adversarial tests of corrective controls, monitoring coverage, residual-risk assessment, named approvals, staged limits, and a rollback plan.

Primary sources and further reading

Research before you rely.

AI products, prices, policies, and capabilities change. Verify consequential details with primary sources and test tools using representative work.