Current status: This playbook is a planning aid, not legal advice, emergency guidance, or a substitute for district crisis, safety, cybersecurity, or breach-response procedures.
A K-12 AI incident response playbook should extend the district’s existing response system, not create a parallel emergency bureaucracy. The district already has processes for cybersecurity, student privacy, student safety, civil rights, accessibility, employee matters, academic integrity, vendor management, records, and public communication. AI changes the fact pattern, evidence, and containment options. It does not erase those established responsibilities.
This playbook provides an operating structure for preparation, triage, containment, investigation, communication, recovery, and improvement. Adapt roles, notification requirements, time frames, and authority with district counsel and existing plans.
Define an AI incident
An AI incident is an event involving an AI product, feature, model, application, or AI-assisted decision that may have:
- used data outside an approved boundary;
- exposed information to an unauthorized person, provider, model deployment, tool, or repository;
- operated under the wrong user, application, grade, role, purpose, or policy context;
- produced harmful, discriminatory, inaccessible, fabricated, or materially misleading output that affected a person or district action;
- taken or recommended an action without required human review;
- searched or returned documents beyond an authorized collection;
- violated a contract, DPA, retention rule, training-use restriction, or provider configuration;
- used an unapproved product, account, model, connector, or feature;
- experienced credential compromise, cross-tenant exposure, malicious manipulation, or service abuse;
- failed in a way that materially disrupted instruction or operations;
- revealed a policy, training, approval, or technical-control gap.
Not every inaccurate answer is a major incident. Not every policy question is harmless. Use triage to evaluate actual and potential impact.
Keep emergency and safety paths primary
If a report concerns immediate danger, a student safety emergency, abuse, a threat, or another urgent condition, follow district emergency and student-support procedures first. The AI system is a source of information, not an emergency-response authority or a clinical assessment.
Do not delay established reporting duties while debating whether an output is “an AI incident.”
Prepare before an incident
NIST’s current incident-response guidance places preparation, response, and recovery throughout cybersecurity risk management. CISA recommends that K-12 organizations regularly exercise an incident response plan. AI readiness should become part of that work.
Build the incident roster
Name a primary and backup for:
| Function | AI incident responsibility |
|---|---|
| Incident lead | Sets scope, cadence, owners, decisions, and closure criteria |
| Technology or application owner | Identifies deployment, dependencies, logs, and disable options |
| Security | Handles compromise, malicious activity, credentials, systems, and forensics |
| Privacy or data governance | Identifies data, affected people, authority, access, and privacy obligations |
| Legal | Advises on privilege, duties, notification, contracts, holds, and external engagement |
| Curriculum or program owner | Assesses educational context, student impact, and instructional correction |
| Student services | Coordinates appropriate student support through established procedures |
| Accessibility or civil rights | Assesses barriers, discrimination, disparate impact, and corrective access |
| Communications | Coordinates accurate internal, family, board, media, and public messages |
| Procurement or vendor manager | Activates contract contacts, cooperation, evidence, and remedies |
| Records manager | Applies retention, legal hold, and secure disposition requirements |
| Executive sponsor | Accepts major operational decisions and escalates to cabinet or board |
Smaller districts may combine roles, but the responsibilities still need named owners.
Maintain a response-ready register
For each material AI use, record:
- accountable and technical owner;
- purpose and eligible users;
- product, account type, provider, and model deployment;
- source systems and data boundary;
- connectors, retrieval sources, tools, and downstream actions;
- human-review rule;
- provider and contract incident contacts;
- identity, access, logging, and retention configuration;
- disable, credential-revocation, rollback, and alternate-workflow steps;
- review date and material-change triggers.
The district AI application register supplies a structure. An incident team cannot contain a system it cannot identify.
Establish evidence rules
Define in advance:
- which district and provider logs exist;
- who may access them;
- how time, identity, policy, deployment, and request correlation are recorded;
- when prompts, outputs, files, or screenshots may be collected;
- how sensitive incident material is transferred and stored;
- how chain of custody, legal holds, access, retention, and deletion work;
- how low-volume or identifying analytics are protected.
Do not turn routine governance into an unlimited transcript archive. An incident can justify targeted preservation, but evidence collection should remain necessary, authorized, and controlled.
Test containment before launch
Confirm that authorized staff can:
- disable the AI feature or application;
- remove a user, group, or service identity;
- revoke or rotate a key or token;
- disconnect a repository, tool, or source system;
- stop a scheduled job or action path;
- change an approved route or model deployment;
- preserve required logs before they expire;
- move users to a documented alternate workflow;
- contact the provider through a tested urgent channel.
A vendor promise to “support incidents” is not a tested district procedure.
Triage a report
Step 1: protect people and stop immediate spread
Use established emergency, safety, or security procedures when required. Limit further sharing of harmful or sensitive content. Do not forward a screenshot or transcript broadly merely to ask whether it matters.
Step 2: open one incident record
Assign an incident identifier, lead, start time, reporter contact, initial facts, systems in scope, current status, and next update time. Separate verified facts, reported facts, and hypotheses.
Step 3: classify the event
An event may belong to more than one category:
| Category | Examples |
|---|---|
| Privacy or data | Unapproved student information, excessive retrieval, retention, or disclosure |
| Security | Compromised account or key, malicious manipulation, cross-tenant access, or exposed integration |
| Safety | Harmful content or behavior requiring established student-support review |
| Civil rights or accessibility | Discriminatory impact, inaccessible workflow, unequal access, or missing alternative |
| Instructional or academic | Assignment-rule failure, fabricated source used in work, or inappropriate automation of judgment |
| Quality or consequential decision | Materially inaccurate output used in grading, placement, services, communication, or operations |
| Contract or vendor | Model, subprocessor, data practice, feature, or term outside the approved agreement |
| Policy or unauthorized use | Unapproved product, account, purpose, user group, connector, or data |
| Availability or operations | AI dependency disrupts teaching, service delivery, or required records |
Step 4: assign an initial severity
Use a district-defined matrix. Consider:
- immediate safety or welfare;
- sensitivity and volume of data;
- number and vulnerability of affected people;
- whether the event is continuing;
- whether an output changed a consequential decision or record;
- ability to reverse, correct, or contain the effect;
- public, legal, contractual, and operational impact;
- evidence reliability and uncertainty;
- whether the same issue can recur across the district.
Example routing:
| Severity | Working definition | Initial handling |
|---|---|---|
| 1: critical | Immediate safety, active major exposure, widespread compromise, or continuing consequential harm | Activate executive incident command and established emergency or breach procedures |
| 2: high | Sensitive data, multiple people, serious control failure, or material decision impact | Cross-functional response, urgent containment, frequent executive updates |
| 3: moderate | Bounded event with limited impact but required correction, notification review, or control change | Named lead, scoped containment, documented review |
| 4: low | No sensitive data or material impact found, but policy, quality, or training improvement is needed | Owner correction and trend review |
Severity can change as facts develop. Record why it changed.
Contain without destroying evidence
Choose the narrowest effective action that protects people and stops continued impact. Options include:
- pause a feature, connector, application, model route, or user group;
- revoke a compromised credential;
- remove an unauthorized document collection;
- change permissions or identity mapping;
- stop automated downstream actions;
- suspend an approval or exception;
- preserve expiring logs;
- notify staff not to use or redistribute affected content;
- move to a manual or non-AI workflow;
- ask the provider to preserve evidence and confirm containment.
Document who authorized the action, what changed, when it took effect, and how rollback will be evaluated. Do not delete potentially relevant records without coordinating security, privacy, legal, and records owners.
Investigate the actual request path
Reconstruct the event from source to consequence:
- Actor: Which human or application identity acted? How was it verified?
- Authority: Which approval, policy, role, and purpose should have applied?
- Input: What text, file, image, audio, record, metadata, or retrieved content entered the flow?
- Context: What history, system instruction, profile, roster, connector, or document collection was added?
- Route: Which product, account, provider, model deployment, region, and subprocessor handled the request?
- Output: What did the system return, recommend, classify, or generate?
- Action: Was the output displayed, copied, stored, communicated, or used to change a record or decision?
- Evidence: Which logs, records, provider notices, or user reports support the timeline?
- Control: Which expected policy, configuration, review, or technical safeguard worked or failed?
- Impact: Who and what were affected, for how long, and can the effect be corrected?
Use the data-boundary worksheet when the route includes several systems or document stores.
AI-specific investigation questions
Wrong or harmful output
- Was the output outside the approved use or an expected model limitation?
- Did a person verify it before use?
- Did the interface communicate uncertainty and source limits?
- Were citations present, and did they actually support the claim?
- Did retrieval supply relevant and authorized material?
- Was the model evaluated for the language, age, subject, and task in scope?
- Did the output change a record, grade, service, message, or other decision?
- Can affected people receive correction, explanation, review, or appeal?
Data exposure or excessive retrieval
- Which exact data elements and people were involved?
- Did the model provider receive the data, or did exposure remain inside another system?
- Did filenames, metadata, attachments, hidden content, or logs add information?
- Did access exceed the acting person’s authority?
- Were inputs or outputs retained, reviewed by people, or used for service improvement?
- Which subprocessors, regions, and backup paths are involved?
- Can the provider identify deletion, preservation, and notification steps?
Application identity or policy failure
- Did a shared key identify only an application, while the use required a separately verified person?
- Did the application receive the correct role, grade, purpose, data, and policy context?
- Was an exception expired, too broad, or used by the wrong identity?
- Did a configuration change alter the approved model route or control?
- Was failure behavior allow, deny, retry, or silent fallback?
- Did a downstream action proceed without required approval?
Provider or model change
- What changed and when?
- Was advance notice required and received?
- Did the change affect data use, retention, provider, subprocessor, region, behavior, or action capability?
- Did the district revalidate the deployment?
- Can the prior configuration be restored or the feature remain disabled?
Communication and notification
Communication should be accurate, coordinated, and appropriate to the audience. Do not promise facts that are not yet known, speculate about legal status, identify students unnecessarily, or let separate teams send conflicting messages.
Create a notification decision table before incidents occur:
| Audience | Decision owner | Trigger and authority | Initial content | Update cadence |
|---|---|---|---|---|
| Executive team | Incident lead | District severity threshold | Facts, impact, action, decisions needed | Scheduled until stable |
| School and staff leaders | Program and communications leads | Operational or instructional need | What changed, what to do, support path | As actions change |
| Students and families | District-authorized leader | Applicable duty or material impact | Known facts, protective steps, correction, contact | As verified facts develop |
| Board | Superintendent or designee | Policy, materiality, or board protocol | Scope, response, risk, next steps | Per governance process |
| Vendor or model provider | Vendor manager and technical lead | Contract or operational need | Incident ID, facts, preservation, containment request | Until provider actions close |
| Insurer, counsel, regulator, or law enforcement | Authorized district role | Applicable policy, contract, or law | Required and verified information | Per established procedure |
| Public or media | Communications lead | Public interest or district decision | Coordinated factual statement | As appropriate |
Notification requirements vary by event, data, jurisdiction, contract, and affected population. Qualified district reviewers should determine whether, when, how, and by whom notice is required.
Recover and validate
Do not restore an AI use only because the vendor says the immediate error is fixed. The decision owner should confirm:
- containment is complete;
- affected data, records, and decisions are corrected where possible;
- credentials, permissions, connectors, and routes are valid;
- the original cause and contributing conditions are addressed;
- required patches, settings, contract actions, training, or policy changes are complete;
- the workflow has been tested with non-sensitive cases;
- monitoring and rollback are ready;
- affected users know what changed;
- executive or Committee approval to restore is recorded.
Use a phased restoration for high-impact systems when practical.
Close with corrective action
A closure record should include:
- verified timeline;
- affected systems, people, data, and decisions;
- root cause and contributing conditions;
- containment and recovery actions;
- notification decisions and authority;
- evidence location, access, and retention;
- corrected records or decisions;
- vendor commitments and deadlines;
- policy, contract, training, accessibility, security, and technical changes;
- owner and due date for each action;
- approval status and next review date;
- lessons for other registered AI uses.
Focus on system improvement as well as individual conduct. If several trained people made the same mistake, the rule, workflow, interface, or control may be the deeper failure.
Tabletop exercises
Run at least one AI scenario inside the district’s regular incident exercise.
Scenario 1: student information in an unapproved account
An educator uploads a class file to a free consumer AI account while preparing an activity. The educator reports it an hour later.
Test:
- reporting and nonpunitive early escalation;
- account, file, provider, and data identification;
- evidence minimization and provider contact;
- privacy, legal, and notification review;
- deletion and verification limits;
- alternate workflow and training correction.
Scenario 2: inaccurate recommendation affects services
A district application summarizes several records and recommends outreach. Staff learn that the model omitted a relevant record and the summary was used in several cases.
Test:
- identification of affected decisions;
- pause and manual review;
- correction, communication, and appeal;
- retrieval and evaluation evidence;
- application-owner and human-review accountability;
- criteria for restoration.
Scenario 3: provider changes the model route
A vendor notice reveals that requests now use a different model provider and region than the approved deployment.
Test:
- contract and material-change triggers;
- ability to disable or restrict the feature;
- data-flow and subprocessor review;
- decision authority and vendor escalation;
- communication to users;
- revalidation or exit.
Scenario 4: connected repository returns restricted content
A public-facing assistant cites an internal draft that was stored in a broadly connected repository.
Test:
- authorization before retrieval;
- source permissions and index scope;
- public cache and log review;
- document-owner notification;
- correction and removal;
- cross-application search for the same configuration gap.
After the exercise, record which contact, permission, log, contract clause, or decision was missing. Assign and verify corrective actions.
Frequently asked questions
What counts as an AI incident in a school district?
An AI incident is an event that may violate an approved purpose, data boundary, security control, policy, contract, accessibility requirement, or human-review rule, or that causes material harm through model output, retrieval, automation, or misuse. Not every error is an incident, but every report needs a defined triage path.
Should AI incidents use the district’s cybersecurity incident plan?
Use the existing plan and incident-command structure when applicable, then add AI-specific roles and questions. Some AI incidents are security or privacy events; others are instructional, accessibility, civil rights, vendor, or quality issues that need different specialists.
What should a district do first after a suspected AI data exposure?
Protect people, open the district’s established incident process, name an incident lead, preserve necessary evidence, limit further exposure, identify the product and data flow, and involve privacy, security, legal, and vendor contacts as required. Do not make public or legal conclusions before facts are verified.
Should a district save full prompts and model responses as incident evidence?
Only when necessary and authorized. Preserve the minimum evidence needed to understand and respond, restrict access, document the source, and apply retention and legal-hold rules. Full content can create an additional sensitive record.
How often should a district exercise its AI incident playbook?
Exercise it at least annually and after a major change to products, data flows, vendors, or incident roles. Include AI scenarios in existing privacy and cybersecurity exercises rather than creating a disconnected process.
Sources
- Incident Response Recommendations and Considerations for Cybersecurity Risk Management, NIST SP 800-61 Rev. 3
- Generative Artificial Intelligence Profile, NIST
- Data Breach Response Checklist, U.S. Department of Education
- Protecting Our Future: Cybersecurity for K-12, CISA
This playbook is educational information. Districts should use qualified emergency, student safety, legal, privacy, security, accessibility, communications, procurement, instructional, and records professionals for their circumstances.