The US Federal Trade Commission (FTC), the federal agency that enforces consumer protection and privacy rules, spent this summer closing a case over an edtech breach that exposed data belonging to 10.1 million students (FTC, 2026).
Three months later, the argument over how much trust schools should place in technology companies has moved squarely onto AI. New York City has imposed a one-year moratorium on student-facing generative AI for nearly 600,000 pupils from 2-K through eighth grade, while disabling AI features in dozens of previously approved products (NYC Mayor’s Office, 2026). A week after that announcement, Microsoft and two major teachers’ unions chose a different lever: instead of restricting access to AI, they wrote limits on what the technology can do with student data into the contracts schools sign with vendors.
Microsoft, the American Federation of Teachers and the United Federation of Teachers have unveiled a National AI Safety & Privacy Standard that US districts can incorporate directly into Microsoft customer agreements. Student and educator data cannot be used to train AI models, tracking is prohibited, automated decisions require human oversight, and vendors must explain in plain language what their systems collect and how they operate (Microsoft, 2026; AFT, 2026).
The rules cover some of the questions that have become harder as generative AI moves into education products:
The agreement also requires security controls and safeguards against manipulative or harmful experiences, while schools retain authority over when AI is used (AFT, 2026).
New York City’s new policy blocks student-facing generative AI through eighth grade, bans companion chatbots across all grades, and limits high-school AI to vetted pilots and supervised use. The district is also disabling AI features in more than 38 previously approved programs that failed its new standards. High-school students will instead receive twice-yearly AI literacy modules, while selected pilots are capped at 45 minutes a week under teacher supervision (NYC Mayor’s Office, 2026).
Microsoft’s agreement assumes AI will still enter many classrooms, but tries to narrow what vendors can do once they are there. That fills a gap left by older privacy law. FERPA protects education records and restricts disclosure of personally identifiable information, yet the US Department of Education explicitly notes that the law does not prescribe specific security controls. Schools can also allow outside vendors to handle student data under certain conditions, including when vendors act as school officials and remain under the school’s control (US Department of Education, 2026).
COPPA provides another layer for children under 13. The FTC strengthened the rule in 2025, adding tighter limits around data sharing, targeted advertising and retention, while expanding the definition of personal information to include biometric and government-issued identifiers. Schools can still consent on parents’ behalf when a service is being used solely for an educational purpose (FTC, 2025).
Recent enforcement shows why contractual detail matters. In June, the FTC finalized an order against Illuminate Education after alleging that security failures exposed data belonging to 10.1 million students. The settlement requires a formal security program, tighter retention and deletion of unnecessary information (FTC, 2026).
The Microsoft-AFT agreement moves the school AI debate from broad promises to procurement language: who owns the data, who can reuse it, which decisions require a human, and what happens when a vendor violates the terms. For districts that continue adopting AI, those clauses may end up carrying more weight than the product’s safety page.