Sponsor MessageBecome a KQED sponsor
upper waypoint

New State Law Requires Additional Safeguards When Police Use Generative AI

Save ArticleSave Article
Failed to save article

Please try again

Fresno Police Officer Gregory Colon-Reyes demonstrates Draft One on Sept. 24, 2024, an AI tool that uses the audio from bodycam footage to generate police reports. A new law requires written disclosures and supporting materials whenever departments use AI to generate police reports. (Courtesy of Gary Kazanjian)

FRESNO, Calif. — Gov. Gavin Newsom this month signed a first-of-its-kind California law requiring police to disclose how they use generative artificial intelligence, a move aimed at boosting transparency and public trust.

California is among the first states to address the issue. KQED first reported last October on local departments adopting AI tools. The reporting was cited in the legislative analysis of the bill that ultimately became the law.

Senate Bill 524, signed into law Oct. 10, requires police officers to disclose when they use AI to write police reports. The tools include Draft One, an AI assistant that transcribes and summarizes body-camera footage to produce a draft report, which officers can then revise and edit.

Sponsored

Specifically, the law now requires a written disclosure to appear at the bottom of each page of a police report for which Draft One or other similar tools were used. The legislation also requires an “audit trail” that would preserve the original draft as well as identify the source bodycam footage or audio.

Police departments in East Palo Alto and Fresno were among the first in the state to adopt the technology.

Axon, the company that developed Draft One, told KVPR and KQED last year that its developers built safeguards into their software. For example, officers must fill in prompts within the generated report, then sign off on the report’s accuracy before it can be submitted. The tool also includes a disclaimer that Draft One was used, though police agencies have thus far been able to customize where in a report it’s placed.

The torso of a person dressed in a police uniform holds a hand over a body camera.
An East Palo Alto Police officer’s Axon body camera in East Palo Alto on Sept. 23, 2024. (Martin do Nascimento/KQED)

Police departments have said the technology saves officers significant time, and even that some AI-generated reports are better than the ones written entirely by officers. Nevertheless, the bill arose out of concerns that bias or errors generated by AI software could make their way into final incident reports, which play a key role in charging, detaining and sentencing suspects.

Kate Chatfield, executive director of the California Public Defenders Association, which sponsored the bill, said she’s grateful the measure became law.

“Due process requires transparency,” Chatfield wrote in a public statement. “Everyone in the legal system — judges, juries, attorneys and the accused — deserve to know who wrote the police report.”

“With SB 524, California is sending a clear message: Innovation in policing must be tethered to accountability,” state Sen. Jesse Arreguín, who wrote the bill, said in the statement. “No more opaque reports, no more guessing whether AI shaped the narrative.”

Kevin Little, a defense attorney in Fresno, said the law is a step in the right direction, but not a true remedy.

“My own experience with AI in an unrelated context leads me to conclude that AI platforms have a significant amount of user bias and tend to support the agendas of the user,” he said.

Larry Bowlan, a spokesperson for the Fresno Police Department, said the agency had already implemented some of the safeguards now required by the law and does not expect the new rules to be especially burdensome.

“Our AI-powered narrative assistant … already generates a disclosure and requires our users to sign acknowledgements. Draft One also already produces the requisite audit trail,” he wrote in an email. “We are actively working with our vendor on the best solution for preserving and storing the first draft provided by the assistant, as well as a minor tweak to ensure the disclosure is present on each printed page, rather than just the first page as it is now.”

A spokesperson for the East Palo Alto Department said his agency has no official response to the law at this time.

A woman dressed in a police uniform sits at a desk in an office looking at a computer screen.
East Palo Alto Police Officer Wendy Venegas reviews body camera footage and uses Axon’s Draft One AI-based system to draft reports based on the audio from the camera at police headquarters in East Palo Alto on Sept. 23, 2024. (Martin do Nascimento/KQED)

Opponents of the bill included the California Police Chiefs Association and the Police Officers Research Association of California, a police union advocacy and lobbying group. The Chiefs Association did not respond to a request for comment. In a statement, PORAC President Brian R. Marvel said the signed version of the law is an improvement over earlier drafts.

“In its original form, SB 524 would have put significant administrative burden on already short-staffed police forces and created broad liability by requiring agencies to retain every AI-generated draft, interim, and final version of a report, each labeled with AI disclosure language,” he wrote. “PORAC advocated to amend this bill … We were pleased to see several of these amendments taken, with the final version of the bill significantly narrowed.”

Axon representative Victoria Keough said the company is committed to complying with all state and federal laws, including SB 524.

“When developing AI for public safety, transparency and accountability are essential,” Keough wrote in a statement. “Responsible innovation remains at the core of how Axon designs and delivers new technology.”

The new requirements go into effect on Jan. 1, 2026.

Kerry Klein is a reporter and editor with KVPR in Fresno. Additional reporting was provided by KQED’s Sukey Lewis. The story was produced with support from The California Newsroom, a collaboration of public media outlets throughout the state. 

lower waypoint
next waypoint