Newsom Orders Oversight of AI Companies
Newsom Orders Oversight of AI Companies; Sen. Weber Pierson’s AI Health Care Bias Bill Awaits His Action
Antonio Ray Harvey | California Black Media
As artificial intelligence rapidly expands into health care and other high-stakes fields, Gov. Gavin Newsom issued an executive order Sept. 18 accelerating California’s efforts to independently oversee AI companies and strengthen safety guardrails.
The order advances the development of an emergency shutdown mechanism, or “kill switch,” for advanced frontier AI models and expedites independent safety audits. It also directs a newly formed group of global AI safety experts to submit a formal legal and technical roadmap by Nov. 16.
Newsom said the order was prompted by the federal government’s resistance to regulating the industry and efforts to prevent states from enforcing their own AI regulations.
“We’re not waiting to act. We’re going to speed up our work on substantial and responsible AI oversight before it’s too late,” Newsom stated. “We’re going to do this thoughtfully but with urgent velocity; the stakes are too high to wait or delay action.”
The executive order is part of a broader effort by California State Government to keep regulations in step with the accelerating development and use of AI. Those efforts include legislation addressing independent audits, transparency and the potential for algorithms to perpetuate discrimination.
Among the measures awaiting Newsom’s decision is Senate Bill 503, authored by Sen. Akilah Weber Pierson (D-San Diego). The bill would require AI developers and deployers to make reasonable efforts to identify, monitor and guard against unfair bias or prejudice in medical algorithms that affect patient care.
SB 503 passed the Assembly on Aug. 24 by a 70-1 vote. The Senate concurred in Assembly amendments the following day by a 39-0 vote. The measure, a priority of the California Legislative Black Caucus, was introduced during the first year of the 2025-2026 legislative session.
“This bill will ensure that AI tools are safe, fair and equitable,” Weber Pierson said. “Especially in high-stakes health care settings, by specifically requiring AI technology used in healthcare facilities be identified if it supports clinical decision-making or healthcare resource allocation, ensuring such tools are mitigated for bias and requiring ongoing monitoring of those tools.”
Weber Pierson, a physician, has emphasized that AI tools used in health care and medicine are not inherently neutral. She told the Assembly Privacy and Consumer Protection Committee in July that algorithms trained on historically biased medical data could reproduce or worsen the same disparities.
Because commercial health care algorithms frequently rely on existing data, Weber Pierson said, AI systems could inherit unscientific and systemic assumptions, including the false premise that Black people naturally have lower lung capacity.
“In the analysis of this bill, there were multiple examples that pointed out how AI bias has impacted patient care directly or indirectly,” she said. “Health care is an area where we should be striving to ensure that everyone is getting the best care and becoming the healthiest Californian that they can be, regardless of their race, gender, socioeconomic status.”
Weber Pierson added, “unfortunately, what we have seen in the past is that those biases have come into play. What you see with AI is that it is only as good as what kind of information has been placed into the system initially. And, so, if biases have been placed in the system initially, then the outcomes will be biased.”
Newsom has already signed other measures intended to strengthen California’s AI oversight framework. Senate Bill 813, authored by Sen. Jerry McNerney (D-Pleasanton), establishes a network of trusted AI evaluators to help ensure advanced technologies are developed and deployed safely.
On Sept. 9, Newsom also signed Assembly Bill 1405, authored by Assemblymember Rebecca Bauer-Kahan (D-Orinda). The law establishes third-party auditing requirements and transparency standards intended to prevent conflicts of interest involving auditors and the technology companies they review.
Bauer-Kahan, an attorney and chair of the Assembly Privacy and Consumer Protection Committee, has focused much of her legislative work on algorithmic discrimination and ethical AI. Her committee vetted SB 503.
“I think the bill is really important. I mean, it covers all protected classes. So, it covers me as a woman who seeks health care as well,” Bauer-Kahan said. “I know it’s Black Caucus’s priority (bill) because it’s critically important to the Black community, but it’s important to all protected classes.”
