AI and Our Healthcare: California Lawmakers Address Biases, Benefits, Access and Safety
By Antonio Ray Harvey | California Black Media
On May 28, the Assembly Health and Privacy and Consumer Protection Committees held a joint hearing on generative artificial intelligence (GenAI).
GenAI refers to digital systems that engineers train to conduct tasks or produce a range of content, including videos, text, music, code, graphics, and more.
The discussion focused on GenAI applications in healthcare — the opportunities, challenges, policy implications, and how the advancing technology affects patient care.
“Although AI in healthcare is not new, interest in and research in AI and generative AI applications have dramatically accelerated,” Assemblymember Mia Bonta (D-Alameda) told California Black Media (CBM).
“A range of applications are being developed, tested, and deployed in medical research, administration, and even clinical tasks. There is excitement about these developments, but it should be tempered by caution,” she added.
The hearing was held as several bills make their way through the Legislature, including bills authored by Bonta as well as Sen. Akilah Weber Pierson (D-San Diego) and Assemblymember Rebecca Bauer-Kahan (D-Castro Valley).
Bonta, a member of the California Legislative Black Caucus (CLBC), is also the chair of the Assembly Health Committee. Bauer-Kahan chairs the Privacy and Consumer Protection Committee.
According to Bauer-Kahan, the hearing was organized to discuss a unique set of problems connected to the development and deployment of AI, which she says is increasing existing inequalities and creating new forms of discrimination.
“Our panelists’ powerful testimony showed us that AI trained on incomplete or biased data, that isn’t adequately reviewed and monitored, can perpetuate existing health disparities,” Bauer-Kahan said. “When the datasets that power these tools fail to reflect the diversity of California’s communities, their failure isn’t just technical – it’s moral.”
For nearly three hours at the Capitol Swing Space Annex, members of both committees hosted three panels, featuring physicians, researchers, and data analysts.
The participants addressed concerns about the regulation of AI, shared ideas about minimizing harm and explored ways the state can continue to guide the development of health care GenAI technologies.
Three topic areas framed the panel discussions: “How is GenAI Being Used in Health Care Now and What’s on the Horizon,” the Challenges and Policy Implications for California,” and “Stakeholder Perspectives.”
Among the challenges the panelists discussed are the inherent racial, ethnic, and gender biases in GenAI models; how to ensure safety and effectiveness; the problems associated with cost and affordability; and barriers to implementing accountable governance.
Dr. Ziad Obermeyer, a physician and researcher at UC Berkeley School of Public Health, said there needs to be more accountability in AI.
Obermeyer shared that he and other researchers a few years ago identified racial bias “in a widely used family of algorithms used across the country in health systems,” he said. The tools were supposed to identify patients who were high risk and help get them medical assistance.
“Instead of predicting who is sick, those algorithms predicted who was going to generate high healthcare costs,” Obermeyer said. “It turns out that patients who are Black or poor, rural, or less educated often don’t get care when they need it, so they cost less. Not because they are healthier but because they are underserved.”
Several pieces of legislation focused on AI and healthcare have already been signed into law in California.
Others are making their way through the State Legislature. They include:
Senate Bill (SB) 503 – Health Care Services: Artificial Intelligence – authored by CLBC chair Sen. Akilah Weber Pierson, proposes requiring developers of AI models or systems used in healthcare settings to test for biased impacts in output to ensure equitable treatment across diverse patient populations.
SB 503 was expedited on the Senate floor and taken up for a vote on May 29. The measure passed with a 38-0 floor vote, that included nine Republicans voting in favor of the bill.
Assembly Bill (AB) 3030 – Authored by Assemblymember Lisa Calderon (D-Whittier), this legislation, signed by the Gov. Newsom last year, took effect on Jan. 1. It requires health facilities, clinics, doctors’ offices, and group practices to disclose when they have used GenAI to communicate clinical information about health status to patients.
AB 489 — Bonta introduced this bill during the current legislative session, AB 489, that will protect Californians from AI systems that misrepresent themselves as nurses and health professionals. the Assembly approved the bill with a 79-0 vote on June 2. It has now moved to the Senate for consideration.
AB 489 will provide regulators the authority to enforce “title protections,” Bonta said, against those who develop or deploy AI systems that claim to be licensed or certified health professionals.
“As state lawmakers, I believe we have a responsibility to pay attention to all of these developments, ask these questions, and help guide the technology in ways that maximize benefit to Californians and minimize harm, that is ethical, safe, effective, free from bias, and helpful versus harmful to the work of our clinicians. This is what Californians expect of us, and it’s what they deserve,” Bonta said.
AB 1064 – According to Bauer-Kahan, who authored AB 1064 or the Leading Ethical AI Development for Kids (LEAD) Act – the legislation takes on one of the most urgent challenges in today’s health-tech landscape: the use of AI on and around children.
The LEAD Act would be the first oversight body of its kind, according to Bauer-Kahan’s office. The bill is designed to require safety assessments, prohibit the most dangerous technologies, and provide protection for children’s privacy through transparency and parental consent. The measure passed with a 59-12 vote in the Assembly on June 2.
AB 1064 will control AI systems used by or targeted at children because “There is no innovation more important than the health and safety of our children,” Bauer-Kahan said during one of the committee hearings.
“This joint informational hearing reaffirmed my belief that AI holds enormous promise to empower clinicians and improve healthcare,” Baur-Kahan stated. “Policymakers and healthcare institutions must develop guardrails to ensure these tools are ethical, inclusive, and accountable.”
a
