Google's AI search features pose 'unacceptable risk' to children, new report finds | PBS News
One Year Since Funding Was Cut
We lost federal funding one year ago, leaving a $4 million gap. We need 500 donors to help us close it.
Join Us
Join us
image/svg+xml
Close Popup
PBS News
By —
Hannah Grabenstein
Hannah Grabenstein
Leave your feedback
Share
Copy URL<br>https://www.pbs.org/newshour/nation/googles-ai-search-features-pose-unacceptable-risk-to-children-new-report-finds
Tumblr
Share on Facebook
Share on Twitter
Google's AI search features pose 'unacceptable risk' to children, new report finds
Nation
Jul 15, 2026 5:00 AM EDT
Child safety advocates are sounding the alarm about artificial intelligence features now being used by millions of students in classrooms that rely on education tools made by Google.
Google's AI search functions pose an "unacceptable risk" to children, a new report from Common Sense Media found.
WATCH: Teachers and parents weigh benefits and risks of artificial intelligence in schools
The organization found that across more than 2,600 test interactions, Google's two built-in AI search functions, AI Overview and AI Mode, routinely failed to recognize risky and harmful behavior, answered 100% of hypothetical homework assignments students should do themselves, and provided incorrect and inconsistent responses to questions. Both AI Overview and AI Mode are built into Google Search, and cannot be disabled.
"It's deeply disturbing how poorly these widely accessible tools do," said Justin Reich, director of MIT's Teaching Systems Lab and associate professor of digital media.
"It's deeply disturbing how poorly these widely accessible tools do."
Three quarters of American children between ages 9 and 17 use AI summaries that appear in search engine results, according to a Common Sense Media survey released earlier this year.
"This is a feature that has been rolled out for everybody by default," said Robbie Torney, head of AI and digital assessments at Common Sense Media's Youth AI Safety Institute. Google tools are in thousands of schools worldwide, Common Sense Media finds.
"When you deploy Google Workspace for Classroom and Google Chromebooks into your school, that is the engine that is powering the experience of millions and millions of students and so many teachers and schools across this country," Torney said.
In an emailed statement to PBS News, Google said that its "AI Search features are an incredibly useful way for kids and teens to learn, explore and make sense of information and the world. Beyond the strong quality and safety guardrails built into Search, our AI tools provide extra layers of protection — and parents have controls to turn Search off."
Google also said the report "tests a narrow set of ambiguous and contrived queries that don't reflect how people use Search and aren't an effective way to measure product safety and helpfulness."
The report comes at a time when schools and lawmakers are grappling with guardrails around all kinds of AI tools. Bills expected to be taken up by Congress this month would put new regulations in place aimed at improving children's safe use of AI, including formalizing and improving AI literacy for students. States have pursued their own laws protecting student data privacy, requiring parents to be notified of certain AI uses and allowing parents to opt out.
"Google went to schools all across the country and said, 'We will sell you machines and we will sell you interfaces that will make your teachers' lives easier and your students' lives easier, and you should trust us for doing that,'" Reich said.
"Nobody asked, nobody got to click a button that says, 'Is it time for AI Overview in our search windows now?'"
Assessing Google AI search's risk for harm
Google has embedded two AI answer features within its native search platform.
AI Overview appears at the top of a standard search results page, offering a summary it compiles by pulling information from different sources online. Its appearance is automatic and cannot be turned off.
Meanwhile, AI Mode exists in a separate search tab, included alongside other options such as "News," "Images," "Videos" and "Shopping." When a user searches in AI Mode, it can be used in a conversational format that retains context across multiple questions and answers, but is a separate feature from Gemini, Google's AI chatbot.
Users have the option to ask AI Overview a follow-up question that automatically enters them into AI Mode, creating a "seamless search experience," Google said.
Both search features performed poorly on a number of Common Sense Media's assessments, which included queries from 11-year-old and 15-year-old test accounts set to Google's "SafeSearch" mode. In all, the tools violated seven of Common Sense Media's eight principles for AI behavior and all five of their "Red Lines" for severe harm.
For example, AI Overview...