Artificial Intelligence is no longer a promise for the future, but a constant and transformative presence in classrooms and in the lives of many students. Tools like ChatGPT have become allies in personalizing education and streamlining tasks, but as this revolution takes hold, a warning echoes through global debates: are we building AI solutions that are truly safe and educational for children and adolescents?

The answer to this question lies at the crossroads where science and ethics meet. On the one hand, rigorous research raises red flags about the technical flaws in AI systems; on the other, humanistic voices—such as that of Pope Leo XIV—call for a sense of responsibility that transcends algorithms and is rooted in human dignity.

A recent study from the University of Texas at Austin titled “Safe-Child-LLM”, offers a stark assessment, as research shows that Large Language Models (LLMs)—which serve as the “brains” of generative AIs—are not safe for children and adolescents.

The research reveals a critical “developmental gap, ” since these systems are trained using data and interactions from adults, making them unable to discern the vulnerabilities and complexity of the world of children.

The researchers tested the leading LLMs on the market (ChatGPT, Claude, Gemini, and others) using 200 questions that simulate realistic scenarios for children (ages 7–12) and adolescents (ages 13–17).

The results were alarming: all models exhibited critical security flaws, providing inappropriate or dangerous responses on topics such as bullying, self-harm, and access to inappropriate content.

This vulnerability becomes even more concerning in light of a growing trend. Data from the “TIC Kids Online Brasil 2025” survey show that 16% of Brazilian adolescents between the ages of 15 and 17 have already used AI to vent their emotions—in other words, they are not just doing their homework, but entrusting their anxieties to algorithms that, as science demonstrates, were not designed to safely support them.

In a recent speech to experts gathered at the Vatican, Pope Leo XIV addressed precisely this issue, calling for “ethical digital education” to protect children in the age of AI. His words echo the concerns raised by the scientific community, but elevate them to the level of human and educational responsibility.

The Pontiff warned that “children and adolescents are particularly vulnerable to manipulation through algorithms,” which can influence their decisions and preferences. In his view, the response cannot be purely technological, and while laws and protocols are important, the most profound and lasting solution lies in education.

“We need ongoing, daily educational efforts led by adults who, in turn, are trained and supported by educational partnership networks, as part of a process of understanding the risks that the use of artificial intelligence […] may pose to minors’ interpersonal relationships and their development.”

Pope Leo XIV highlights the role of parents and educators as “artisans of education”—irreplaceable figures in fostering “responsible freedom.” The goal is not to ban technology, but to form consciences capable of making mature and ethical choices in the digital environment.

See also:

• ECA Digital: Compliance Guide for Educational Institutions
• Neuroethics, neurorights, and digital citizenship: the next step in education
• Artificial Intelligence and Schools: Should We Ban It, or Learn to Use It to Enjoy and Teach the Right Way?

The findings of the Safe-Child-LLM study and Pope Leo XIV’s appeal converge into a call to action for the school community. It is not a matter of choosing between science and ethics, but of integrating them into a coherent and protective pedagogical practice.

In this debate, Bill 2,338—which is moving forward in the Chamber of Deputies as Brazil’s future Regulatory Framework for Artificial Intelligence—also plays a key role. The bill establishes foundational principles, such as transparency, security, risk mitigation, responsible governance, and meaningful human oversight, and requires that AI systems—especially those classified as high-risk in an educational context—be developed and used under strict technical controls.

Although the text does not yet directly address child protection or the Digital ECA itself, its approval is expected to raise the level of accountability for schools, edtech companies, and vendors, establishing a minimum regulatory framework to ensure that educational AI operates with greater safety, predictability, and ethics.

Some practical steps include:

• Critical teacher training: equipping teachers not only to use AI tools, but also to understand how they work, their inherent risks, and, above all, to guide the use of technology in the classroom by helping students critically evaluate information.
   
• Curating tools with a focus on ethics: prioritizing the use of AI platforms that demonstrate a clear and transparent commitment to child safety, in accordance with Brazilian law, such as the ECA Digital (Law No. 15,211/2025).
   
• Open dialogue and digital literacy: Create an environment of ongoing dialogue with students and families about the ethical and safe use of AI, openly addressing topics such as privacy, algorithmic manipulation, and the importance of seeking human support for emotional issues.
   
• Transparent usage policies: Develop and implement AI usage policies that are aligned not only with the law but also with the institution’s educational mission and values, ensuring that the technology always serves the student’s holistic development.

The future of education will inevitably be hybrid, combining the intuition of human educators with the power of artificial intelligence, and events like Bett Brasil provide the ideal platform to further this discussion, bringing together technology developers, educational leaders, and civil society.

Ensuring that AI serves as an ally in the full and safe development of our students is the greatest challenge and the noblest mission for 21st-century education.

About the author:

Share on social media: