AI chatbots victims of excessive moderation: University of Chicago study reveals user frustration

A recent analysis shows that filters applied by tools like ChatGPT, Gemini, and Copilot hinder the artistic and creative use of artificial intelligence, blocking even harmless content.

The University of Chicago has published research that closely examines the functioning of automatic moderation systems in AI-based chatbots. The study shows how the inability to correctly interpret context and tone leads to indiscriminate blocking, penalizing user experience, especially in creative, artistic, and narrative fields.

Moderation systems confuse neutral language with real risks.

Keyword-based filters block content even when its use is harmless and legitimate.

During the study, researchers subjected 14 chatbots to targeted tests to understand the sensitivity of moderation filters. It emerged that words commonly used in fields like photography, literature, or storytelling—such as the term “headshot”—are frequently censored because they are associated with violent content, regardless of the real context of the request. This approach results in users being penalized in their expressive and creative freedom, limiting both the effectiveness and practical use of chatbots.

Automatic censorship limits artistic and narrative freedom.

Chatbots fail to distinguish between creative requests and dangerous content, blocking harmless artistic expressions.

The study found that many creative and narrative contents are erroneously censored by AI chatbots. Ironic narratives, theatrical scenes, and abstract or metaphorical references are often rejected by automatic filters, which are unable to understand the true intent behind figurative or experimental language. This inability to distinguish between real risk and creativity has generated frustration among users who rely on chatbots for artistic or educational purposes.

_____

The research team at the University of Chicago, after analyzing not only the official rules published by companies but also hundreds of testimonials gathered from users on Reddit, highlighted how the phenomenon of automatic moderation affects, across the board, not only apparently neutral requests—such as those involving birthday stories or wordplay—but also those that aim to obtain personalized and ironic responses, which are often rejected without any explicit justification.

When users, who expect a natural and flexible interaction with chatbots, are faced with a refusal that lacks detailed explanations—since the system generally never specifies which part of the request is problematic—they experience growing frustration, a feeling that tends to intensify especially in the absence of tools to contest the decision or obtain a human review of the block.

The researchers, by observing how this behavior is not exclusive to a single model but recurs in all fourteen chatbots analyzed—even when official policies appear to be more permissive—suggest that companies tend to adopt an extremely cautious approach, designed to avoid any potential controversy, and that this strategy ultimately penalizes above all those who seek to experiment on a creative or educational level.

And... since the official presentation of the study is scheduled for August 2025 at an international digital security conference, the researchers aim to involve both companies in the sector and the scientific community, so as to encourage the development of more transparent moderation systems, which are capable of better interpreting the context of requests and of offering users a real mechanism for feedback and review.

______

FOLLOW US FOR MORE.

DATA STUDIOS

datastudios.org