I should call out that NSFW AI chat systems, with their ability to be advanced in such a way, have some serious difficulties living up to human moderation as well. This is why companies deploy these AI models, which can then process huge quantities of data at blitz-like speeds — millions upon millions messages or images in seconds to either pass through allowed content if nothing amiss found (i.e. false positive) OR be able to catch posted digusting materials quickly! In practice, though, AI remains imperfect in certain contextual and nuance aspects of human language. If we look at it, 95% sounds high but the remaining 5% have escaped through this is what shows us where AI also stops and that it has its own limitations.
In the case of human moderators, they provide context — understanding and cultural analysis that AI can not. Professionals in the field argue that AI fails to detect subtle signals like sarcasm or idiomatic regional slang — elements for which human labor remains crucial. Thus, in cases where accuracy is critical — due to legal implications or more complex social interactions — it still mandates human moderators. And companies like Facebook and Youtube spend billions a year more to keep that AI (un) well-oiled with some form of human moderation, demonstrating the large cost in terms of scaling content coordination.
One of many famous instances that demonstrate the issues with AI is what happened in 2021 when automated moderation systems from major social media platforms considered unnecessary content as offensive and received a severe response. The incident served to remind us that even the most advanced AI systems require oversight from human beings.
Figures as leading in the tech space as Sundar Pichai has mentioned AI moderation tools can help immensely with efficiency and reducing costs, but is not a practical solution to depend on them entirely due to these nuances gaps of understanding. In reality, companies generally apply AI as an initial screen to disqualify plain violations of their standards with human moderators deciding more difficult cases that demand judgment and context.
There is some nuance in this, as much of the effectiveness AI models present with moderation can be attributed to how they are customized and adapted. Depending on the agent's precision (the percentage of correct flagged content identified) and recall (how thorough an inappropriate-content dragnet), these systems can operate independently or require intervention by a human. Those parameters are in a constant state of flux, with companies using reinforcement learning to twiddle them just so, but even small changes can be time-consuming and expensive improvements sometimes taking several months between development cycles.
If you are curious to look at how AI deals with these kinds of problems, platforms such as nsfw ai chat give an idea about the right amount automatic vs human managed content moderation. While there are technological advancements, human moderation is irreplaceable — particularly in high-stakes or nuanced situations (exemplifying how the process of managing explicit content on the internet remains complex to this day).