Safe, Harmful and Problematic: One challenge of NSFW AI chat is determining the gradation of harm across conversations. While they often do well detecting explicit content through keyword matches using natural language processing (NLP) algorithms, these AI systems still struggle to understand the subtleties of harm. MIT Technology Review reports that nearly a third of AI moderation tools strayed into incorrect moderate severity territory — either flagging non-abusive comments or missing low-severity but abusive behaviour, such as trolling.
The big reason why this is so challenging Is that NSFW AI chat systems need a lot of data to train their models. These datasets often contain direct, black-and-white examples of what is harmful behavior but fail to give a significant amount or even any at all more fine-grained/in-between type evidence. The AI can therefore easily detect clear threats or explicit language but may fail to address more subtle forms of emotional and psychological danger. The Verge reported, for example, that when testing AI systems to detect emotional abuse over conversations they were accurate only 65% of the time — revealing a major disparity in recognising shades of harm.
The other limitation is also due to efficiency in processing. Artificial IntelligencePowered by conversational AI, these systems can handle thousands of conversations simultaneously and process them in real-time due to their ability to crunch large amounts of data quickly. This scales, but don't add context to each and every conversation AI. Gartner also forecasts that AI will be responsible for 75% of online content moderation by 2025, but adds this needs to work better in tandem with humans who should oversee the rules and step manually handle any edge cases where more nuanced harms have been detected (such as hate speech). When the harm is indirect, or multi-layered as above and on charts like these below (basically most social issues), my argument boils down to: points for brevity are great too but they may tend towards missing non-immediate cues that could alter how impactful those harms truly exist.
A further problem is the complex question of what constitutes harm in different contexts. Harm can be physical, emotional and psychological as well social, manifestations of which may differ in conversation. NSFW AI chat, for instance might automatically tag posts as having explicit sexual content but not understand that manipulation and gaslighting touch on a completely different level of harm. Elon Musk also famously remarked on the subject as well: 'AI could be more dangerous than nukes.' Although he is talking about AI at large, his statement helps illustrate that AI does not have the moral and emotional judgement necessary to evaluate different levels of harm. This limitation is essential in any platform that deals with private or dangerous content.
Cultural differences also get in the way of Rooster and Plaisir's tech. Something that might seem in one culture to be dangerous can appear nonthreatening at best or amusingly stupid at worst. If an AI System is trained on data from some cultural chrono where the root cause for its inception was based, then it might not do a fine job with interpreting content [E.g. Preference] of other regions — leading to wrong assessment about harm done by that piece of work at 3AM in night having MC Rightfully Roaring Haha Account]--> Nonetheless, in non-Western contexts they have close to a 20% error rate and this is largely because of the difficulty with interpreting language specific cultural nuances as Forbes explains.
At the same time, AI is getting better at recognizing emotional cues and context. NSFW AI chat systems, which are becoming more and more common in our world today use techniques such as sentiment analysis to measure the emotional tone of a conversation. Sentiment analysis tools will augment AI content moderation accuracy by 15 percent, especially around areas like emotive or psychological harm according to Statista. This represents progress, but the truth is that these tools are still some way off being perfect especially in more complex or multilayered conversations.
To dive deeper into the ways in which NSFW AI chat manages content moderation and different related harms, download nsfw ai chat where I explore how AIs pick up on subtle conversations.