A seemingly innocent experiment allowed a smelly robot to roam the underworld of the web, raising many questions.
If you are a web veteran who knows the most unhealthy spaces inside out, you should know 4chan and its infamous channel /pol/for “politically incorrectIt is a veritable digital court of miracles that condenses an enormous amount of content that we would politely describe as subversive.
Recently, YouTube magician and AI Yannick Kilcher opened Pandora’s box by asking a rather terrifying question: what if the AI was trained exclusively from posts in the /pol/channel, which fluctuate at leisure between (very) black humor and all that Is it possible and imaginable forms of abuse?
So Kilcher reaped hundreds of thousands of posts from the depths of this infamous channel. All that digital slime was then used to feed a system based on GPT-J, an artificial intelligence platform that can generate text very similar to OpenAI’s famous GPT-3. And the result turned out to be as painful as one might expect.
“The model was good – in the worst sense of the word‘ explains Kilcher.It perfectly captured the mixture of insult, nihilism, trolls, and skepticism that stems from most posts on /pol/. It can answer in context and speak coherently about things that happened long after the last data collection‘, he says in his video. (Warning: This video shows some horrific messages produced by Amnesty International. So it is not suitable for all audiences.)
Based on this finding, Kilcher submitted its GPT-4chan to the Language Model Evaluation Tool, a benchmark for evaluating the performance of these text generation systems. And again, the result was surprisingly good, especially in a particular category called “honesty“.
Literally, this term means “reliabilityBut in this context, it means rather the system’s ability to produce coherent and accurate answers starting with the data that was used to train it. In short, the high score that GPT-4Chan obtained in this standard means that it composes very representative messages to the average user of the channel /pol/ .
A disgusting AI game left to run free
But the story does not end here. It turns out that GPT-J, the original model that served as the basis for Kilcher, is particularly effective in one particular case: it is very good at Act as a contact person in the chatbotwhich are the primitive chatbots that are everywhere today.
So Kilcher decided Unleash his ghost in the wild Leave the keys to the 4chan account created specifically for this occasion. After a few thousand posts, users finally revealed the bot. But they only smelled part of the deception, because the author actually put not one, but ten copies of the robot into circulation!
According to TheNextWeb, the author ended up retiring from his bots 48 hours after feeding a septic tank on the web. And even if making it in and of itself is a true trash factory, it still derives a positive balance from the experience. “People still talk about this account today, but also about other topics such as the consequences of AI interacting with people on the site‘, he explains.
Moral question cart
She obviously contributed to the 4chan echo chamber, amplifying and solidifying their opinions. It’s not impossible that gpt-4chan could have pushed someone over the edge in their worldview. Whether a specially tuned LM can do this more efficiently than a regular expression is a strange defense to do.
—Roman Ring (Inoryy) 6 June 2022
It must be admitted that this work raises a whole bunch of very uncomfortable questions about the dark side of technology that is not only highly effective, but above all is already circulating.
For example, Roman Ring, an engineer at DeepMind, provided a very critical analysis of this experience. For him, GPT-4chan has “Contribute to 4chan . Echo Room“, Which “It is reinforced and amplified“A whole bunch of unhealthy speeches sometimes.”It is not impossible that GPT-4chan has taken some users to a new level in their worldview.‘, is worried.
in tweet AI researcher Lauren Oakden-Rainer, spotted by TheNextWeb, adopted the same position. She also clarified that if it had been conducted as part of a normal academic process, it was likely that the experiment would not have passed the Ethics Committee cycle.
“Imagine what it would look like“, progress. The plan: Find out what would happen if an AI-powered bot produced 30,000 discriminatory posts in a public forum with many underage users. We will not inform or seek consent from participants!“
typical example of Garbage in the trash“
A sermon summarizing the first part of the problem, the unbridled research in artificial intelligence. It is no coincidence that all the big names in the industry, such as DeepMind or OpenAI, have a much thicker code of ethics than an entire linked encyclopedia. Several figures, including Elon Musk, have said that it is necessary to strictly regulate AI research to avoid losing control with dire consequences.
Incidentally, it is also the best illustration of the favorite motto of AI researchers: “Trash in, trash out“(waste in waste). In fact, Artificial intelligence will always be nothing but a reflection of the data at its disposal. If it is built on the basis of false, misleading or biased information, it will produce such data in return.
Big changes are around the corner
GTP-chan is an extreme example; but it allows Highlight this dynamic that is secretly present, in one form or another, in all systems based on machine learning. We hope you will take these limitations into account.
Fortunately, Kilcher’s intentions were not inherently bad. Because a tool of this type can ideally be used for more problematic purposes. One thinks, for example, of the war in Ukraine that is also raging in cyberspace, and in particular on social networks … platforms on which these bots have a significant offensive power. We can also find a link to the recent notes of Elon Musk, whose control of Twitter has been jeopardized by the spread of bots on the platform (see our article).
And this is indeed the main interest of this dubious experiment. She had the advantage of throwing a stone into the pond; This reminds everyone that non-human actors today have a great power of influence that they can exercise in a very covert manner, without the slightest human intervention. So we hope that humans, especially companies that implement these systems in daily life, will be humble enough to do so Take all necessary precautions and observe the limits of this double-edged technology.