OpenAI has announced the launch of a tool to distinguish between human-generated texts and the created by Artificial Intelligence systems, such as ChatGPT and GPT-3. Is named OpenAI AI Text Classifier, and for now it is not excessively accurate, since it has a success rate of 26%. However, according to OpenAI, if used in combination with other systems, it can be useful in preventing users of text-generating AI systems from abusing them.
Until now, mostly because students were starting to cast texts created with ChatGPT as part of their homework. They have also used them in other areas, which has led various institutions to prohibit the use of ChatGPT on their networks and equipment. This is the case of several schools in the United States. They have done so out of fears of the impact they may have on student learning.
Among the entities that have banned ChatGPT is Stack Overflow, whose managers have decided that their users cannot share content generated with ChatGPT, alleging that Artificial Intelligence makes it very easy for users to flood their discussion forums with more than dubious answers to the questions posed.
In other words, their mission will be to help discover those who use texts generated by Artificial Intelligence, ensuring that they have written them, or any other human. Of course, as the company points out, it is best to use it as a complement to other systems for identifying the text source, instead of using it as a primary tool to identify if it is written by humans.
In addition, it is a classification tool considered initial by OpenAI, although the company has decided to launch it to obtain information from its users about its utility. In OpenAI they also assure that they hope to share improved systems in the future, although without specifying.
As with ChatGPT, OpenAI AI Text Classifier is an Artificial Intelligence language model trained with a multitude of publicly available text examples on the web. But unlike said tool, it is tuned to predict the probability that the text it analyzes is generated by any Artificial Intelligence model capable of generating text.
Specifically, OpenAI has trained this tool with text from 34 text generation systems from five different organizations, including OpenAI. This text was matched with somewhat similar human-written text on Wikipedia, websites pulled from links shared on Reddit, and a set of human-written samples compiled by an older OpenAI text generation system.
Text Classifier does not work, yes, with any type of text. You need me to have a minimum of 1,000 characters, or between 150 and 200 words. In addition, it does not detect plagiarism, and according to the company, it is more likely to fail to detect texts written by an AI if the text is written by children or in a language other than English. The latter is mainly due to the fact that the data with which the tool has been trained is fundamentally in English.
The tool, when analyzing a text, displays a response, which determines the probability that the text is or is not generated by Artificial Intelligence. Thus, it will classify it as “very unlikely” that it is generated by an AI when it believes that this probability is less than 10%; “unlikely”, when the probability of this being the case is between 10% and 45%; “not clear” if the probability that it was not written by a human is between 45% and 90%; “possibly” generated by AI if it has between 90% and 98% probability that it is so, and “probably” generated by an AI if the percentage probability exceeds 98%.