The researchers conclude that the available detection tools are neither accurate nor reliable and have a main bias towards classifying the output as human-written rather than detecting AI-generated text. Furthermore, content obfuscation techniques significantly worsen the performance of tools.
Weber-Wulff, et al. (2023). Testing of detection tools for AI-generated text. International Journal for Educational Integrity, 19(1), 26.
Stop using AI detection services.
At this point it seems farcical to continue using AI detection, since they all suffer from a range of problems:
- They have an unacceptably high level of false positives i.e. finding AI-generated text that is, in fact, written by a person.
- They disproportionately discriminate against non-English first language speakers.
- They are not accurate or reliable.
- They are easily fooled by simple text obfuscation.
There are plenty of articles written about the problems AI detection (here and here, for example), but the most damning is that OpenAI – the company that builds ChatGPT – has withdrawn it’s own AI detection system because it’s not accurate.
If the company creating ChatGPT can’t accurately detect text written by ChatGPT, I’m not confident that anyone else can make the claim that they’ve figured it out.