Generative AI text detectors can easily be tricked – Study

There are significant reductions in the accuracy of generative AI (GenAI) text detectors faced with simple techniques to manipulate AI generated content. This indicates that they cannot currently be recommended for determining academic integrity violations, due to accuracy limitations and potential false accusations that undermine fair and inclusive assessment practices.

This is the main message to emerge from a study titled “Simple techniques to bypass GenAI text detectors: Implications for inclusive education”, published in the International Journal of Educational Technology in Higher Education last month.

The study was authored by Mike Perkins, Binh Vu, Darius Postma, Don Hickerson, James McGaughran and Huy Khuat at British University Vietnam along with Jasper Roe at James Cook University Singapore.

It aims to guide educators and institutions in the critical implementation of AI text detectors in higher education, and to highlight the importance of exploring alternatives to maintain inclusivity in the face of emerging technologies.

Key message to academic community

Lead author of the study Professor Perkins, head of the centre for research and innovation at British University Vietnam, told University World News: “Our findings reveal that these AI text detectors can be easily tricked, leading to a false sense of security from universities using these tools.

“The most important finding of our study is that AI text detection tools can be easily deceived by students who intentionally try to do so. This significantly reduces the reliability of these tools and challenges the claims made by their creators about their effectiveness.”

He continued: “Our key message to the academic community and higher education policy-makers is that attempting to detect and punish students for using AI is not a sustainable approach.

“We can’t rely on AI text detection tools to ensure academic integrity. It’s like trying to hold back a flood with outdated methods. Instead, we need to adapt our educational practices to incorporate AI responsibly.”

He warned: “The problem higher education faces is substantial if we continue to rely on ineffective AI detection tools. Universities and academics who don’t adapt to new technologies will face significant challenges to the overall integrity of the qualifications they provide.

“Relying on traditional assessment approaches and assuming AI detectors will catch ‘cheating’ gives a false sense of security, potentially compromising the value and credibility of education.”

Lack of comprehensive studies

Despite growing interest in AI text detectors, “there is a lack of comprehensive studies that evaluate their performance against various GenAI tools and adversarial techniques, particularly in the context of educational inclusivity and equity”, the research article noted.

“This gap in knowledge hinders educators’ ability to make informed decisions about implementing these tools in higher education settings,” it stated.

The study seeks to close this gap with systematic analysis of AI text detectors’ efficacy and potential impact on inclusive education practices.

The research investigated the efficacy of six major generative AI text detectors when confronted with machine-generated content modified to evade detection, to assess their reliability in identifying AI-generated text in educational settings.

To achieve that, the study used three popular GenAI tools to generate short samples of text. Altered versions of the samples were created by applying six adversarial techniques. Ten samples written by humans were used as controls.

Late last year, all of the developed samples were tested against seven popular AI text detectors to determine the effect of adversarial techniques on text detection accuracy.

A weakness of the research, the article said, was that it used a relatively small number of samples, three GenAI tools and seven AI detectors; these do not cover the spectrum of possible writing styles, types of manipulation, and available tools.

Study outcomes

The AI detectors tested had a mean accuracy rating of only 39.5% when evaluating unmanipulated AI-generated content. “Importantly, regarding the human-written control samples, only 67% of the tests were accurate, leading to significant concerns regarding the potential for false accusations from these tools,” the study pointed out.

Perkins and colleagues wrote: “When adversarial techniques were applied to the AI-generated samples, the average accuracy of the detectors dropped further to 22.14%, with some techniques, such as adding spelling errors and increasing ‘burstiness’, proving highly effective in evading detection.”

‘Burstiness’ refers to abrupt shifts in quality, coherence or relevance in AI generated content. The high variations between tools ranged from 1.5% to more than 42% when the AI generated samples were subjected to adversarial techniques.

Error analysis highlighted the risk of false accusations and undetected cases. “With the rate of false accusations at 15%, considering the major impact that this could have on student outcomes, we consider this to be a major concern for student equity,” stated the study.

“These findings underscore the limitations of current AI text-detection tools in accurately determining the authorship of a given piece of text, particularly when faced with deliberate attempts to obscure the nature of the sample,” the study noted.

Impacts and recommendations

The high sensitivity of AI detectors to the application of adversarial techniques has implications, according to the research.

First, people with “technological sensibility and resources, as well as inclination, will be able to disguise GenAI content relatively easily for the purpose of misrepresenting authorship. This reduces the equality of assessment (or publication of research) and thus advantages some groups over others, acting as a barrier to inclusion”, the study stated.

Second, there is potential for AI detection tools to unfairly penalise students who write with a less complex textual structure – including non-native English speakers or lower-proficiency English speakers. This suggests that currently, the use of GenAI detection software to identify academic misconduct may produce barriers to inclusive assessment.

Perkins told University World News: “We propose that universities stop using AI text detection tools for punitive measures. Instead, they should focus on changing assessments to allow for the responsible use of AI.

“We recommend implementing frameworks like the AI Assessment Scale to support academics in effectively integrating AI into their teaching and assessment practices.”

He explained: “The main challenge universities might face in implementing our recommendations is time constraints on educators, who already have many demands. To address this, universities need to provide dedicated time, space and training for faculty to understand these new tools and technologies.

It is crucial to help educators grasp the implications of AI for assessment design and execution. This might involve restructuring workloads or providing additional resources to support the transition to integrating AI into teaching and assessment.”

He concluded: “It’s important to note that there may still be a place for AI detection tools in the learning process, particularly in helping students develop their own voice and writing style. However, these tools should be used for formative work rather than punitive measures.

“The goal should be to educate students on effective and ethical AI use, not to catch and punish them. This approach aligns with our broader recommendation of adapting to, rather than resisting, the integration of AI in education.”