As generative AI tools like GPT-4 and Claude continue to reshape higher education, a new study argues that it’s time to fight machine with machine. The paper, titled “Machine vs Machine: Using AI to Tackle Generative AI Threats in Assessment”, was presented at the Learning, Teaching & Student Experience 2025 conference hosted by the Chartered Association of Business Schools. It proposes a dual-strategy framework that uses AI to evaluate the vulnerability of assessments to AI-generated cheating before exams are even administered.
The approach merges static analysis and dynamic testing to proactively identify which types of assignments and questions are most susceptible to manipulation by AI tools. With 74% to 92% of students reportedly experimenting with generative AI for academic work, the stakes are high. According to the authors, existing solutions like plagiarism detection software and manual rubric revisions are insufficient, flawed by bias and static assumptions that fail to keep pace with AI’s rapid evolution.
What makes assessments vulnerable to generative AI?
The study identifies eight critical dimensions that make educational assessments more or less vulnerable to AI exploitation. These are: specificity and contextualization, temporal relevance, process visibility requirements, personalization elements, resource accessibility, multimodal integration, ethical reasoning requirements, and collaborative components.
Assessments that rely on general prompts or outdated content are especially easy targets for generative AI models trained on large public datasets. By contrast, tasks that require recent events, access to exclusive resources, or draw on personal experiences are harder for AI to fake convincingly.
One example includes requiring students to apply course-specific insights to real-world, current case studies. Another involves integrating multiple forms of expression, such as pairing visual data with reflective writing, to create barriers for text-based AI tools. The framework’s core principle is to exploit what AI models still struggle with: moral agency, context-specific reasoning, real collaboration, and unstructured process thinking.
To illustrate this, the authors introduce a scoring system that quantifies assessment vulnerability across these eight dimensions, visualized using radar charts to pinpoint weak spots in an exam design.
How does the dual strategy work?
The framework consists of two complementary phases. The first, static analysis, involves auditing assessments based on predefined patterns of weakness that generative AI models are known to exploit. This includes evaluating whether tasks are too generic, lack current relevance, or are overly reliant on widely available sources.
However, static analysis alone cannot account for how fast AI models are evolving or how students may creatively prompt them. That’s where the second component, dynamic testing, comes in. It involves actually running assessment prompts through AI models to see how well they perform, revealing real-world vulnerability that static analysis might miss.
The two methods create a feedback loop: static analysis guides where to test, and dynamic testing helps refine static criteria. This integration mirrors cybersecurity principles of red-teaming, in which systems are deliberately attacked to expose flaws before real adversaries do. It also draws from educational theories like constructive alignment and authentic assessment, ensuring the approach enhances both security and pedagogical integrity.
Notably, the framework includes a vulnerability scoring system that maps assessments on a scale rather than a binary classification. Thresholds categorize assessments into red, amber, or green zones, guiding educators on whether to revise, monitor, or maintain existing formats. These thresholds can be adjusted based on course level, disciplinary context, or institutional policy needs.
What does this mean for the future of higher education?
The implications are profound. Instead of treating AI as a threat to be suppressed, the framework repositions it as a partner in building better assessments. This dual-use of AI both identifies flaws and supports educators in making improvement, turning the challenge of academic integrity into an opportunity to improve pedagogical design.
The model also shifts the concept of validity in assessment. Traditional definitions focus on whether an assignment measures the intended learning outcomes. But in an AI-augmented world, educators must also ask whether assessments can distinguish authentic human learning from machine-simulated performance.
In practice, this could mean moving away from standardized essay formats toward assignments that are unique to a course, class, or even an individual student. It could also mean rethinking how group work, ethical reasoning, and real-time collaboration are incorporated into assessments, given that generative AI cannot yet authentically simulate those interactions.
However, the study does acknowledge limitations. As AI tools continue to evolve rapidly, today’s vulnerabilities may become obsolete tomorrow. Students themselves are adapting, using advanced prompt engineering or combining human and machine responses to bypass existing safeguards. Moreover, increased complexity in assessment design must be balanced with accessibility, fairness, and resource constraints.
