A case study on evolving assessment practices evolve to ensure they continue to measure student understanding, judgement and accountability in an AI-enabled world from Southern Cross University and the University of the Sunshine Coast.

If generative AI can produce a database schema in seconds, testing whether students can produce one too has stopped being much of a test at all. A case study from Professor Rania Shibl at Southern Cross University and Dr Judy Watson at the University of the Sunshine Coast (UniSC) tackles this head-on, not by trying to ban AI from the process, but by moving the goalposts entirely. Instead of asking students to generate content, the assessment asks them to evaluate it – to diagnose, correct and take responsibility for a piece of work that isn’t their own. It’s a small reframing with big implications for how assessment might work in an AI-enabled world.

The problem they faced will be familiar to anyone teaching computing or data disciplines. In UniSC’s second-year database design course, ICT211, students were traditionally assessed on their ability to design database schemas and write SQL. But with tools such as ChatGPT now capable of producing convincing code in an instant, that task had stopped measuring much at all. If a chatbot can generate the answer, what exactly is the exam testing?

Shibl and Watson’s solution was “Liam Chat” – a simulated AI client built on OpenAI’s API and deployed through ShinyApps.io. Crucially, Liam Chat wasn’t there to help students. Its only job was to generate a unique SQL script for each student – deliberately riddled with flaws such as wrong data types, missing constraints and logical inconsistencies – along with a matching dataset for a fictional business, The Smart Café. Students then had to find the errors, fix them and get the database working.

The genius of the design is in what Liam Chat doesn’t do. It offers no hints, no explanations, no corrective feedback. It simply hands each student a personalised mess to untangle. That single constraint changes everything about what the assessment measures. Copying an AI-generated answer is pointless, because there isn’t a clean one to copy – only judgement, diagnosis and correction will get the job done. Collusion becomes near-impossible too, since no two students are working from the same script.

The results bear this out. Students with a solid grasp of database fundamentals could reason their way through the flaws. Those who had leaned on AI without understanding the underlying concepts struggled, regardless of how sophisticated their tools were. Failure rates rose compared with previous years, but so did the honesty of what the results actually showed: this was a test of understanding, not typing speed. Tellingly, staff reported no cases of plagiarism or misconduct, and marking became simpler, since assessors were checking whether code ran and rules were followed rather than second-guessing where an answer came from.

The theoretical grounding is solid too, drawing on Wiggins’ authentic assessment theory and constructivist learning principles – the idea that students build understanding by wrestling with genuine, messy problems rather than absorbing tidy model answers. It also reflects a wider shift in thinking about AI in education: not as a threat to police, but as a “cognitive partner” whose presence can be designed around, so that human judgement and accountability remain squarely in the frame.

Perhaps the most useful message here is one of transferability. The same logic – deliberately flawed, individualised artefacts requiring evaluation rather than production – could apply well beyond database design, from programming and analytics to business modelling and beyond. In an AI-enabled world, it seems, the smartest assessments might be the ones that are broken on purpose.