More from this theme
Recent articles
This month saw a significant milestone for artificial intelligence (AI) in education: findings from the first randomised controlled trial investigating the use of generative AI by UK teachers.
The Education Endowment Foundation (EEF) found teachers randomly allocated to use ChatGPT to support lesson and resource creation (while using a detailed ‘how-to’ guide), significatnly reduced lesson planning time compared to a control group.
UK teachers are already embracing AI. One-third now use these tools in their work, double the number five months ago. Conducting research into what works and what doesn’t is obviously vital.
But as the EEF warns, we need to proceed with caution when it comes to off-the-shelf Generative AI tools (e.g. ChatGTP, Gemini and others) in schools. In particular, we must be careful about the quality and accuracy of the content they produce.
With regards to quality, findings in its study were promising. However, as the evaluation makes clear, they were based on a very limited sample of lesson resources generated. As the EEF concluded, many more studies and more work on quality is needed before we can hail AI as a game-changer.
The problem is that off-the-shelf Generative AI tools are simply not yet good enough to generate the high-quality and safe-for-classroom-use content teachers need. In the EEF trial, teachers taking part were given clear guidance on how to use ChatGPT for tasks such as tailoring existing materials and generating ideas.
Generative AI tools have been trained on a huge amount of data that has not been quality-assured (usually the wider internet). They use this data to predict how to respond to questions or finish phrases. As we all know, the predictions are not always accurate or useful – in part because the inputs aren’t always accurate.
So what is it that makes particular AI tools fit for classroom use? Here are three key considerations for teachers in deciding whether an AI tool is good enough to support them.
Prompt engineering
Prompt engineering is the technical term for providing clear instructions to an AI model to maximise the quality, relevance and specificity of what it produces. If we wanted ChatGPT to generate, say, an email, then a typical request (or prompt) would be around a dozen words.
This might be enough to get a half-decent email, but there’s a lot more to consider when it comes to lesson planning. Look out for tools that have much more extensive built-in instructions to make sure AI-generated content for the classroom aligns with the national curriculum, strong pedagogy and what evidence tells us about great teaching.
Retrieval-augmented generation
We know that AI is only as good as what it draws on to generate its outputs. This is where a technique called ‘retrieval-augmented generation’ (RAG) becomes useful.
RAG allows AI tools to prioritise pulling from existing content rather than solely relying on an AI model to create it from scratch. Better quality in means better quality out, so look for AI tools that draw from existing, quality-assured materials.
This means they will be far more likely to produce something that’s fit for use in a classroom and avoid the inaccuracies, bias and unsafe information that exist on the wider internet.
Transparency
Lots of AI tools out there are black boxes. Users don’t know (and can’t find out) how exactly the technology produces what it does. This doesn’t exactly inspire confidence in their quality and safety.
It’s important that information such as the underlying models, technologies and risk assessments, as well as how personal data is used, isn’t hidden from view. It should be out in the public domain for anyone who wants to access it.
Prompt engineering, RAG and transparency may not be as eye-catching as time-saving and other benefits AI tools promise. However, not all AI tools are created equal and it’s important we, as users of these tools, start to understand what to look out for before we get swept away by the hype.
At Oak, we recently released our AI-powered lesson assistant, Aila. We’ve spent a year building in and refining all of these important features, to make sure it delivers not just time-saving, but quality resources.
Aila’s prompt is 9,000 words long and works behind the scenes to make sure it has all the information it needs to generate fit-for-purpose teaching resources. Wherever it can, it also draws on Oak’s own 10,000-plus quality-assured resources rather than the general internet.
Every line of code in our product is open source, enabling anyone to view, use, and build upon it. In addition, as a public sector organisation, we’ve just published detailed information about how Aila works as part of the Government’s new Algorithmic Transparency Recording Standard.
Aila is built to automatically evaluate the quality of what it produces, so that we can keep an eye on how it performs at scale. We’ll also be looking at Aila’s impact on workload and the quality of its output and will, of course, be making those results public.
In time, off-the-shelf Generative AI tools like Chat GPT will develop so they are far more accurate, safe and aligned with what is taught in UK schools, along with a host of new tools for teachers.
In the meantime, getting to know the AI techniques that enhance both the quality of the exciting new tools on offer can help us become smarter consumers – and not settle for anything but the best.
