A special artificial intelligence (AI) edition of ASQA IQ provides an important reminder about AI and what is needed for robust assessment decision making.
ASQA’s five principles for responsible AI use state that providers must ensure that:
- “AI use is supported by strong governance that ensures it does not undermine the quality or integrity of VET.
- Human oversight and accountability are maintained in all AI supported activities, ensuring that decisions affecting students remain the responsibility of qualified trainers, assessors and staff.
- AI systems and tools manage information securely and in accordance with existing privacy, data protection and record keeping obligations.
- AI use supports and enhances student equity, inclusivity, accessibility and wellbeing.
- AI use aligns with training product requirements, industry expectations and the needs of the relevant student cohort.”
These principles do not create additional regulatory obligations on RTOs, but instead explain how the existing requirements in the 2025 Standards for RTOs apply when providers use AI.
ASQA illustrates the issue through a case study of a provider that considered using AI avatars to assess performance in BSBPEF502 Develop and use emotional intelligence. Students had previously participated in role-plays with trainers or volunteers acting as workplace colleagues, which could be difficult to organise. The proposed alternative allowed students to conduct simulated workplace conversations with AI avatars in scenarios involving conflict resolution, performance coaching and supporting a colleague in distress. Before introducing the system, the provider tested it with assessors, instructional designers and industry representatives.
The testing exposed a significant threat to assessment validity when it was discovered that the avatar responded favourably to particular expressions, allowing participants to achieve a successful outcome by repeating the right phrases even when the overall quality of the conversation was poor. The assessment therefore risked measuring students’ ability to work out how the technology operated, rather than their ability to manage a difficult workplace interaction. This is a particularly important consideration as students gain familiarity with AI systems and become increasingly adept at identifying the prompts, keywords and response patterns that produce the desired outcome.
The review also identified concerns about fairness and the sufficiency of the assessment evidence. Some participants with accents experienced difficulties because the avatar misinterpreted or failed to recognise their speech. The system could analyse words and tone to some extent, but could not consistently interpret body language, eye contact, active listening and other non-verbal behaviours – all of which are critical elements in how students can be expected to demonstrate emotional intelligence.
Although the unit permitted simulation, the reviewers concluded that interacting successfully with an avatar did not necessarily demonstrate that a student could manage a comparable interaction with another person.
The provider consequently retained the avatar for training and practice, but did not rely on it as the sole source of assessment evidence. Students could use it to rehearse difficult conversations and receive feedback, while assessment continued to include observations by an assessor, professional discussion and role-play involving real people.
ASQA’s case study stops short of imposing a blanket prohibition on the use of AI in assessments, but it makes it clear that a qualified assessor needs to remain responsible for deciding whether the assessment evidence is valid, sufficient, authentic and fair.
Before adopting an AI simulation, providers should test whether students can manipulate it, whether it works equitably for different cohorts, any evidence it might fail to capture and whether student performance is sufficiently realistic in a workplace context.
