Author: Alan Maguire

40+ years experience as a trainer, instructional designer, quality manager, project manager, program manager, RTO auditor, RTO manager and VET adviser.

AI agents in VET: A shortcut to non-compliance?

Introduction

After recently reviewing a suite of VET training and assessment materials purchased from a well-known commercial supplier, I published an article titled, ‘Human versus AI: The future of assessment design’.

The resources I had reviewed were disappointingly unfit for purpose. I identified several critical issues, including:

  • Overly complex numbering and an excessive amount of fragmented documents made navigation difficult.
  • The content was cluttered with unnecessary instructions and jargon that is neither learner-friendly nor used in actual workplaces.
  • The training and assessment materials lacked details and felt like generic templates had been used rather than materials tailored for the Unit of Competency.

The overall quality was bland and disconnected. This is highly characteristic of AI generated content. I later confirmed that this supplier is a ‘leading’ user of AI agents to produce their materials.

This is a following-on article warning all who use, or are considering to use, an AI agent to develop training and assessment materials. Also, it is a warning to RTOs who are intending to purchase training and assessment materials that have been produced by an AI agent.

I am not against using AI. I design and develop training and assessment materials, and I use an AI chatbot to assist me.

Let’s first look at the difference between an AI chatbot, AI assistant, and AI agent.

What is the difference between an AI chatbot, AI assistant, and AI agent?

An AI chatbot describes the ‘chat’ format or interface with AI. An AI assistant describes the overall role of helping the user. And an AI agent describes an AI that can act autonomously.

In the Australian VET system, the distinction between these three tools is defined by their autonomy and integration into an RTO compliance workflow.

Here is one specific example of how an instructional designer might use each of the three AI applications.

AI chatbot: The conversational researcher

When unpacking a new unit of competency, a chatbot acts as a reactive sounding board. You manually copy technical jargon or Performance Criteria into a separate window to request plain-English explanations or workplace scenarios. It requires a constant back-and-forth exchange, where the AI only knows what you explicitly provide in the chat. This manual ‘copy-paste’ workflow makes it a useful external tool for brainstorming and simplifying complex training requirements.

AI assistant: The integrated co-writer

As you draft learner guides or assessment tools within your word processor, an AI assistant works alongside you in real-time. Because it is context-aware, it ‘sees’ your active document, allowing it to suggest knowledge checks or generate marking rubrics based on your specific text. You can refine your tone or create content without switching windows. This integrated approach streamlines the design process by providing immediate, relevant support inside your workspace.

AI agent: The autonomous worker

For complex tasks like gap analysis, an AI agent operates with high autonomy. Once you set a goal, such as auditing assessment documents against a unit’s requirements from training.gov.au, it proactively executes a multi-step workflow. The agent navigates sites, downloads requirements, and identifies evidence gaps across files without further prompting. Unlike reactive tools, it completes the entire project independently and delivers a finished mapping matrix directly to your inbox.

The following is a summary comparing the above three AI applications.

Using AI agents to develop training and assessment materials

While AI agents offer significant efficiency in automating high-volume tasks, their use within the Australian VET sector, specifically under the 2025 Standards for RTOs, poses significant risks when developing training and assessment materials.

Here are five ways that relying on an AI agent can degrade the quality of training and assessment materials

The compliance illusion

AI agents excel at keyword matching but lack the expert judgment to determine if a task measures competency. An agent might incorrectly flag an assessment tool as ‘fully mapped’ just because it identifies specific terms from a Performance Criteria. However, it cannot determine if the task actually represents a valid or authentic measure of competency in a real-world workplace. This creates a ‘compliance illusion’ that can lead to non-compliance during a compliance audit.

Compromised intellectual property

Developing high-quality, training and assessment materials requires significant investment. Unless you are using a private AI system, uploading an RTO’s documents can mean your IP is used to train external AI models. For many RTOs, this is not just a quality issue but a major breach of data sovereignty and a loss of competitive advantage.

Pedagogically flawed

Training Packages on training.gov.au are complex and frequently updated. An AI agent may inadvertently pull historic definitions or draw from outdated datasets. Furthermore, it often lacks the ability to interpret the Companion Volume Implementation Guide, which provides the essential context for how a unit should actually be delivered and assessed, leading to mapping that may be technically correct but pedagogically flawed.

Lack of accountability for ‘hallucinated’ mapping

If an AI agent produces a mapping matrix that claims a specific content or assessment item covers a Performance Criteria or Foundation Skill when it actually doesn’t, the responsibility still rests entirely with the RTO. Unlike a human instructional designer who can provide an evidence-based rationale, an agent cannot justify its professional judgment. This lack of accountability results in unreliable mapping.

Erosion of contextualisation

A core requirement of the VET sector is contextualisation. This means tailoring training and assessments to a specific industry or learner cohort. AI agents tend to produce generic, one-size-fits-all training and assessment materials. Relying on an autonomous agent risks producing ‘cookie-cutter’ materials that fail to meet compliance or contextualised requirements.

Conclusion: Efficiency must not replace expertise

The allure of ‘set and forget’ AI agents for resource generation and compliance mapping is tempting for the time-poor VET sector. However, there is a vast chasm between functional automation and quality materials. Speed is irrelevant if the output fails a compliance audit.

Outsourcing instructional design to autonomous AI agents risks sacrificing human professional judgment. While AI can complete complex tasks at lightning speed, it lacks the capacity to understand workplace nuances, specific learner cohorts, or the pedagogical depth of a Training Package.

For RTOs, the warning is clear. Investigate how developers of training and assessment materials have used AI. Is it a chatbot for research, an assistant for drafting, or an agent for autonomous creation? As human oversight decreases, the risks to compliance and learner outcomes increase.

Technology should be embraced as a tool, not a replacement. Use chatbots to brainstorm or assistants to refine prose, but keep the human instructional designer at the centre of the development process. In an era of AI agents, human expertise is the only safeguard against a ‘cookie-cutter’ future.

Please tell me what you think!

Human versus AI: The future of assessment design

Introduction

Recently, I reviewed a suite of VET training and assessment materials purchased from a well-known commercial supplier. Despite the provider’s reputation, the resources were disappointingly unfit for purpose. Focusing specifically on the assessment components, I identified several critical issues:

  • Poor usability: Overly complex numbering and an excessive amount of fragmented documents made navigation difficult.
  • Language and literacy barriers: The content was cluttered with unnecessary instructions and jargon that is neither learner-friendly nor used in actual workplaces.
  • Lack of context: Assessments lacked specific scenario details and felt like generic templates rather than materials tailored to the unit of competency being assessed.

The overall quality was bland and disconnected. This is highly characteristic of AI generated content. I later confirmed that this supplier is indeed a ‘leading’ user of AI to produce their materials. This serves as a stark reminder: while AI is a powerful tool, it cannot replace the human expertise required to create meaningful, compliant VET resources.

Structuring assessment tasks

While there are typically multiple ways to structure assessment tasks, the quality of that design varies significantly. At the highest level, a structure is effective, efficient, and compliant, balancing regulatory requirements with a smooth user experience. Other designs may be adequate and compliant but ultimately burdensome, creating unnecessary hurdles for both the learner and the assessor. More concerning are structures that are inadequate but appear compliant on the surface, masking deeper flaws. Finally, some structures are simply inadequate and obviously non-compliant, failing to meet the basic standards required for a valid assessment.

To illustrate these differences in practice, I have provided the following three distinct comparisons between AI-generated and human-designed assessment tasks across various industry sectors. These three examples highlight how a human-led strategy ensures that the structure remains both pedagogical and practical. While the AI versions may tick boxes in a literal sense, the human-designed versions demonstrate a deeper understanding of how to weave complex requirements into a logical, streamlined workflow that supports an effective, efficient and compliant assessment process.

Example 1. BSBCMM411 Make presentations

The following is the Performance Evidence for the BSBCMM411 Make presentations unit of competency.

The following are assessment tasks generated by AI.1

The following show the assessment tasks generated by a human.2

The following is a list1 of five reasons why the human-generated assessment structure for BSBCMM411 unit is superior to the AI-generated version.

  • Logical chunking of workflow: The human version groups the planning, delivery, and review into a single cohesive task for each presentation (Task 2 and Task 3), whereas the AI splits the planning and delivery into entirely separate tasks.
  • Reinforcement of the full cycle: By requiring the candidate to complete the entire cycle (Plan-Deliver-Review) for the first presentation before moving to the second, the human structure allows for immediate application of “lessons learned”.
  • Explicit material development: The human-generated structure explicitly includes the “development of presentation aids” within the planning phase, ensuring this critical requirement is not overlooked, while the AI description is more generic.
  • Clarity on “different” scenarios: The human structure clearly mandates that Task 3 must be a second presentation that is “different to the presentation delivered in Task 2”, providing a clear instruction for meeting the unit’s diversity requirements.
  • Reduced administrative confusion: In the AI structure, an assessor must jump back and forth between Task 2 (Planning) and Task 3 (Delivery) to grade one presentation. The human structure allows an assessor to finalise all evidence for “Presentation 1” within a single task block.

Example 2. CHCECE037 Support children to connect with the natural environment

The following is the Performance Evidence for the CHCECE037 Support children to connect with the natural environment unit of competency.

The following are assessment tasks generated by AI.1

The following show the assessment tasks generated by a human.2

The following is a list1 of three reasons why the human-generated assessment structure for CHCECE037 unit is superior to the AI-generated version.

1. Direct alignment with assessment requirements

The Performance Evidence explicitly requires evidence of supporting children’s knowledge on three occasions.

  • Human Design: Tasks 2, 3, and 4 in the human version clearly provide these three distinct opportunities (Indoor, Outdoor, and Aboriginal/Torres Strait Islander focused).
  • AI Design: The AI version only lists two clear implementation experiences (Experience A and B) in Task 3, potentially failing to meet the “three occasions” mandate.

2. Specific inclusion of cultural perspectives

The unit requires that at least one occasion must involve Aboriginal and/or Torres Strait Islander peoples’ use of the natural environment.

  • Human Design: Dedicates a specific, standalone task (Task 4) to ensure this mandatory requirement is met and observed.
  • AI Design: Completely omits this specific cultural requirement in its brief descriptions, focusing instead on generic activities like “seed growing” or “scavenger hunts”.

3. Clear Indoor/Outdoor distinction

The unit requires one indoor and one outdoor opportunity.

  • Human Design: Explicitly structures Task 2 as an indoor activity and Task 3 as an outdoor activity, ensuring the candidate covers both environments.
  • AI Design: Focuses heavily on the outdoor environment (Task 2 audit and Task 3 “nature play”), without clearly designating or requiring a specific indoor engagement.

Example 3. CPCCCA3010 Install windows and doors

The following is the Performance Evidence for the CPCCCA3010 Install windows and doors unit of competency.

The following are assessment tasks generated by AI.1

The following show the assessment tasks generated by a human.2

The human-generated assessment tasks ensure full compliance with the specific Performance Evidence for CPCCCA3010 unit. The following is a list1 of three reasons why the human-generated assessment structure is superior to the AI-generated version.

1. Inclusion of specific door types

The Performance Evidence requires the installation of a sliding cavity door unit and door and a pair of doors.

  • Human Design: Includes “Task 4” specifically for the sliding cavity door and “Task 5” for the pair of doors.
  • AI Design: Uses generic categories like “External Door” and “Internal Door”, which fails to explicitly require these two specialised installation types.

2. Accurate quantity of installations

  • Human Design: The human-generated tasks align perfectly with the requirement to install “a” (single) standard window
  • AI Design: The AI-generated Task 2 requires the candidate to install two windows, which adds an unnecessary burden not specified in the performance evidence.

3. Integration of planning and installation

  • Human Design: Integrates the “plan” and “prepare” requirements directly into every individual practical task (Tasks 2, 3, 4, 5, and 6). This ensures that the planning is context-specific to the unique requirements of a window, a sliding cavity door, or a pair of doors.
  • AI Design: Separates “Planning & Compliance” into a standalone Portfolio (Task 3). By treating planning as a generic administrative exercise rather than an embedded part of the installation process, the AI version risks a disconnect between the candidate’s theoretical plan and the actual technical preparation required for different types of frames and doors.

Conclusion: Why the human designer is irreplaceable

The examples above highlight a consistent pattern: while AI can generate a list of tasks that look like an assessment, it lacks the professional judgment to design a strategy that is actually fit for purpose.

The disparity between these two approaches boils down to three critical factors:

  • Nuance and compliance: As seen in the CPCCCA3010 and CHCECE037 examples, AI frequently misses specific requirements that are essential for a finding of competency. A human designer reads between the lines of a Training Package to ensure no mandatory evidence is overlooked.
  • Pedagogical workflow:  AI tends to “atomise” tasks into clinical, disconnected steps. In contrast, human designers understand how a job actually functions. By grouping planning, execution, and review into a single cohesive task, as seen in the BSBCMM411 example, humans create a natural assessment flow that mirrors real-world workplace practice rather than a fragmented digital checklist.
  • The “Goldilocks” principle of evidence: AI often oscillates between two extremes: providing too little detail or creating “assessment bloat” by requiring more work than is necessary. A human expert knows how to design a strategy that is “just right”, meeting every requirement specified by the unit of competency without placing an unnecessary administrative burden on the learner or the assessor.

AI is a powerful assistant for brainstorming or drafting, but it is a poor architect. In the high-stakes environment of VET compliance, an assessment strategy is more than just a document. It is a roadmap that needs to be accurate and compliant. The “human-in-the-loop” must remain the “human-at-the-helm.”

Investing in human-led design isn’t just about avoiding “bland” materials; it’s about ensuring that our VET students are truly competent and that our RTOs remain compliant.

Footnotes:

1 On the 2nd of March 2026, Gemini was the AI platform used to generate the assessment tasks for the three examples. It was also used to compare the assessment structure generated by AI and the human.

2 Alan Maguire was the human who generated the assessment tasks for the three examples. He has had more than 40 years experience designing training and assessment. Alan may be getting older, but he is not yet redundant.

Dissatisfaction with the TAE40122 qualification

It is no secret that the TAE40122 Certificate IV in Training and Assessment is disliked by many people.

Every six months over the past 2 years I have conducted a poll to find out if people were enjoying their Certificate IV in Training and Assessment.

The following graph shows the most recent poll result and the results from previous polls.

And here is an analysis of the most recent poll compared with previous polls.

The result from November 2023 shows that 50% of people studying for their Certificate IV in Training and Assessment were enjoying it and 50% were not enjoying it, or only sometimes enjoying it. This is when most people were doing the TAE40116 qualification.

The results in July 2024 and November 2024 shows a massive decrease in satisfaction and massive increase in dissatisfaction. This is the year when the TAE40122 qualification began to be implemented by most RTOs.

The two results for 2025 shows an increasing satisfaction and subsequent decreasing dissatisfaction. I have assumed this is because RTOs have been improving the way they deliver the TAE40122 qualification. The November 2025 result shows 33% are satisfied. However, this is not a good result since two thirds of people are dissatisfied.

Does it matter if people enjoy doing their Certificate IV in Training and Assessment? Yes, it matters. If people are not enjoying it, then they become dissatisfied, and some get confused, frustrated, experience self-doubt, and the barriers to learning are increased.

Sadly, if you are not enjoying your Certificate IV in Training and Assessment, you are not alone.

Do you need help with your TAE studies?

Are you a doing the TAE40122 Certificate IV in Training and Assessment, and are you struggling with your studies? Do you need help with your TAE studies?

Ring Alan Maguire on 0493 065 396 to discuss.

Contact now!

logo otws

Training trainers since 1986

Is Australia’s VET system changing?

This afternoon I was preparing the conclusion for a TAE40122 training program, which started in March 2025 and concludes this Thursday. At the commencement of the training program, I stated that “the VET system is complex, and it is continuously changing”.

I thought it would be appropriate to revisit and identify what has changed since this training program started eight months ago.

VETNet.gov.au website

The vetnet.gov.au website has been decommissioned. For the past ten years, this website had been used to give access to Training Package Implementation Guides and other relevant documents that support the implementation of Training Packages.

Training.gov.au website

There have been ongoing changes to the training.gov.au website, including:

  • Providing access to Training Package Implementation Guides
  • Combining the Unit of Competency and Assessment Requirements into one downloadable document.

The single Unit of Competency document is a return to what we had prior to the implementation of the Standards for Training Packages 2012. This is a ‘back to the future’ moment, in other words, something we had in the past that has been recycled in the present.

Standards for RTOs

The Standards for RTOs 2015 has been replaced by the Standards for RTOs 2025. There are now three pieces of legislation rather than one. This tends to make it more difficult to find information.

Also, ASQA has replaced the ‘Users’ Guide to the Standards’ with ‘Practice Guides’. One comprehensive document has been replaced by nineteen documents, and the Practice Guides are vague compared to the Users’ Guide to the Standards. This tends to make it more difficult to find and understand the information relating to compliance.

Training Package Organising Framework

The Standards for Training Packages 2012 has been replaced by the Training Package Organising Framework 2025. The new Training Package Organising Framework includes:

  • Two templates for Units of Competency (instead of one template)
  • One of the two templates removes elements and performance criteria
  • Foundation Skills have been re-defined as ACSF skills only, and digital literacy skills are optional for Training Package developer to include.

The release of Units of Competency using the new templates has yet to occur. Therefore, it is too early to say what will be the impact. However, this could be the beginning of the end for competency-based training and assessment in the Australian VET system.

Vocational Degree

The ‘Vocational Degree’ has been introduced as a new VET qualification in the AQF. Only higher education providers could deliver qualifications at the AQF Level 7 until this introduction of the ‘Vocational Degree’.

Artificial intelligence (AI)

AI has improved and is popping up everywhere. Also, search engines are incorporating AI-generated summaries, for example, Google often presenting an AI Overview when a person conducts a search. These have contributed to an increased use of AI by VET students, in particular when answering questions designed to gather evidence of a person’s knowledge.

Generally, RTOs have been slow to develop assessment strategies to ensure authenticity of evidence. I assume cheating services are declining because AI has become user-friendly and freely available.

In conclusion

Over the past eight months there has been many changes to the VET system.

Some changes have an immediate impact, such as, providing access to Training Package Implementation Guides on training.gov.au website instead of vetnet.gov.au website. Also, this seems to be relatively insignificant.

Some changes are significant and the impact of those changes are delayed. A significant change that has the potential to disrupt the entire VET system are the new templates for Units of Competency as specified by the Training Package Organising Framework. Although the change has been released, the implementation has yet to occur.

Some people have been lobbying for Australia’s competency-based training system to be replaced by a tertiary education system with characteristics like those that we had prior to 1993 when the current VET system was introduced. This is truly a massive ‘back to the future’ moment.

How to analyse or unpack a unit of competency

My advice to people who need to analyse or unpack a unit of competency is to highlight text and write text. But if you don’t know why you are highlighting, then you don’t know what to highlight. And if you don’t know why you are writing text, then you don’t know what text to write.

This article aims to demonstrate how to analyse or unpack a unit of competency and explain why we highlight and write text when we analyse or unpack a unit.

Introduction

I deliver the TAE40122 Certificate IV in Training and Assessment qualification. And frequently I find some people have difficulty understanding what they need to do to analyse or unpack units of competency.

The following shows the original BSBTEC201 Use business software applications unit of competency download from the training.gov.au website.

The next three examples show how some people have unpacked this unit.

Example 1

This example shows parts of the content being highlighted. However, this highlighting has no or limited value in analysing the unit.

Example 2

This example shows different colour highlighting has been used. However, this highlighting has limited value.

In this example, it also shows that the analysis is incomplete. There is no evidence (no highlighting) to show that the Foundation Skills, Knowledge Evidence or Performance Evidence have been analysed.

Example 3

This example shows general comments about the unit rather than an analysis of the unit’s content. In the black text boxes, I’ve tried to make it a bit easier to read the green highlighted text. These comments do not help analyse or unpack the contents of the unit.

The above examples are indicators that someone doesn’t yet understand why they are highlighting or writing text.

Why do we highlight or write text?

Units of competency are designed to be ambiguous. This ambiguity allows us to interpret and contextualise the unit to meet different situations and environments that occur in different workplaces.

What does ‘interpret’ mean?

To ‘interpret’ means to explain the meaning of something, to understand it in a particular way, or to translate it from one language to another. Units of competency have often been written in VET language, and we need to be translated into plain English or into words that are clear and unambiguous for ourselves and others to understand.

For example, let’s interpret ‘software applications’.

The first thing to notice is that it is plural rather than singular. In other words, more than one software application. Therefore, we can interpret ‘software applications’ to mean word processing application, spreadsheet application, presentation application, email application, etc.

What does ‘contextualise’ mean?

The verb ‘contextualise’ means to place something (like a word or piece of information) within a context. The main purpose is to make something clearer, easier to understand, or more relevant.

The BSBTEC201 Use business software applications unit of competency can be contextualised for workplaces that use different software applications.

For example, let’s contextualise ‘software applications’ for three different contexts.

Techniques used to analyse or unpack a unit

Start with a Microsoft Word version of the unit of competency downloaded from the training.gov.au website. Then use the following four techniques when analysing or unpacking the unit:

1. Highlighting

Use different coloured highlights to visually connect similar or related information. Conversely, use distinct coloured highlights to visually separate unrelated information. Essentially, you are using colours to highlight which pieces of information belong together and which do not.

2. Use shapes and lines

Shapes and lines can be used as an alternative or complement highlighting to visually connect related information or visually separate information that is not related. For example, you could draw a box around all related items or use a connecting line (like an arrow) to show the flow between them.

3. Numbering

Sequencing information

Use numbers to establish the logical sequence of information. For instance, even if the Performance Criteria aren’t listed in the order they’re executed, assigning numbers will clearly identify the correct, step-by-step progression (step 1, step 2, etc.).

Cross-referencing information

Also, to simplify cross-referencing information, we can assign a number to Foundation Skills, Knowledge Evidence, and Performance Evidence, as they are not currently numbered. For example:

  • FS plus a sequential number to be allocated to each Foundation Skill (FS1, FS2, etc.)
  • KE plus a sequential number to be allocated to each Knowledge Evidence (KE1, KE2, etc.
  • PE plus a sequential number to be allocated to each Performance Evidence (PE1, PE2, etc.)

We do not need to number the Performance Criteria since they are already numbered (PC1.1, PC1.2, PC2.1, etc.)

This numbering system makes it easier to refer from one section to another without needing connecting lines.

4. Writing text

Add text in the downloaded Microsoft Word version of the unit of competency. The purpose of this text is to record your interpretations and contextualisation. Use red or blue coloured text so that your notes are easily visible and cannot be confused with the original black text of the download unit.

An example of an analysed or unpack unit

Here is an example showing how highlighting and text has been used to analyse a unit of competency. Visually, it obviously shows that the entire unit of competency and assessment requirements (from start to end) has been analysed.

The above example is a bit hard to see the details.

Here is the first page…

Here is the second page…

And here is the third page.

From the analysed or unpacked unit, we can begin to clarify the work task or tasks to be performed. The following is a task breakdown (step-by-step procedure) for the BSBTEC201 Use business software applications unit of competency.

Also, it is important to note that this unit requires a person to be competent at using at least three different software applications.

In conclusion

This article started with an explanation about why we highlight and write text when we analyse or unpack a unit. Then an example has been used to demonstrate how to analyse or unpack a unit of competency.

Units of competency are download from the training.gov.au website as a Microsoft Word file. The following Microsoft Word skills are required to analyse or unpack the unit:

  • Enter text
  • Change text colour
  • Bold and un-bold text
  • Use highlighter tool
  • Change colour of highlighter
  • Insert shapes (select text box, shape or line to be inserted)
  • Change size, location, and colour of a text box, shapes and lines.

Units of competency vary. Each will need to be analysed or unpacked before we can determine the appropriate assessment methods and tasks, and then we can identify the training content and sequence to be delivered. The Australian VET system is competency-based and the ability to unpack a unit of competency is essential for all TAFE teachers, trainers and assessors working for RTOs.

Do you need help with your TAE studies?

Are you a doing the TAE40122 Certificate IV in Training and Assessment, and are you struggling with your studies? Do you need help with your TAE studies?

Ring Alan Maguire on 0493 065 396 to discuss.

Contact now!

logo otws

Training trainers since 1986