Tag: assessment validation

Do RTOs expect trainers to participate in pre-assessment validation?

I conducted a LinkedIn poll during May 2025 asking, “Does your RTO expect trainers to participate in assessment validation?”

Here are the results:

The result of this poll means that the TAE40122 Certificate IV in Training and Assessment qualification should include a core unit about participating in assessment validation – and it does. The relevant unit of competency covering assessment validation is TAEASS413 Participate in assessment validation. This unit requires a person to not only participate in assessment validation but also participate in pre-assessment validation.

‘Assessment validation’ is different from ‘pre-assessment validation’. This raises another question, “Do RTOs expect trainers to participate in pre-assessment validation?” Let’s explore this topic a little before answering the question.

What is pre-assessment validation?

Over the past few years, the term ‘pre-assessment validation’ has been introduced to describe the review of an assessment tool prior to implementation. This review activity has been around for much longer than the term ‘pre-assessment validation’.

In 2015, ASAQ published a guide to the development of assessment tools. This guide was structured around a 3-step process:

ASQA’s guidelines states that the quality checking should occur before implementing the assessment tool. This quality check has recently become known as ‘pre-assessment validation’.

The elements of the TAEASS512 Design and develop assessment tools unit of competency is consistent with the 3-step process published by ASQA, and it includes ‘undertake a systematic review of the assessment tool [before implementing it]’ as a performance criteria.

Currently, the Australian VET system is using three different ways to describe the same activity that should be conducted before an assessment tool is implemented:

‘Pre-assessment validation’ is the review of an assessment tool prior to implementation. This includes checking for:

  • Relevance to the current workplace and industry requirements
  • Compliance with the principles of assessment
  • Compliance with the rules of evidence
  • Usability of the assessment tool.

What’s the difference between pre-assessment validation and assessment validation?

‘Pre-assessment validation’ is not the same thing as ‘assessment validation’. In an attempt to avoid confusion, it is useful to refer to ‘assessment validation’ as ‘post-assessment validation’. Using the prefix ‘pre’ or ‘post’ helps to identify these two different activities as being different.

The TAEASS413 Participate in assessment validation unit of competency describes the following:

  • Pre-assessment validation is the validation of an assessment tool before it is first used.
  • Post-assessment validation is the validation of the assessment tool, practices and judgements after it has been used to conduct assessments.

Most RTOs expect their trainers to participate in post-assessment validation (93% of poll respondents). However, it is highly unlikely that trainers will participate in pre-assessment validation. Pre-assessment validation is an activity conducted when developing an assessment tool or purchasing an assessment tool.

Developing an assessment tool

The person developing an assessment tool should conduct a pre-assessment validation. The committee with the responsibility for developing the TAE40122 qualification said that most trainers would not be involved in developing assessment tools. This was the reason for removing the ‘design and develop assessment tools’ unit of competency as a core unit for the Certificate IV in Training and Assessment.

Very few trainers will be the developers of assessment tools or have the responsibility for the RTO’s compliance. Therefore, it is highly unlikely that trainers will participate in pre-assessment validation when assessment tools are developed by the RTO.

Purchasing an assessment tool

An RTO may purchase an assessment tool rather than develop it. The person with the authority to purchase assessment tools will usually check the assessment tool before it is purchased. (If they don’t, they should!)

Very few trainers will have the authority to purchase assessment tools or have the responsibility for the RTO’s compliance. Therefore, it is highly unlikely that trainers will participate in pre-assessment validation when assessment tools are being purchased by the RTO.

In conclusion

I wish we weren’t using the term ‘pre-assessment validation’ because it gets confused with ‘assessment validation’. To avoid some of the confusion, it is best to refer ‘assessment validation’ as ‘post-assessment validation’.

In this article, I have shown that trainers are highly likely to participate in ‘post-assessment validation’, but highly unlikely to participate in ‘pre-assessment validation’.

I think the committee responsible for the TAE40122 Certificate IV in Training and Assessment qualification got it wrong. Very few trainers will participate in pre-assessment validation, and it should not have been included within a core unit for the qualification. There are many things wrong with the TAE40122 qualification. This article has only addressed one of those things.

Let’s hope that a future committee with the responsibility for updating the TAE40122 Certificate IV in Training and Assessment will not make the mistakes made by the previous committee. I had predicted that the previous committee would make mistakes, and it did. It did not want to listen to me. Maybe it should’ve. There is no timeline for reviewing and updating the Certificate IV in Training and Assessment qualification. However, the qualification or credential designed for trainers working in the Australian VET sector has been updated every 6 years: 1992, 1998, 2004, 2010, 2016, and 2022. If this pattern continues, the next Certificate IV in Training and Assessment qualification is due in 2028. Usually, it takes about 2 years to gain project approval, obtain project funding, review, design, and develop the qualification and associated units of competency.

Prepare for assessment validation by unpacking the unit

We will unpack a unit of competency as a starting point for various activities, such as:

  • Designing and developing competency-based assessment
  • Designing and developing competency-based training
  • Preparing for assessment validation.

The unpacking of a unit of competency will be slightly different depending on what activity we are doing. In this article, I will focus on unpacking units of competency when we are preparing to conduct assessment validation.

Unpacking units of competency when preparing for assessment validation

‘Unpacking’ means reading, analysing, and understanding the contents of a unit of competency. Here is a 4-step process that can be used to unpack a unit of competency when preparing for assessment validation:

  • Step 1. Read the Application statement
  • Step 2. Check the Performance Evidence
  • Step 3. Check the Assessment Conditions
  • Step 4. Quick review of the Elements and Performance Criteria.

Step 1. Read the Application statement

The Application statement is often a summary of the elements of competency. This can give us a quick overview of the unit of competency. Also, other useful contextual information may be found in the Application statement.

Example 1. BSBCMM411 Make presentations

In the following example, the Application statement uses the elements of competency to give an overview of the unit of competency.

Example 2. CHCECE037 Support children to connect with the natural environment

In the following example, the Application statement includes three relevant pieces of information about this unit of competency.

Nowhere else in the unit does it refer to the curriculum planning process. This is the first important piece of contextual information. A second piece of contextual information refers to the early childcare educator performing work under the guidance of others. And a third piece of contextual information refers to performing work in accordance with relevant legislation and industry standards. Early childcare and education is a highly regulated industry.

Step 2. Check the Performance Evidence

The Performance Evidence will consist of essential evidence of performance that must be gathered. You can use the following questions to identify relevant evidence requirements.

  • Is the quantity of performance evidence specified?
  • Is the type of performance evidence specified?
  • Are Foundation Skills accessible items?

Example 3. BSBCMM411 Make presentations

In the following example, a quantity of performance evidence is specified. It requires evidence that at least two presentations. Also, the presentations must be different.

Therefore, during the assessment validation we will be checking that there is performance evidence of at least two presentations is planned to be gathered (pre-assessment validation) or has been gathered (post-assessment validation). And checking that the presentations are different.

Example 4. CHCECE037 Support children to connect with the natural environment

In the following example, a quantity and type of performance evidence is specified. It requires performance evidence to be gathered on three occasions. At least one of those occasions must involve Aboriginal and/or Torres Strait Islander peoples’ use of the natural environment. Also, at least one occasion must be indoors and at least one occasion must be outdoors.

Therefore, during the assessment validation we will be checking that the specified quantity and type of performance evidence is planned to be gathered (pre-assessment validation) or has been gathered (post-assessment validation).

Example 5. SITHCCC025 Prepare and present sandwiches

In the following example, it shows a very detailed specification relating to the quantity and type of performance evidence. Also, it includes other details such as completing the task within commercial time constraints.

Therefore, during the assessment validation we will be checking that the specified performance evidence is planned to be gathered (pre-assessment validation) or has been gathered (post-assessment validation).

Example 6. BSBCMM411 Make presentations

Some units of competency may specify that Foundation Skills are accessible items. In the following example, it states that performance evidence relating to elements, performance criteria and foundation skills must be gathered.

Therefore, during the assessment validation we will be checking that evidence of all specified Foundation Skills are planned to be gathered (pre-assessment validation) or has been gathered (post-assessment validation). This is in addition to checking that evidence of all Performance Criteria are planned to be gathered (pre-assessment validation) or has been gathered (post-assessment validation).

Example 7. CHCECE037 Support children to connect with the natural environment

In the following example, it does not state that performance evidence relating to Foundation Skills must be gathered.

This is an example when there is no information about Foundation Skills being assessable items. We may need to check our RTO’s policies and procedures to determine if Foundation Skills are to be assessed.

Step 3. Check the Assessment Conditions

The Assessment Conditions may consist of relevant information to be checked during assessment validation. You can use the following questions to identify any relevant requirements.

  • Is the location for performing assessment tasks specified?
  • Is the access to equipment, materials, or other resources specified?
  • Is a method of assessment specified?
  • Do assessors need to satisfy any specified requirements?

Example 8. BSBCMM411 Make presentations

In the following example, the Assessment Conditions permit the performance specified by the unit of competency to occur in a workplace or simulated workplace. It is important to note that the simulated workplace must have conditions that are typical of those in a real workplace.

Example 9. CHCECE037 Support children to connect with the natural environment

In the following example, the Assessment Conditions states that the performance specified by the unit of competency must occur in a workplace. The workplace must be a regulated children’s education and care service, and it must be in Australia. Also, it should be noted that there must be children involved.

Therefore, during the assessment validation we will be checking that the assessment has occurred in a regulated children’s education and care service in Australia, and children were involved.

Example 10. SITHCCC025 Prepare and present sandwiches

In the following example, the Assessment Conditions permit the performance specified by the unit of competency to occur in a workplace or simulated workplace. It should be noted that the simulated workplace needs to be an industry-realistic kitchen, and it must be serving customers.

Therefore, during the assessment validation we will be checking that assessment has occurred in an operational commercial kitchen.

Example 11. SITHCCC025 Prepare and present sandwiches

In the following example, the Assessment Conditions has a very long list of items that must be available in the operational commercial kitchen for the assessment.

Therefore, during the assessment validation we will be checking that assessment has occurred in an operational commercial kitchen with access to all listed items of equipment, materials and other resources.

Example 12. CHCECE037 Support children to connect with the natural environment

In the following example, the unit of competency that specifies methods of gathering evidence.

Therefore, during the assessment validation we will be check that at least one occasion has been directly observed by the assessor. Also, we will check that observations and third-party reports are supplemented by other forms of evidence.

Example 13. SITHCCC025 Prepare and present sandwiches

In the following example, the Assessment Conditions specify additional requirements for assessors.

Therefore, during the assessment validation we will be checking that the assessors comply with the specified requirements to be an assessor.

Step 4. Quick review of the Elements and Performance Criteria

When preparing to conduct an assessment validation, I will not spend much time reading the Elements, Performance Criteria, Foundation Skills or Knowledge Evidence. This is because I will be spending a lot of time checking these in detail during the assessment validation.

However, it is worthwhile to quickly review the Elements and Performance Criteria. You may find something relevant without getting into the details. For example:

  • Is the unit describing the performance of one work task?
  • Is the unit describing the performance of more than one work task?
  • Is the unit describing a behavioural or interpersonal skill?
  • Is the unit primarily describing knowledge?

In conclusion

Effectively ‘unpacking’ a unit of competency is an essential activity in preparing for thorough and meaningful assessment validation. The outlined four-step approach provides a practical framework for this crucial preparatory work. By methodically reading the Application statement (Step 1), checking the Performance Evidence and Assessment Conditions through targeted questions (Steps 2 and 3), and conducting a preliminary review of the Elements and Performance Criteria (Step 4), assessment validators can gain a crucial understanding of the unit’s scope and requirements.

In short, taking the time to understand the unit first makes assessment validation clearer and more effective.

Do you need help with your TAE studies?

Are you a doing the TAE40122 Certificate IV in Training and Assessment, and are you struggling with your studies? Do you need help with your TAE studies?

Ring Alan Maguire on 0493 065 396 to discuss.

Contact now!

logo otws

Training trainers since 1986

Risk-based approach: How to determine sample size for assessment validation

Introduction

The Standards for Registered Training Organisations (RTOs) 2015 required an RTO to review a statistically valid sample of the assessments. The national VET regulator, Australian Skills Quality Authority (ASQA) provided an online calculator to determine the sample size so that it would be statistically valid.

ASQA’s Validation sample size calculator has been used to calculate the statistically valid sample size for the following two examples. [1]

Example 1

Example 2

The new Standards for RTOs 2025 has introduced a significant change to assessment validation. Instead of a fixed requirement, RTOs are now required to adopt a risk-based approach to determine their validation sample size. This means the number of assessments validated will vary considerably across RTOs, reflecting their individual risk assessments.

Select the units to be validated

The new Standards for RTOs 2025 states that “every training product on the organisation’s scope of registration is validated at least once every five years and on a more frequent basis where the organisation becomes aware of risks to training outcomes, any changes to the training product, or receives relevant feedback from VET students, trainers, assessors, and industry.” [2]

What is a training product?

The new Standards for RTOs 2025 defines training products as:

  • VET Qualification
  • Skill set
  • Unit of competency
  • Accredited short course or module.

How many units per qualification should be validated?

ASQA has provided the following guidance for RTOs: [3]

“At least two units from each qualification must be validated; however, your RTO may choose to validate more if validation of the two units identifies risks or a potential harm to learners who may not have met the required assessment outcomes, inconsistent assessment judgements have been made by assessors or assessment has not been conducted in accordance with the Principles of Assessment or the Rules of Evidence.”

Prioritising high-risk units

When RTOs prioritise the validation of high-risk units over low-risk ones, they are strategically focusing their quality assurance efforts where they matter most. High-risk units often involve complex skills, critical safety implications, or significant industry impact. By concentrating validation on these areas, RTOs can identify and rectify potential assessment flaws that could lead to serious consequences, such as workplace accidents or compromised professional standards. This approach ensures that training quality is rigorously maintained in the most crucial areas, safeguarding both learner outcomes and industry integrity. Essentially, it’s about maximising the impact of validation resources by addressing the areas with the greatest potential for negative consequences.

Identifying risks

The new Standards for RTOs 2025 states that a risk-based approach should be used to determine the sample size of assessments that should be validated. It’s important to understand that the risk-based approach in the Australian VET sector is about ensuring quality and compliance. Therefore, the risks considered relate to factors that could negatively impact those outcomes. Here are five risks that RTOs could consider when determining assessment validation sample sizes:

  • Type of unit
  • Experience of assessors
  • Changes to assessment practices
  • Volume of assessments
  • Historical compliance and validation outcomes.

Risk 1. Type of unit

Units involving high-risk activities, complex skills, or critical safety components require more rigorous validation. The potential consequences of incompetent performance are higher.

Risk 2. Experience of assessors

If assessors are new, less experienced, or are not fully qualified, there is a higher risk of inconsistent or inaccurate assessments. This necessitates a larger validation sample.

Risk 3. Changes to assessment practices

Any recent changes to assessment tools or assessment procedures can introduce inconsistencies. A larger validation sample size helps identify any unforeseen issues.

Risk 4. Volume of assessments

A high volume of assessments within a short period can increase the risk of errors or inconsistencies. Larger sample sizes are needed to maintain quality assurance.

Risk 5. Historical compliance and validation outcomes

A history of non-compliance or poor validation outcomes should lead to a more conservative approach with larger sample sizes. This allows for closer scrutiny and helps build confidence in the RTO’s assessment practices.

The above five risks are examples, not a complete list, of risks that may influence an RTO’s risk assessment. In essence, the risk-based approach should encourage an RTO to prioritise validation efforts where the potential for errors or negative impacts is greatest.

Determining sample size

Let’s look at how a risk-based approach to assessment validation sample sizes might work with some numerical examples. Here are three scenarios.

Scenario 1. High-risk unit

Scenario 2. Medium-risk unit

Scenario 3. Low-risk unit

The numbers in the above three scenarios are examples. The exact percentages will vary depending on the RTO’s own risk assessment and validation policies.

The following table compares the statistically valid sample size with the sample size for the three previous scenarios.

High-risk units should be selected for validation rather than low-risk units. Therefore, the new risk-based approach should not significantly reduce the sample size of assessments to be validated.

Selecting units to be validated

A VET qualification consists of many units of competency. The RTO will need to select at least two units to be validated. The following is a three-step process that can be used for risk-based selection of unit.

  • Step 1. Select the risk assessment criteria
  • Step 2. Create a risk assessment table
  • Step 3. Conduct and document the risk assessment.

Step 1. Select the risk assessment criteria

Here are some examples of risk assessment criteria:

  • Complex skills
  • High-risk activities
  • New, inexperienced or partly qualified assessors
  • New or changed assessment tools
  • Feedback or complaints from students, trainers, assessors, or industry.

Step 2. Create a risk assessment table

The following risk assessment table show an example with four risk assessment criteria. The number of risk assessment criteria shall be determined by the RTO, and this shall determine the number of columns required.

Step 3. Conduct and document the risk assessment

Here are risk assessment examples for two different qualifications.

Example 1

Selection of units to be validated based on the above risk assessment table should consider:

  • Units with newly implemented assessment tools (for example, BSBSUS211 Participate in sustainable work practices)
  • Units assessed by new assessors (for example, BSBTEC201 Use business software applications)
  • Units related to critical areas like safety (for example, BSBWHS211 Contribute to the health and safety of self and others).

Example 2

Unit selection for validation based on the above risk assessment table may prioritise two of the following:

  • SITHFAB025 Prepare and serve espresso coffee
  • SITHACS009 Clean premises and equipment
  • SITXFSA005 Use hygienic practices for food safety
  • SITXWHS005 Participate in safe work practices.

What assessment items must be kept? And how long do these items need to be kept?

ASQA has provided the following guidance for RTOs: [4]

“An RTO must keep all completed assessment items for each student for a period of six months from the date on which the judgement of competence for the student has been made. Completed student assessment items include the actual work completed by a student or evidence of that work, including evidence collected for a Recognition of Prior Learning (RPL) process.

If a student’s actual work is unable to be retained, an assessor’s completed marking guide, criteria, and observation checklist for each student may be sufficient. However, this evidence must have enough detail to demonstrate the assessor’s judgement of the student’s performance.”

Assessment items must be kept for at least 6 months. Some state and territory governments may require RTOs delivery government-funded or subsidised training to keep assessment items for a longer period of time.

Therefore, completed assessment items should be available for conducting assessment validation.

Random selection of assessments

While random selection is a common approach to assessment validation, best practice dictates including assessments conducted by new, inexperienced, or partially qualified assessors. Additionally, a sample of any Recognition of Prior Learning (RPL) assessments should always be included in the validation process.

In conclusion

The Standards for RTOs 2025 replace the previous fixed statistically valid sample size requirements with a risk-based approach. RTOs must now determine their own sample size based on their risk assessment.

Apart from determining the validation sample size, the RTO must select the units to be validated. An RTO should select units that are high risk rather than low risk. Prioritising high-risk units for validation allows RTOs to focus quality assurance where it’s most critical. By concentrating on complex skills and high-impact areas, RTOs can ensure assessment quality is maintained and mitigate potential serious consequences.

References

[1] https://www.asqa.gov.au/resources/tools/validation-sample-size-calculator accessed 15 March 2025

[2] Standard 1.5 (2) (b) https://www.legislation.gov.au/F2025L00354/asmade/text accessed 15 March 2025

[3] https://www.asqa.gov.au/faqs/how-many-units-qualification-should-be-validated accessed 15 March 2025

[4] https://www.asqa.gov.au/faqs/what-student-assessment-items-do-i-need-keep-and-how-long-do-i-need-keep-them accessed 15 March 2025

Do you need help with your TAE studies?

Are you a doing the TAE40122 Certificate IV in Training and Assessment, and are you struggling with your studies? Do you want help with your TAE studies?

Ring Alan Maguire on 0493 065 396 to discuss.

Contact now!

logo otws

Training trainers since 1986