Introduction to assessment validation

Validation is defined as the quality review of the assessment process. It involves checking that the assessment tool produces valid, reliable, sufficient, current and authentic evidence to enable reasonable judgements to be made as to whether the requirements of the training package or VET accredited courses are met. It includes reviewing a statistically valid sample of the assessments and making recommendations for future improvements to the assessment tool, process and/or outcomes and acting upon such recommendations. [1]

Assessment validation has two distinct parts:

• Part 1. Check the assessment tool for compliance
• Part 2. Review a sample of the assessments.

This article covers the first part only.

If you want to know more about the second part, then I recommend reading the information published by ASQA about how to conduct assessment validation. This information covers: [2]

• Who conducts validation?
• Scheduling validation
• Statistically valid sampling and randomly selecting samples to be validated
• Effective validation
  • Reviewing assessment practice
  • Reviewing assessment judgements
• Validation outcomes and the implementation of recommendations for improvement.

Part 1. Check the assessment tool for compliance

The assessment tool must be checked to ensure it complies with the requirements specified by the Standards for RTOs, in particular: [3]

• Compliance with the principles of assessment and the rules of evidence
• Compliance with the requirements specified by the training package or VET accredited course.

The following 6-step process can be used to check the assessment tool for compliance:

• Step 1. Read the assessment requirements
• Step 2. Review the assessment plan
• Step 3. Review the assessment matrix (mapping)
• Step 4. Check the details about how the knowledge evidence is planned to be being gathered
• Step 5. Check the details about how the performance evidence is planned to be being gathered
• Step 6. Check the overall quality of the assessment tool.

Step 1. Read the assessment requirements

This is a quick step to perform. You will read and re-read the unit of competency and its assessment requirements many times during the assessment validation process. During this first step, have a quick read of the assessment requirements and answer the following questions:

1. What is the volume or frequency of performance evidence?
2. Is the location, facilities, equipment, or other assessment conditions specified?

Step 2. Review the assessment plan

This step should also be quick. The purpose of this step is to get an overview of what is the planned assessment approach During this second step, answer the following questions:

1. Has the correct unit code and title been used?
2. How many assessment tasks are planned?
3. Is there a plan to gather the knowledge evidence?
4. Does there appear to be sufficient assessment tasks for gathering the volume or frequency of performance evidence?
5. Does the planned assessment approach seem to be simple or complex?

Note: This planned assessment approach may be found in the Training and Assessment Strategy (TAS) or other documents covering how the RTO plans to implement the delivery of the training and assessment for a unit or cluster of units.

Step 3. Review the assessment matrix (mapping)

This step should be a relatively quick step. The assessment matrix is an important document used to display how the RTO plans to gather evidence that comply with the requirements specified by the training package or VET accredited course. The assessment matrix will be used during Step 4 and Step 5 to cross-check the RTO’s planned assessment approach and the assessment instruments being used to gather evidence.

During this third step, answer the following questions:

1. Has the correct unit code and title been used?
2. Has the entire unit of competency and its assessment requirements been copied into the matrix? Are the number of items the same? For example, if the unit has five elements does the matrix have five elements? And scan the wording to ensure the matrix has the exact words as the unit of competency and its assessment requirements.
3. Is there one column for each planned assessment task?
4. Are the titles or descriptions of the assessment tasks the same in the assessment plan and assessment matrix?
5. Is every item from the unit of competency and its assessment requirements planned to be assessed? For example, is there at least one ‘tick’ in every row?

Note: Some assessment matrices will provide information or numerical indicator about the assessment item instead of using a ‘tick’. For example, the matrix may indicate that a piece of knowledge evidence will be gather by Question 1.

Step 4. Check the details about how the knowledge evidence is planned to be being gathered

This step requires an attention to details. The purpose is to ensure that the assessment tool will gather the required knowledge evidence. During this fourth step, answer the following questions:

1. Is there an assessment instrument for gathering the knowledge evidence?
2. Are the instructions to the assessor clear and concise?
3. Are the instructions to the candidate clear and concise?
4. Is the structure, format, and layout of the assessment instrument easy to follow? This includes headings, sub-headings, page numbers, and numbering of questions.
5. Is there consistency between the assessment plan, assessment matrix and assessment instrument? For example, if the assessment plan states that there are 17 questions, does the assessment instrument have 17 questions?
6. Is every item of knowledge evidence being adequately gathered? A judgement about ‘adequately’ will need to be made.

Step 5. Check the details about how the performance evidence is planned to be being gathered

This step requires an attention to details and it can take time to examine the assessment documents for compliance. The purpose is to ensure that the assessment tool will gather the required performance evidence. During this fifth step, answer the following questions:

1. Is there one or more assessment instruments for gathering the performance evidence?
2. Are the assessment conditions compliant with those stated in the Assessment Requirements for the unit of competency? This may include assessment location, facilities, equipment, and access to specified documents. For example, if the assessment conditions state that the assessment occurs in the workplace, then the assessment tasks must state that the evidence must be gathered from a workplace (not from a simulated workplace).
3. Are the instructions to the assessor clear and concise?
4. Are the instructions to the candidate clear and concise?
5. Are the items of performance evidence clearly listed or identified?
6. Is the structure, format, and layout of the assessment instrument or instruments easy to follow? This includes headings, sub-headings, and page numbers.
7. Is there consistency between the assessment plan, assessment matrix and assessment instrument? For example, if the assessment matrix states that evidence for Performance Criteria 1.1 will be gather during Assessment Task 2, then Assessment Task 2 must cover the gathering of evidence for Performance Criteria 1.1.
8. Is every item of performance evidence being adequately gathered? A judgement about ‘adequately’ will need to be made. This includes a check that the amount of evidence being gathered is compliant with the specified volume or frequency of performance evidence.

Note: Verbs are important. For example, if performance criteria says, ‘negotiate and agree with a supervisor’, then there needs to be evidence that the candidate has negotiated and agreed with a supervisor’. Also, the letter ‘s’ is important. A item of performance evidence may specify plural rather than singular. For example, if it states ‘write reports’, then more than one written report is required for evidence.

Step 6. Check the overall quality of the assessment tool

This step can take time to examine the assessment tool for compliance, readability, and usability.

1. Are there sample answers and assessment decision criteria for assessors?
2. Is the structure, format, and layout of all assessment documents easy to follow?
3. Are all instructions written clearly and concisely?
4. Are there any grammar, spelling and typo errors?
5. Is there a list of all the assessment documents required for the assessor?
6. Does the assessment tool have all the documents required for the assessor?
7. Is there a list of all the assessment documents required for the candidate?
8. Does the assessment tool have all the documents required for the candidate?
9. Has the correct unit code and title been used throughout all the assessment documents? This may include release number.
10. Do all the assessment documents have version control information?

In conclusion

Assessment validation has two distinct parts:

• Part 1. Check the assessment tool for compliance
• Part 2. Review a sample of the assessments

Assessment validation can be time-consuming and mind-bending.

Preparation before an assessment validation meeting can reduce the time at the assessment validation meeting. However, you can expect a typical assessment validation meeting to require anywhere between a few hours and an entire day. The duration of the assessment validation meeting can depend on the quality of the assessment tool and number of assessment samples to be reviewed. I regularly see poor quality assessment tools, and it takes time to properly check large numbers of assessment samples.

Clear and critical thinking is required by people participating in an assessment validation meeting. There are usually many documents to be reviewed and checked. Printing paper copies of documents (or some documents) and using ‘split screens’ on computers will help when comparing information from two or more documents, such as:

• unit of competency
• assessment requirements
• assessment plan
• assessment matrix
• assessment instructions
• assessment instruments.

Frustration and fatigue can be experienced during long assessment validation meetings. Breaks will be needed (and sometimes chocolate helps). It is a good idea to assign an experienced VET practitioner to lead the assessment validation meeting.

References

[1] https://www.asqa.gov.au/standards/appendices/glossary accessed 2 September 2021

[2] https://www.asqa.gov.au/resources/fact-sheets/conducting-validation accessed 2 September 2021

[3] https://www.asqa.gov.au/standards/training-assessment/clauses-1.8-to-1.12 accessed 2 September 2021

Do you need help with your TAE studies?

Are you a doing the TAE40122 Certificate IV in Training and Assessment, and are you struggling with your studies? Do you want help with your TAE studies?

Ring Alan Maguire on 0493 065 396 to discuss.

Contact now!

Training trainers since 1986

I have found that some people studying for their Certificate IV in Training and Assessment qualification struggle with understanding what a task breakdown is, and how to develop a task breakdown. In this article, I aim to answer the following questions:

• What is a task breakdown?
• How can a task breakdown be developed?
• When can a task breakdown be used?

Where do we start

Australia’s VET system is competency-based. A primary objective of VET is to help people learn how to perform work tasks and activities.

The four stages of competence, also known as the ‘conscious competence learning model’, describes the process of progressing from incompetence to competence in performing a task or activity. [1]

Stage 1. Unconscious incompetence

At this ‘unconscious incompetence’ stage, a person is unaware of their inability to perform a particular task or activity.

Stage 2. Conscious incompetence

At the ‘conscious incompetence’ stage, a person becomes aware that they are unable to perform a particular task or activity.

Stage 3. Conscious competence

A person who is aware of their incompetence may decide it is worth their time and effort to learn how to perform the particular task or activity. At the ‘conscious competence’ stage, the person can perform the task but they will require concentration and may need to think about each step.

Stage 4. Unconscious competence

In the final stage of ‘unconscious competence’, performing the task becomes second nature because the person has learnt and practiced performing the task so much. They can perform the task easily and without thinking about it. As a result, the task may be performed while executing another task.

An experienced or skilled worker often performs many work tasks and activities without needing to think about how or what to do.

Becoming conscious again

As a trainer, we often have mastered skills and we are no longer conscious about how we perform work tasks or activities. We just do it.

But as a trainer, we must explain and demonstrate to others how to perform work tasks, and to do this we have to again become conscious about the steps required to perform the task. The development of a task breakdown is a method that helps us become conscious again about how to perform a work task or activity.

A task breakdown is a step-by-step description about how to perform a task. It may also be known as a procedure or work instructions.

What is a task breakdown?

A ‘task breakdown’ is a document that breaks down a task. It describes the steps in a logical sequence to perform the particular work task or activity.

I use three analogies to help people understand and appreciate the value of a task breakdown: Lego instructions, Ikea instructions, and a cooking recipe.

The following three examples have three common features:

• Steps are numbered
• Illustrations are used to visually communicate
• The outcome or desired result is shown.

Lego instructions

Ikea instructions

Cooking recipe

How can a task breakdown be developed?

Use the following six steps to develop a task breakdown.

Step 1. Create 3-column table

Open a new Microsoft Word document and create a 3-column table. For example:

Step 2. Add table headings and adjust column width

The following illustrates the recommended layout and format for the task breakdown. For example:

Note: Layout and format can be modified later, if required.

Step 3. Enter step numbers

Use the first column to enter step numbers. The numbers imply a logical sequence to followed, and repeatedly use the word ‘step’ because this reinforces that each row is a step to be performed. For example:

Note: Table rows can be added or removed later, as required.

Step 4. Write brief description for each step

Perform the task or observe someone performing the task. Write a brief description for each step. For example:

Step 5. Write explanations

Write a clear and concise explanation for each step. Use illustrations, diagrams, and photos to show examples. Do not decorate or use unnecessary visuals. For example:

Important note: Remember to highlight safety requirements for the task or for a particular step.

Step 6. Review and finalise task breakdown

Ask someone to use your draft task breakdown to perform the task. Observe the person performing the task, checking the sequence of steps, and seek feedback about readability.

Check your task breakdown for grammar and spelling. Add task title, version control, and page numbers before finalising the document. For example:

Download the finished product

You can download a PDF version of the finished task breakdown for creating a task breakdown from here.

When can a task breakdown be used?

The task breakdown has multiple uses:

• Used as a planning tool to identify required knowledge, required skills, safety requirements, and resources required to perform a task
• Used to explain the process or performance of the task
• Used to demonstrate the process or performance of the task
• Used by the learner to guide them when they learn or practice performing the task
• Used to check or assess the performance of the task
• Used as a diagnostic tool when a person is struggling with learning or performing the task (used to identify what step the learner is stuck on).

Training session plan

A task breakdown describes a task. A session plan is used to describe the training to be delivered. The task breakdown should not make reference to the training process. It should only describes the work task or activity.

A training session plan can be developed based on the steps described by the task breakdown.

Observation checklist

A task breakdown can be converted into an observation checklist. For example:

A procedure that can be used in the workplace

A task breakdown should be written to guide performance of a work task or activity. It should describe the step-by-step procedure to be followed in a training environment or in a real workplace. The task breakdown should be the procedure to follow, no matter where the task is performed.

In conclusion

Creating a task breakdown is a basic skill that all trainers and TAFE teachers must have. A task breakdown describes the steps in a logical sequence to perform a particular work task or activity.

A task breakdown has a multitude of uses:

• Used by the trainer as a planning tool
• Used by the trainer when explaining and demonstrating the performance of the task
• Used by the learner to guide them when they learn or practice performing the task
• Used by the trainer when checking or assessing the performance of the task
• Used by the trainer as a diagnostic tool.

Reference

[1] https://en.wikipedia.org/wiki/Four_stages_of_competence accessed 2 March 2023

Do you need help with your TAE studies?

Are you a doing the TAE40122 Certificate IV in Training and Assessment, and are you struggling with your studies? Do you want help with your TAE studies?

Ring Alan Maguire on 0493 065 396 to discuss.

Contact now!

Training trainers since 1986

Introduction

Over the past few weeks I have been thinking about what tasks do trainers perform and what credentials do trainers need. This was triggered by the release of the Australian Skills Classification by the National Skills Commission. The occupation profile for a vocational trainer described in the Australian Skills Classification was appalling. My various thoughts have been recently published in four articles:

• What is the Australian Skills Classification? And how does it describe the tasks performed by a trainer working in the VET sector?
• Is training the same thing as teaching?
• What tasks do trainers do?
• What AQF level is appropriate for a qualification that targets a trainer working in the VET sector?

I would like to thank the many people who commented in response to these previous articles on this topic. In this article, I want to put together my many and varied thoughts.

What tasks are performed by trainers?

Qualifications must have an occupational outcome. Therefore, we need to develop an occupation profile based on work tasks and activities performed by a trainer. There are different types of trainers and not all trainers perform the same range of tasks.

What are the different types of trainers?

Some trainers work as part of Australia’s vocational education and training (VET) system. And some do not. This is the first significant difference between trainers:

• VET trainers
• Non-VET trainers.

Another difference is the job role performed: [1]

• Employed as a VET trainer (TAFE, private RTO, other RTO)
• Employed in a job role other than a VET trainer (experienced worker, workplace supervisor, workplace manager, etc.)
• School teacher delivering a VET in Schools program.

And another difference is the employment status: [1]

• Permanent employee (full-time and part-time employee)
• Non-permanent employee (casual, sessional or employed under fixed-term or short-term contract)
• Volunteer.

What are the different types of RTOs?

It has been said that VET trainers perform different tasks depending on the type of RTO they work for. The Education IRC identified six types of RTOs: [2]

• TAFE Institute
• University RTO
• Private RTO
• School RTO
• Community RTO
• Enterprise RTO.

I am not convinced that a person delivering training and assessment services for different types of RTOs actually perform profoundly different tasks:

• Are trainers expected to deliver training?
• Are trainer expected to keep training records?
• Are trainers expected to monitor progress?
• Are trainers expected to resolve issues?
• Are trainers expected to assess competence?
• Are trainers expected to keep assessment records?

Fundamentally, I think trainers will perform the same core tasks regardless of the type of RTO they work for. Some people may disagree with me, but they haven’t provided any details to persuade me otherwise.

I do acknowledge that trainers may operate under different conditions:

• Different types of learners, with different characteristics and needs
• Different types of content for different industry sectors
• Different types of training and assessment resources
• Different types of locations where training and assessment is conducted
• Different technology to be used (Moodle, Canvas, other LMS, Zoom, Cisco Webex, other web conferencing platform, etc.).

The different conditions do not essentially alter the work tasks or activities performed by a trainer. One important feature of the Australian VET system is ‘contextualisation’, and this allows RTOs to customise what they deliver to respond to different conditions. Therefore, an RTO delivering a TAE Certificate IV qualification can tailor content for a particular learner group need.

What qualifications, skill sets, and units of competency are needed?

Units of competency are used to describe the performance of work tasks and activities. They are then used as the building blocks for qualifications and skill sets. The needs of industry determines qualifications, skill sets, and units of competency. And the VET industry is mainly represented by the VET regulators, in particular ASQA, lobby groups, and the employers of VET trainers.

What are the regulatory requirements?

VET is highly regulated to ensure quality and to protect clients. The regulations specify the credentials that must be held by a VET trainer.

Deliver training within the VET context

An RTO’s training must be delivered only by a person who has the appropriate credential. Currently, this credential is the TAE40116 Certificate IV in Training and Assessment qualification or a diploma or higher-level qualification in adult education. [3]

Where the RTO engages an individual, who does not hold the appropriate credential, the individual can deliver training but they must be supervised by a person who does have the appropriate credential. The supervisor must monitor the individual under their supervision. And the supervisor is accountable for the quality of the training provided and the quality of assessment evidence collected. [3]

Assess competency within the VET context

Assessment must be conducted by a person who has attained the following units of competency: [3]

• TAEASS401 Plan assessment activities and processes
• TAEASS502 Design and develop assessment tools
• TAEASS402 Assess competence
• TAEASS403 Participate in assessment validation.

Alternatively, the person conducting assessments can have a diploma or higher-level qualification in adult education. [3] Do all higher-level qualifications in adult education ensure competence in the TAEASS401, TAEASS402, TAEASS403, and TAEASS502 units?

Where the RTO engages an individual, who does not hold the appropriate credential, the individual can contribute to assessment by collecting assessment evidence and providing it to a ‘qualified assessor’. It is the ‘qualified assessor’ who is accountable for the assessment process and determines the assessment outcome. The ‘unqualified individual’ is prohibited from determining assessment outcomes. [3]

Industry experts may also be involved in the assessment judgement, working alongside the ‘qualified assessor’. Again, it is the ‘qualified assessor’ who is accountable for the assessment process and determines the assessment outcome. The ‘unqualified individual’ is prohibited from determining assessment outcomes. [3]

What tasks are performed by a trainer?

The following diagram shows the relationship between:

• Tasks performed by a trainer
• Regulatory requirements
• Qualifications, skill sets, and units of competency.

I acknowledge that not all trainers perform the same range of tasks. Some work for RTOs and others do not, and maybe some RTOs expect their trainers to do things that are not required when working for other RTOs. Therefore, I have created five categories of trainer.

Category 1 trainer

• Deliver training within a non-VET context
• Deliver training under supervision within a VET context
• Prohibited from conducting assessment within a VET context

Category 2 trainer

• Deliver training under supervision within a VET context
• Contribute to assessments within a VET context
• Prohibited from conducting assessments within a VET context

Category 3 trainer

• Deliver training under supervision within a VET context
• Conduct assessments within a VET context

Category 4 trainer

• Perform the role of trainer or TAFE teacher
• Deliver training within a VET context
• Conduct assessments within a VET context
• Has the capability to design and develop training and assessment resources
• Has the capability to participate in assessment validation and moderation activities

Category 5 trainer

• Perform the role of senior trainer or senior TAFE teacher
• Has all the capabilities to perform the role of a Category 4 trainer
• Has the capability to design and develop training and assessment strategies
• Has the capability to lead assessment validation and moderation activities
• Has the capability to evaluate training and assessment services
• Supervise or mentor trainers or TAFE teachers.

The following table identifies the typical tasks performed by each category of trainer.

Tasks performed will determine the category. For example:

What category would match the requirements of a school teacher who is delivering a VET in Schools program? Probably, most would be a Category 3 trainer. Some would be a Category 4 trainer.

What category would match the requirements for a volunteer trainer working for a community-based RTO? Depending on the RTO requirements and supervision arrangements, the volunteer could be Category 1, 2, 3, or 4 trainer.

How to select the appropriate qualification or skill set

The following diagram can be used to help select the appropriate qualification or skill set.

A person only needs the TAE Certificate IV if:

• They are a trainer working in the VET system
• They will be responsible for conducting assessments
• They will not work under supervisions (as per the Standards for RTOs)
• They a new to the VET system.

Is AQF Level 4 appropriate for VET trainers?

The qualification for new trainers has been at the AQF Level 4 for the past 23 years. Some people think that the current qualification for VET trainers is at the wrong AQF level. I think it is the right level because of the supervision responsibilities of a trainer: supervise learners, supervise their safety, supervise their progress, etc.

Should there be more than one TAE Certificate IV qualifications?

Some people suggest that more than one TAE Certificate IV qualification is needed. How many qualification are needed? What units of competency would be included in each qualification? What units of competency would be the same? And what units of competency would be different?

I am not convinced that we need multiple qualifications. If people think some tasks are not being covered, then maybe the solution is have a few elective units. Currently, the TAE40116 Certificate IV in Training and Assessment qualification only has provision for one elective unit. If there was two or three elective units, then RTOs could select electives to meet different needs. Having said that, most RTOs would probably continue to deliver a fixed program.

In conclusion

In this article, I have introduced the concept of five categories of trainers.

Each category is supported by a different credential. Category 1, 2, and 3 trainers are covered by skill sets. Category 4 trainers are covered by a Certificate IV, and Category 5 trainers are covered by a Diploma. A pathway from Category 1 trainer to Category 5 trainer is provided with minimal duplication. In other words, one credential leads onto the next.

References

[1] https://www.ncver.edu.au/research-and-statistics/publications/all-publications/understanding-the-australian-vocational-education-and-training-workforce accessed 20 April 2021

[2] Education IRC, Industry Skills Forecast and Proposed Schedule of Work, March 2019 assessed 20 April 2021

[3] https://www.legislation.gov.au/Details/F2019C00503 accessed 20 April 2021