This article was written in March 2022.

I frequently see assessment instruments being used by TAFEs and private RTOs that do not comply with the requirements specified by the Standards for RTO. The national VET regulator, ASQA, was established in 2011. It is astonishing that after more than a decade, ASQA has been unable to encourage or enforce compliance. And what makes it even more astonishing is a move towards self-regulation. This will allow the many RTOs that are not currently compliant, to self-regulate their future compliance (or should I say, self-regulate their non-compliance).

Recently, I assisted a TAFE employee to develop assessment instruments for the CPCCCA3010 Install windows and doors unit of competency. Consequently, I was prompted to write this article as a case study to explore:

• How to determine the number of assessment tasks
• How to develop an assessment instrument to gather performance evidence
• How to develop an assessment instrument to gather knowledge evidence.

How to determine the number of assessment tasks

I always start with determining how many assessment tasks will be needed to cover the assessment requirements. This must be done before designing and developing the assessment instruments.

As a general rule, there should be a plan for at least two assessment tasks:

• An assessment task to gather the knowledge evidence
• At least one other assessment task to gather the performance evidence.

It is common to need more than one assessment task to gather the frequency or volume of performance evidence that is specified by the Assessment Requirements.

Let’s look at how many assessment tasks are required to gather the performance evidence for the CPCCCA3010 Install windows and doors unit of competency. The Performance Evidence for this unit states:

A candidate can not do all these tasks at the same time. Therefore, I would plan for four (4) assessment tasks to gather the specified performance evidence.

I usually assign ‘Assessment Task 1’ for the gathering of knowledge evidence. Therefore, I would have a total of five (5) assessment tasks for the CPCCCA3010 Install windows and doors unit of competency.

The CPCCCA3010 Install windows and doors unit of competency has six (6) elements of competency. The first and last elements are common for installing all windows and doors. The other elements of competency are only relevant to specific tasks determined by the type of door or window being installed. The following matrix shows the connection between each element of competency and each assessment tasks.

The design and development of an assessment tool is a creative process. Sometimes, we may need to change of initial assessment plan in regard to the number of assessment tasks.

How to develop an assessment instruments to gather performance evidence

So far, we have determined the need to have four (4) assessment tasks to gather the performance evidence for the CPCCCA3010 Install windows and doors unit of competency. Each of these assessment tasks may follow a 3-part process:

• Part 1. Plan and prepare
• Part 2. Install
• Part 3. Clean up

An observation checklist may be used to gather the performance evidence. I would consider the need to develop an observation checklist for each assessment task. Therefore, there will be four (4) observation checklists developed. However, some observation items will be the same for each checklist.

For example, the same observation items can be used for:

• Plan and prepare (as specified by Element 1 of the CPCCCA3010 unit)
• Clean up (as specified by Element 6 of the CPCCCA3010 unit).

But some observation items will be different because the installation of different types of doors and windows are different, and there are different Performance Criteria specified. For example, the observation checklist for Assessment Task 2 would include the Performance Criteria for Element 2, but not the Performance Criteria for Elements 3, 4 and 5.

The following matrix highlights Assessment Task 2. It shows what elements of competency are the same or different to the other assessment tasks used to gather performance evidence.

The next step will be to develop the assessment instruments.

For the purpose of this example, a third-party report is not being used to gather evidence. It is common for building and construction apprenticeships to include the CPCCCA3010 Install windows and doors unit of competency. Therefore, the apprentice would perform work under the supervision of a builder or building site supervisor. These people would ideally contribute to assessment by providing third-party evidence.

This example is only showing assessment tasks conducted by an assessor to gather performance evidence.

Developing the observation checklist

The items of an observation checklist should not be a ‘copy and paste’ of the performance criteria. Performance criteria can be vague or ambiguous. And performance criteria can lack sufficient details to clearly communicate the benchmark or expected standard of performance.

The following table show how the Performance Criteria could be analysed. It only uses the first four performance criteria of Element 1 as an example.

The technique is basically about you asking yourself questions. These questions aim to seek clarity. And these question help to interpret and contextualise.

The next thing is to draft the observation items for the checklist. The following table shows the development of observation items for the first four (4) performance criteria of Element 1.

The above table also shows how additional evidence could, or should, be gathered. These would be in addition to the observation checklist.

Many RTOs will have their own templates to be used for assessment instruments. The following an example of an observation checklist for the first four (4) performance criteria of Element 1

Sometimes, the sequence of items on the observation checklist can be different than the numerical order of performance criteria. Sometimes one performance criteria may need more than one observation items, and sometimes several performance criteria may be adequately covered by one observation item.

We need to be flexible in our approach to the development of an observation checklist. Each unit of competency is different. Therefore, the observation checklist must be created to meet the unique requirements specified by each unit of competency.

Developing a product review checklist

An observation of a person performing work tasks will often be used to gather direct evidence of performance. Sometimes, the review of a product that has been produced during the performance of the work task will be used as evidence. A product review checklist would need to be developed for an assessor to use during the assessment process.

The product to be reviewed may be:

• A document
• A completed form
• A physical item
• A non-physical item.

How to develop an assessment instruments to gather knowledge evidence

I recommend the development of an assessment task to gather all the specified Knowledge Evidence. Create a table to ensure you write at least one question to gather the knowledge evidence specified.

The following is an example for a table created for the CPCCCA3010 Install windows and doors unit of competency.

The next step is to start writing questions. Sometimes, one question will not be enough to cover the breath or depth of knowledge required. Therefore, a set of questions may need to be asked. The following are two examples.

Example 1: This is an example for the Knowledge Evidence item 5.

Example 2: This is an example for the Knowledge Evidence item 6.

Numbering the questions

Consider how the questions are going to be numbered.

Let’s say, there are four questions to be asked to gather evidence for one Knowledge Evidence item. The following are three different ways that the questions could be numbered:

• Q6a, Q6b, Q6c, Q6d.
• Q6.1, Q6.2, Q6.3, Q6.4
• Q6, Q7, Q8, Q9

Sequencing the questions

Consider the sequence the questions. Sometimes, the questions may be re-sequenced to present them in a logical order or asking questions that relate to a similar topic. For example, asking all questions relating to safety one after the other rather than scattered.

Structuring the questions

Consider giving structure to the questions. This can be done by using ‘topic headings’ with all questions relating to the topic under that heading. This will make a large number of questions easier to navigate.

Attributes needed to design and develop assessment instruments

The following attributes are required by people wanting to design and develop assessment tools:

• Good word processing skills
• Analytical skills and the willingness to spend time conducting the analyst
• Subject matter expertise.

Good word processing skills

Many RTOs have templates to be used. Unfortunately, many of these templates have been developed by word processing wizards, making them difficult to use or modify for people with limited word processing skills.

Analytical skills and the willingness to spend time conducting the analyst

Earlier this week, one of my TAE Students said,

“We have been working on planning, designing and developing an assessment tool for a day and half. And we still are not yet finished. It takes a long time to create the assessment documents just to assess if some can or cannot use a hand-held radio.”

The time required to conduct an assessment to determine if a person is competent at using a hand-held radio is typically going to be less than an hour. It is a relatively low level skill. But it takes a long time to develop a compliant assessment tool. People need time to analyse the unit of competency and associated assessment requirements. And then more time to draft assessment documents, review and trial assessment documents, and finalise assessment documents prior to implementation.

People need to know how to conduct the analysis and be willing to spend their time doing it. It can take a lot of time to design and develop compliant assessment instruments.

Subject matter expertise

The design and development of assessment instruments require subject matter expertise. It the developer is not a subject matter expert, then they will need to work with one.

In conclusion

Gathering valid and sufficient evidence of competency is a fundamental part of the assessment process. It takes time and effort to design and develop the assessment instruments to gather the required knowledge evidence and required performance evidence. Two essential sources of information are:

ASQA’s Guide to assessment tools

ASQA’s Conduct effective assessment

Other articles

And you may be interested in other related articles that I have written:

Guide to developing assessment tools

How to develop an assessment tool to gather knowledge evidence

Knowledge evidence and the Australian VET system

Do you need help with your TAE studies?

Are you a doing the TAE40122 Certificate IV in Training and Assessment, and are you struggling with your studies? Do you want help with your TAE studies?

Do you want more information? Ring Alan Maguire on 0493 065 396 to discuss.

Contact now!

Training trainers since 1986

The Education Industry Reference Committee (IRC) is exploring ‘stackable’ skill sets. The aim is to support entry-level trainers and assessors working under supervision prior to attaining their full qualification. In this article, I will answer the following questions:

• What knowledge and skills would benefit a new trainer starting work at an RTO?
• Why would a person do ‘stackable’ skill sets instead of the full qualification?
• Do we need ‘stackable’ skill sets developed for the TAE Training Package?

What knowledge and skills would benefit a new trainer starting work at an RTO?

The immediate knowledge and skills that would benefit a new trainer who is starting work at an RTO are:

• Knowledge about the Australian VET system
• Training skills
• Assessment skills

Knowledge about the Australian VET system

The TAEDES402 Use training packages and accredited courses to meet client needs unit of competency can be used to introduce new trainers to the Australian VET system.

Training skills

A trainer needs the knowledge and skills to plan, organise and deliver group-based learning. This is covered by the TAEDEL401 plan, organise and deliver group based learning unit of competency.

The Standards for RTOs allow an RTO to supervise a new trainer prior to them attaining their TAE qualification, TAE skill set or TAE units of competency. The new trainer can deliver training under supervision, but can not conduct assessments.

Assessment skills

The Standards for RTOs prohibit people without the four specified assessment units of competency from conducting assessments. Therefore, it is good if a new trainer can attain these units as soon as possible:

• TAEASS401 Plan assessment activities and processes
• TAEASS502 Design and develop assessment tools
• TAEASS402 Assess competence
• TAEASS403 Participate in assessment validation.

The ‘stackable’ TAE40116 qualification

I have developed resources for the TAE40116 Certificate IV in Training and Assessment qualification. My TAE40116 training program has been given a ‘stackable structure’, starting with The VET Framework. This covers the essential knowledge to help the new trainer understand VET terminology and concepts about competency-based training. The next topic covered is Competency-based assessment. And the following diagram shows the ‘stackable sequence’ of topics.

The ‘strawberry on top of the stack’ is the elective unit. My recommendation is to select the TAEDEL301 Provide work skill instruction unit of competency and integrate it with the TAEDEL402 Plan, organise and facilitate learning in the workplace unit of competency.

What is being proposed by the Education IRC?

The following has been published to inform us about what is being proposed by the Education IRC.

Reference: TAE Town Hall Series November 2021 (PowerPoints, Slide 10)

There are several issues with what has been proposed. It does not provide a quick pathway to skill-up new trainers to allow them to deliver training and assessment services. And the proposed Assessment Skill Set is missing one of the four specified units of competency that are required to conduct assessments.

The following diagram is an alternative approach. It would provide a quick pathway to skill-up new trainers to allow them to deliver training and assessment services for an RTO.

The above approach does not formally cover VET terminology and concepts about competency-based training. Therefore, the following diagram shows another alternative approach that would be better.

And finally, if we continue with the madness of creating TAE skill sets, the following diagram provides the ‘stackable’ skill sets to support the up-skilling of new trainer employed by an RTO.

Why would a person do ‘stackable’ skill sets instead of the full qualification?

Most RTOs prefer to employ a trainer who has previously attained their full TAE40116 Certificate IV in Training and Assessment qualification. I have only come across a small number of people who have been employed as a trainer without having the TAE40116 qualification. These people have been plumbers, TESOL/ESL trainers and one expert from the pest control industry.

The vast majority of people seeking employment as a trainer or TAFE teacher working in the Australian VET system will be required to attained the full TAE40116 Certificate IV in Training and Assessment qualification. Therefore, most people will continue to enrol into the full qualification. They will not want to enrol into a ‘stackable’ skill set because that will not provide the credential required for employment. RTOs that deliver the TAE units of competency will continue to offer the full TAE40116 qualification. It will be unlikely that ‘stackable’ skill sets will be offered because there will not be the demand.

Generally, a person will not want to do ‘stackable’ skill sets. And if they did, then they will struggle to find an RTO offering them.

Do we need ‘stackable’ skill sets developed for the TAE Training Package?

The TAE40116 Certificate IV in Training and Assessment qualification is already ‘stackable’. If it is implemented using my sequence for delivery, then the aim of supporting entry-level trainers and assessors working under supervision prior to attaining their full qualification can be achieved.

I know of one TAFE that delivered the TAE40116 Certificate IV in Training and Assessment qualification for their own newly employed trainers. This program started with the TAEDEL301 Provide work skill instruction unit of competency. Followed by:

• TAEASS401 Plan assessment activities and processes
• TAEASS502 Design and develop assessment tools
• TAEASS402 Assess competence
• TAEASS403 Participate in assessment validation

And the other core units of competency were delivered to complete the full qualification. This provided a ‘fast-track’ approach to getting their new trainers delivering training and conducting assessments as soon as possible.

The solution is for RTOs to appropriately sequence the delivery of TAE units of competency. The solution is not about creating ‘stackable’ skill sets. There is no need for the Education IRC to introduce ‘stackable’ skill sets. This will be a waste of time and effort. And this will be a waste of tax-payers’ money.

It seems that the members of the Education IRC have lacked the knowledge and experience about the TAE Training Package, how it can be implemented, and how it is actually implemented. Stupid suggestions, such as ‘stackable’ skill sets, should be stopped. Did anyone on the Education IRC try to stop this initiative? And, what other stupid suggestions are going to be pursued during the holistic review of the TAE Training Package?

Further information and advice

Are you an RTO, and do you want more information about purchasing high-quality and compliant TAE40116 training and assessment resources?

Contact now!

Training trainers since 1986

Introduction

A decade ago, many RTOs were getting away without explicitly gathering evidence of the Required Knowledge specified by the unit of competency. If challenged by an auditor, the RTO would argue that if a person could perform the work task, then they obviously must have the required knowledge. Over the past 10 years it has become abundantly clear that this argument is unacceptable. Most RTOs will now explicitly gather the Knowledge Evidence specified in the unit of competency.

In 2012, the Standards for Training Packages were changed, and a ‘new format’ for units of competency was implemented. It has taken until 2021 before most units of competency have be converted into the ‘new format’. An exception has been units of competency from accredited course that have remained in the ‘old format’. The Standards for Accredited Courses were changed in 2021, and over time, this will make all units of competency follow the same format.

Old format

The following is an example of ‘old format’:

BSBWOR202A Organise and complete daily work activities 

New format

The following is an example of ‘new format’:

BSBPEF202 Plan and apply time management (superseded BSBWOR202A Organise and complete daily work activities)

In this example, the ‘new format’ has include the statement that the candidate must be able to demonstrate knowledge. This has meant that RTOs must gather the specified Knowledge Evidence. (No more arguments with auditors.)

Note: Currently, a completely different way of describing industry skill standards is being explored with a possible introduction in 2022 or 2023. See the Australian Government’s Skill Reforms for more details about this initiative. It took a decade to fully implement the previous change. I wonder how long it will take to fully implement the next change.

Why is knowledge important?

A person needs knowledge and skills to perform work tasks. Therefore, without the required knowledge a person could not perform their work.

Also, it is said that if a person does not have the appropriate depth of knowledge they cannot respond to non-routine or emergency situations.

What knowledge is required?

The developer of a Training Package has consulted with industry to identify the required knowledge to perform work tasks. And a list of Knowledge Evidence items is given to us. Here is an example. The BSBPEF202 Plan and apply time management unit of competency has been used for the purpose of this example.

Knowledge evidence:

• time management techniques and strategies
• features of a time management plan
• organisational standards, policies and procedures relevant to own work role
• relationship between own work goals and plans and organisation’s goals and plans
• factors affecting work progress and performance improvement techniques.

The knowledge required by a person to perform a work task has already been identified for each unit of competency. We do not need to ask, “What knowledge does a person need to perform a work task?” But we may still need to ask, “When is the the knowledge used?” and “How much knowledge is required?”

When is the knowledge used?

The knowledge required to perform a work task is specified. Before the introduction of competency-based training and assessment, I use to work for one of Australia’s largest and most progressive organisations as an instructional designer. I would use the following structure to identify content for the development of curriculum.

The concept that people need knowledge and skills have carried over to the competency-based training and assessment approach embraced by the Australian VET system.

A unit of competency specifies the required knowledge but does not identify when that knowledge is used. Extra analysis is required to identify when each item of required knowledge is used to perform the work task. Here is an example. The BSBPEF202 Plan and apply time management unit of competency has been used for the purpose of this example.

Another way of presenting the above mapping information is:

It is helpful to know when the knowledge is used. This can provide a context for determining what and how much knowledge is required.

How much knowledge is required?

Often the required knowledge specified in a unit of competency is a broad description with very little details. Additional research and analysis may be need to determine how much knowledge is required.

Industry engagement

The Standards for RTOs, Clauses 1.5 and 1.6 requires an RTO to engage with industry to ensure the assessment resources are relevant and current. The aim of this engagement should include the identification of industry’s expectations for the knowledge required.

A person who is developing an assessment instrument to gather the specified Knowledge Evidence may seek further details from employers and industry representatives. Here is an example. The BSBPEF202 Plan and apply time management unit of competency has been used for the purpose of this example.

• What time management techniques and strategies are used in your workplace? (Can you give me examples about how these time management techniques and strategies are used in your workplace?)
• What are the features of a time management plan used in your workplace? (Can you show me a time management plan that is used in your workplace?
• What organisational standards, policies and procedures do you have in your workplace that relate to time management, development of work plans and communicating the progress on work plans to relevant personnel? (Can you show me a copy of these organisational standards, policies and procedures?)
• In your workplace, how are the work goals and plans of individuals aligned with organisation’s goals and plans? (Can you show me a copy of documented work goals and plans used in your workplace?)
• In your workplace, what factors affect work progress? (Can you give me some recent examples about these factors and how they have affected work progress?)
• In your workplace, what performance improvement techniques are used? (Can you give me some recent examples about how these performance improvement techniques have been used?)

Types of knowledge

The Standards for RTOs gives us the following definition:

“Competency means the consistent application of knowledge and skill to the standard of performance required in the workplace. It embodies the ability to transfer and apply skills and knowledge to new situations and environments.”

This acknowledges the need for a person to have knowledge to perform work tasks. There are different types of knowledge. Wikipedia gives us the following description:

“The term ‘knowledge’ can refer to a theoretical or practical understanding of a subject. It can be implicit (as with practical skill or expertise) or explicit (as with the theoretical understanding of a subject); formal or informal; systematic or particular.“

Our understanding of knowledge can become philosophical or esoteric if we think too deeply about what it is. As a trainer or assessor working in the Australian VET system, we need a basic understanding about what knowledge is, without getting immersed in abstruse thoughts.

Also, Wikipedia identifies descriptive knowledge (facts) and procedural knowledge (skills). The Knowledge Evidence items from a unit of competency are descriptive knowledge. I would like to give another way of looking at different types of knowledge:

• Undefined knowledge
• Defined knowledge

Undefined knowledge

Knowledge that is undefined is not exact. It may vary depending upon various situations or circumstances. It could be disputed or debated. The following is an example of a question seeking an answer for undefined knowledge:

What is knowledge?

The answer to this question can vary. There may be more than one acceptable answer.

A person may need some knowledge that is defined and some knowledge that is undefined when performing a work task. The following are some considerations when the knowledge is undefined:

• A developer of an assessment instrument being designed to gather evidence of undefined knowledge will need to develop a range of sample answers that could be acceptable.
• An assessor will need to exercise flexibility when a candidate gives an acceptable answer that has not been provided by the developer of the assessment instrument.

Defined knowledge

Knowledge that is defined cannot be disputed or debated. Therefore, the following is an example of a question seeking an answer for defined knowledge:

What is the definition of competency in the Australian VET system?

There is only one answer to this question. The Standards for RTOs gives us the definition. Therefore, the answer is defined and indisputable.

Knowledge is usually defined when it relates to legislation or regulations, such as, workplace health and safety (WHS) legislation, regulations, and codes of practice.

Bloom’s Taxonomy

The Taxonomy of Educational Objectives , also known as Bloom’s Taxonmoy, is a framework for categorising educational goals. The following illustrates the ‘original’ Bloom’s Taxonomy.

Having the ‘Knowledge’ involves the recall of it. ‘Comprehension’ refers to having a type of understanding and making use of knowledge without necessarily being able to apply it.

The Australian VET system requires ‘Knowledge Evidence’ to be demonstrated. But the problem is we can not see knowledge. It is hidden inside a person’s brain.

How can a person demonstrate they have knowledge?

We will ask a question or give an instruction to elicit a response. This is how we can determine if a person has knowledge, or not. The Bloom’s Taxonomy give us a useful approach to developing questions or instructions so that a person can demonstrate they have the required knowledge.

Bloom’s Taxonomy provides lists of action verbs that can be used to write educational goals or objectives. The following is a sample.

We can select verbs from these lists to write an instruction to elicit a response. For example:

“Define competency within the context of the Australian VET system.”

As an alternative, we can write a question rather than writing an instruction to elicit a response.

“What is the definition of competency within the context of the Australian VET system?”

Scenario-based questions

Scenario-based questions ask a candidate to describe how they might respond to a workplace situation. The following is an example.

A scenario would be given and the person would be asked questions or given instructions designed to elicit a response. For example:

“The above cartoon illustrates a workplace:

• Identify five hazards
• Assess the risks for each hazard you identified
• State a control measure to address each identified hazard
• Explain the procedure for reviewing the effectiveness of control measures.“

A person’s response can be used to determine if they have the required knowledge and to what extent they comprehend it. This scenario-based approach should not be used as performance evidence. It may be true to say that performance of a work task will require the application of knowledge but performance evidence should be gathered when the person performs the task in a real or simulated workplace (not by looking at a cartoon).

Explicit or implicit evidence

Explicit evidence is evidence that clearly has been gathered, leaving nothing implied. Gathering explicit evidence will leaves no room for debate about whether knowledge evidence has or hasn’t been gathered. However, if an RTO says that evidence of knowledge is implied when a person performs a work task, then there will be no record of knowledge evidence being gathered.

It is best to always develop an assessment task designed to explicitly gather all items of knowledge evidence. As a minimum, one question for each knowledge evidence item. However, sometimes one question is insufficient to gather the breath and depth of knowledge that is required. Ask as many questions as necessary to gather sufficient evidence.

In conclusion

This article is a follow-on from a previous article titled:

How to develop an assessment instrument for gathering knowledge evidence

The previous article was based around a 5-step process. It was a simplified process that may not have communicated some of the complexity of developing an assessment instrument for gathering knowledge evidence.

Questioning is the common assessment method for gathering evidence for the required knowledge (listed as Knowledge Evidence). It can take time to:

• Analyse the required Knowledge Evidence items
• Research sample answers for each item of knowledge
• Develop appropriate questions and sample answers.

As a reminder, the assessment instrument should be reviewed and trialled before being finalised and implemented.

Do you need help with your TAE studies?

Do you need help with your TAE studies?

Are you a doing the TAE40122 Certificate IV in Training and Assessment, and are you struggling with your studies? Do you want help with your TAE studies?

Ring Alan Maguire on 0493 065 396 to discuss.

Contact now!

Training trainers since 1986