This article was written in March 2022.
I frequently see assessment instruments being used by TAFEs and private RTOs that do not comply with the requirements specified by the Standards for RTO. The national VET regulator, ASQA, was established in 2011. It is astonishing that after more than a decade, ASQA has been unable to encourage or enforce compliance. And what makes it even more astonishing is a move towards self-regulation. This will allow the many RTOs that are not currently compliant, to self-regulate their future compliance (or should I say, self-regulate their non-compliance).
Recently, I assisted a TAFE employee to develop assessment instruments for the CPCCCA3010 Install windows and doors unit of competency. Consequently, I was prompted to write this article as a case study to explore:
- How to determine the number of assessment tasks
- How to develop an assessment instrument to gather performance evidence
- How to develop an assessment instrument to gather knowledge evidence.
How to determine the number of assessment tasks
I always start with determining how many assessment tasks will be needed to cover the assessment requirements. This must be done before designing and developing the assessment instruments.
As a general rule, there should be a plan for at least two assessment tasks:
- An assessment task to gather the knowledge evidence
- At least one other assessment task to gather the performance evidence.
It is common to need more than one assessment task to gather the frequency or volume of performance evidence that is specified by the Assessment Requirements.
Let’s look at how many assessment tasks are required to gather the performance evidence for the CPCCCA3010 Install windows and doors unit of competency. The Performance Evidence for this unit states:
A candidate can not do all these tasks at the same time. Therefore, I would plan for four (4) assessment tasks to gather the specified performance evidence.
I usually assign ‘Assessment Task 1’ for the gathering of knowledge evidence. Therefore, I would have a total of five (5) assessment tasks for the CPCCCA3010 Install windows and doors unit of competency.
The CPCCCA3010 Install windows and doors unit of competency has six (6) elements of competency. The first and last elements are common for installing all windows and doors. The other elements of competency are only relevant to specific tasks determined by the type of door or window being installed. The following matrix shows the connection between each element of competency and each assessment tasks.
The design and development of an assessment tool is a creative process. Sometimes, we may need to change of initial assessment plan in regard to the number of assessment tasks.
How to develop an assessment instruments to gather performance evidence
So far, we have determined the need to have four (4) assessment tasks to gather the performance evidence for the CPCCCA3010 Install windows and doors unit of competency. Each of these assessment tasks may follow a 3-part process:
- Part 1. Plan and prepare
- Part 2. Install
- Part 3. Clean up
An observation checklist may be used to gather the performance evidence. I would consider the need to develop an observation checklist for each assessment task. Therefore, there will be four (4) observation checklists developed. However, some observation items will be the same for each checklist.
For example, the same observation items can be used for:
- Plan and prepare (as specified by Element 1 of the CPCCCA3010 unit)
- Clean up (as specified by Element 6 of the CPCCCA3010 unit).
But some observation items will be different because the installation of different types of doors and windows are different, and there are different Performance Criteria specified. For example, the observation checklist for Assessment Task 2 would include the Performance Criteria for Element 2, but not the Performance Criteria for Elements 3, 4 and 5.
The following matrix highlights Assessment Task 2. It shows what elements of competency are the same or different to the other assessment tasks used to gather performance evidence.
The next step will be to develop the assessment instruments.
For the purpose of this example, a third-party report is not being used to gather evidence. It is common for building and construction apprenticeships to include the CPCCCA3010 Install windows and doors unit of competency. Therefore, the apprentice would perform work under the supervision of a builder or building site supervisor. These people would ideally contribute to assessment by providing third-party evidence.
This example is only showing assessment tasks conducted by an assessor to gather performance evidence.
Developing the observation checklist
The items of an observation checklist should not be a ‘copy and paste’ of the performance criteria. Performance criteria can be vague or ambiguous. And performance criteria can lack sufficient details to clearly communicate the benchmark or expected standard of performance.
The following table show how the Performance Criteria could be analysed. It only uses the first four performance criteria of Element 1 as an example.
The technique is basically about you asking yourself questions. These questions aim to seek clarity. And these question help to interpret and contextualise.
The next thing is to draft the observation items for the checklist. The following table shows the development of observation items for the first four (4) performance criteria of Element 1.
The above table also shows how additional evidence could, or should, be gathered. These would be in addition to the observation checklist.
Many RTOs will have their own templates to be used for assessment instruments. The following an example of an observation checklist for the first four (4) performance criteria of Element 1
Sometimes, the sequence of items on the observation checklist can be different than the numerical order of performance criteria. Sometimes one performance criteria may need more than one observation items, and sometimes several performance criteria may be adequately covered by one observation item.
We need to be flexible in our approach to the development of an observation checklist. Each unit of competency is different. Therefore, the observation checklist must be created to meet the unique requirements specified by each unit of competency.
Developing a product review checklist
An observation of a person performing work tasks will often be used to gather direct evidence of performance. Sometimes, the review of a product that has been produced during the performance of the work task will be used as evidence. A product review checklist would need to be developed for an assessor to use during the assessment process.
The product to be reviewed may be:
- A document
- A completed form
- A physical item
- A non-physical item.
How to develop an assessment instruments to gather knowledge evidence
I recommend the development of an assessment task to gather all the specified Knowledge Evidence. Create a table to ensure you write at least one question to gather the knowledge evidence specified.
The following is an example for a table created for the CPCCCA3010 Install windows and doors unit of competency.
The next step is to start writing questions. Sometimes, one question will not be enough to cover the breath or depth of knowledge required. Therefore, a set of questions may need to be asked. The following are two examples.
Example 1: This is an example for the Knowledge Evidence item 5.
Example 2: This is an example for the Knowledge Evidence item 6.
Numbering the questions
Consider how the questions are going to be numbered.
Let’s say, there are four questions to be asked to gather evidence for one Knowledge Evidence item. The following are three different ways that the questions could be numbered:
- Q6a, Q6b, Q6c, Q6d.
- Q6.1, Q6.2, Q6.3, Q6.4
- Q6, Q7, Q8, Q9
Sequencing the questions
Consider the sequence the questions. Sometimes, the questions may be re-sequenced to present them in a logical order or asking questions that relate to a similar topic. For example, asking all questions relating to safety one after the other rather than scattered.
Structuring the questions
Consider giving structure to the questions. This can be done by using ‘topic headings’ with all questions relating to the topic under that heading. This will make a large number of questions easier to navigate.
Attributes needed to design and develop assessment instruments
The following attributes are required by people wanting to design and develop assessment tools:
- Good word processing skills
- Analytical skills and the willingness to spend time conducting the analyst
- Subject matter expertise.
Good word processing skills
Many RTOs have templates to be used. Unfortunately, many of these templates have been developed by word processing wizards, making them difficult to use or modify for people with limited word processing skills.
Analytical skills and the willingness to spend time conducting the analyst
Earlier this week, one of my TAE Students said,
“We have been working on planning, designing and developing an assessment tool for a day and half. And we still are not yet finished. It takes a long time to create the assessment documents just to assess if some can or cannot use a hand-held radio.”
The time required to conduct an assessment to determine if a person is competent at using a hand-held radio is typically going to be less than an hour. It is a relatively low level skill. But it takes a long time to develop a compliant assessment tool. People need time to analyse the unit of competency and associated assessment requirements. And then more time to draft assessment documents, review and trial assessment documents, and finalise assessment documents prior to implementation.
People need to know how to conduct the analysis and be willing to spend their time doing it. It can take a lot of time to design and develop compliant assessment instruments.
Subject matter expertise
The design and development of assessment instruments require subject matter expertise. It the developer is not a subject matter expert, then they will need to work with one.
In conclusion
Gathering valid and sufficient evidence of competency is a fundamental part of the assessment process. It takes time and effort to design and develop the assessment instruments to gather the required knowledge evidence and required performance evidence. Two essential sources of information are:
ASQA’s Guide to assessment tools
ASQA’s Conduct effective assessment
Other articles
And you may be interested in other related articles that I have written:
Guide to developing assessment tools
How to develop an assessment tool to gather knowledge evidence
Knowledge evidence and the Australian VET system
TAE Tutoring service
Some people struggle with various topics and terminologies used during their TAE40116 studies. One area that is often found difficult is the planning, designing and developing assessment tools and assessment instruments.
I provide a TAE Tutoring service that has been designed to help you with your studies.
Do you want more information? Ring Alan Maguire on 0493 065 396 to discuss.
Training trainers since 1986
On Target Work Skills 
Sadly the standards of competency assessment have been lowered to enable a quick tick and flick approach in some areas, especially those whose tasks are embedded in the development of complex, evolving relationships. You have clearly and expertly demonstrated the challenges involved in task analysis and the design of assessment tools; please consider providing us with another article on the analysis of the complex interpersonal skills involved in the assessment of person-centred or person-led services like aged-care, child care and career-advice at certificate level.
LikeLike
Thank you, Judith, for your comments. Each article takes me time to create and I am ‘unpaid content creator’. I will consider creating another article covering interpersonal skills but there may be a delay due to other current work commitments. I thank you for your good suggestion.
LikeLike