As part of the TAE Training Package review, a new e-assessment unit has been drafted. The title of of the proposed unit is TAEASS404 Conduct e-assessment of competence.

PwC’s Skills for Australia is currently seeking our feedback about this draft. The following is my feedback about the unit, and I would be happy to hear your comments before I submit it.

(You may also like to read an article that I have previously posted about another proposed unit with the title of TAEDEL405 Plan, organise and facilitate e-learning.)

A definition of e-assessment

E-assessment may mean different things to different people. I have categorised two different types of e-assessment:

• Assessment conducted by technology
• Assessment supported by technology

Assessment conducted by technology

Assessment conducted by technology may also be known as computer-based assessment (CBA) or e-marking. This type of e-assessment is assessment which is both delivered and marked by computer. And the computer may provide feedback automatically to students without the involvement of an assessor (or any other person).

Benefits of conducting assessment using technology

Benefits of using technology to conduct assessments include:

• consistency of assessments decisions
• supports the record keeping of evidence and results
• speed of getting the assessment done and providing feedback
• reduces the workload of assessors.

Limitations of conducting assessment using technology

Assessment conducted by technology can be used for assessing knowledge, but it is not an appropriate method for assessing performance, such as:

• bake a cake
• repair a lawn mower
• care for an elderly person
• unblock a toilet
• shear sheep.

The Australian VET system requires evidence of a person’s ability to perform work tasks and activities. Therefore, this type of e-assessment should only be used to assess knowledge, and in some cases, the breath and depth of knowledge required cannot be covered by this application of technology.

This method of computer-based assessment requires the design and development of valid test questions with answers (and sometimes responses to be sent as automatic feedback). Not every trainer will have the ability to develop effective test questions. Usually, the writing of test questions would be the responsibility of an instructional designer or resources developer.

What skills are needed to develop computer-based assessment?

An instructional designer or resources developer need the following skills to develop computer-based assessment:

• Write questions
• Write answers
• Write feedback
• Upload to computer-based assessment instrument.

The value of computer-based assessment is that there is no involvement of an assessor during the assessment process. The computer-based assessment instrument performs the assessment and records the result.

Some trainers and assessors may be involved in the development of questions, answers and feedback for computer-based assessment. However, this would be outside the scope of normal activities for most trainers and assessors.

Assessment supported by technology

It has become common for RTOs to use technology to support the assessment process. Many RTOs have implemented a learning management system (LMS), such as Moodle or Canvas. It is more than 5 years since I worked for an RTO that wasn’t using an LMS for students to submit their assessments.

What does the ‘e’ in e-assessment stand for? Does is stand for ‘electronic’ or ‘everyday’? Assessment supported by technology has become the norm. The distinction between ‘e-assessment’ and ‘assessment’ has dissolved over the past 5 years. It would be rare for the assessment process not to supported by technology.

The Education IRC believes that ‘e-assessment’ is different to ‘assessment’ . How did the Education IRC identify a need to create a ‘new’ e-assessment unit of competency? Most RTOs use technology for students to submit their evidence and support the assessment process conducted by assessors. A very small number of exceptions to this may include:

• VET in School programs that use print-based workbooks
• Creative arts qualifications, such as, music or dance
• Small RTO that have a paper-based system.

Is it difficult to learn how to use an LMS?

No, it is not difficult to learn how to use an LMS. So far, I have had to learn three different LMS platforms. The first LMS I learnt was Moodle. I attended a 3-hour session about how to use it to perform my role as an assessor. This was enough to get me using Moodle to conduct assessments. Later, I started to work for an RTO that used Canvas. About 30 minutes was taken during my induction to show me how to log-on, navigate, and use this LMS. And later again, another RTO emailed me a 5-page document to show how to log-on, navigate, and use Job Ready for conducting assessments.

Learning an LMS is not difficult. The knowledge and skills required to use an LMS are insufficient and insignificant for the creation of a unit of competency. It is possible that using an LMS or other technologies could be identified as a foundation skill for the TAEASS402 Assess competence unit of competency. However, learning one LMS does not negate the need for a newly qualified assessor to need to learn a different LMS once they get employment.

What skills are need to conduct assessments that are supported by technology?

The skills required by an assessor to conduct assessment that are support by technology include:

• Prepare for assessment
• Brief the candidate
• Gather evidence
• Support the candidate
• Make the assessment decision
• Record and report the assessment decision

These skills are the same as what is covered by the current TAEASS402 Assess competence unit of competency.

TAEASS404 Conduct e-assessment of competence (draft)

Feedback about elements and performance criteria

The following are the performance criteria for the first element.

Answers to questions raised

The Training Package Developer has raised some questions. The following are my answers to the three questions raised.

The following are the performance criteria for elements 2, 3 and 4.

There is very little difference between the draft TAEASS404 Conduct e-assessment of competence unit of competency and the current TAEASS402 Assess competence unit of competency.

What happens if we remove duplicate performance criteria from the TAEASS404 Conduct e-assessment of competence unit of competency? After removing all the performance criteria that duplicates what is already in the TAEASS402 Assess competence unit of competency, we are left with five performance criteria. The following table lists the five performance criteria.

Contrary to the Case for Change, there is very little difference between conducting ‘traditional assessment’ (as described by the TAEASS402 unit) and conducting ‘e-assessment’ (as described by the TAEASS404 unit). The TAEASS404 Conduct e-assessment of competence unit of competency is basically an attempt to contextualise the TAEASS402 Assess competence unit of competency. The minor differences does not warrant the development and implementation of new e-assessment unit of competency.

Feedback about foundation skills

A candidate will be required to demonstrate the ability to complete the tasks outlined in the elements, performance criteria and foundation skills (as per the Performance Evidence statement). This makes the foundation skills assessable.

The following are the foundation skills.

Analysis and comprehensive feedback about the foundation skills would require much more time and effort. However, some of the foundation skills seem to poorly describe or do not describe a skill. For example:

Feedback about performance evidence

The following are the performance evidence.

The performance evidence is poorly written. I am an experienced assessor and for many years I have been conducting e-assessment. I find the performance evidence is complicated, confusing, and difficult to comprehend. For example:

• How is an e-assessment conduct for a group of students? When does this occur?
• What is the size of the group of students? How is an individual’s competence assessed in a group?
• What is the definition for an assessment task? Assessing one assessment task is far short of assessing competence.
• Will sending an email be appropriate for communicating to candidates electronically?
• What does ‘complete feedback for students in relation to their e-assessment completion’ mean?

Why is there a distinction being made between an ‘e-assessment tool’ and an ‘assessment tool’? Why are we using or alternating between the words ‘student’ and ‘candidate? The performance evidence is poorly written and needs to be significantly reworked.

Feedback about knowledge evidence

Analysis and comprehensive feedback about the knowledge evidence would require much more time and effort. However, I believe that a critical analysis of the knowledge evidence should be done prior to implementation.

Feedback about assessment conditions

The following are the assessment conditions.

Why is the term ‘digital assessment tool’ being used? And why is the term ‘e-portfolio being used?

I think the entire Assessment Requirements need to be thoroughly reworked. But I really think that the draft TAEASS404 Conduct e-assessment of competence unit of competency should be scrapped.

In conclusion

Is there a need for a new e-assessment unit of competency? Can the current TAEASS402 Assess competence unit of competency adequately cover the requirements of conducting e-assessment?

Most assessors are using an LMS or other technology when they conduct assessments. Using an LMS or other technology has become fundamental or foundational to conducting assessments. The distinction between conducting ‘e-assessment’ or conducting ‘non-e-assessment’ is blurred. And the knowledge and skills required by assessors to conduct ‘e-assessment’ or ‘non-e-assessment’ are the same.

The current TAEASS402 Assess competence unit of competency adequately covers the requirement of conducting assessment that are supported by technology (e-assessment). At best, the draft TAEASS404 Conduct e-assessment of competence unit of competency is an attempt to contextualise the TAEASS402 Assess competence unit of competency. The minor differences does not warrant the development and implementation of new e-assessment unit of competency.

We do not need a new e-assessment unit of competency.

Please let me know what you think.

Introduction to assessment validation

Validation is defined as the quality review of the assessment process. It involves checking that the assessment tool produces valid, reliable, sufficient, current and authentic evidence to enable reasonable judgements to be made as to whether the requirements of the training package or VET accredited courses are met. It includes reviewing a statistically valid sample of the assessments and making recommendations for future improvements to the assessment tool, process and/or outcomes and acting upon such recommendations. [1]

Assessment validation has two distinct parts:

• Part 1. Check the assessment tool for compliance
• Part 2. Review a sample of the assessments.

This article covers the first part only.

If you want to know more about the second part, then I recommend reading the information published by ASQA about how to conduct assessment validation. This information covers: [2]

• Who conducts validation?
• Scheduling validation
• Statistically valid sampling and randomly selecting samples to be validated
• Effective validation
  • Reviewing assessment practice
  • Reviewing assessment judgements
• Validation outcomes and the implementation of recommendations for improvement.

Part 1. Check the assessment tool for compliance

The assessment tool must be checked to ensure it complies with the requirements specified by the Standards for RTOs, in particular: [3]

• Compliance with the principles of assessment and the rules of evidence
• Compliance with the requirements specified by the training package or VET accredited course.

The following 6-step process can be used to check the assessment tool for compliance:

• Step 1. Read the assessment requirements
• Step 2. Review the assessment plan
• Step 3. Review the assessment matrix (mapping)
• Step 4. Check the details about how the knowledge evidence is planned to be being gathered
• Step 5. Check the details about how the performance evidence is planned to be being gathered
• Step 6. Check the overall quality of the assessment tool.

Step 1. Read the assessment requirements

This is a quick step to perform. You will read and re-read the unit of competency and its assessment requirements many times during the assessment validation process. During this first step, have a quick read of the assessment requirements and answer the following questions:

1. What is the volume or frequency of performance evidence?
2. Is the location, facilities, equipment, or other assessment conditions specified?

Step 2. Review the assessment plan

This step should also be quick. The purpose of this step is to get an overview of what is the planned assessment approach During this second step, answer the following questions:

1. Has the correct unit code and title been used?
2. How many assessment tasks are planned?
3. Is there a plan to gather the knowledge evidence?
4. Does there appear to be sufficient assessment tasks for gathering the volume or frequency of performance evidence?
5. Does the planned assessment approach seem to be simple or complex?

Note: This planned assessment approach may be found in the Training and Assessment Strategy (TAS) or other documents covering how the RTO plans to implement the delivery of the training and assessment for a unit or cluster of units.

Step 3. Review the assessment matrix (mapping)

This step should be a relatively quick step. The assessment matrix is an important document used to display how the RTO plans to gather evidence that comply with the requirements specified by the training package or VET accredited course. The assessment matrix will be used during Step 4 and Step 5 to cross-check the RTO’s planned assessment approach and the assessment instruments being used to gather evidence.

During this third step, answer the following questions:

1. Has the correct unit code and title been used?
2. Has the entire unit of competency and its assessment requirements been copied into the matrix? Are the number of items the same? For example, if the unit has five elements does the matrix have five elements? And scan the wording to ensure the matrix has the exact words as the unit of competency and its assessment requirements.
3. Is there one column for each planned assessment task?
4. Are the titles or descriptions of the assessment tasks the same in the assessment plan and assessment matrix?
5. Is every item from the unit of competency and its assessment requirements planned to be assessed? For example, is there at least one ‘tick’ in every row?

Note: Some assessment matrices will provide information or numerical indicator about the assessment item instead of using a ‘tick’. For example, the matrix may indicate that a piece of knowledge evidence will be gather by Question 1.

Step 4. Check the details about how the knowledge evidence is planned to be being gathered

This step requires an attention to details. The purpose is to ensure that the assessment tool will gather the required knowledge evidence. During this fourth step, answer the following questions:

1. Is there an assessment instrument for gathering the knowledge evidence?
2. Are the instructions to the assessor clear and concise?
3. Are the instructions to the candidate clear and concise?
4. Is the structure, format, and layout of the assessment instrument easy to follow? This includes headings, sub-headings, page numbers, and numbering of questions.
5. Is there consistency between the assessment plan, assessment matrix and assessment instrument? For example, if the assessment plan states that there are 17 questions, does the assessment instrument have 17 questions?
6. Is every item of knowledge evidence being adequately gathered? A judgement about ‘adequately’ will need to be made.

Step 5. Check the details about how the performance evidence is planned to be being gathered

This step requires an attention to details and it can take time to examine the assessment documents for compliance. The purpose is to ensure that the assessment tool will gather the required performance evidence. During this fifth step, answer the following questions:

1. Is there one or more assessment instruments for gathering the performance evidence?
2. Are the assessment conditions compliant with those stated in the Assessment Requirements for the unit of competency? This may include assessment location, facilities, equipment, and access to specified documents. For example, if the assessment conditions state that the assessment occurs in the workplace, then the assessment tasks must state that the evidence must be gathered from a workplace (not from a simulated workplace).
3. Are the instructions to the assessor clear and concise?
4. Are the instructions to the candidate clear and concise?
5. Are the items of performance evidence clearly listed or identified?
6. Is the structure, format, and layout of the assessment instrument or instruments easy to follow? This includes headings, sub-headings, and page numbers.
7. Is there consistency between the assessment plan, assessment matrix and assessment instrument? For example, if the assessment matrix states that evidence for Performance Criteria 1.1 will be gather during Assessment Task 2, then Assessment Task 2 must cover the gathering of evidence for Performance Criteria 1.1.
8. Is every item of performance evidence being adequately gathered? A judgement about ‘adequately’ will need to be made. This includes a check that the amount of evidence being gathered is compliant with the specified volume or frequency of performance evidence.

Note: Verbs are important. For example, if performance criteria says, ‘negotiate and agree with a supervisor’, then there needs to be evidence that the candidate has negotiated and agreed with a supervisor’. Also, the letter ‘s’ is important. A item of performance evidence may specify plural rather than singular. For example, if it states ‘write reports’, then more than one written report is required for evidence.

Step 6. Check the overall quality of the assessment tool

This step can take time to examine the assessment tool for compliance, readability, and usability.

1. Are there sample answers and assessment decision criteria for assessors?
2. Is the structure, format, and layout of all assessment documents easy to follow?
3. Are all instructions written clearly and concisely?
4. Are there any grammar, spelling and typo errors?
5. Is there a list of all the assessment documents required for the assessor?
6. Does the assessment tool have all the documents required for the assessor?
7. Is there a list of all the assessment documents required for the candidate?
8. Does the assessment tool have all the documents required for the candidate?
9. Has the correct unit code and title been used throughout all the assessment documents? This may include release number.
10. Do all the assessment documents have version control information?

In conclusion

Assessment validation has two distinct parts:

• Part 1. Check the assessment tool for compliance
• Part 2. Review a sample of the assessments

Assessment validation can be time-consuming and mind-bending.

Preparation before an assessment validation meeting can reduce the time at the assessment validation meeting. However, you can expect a typical assessment validation meeting to require anywhere between a few hours and an entire day. The duration of the assessment validation meeting can depend on the quality of the assessment tool and number of assessment samples to be reviewed. I regularly see poor quality assessment tools, and it takes time to properly check large numbers of assessment samples.

Clear and critical thinking is required by people participating in an assessment validation meeting. There are usually many documents to be reviewed and checked. Printing paper copies of documents (or some documents) and using ‘split screens’ on computers will help when comparing information from two or more documents, such as:

• unit of competency
• assessment requirements
• assessment plan
• assessment matrix
• assessment instructions
• assessment instruments.

Frustration and fatigue can be experienced during long assessment validation meetings. Breaks will be needed (and sometimes chocolate helps). It is a good idea to assign an experienced VET practitioner to lead the assessment validation meeting.

References

[1] https://www.asqa.gov.au/standards/appendices/glossary accessed 2 September 2021

[2] https://www.asqa.gov.au/resources/fact-sheets/conducting-validation accessed 2 September 2021

[3] https://www.asqa.gov.au/standards/training-assessment/clauses-1.8-to-1.12 accessed 2 September 2021

Do you need help with your TAE studies?

Are you a doing the TAE40122 Certificate IV in Training and Assessment, and are you struggling with your studies? Do you want help with your TAE studies?

Ring Alan Maguire on 0493 065 396 to discuss.

Contact now!

Training trainers since 1986

I have found that some people studying for their Certificate IV in Training and Assessment qualification struggle with understanding what a task breakdown is, and how to develop a task breakdown. In this article, I aim to answer the following questions:

• What is a task breakdown?
• How can a task breakdown be developed?
• When can a task breakdown be used?

Where do we start

Australia’s VET system is competency-based. A primary objective of VET is to help people learn how to perform work tasks and activities.

The four stages of competence, also known as the ‘conscious competence learning model’, describes the process of progressing from incompetence to competence in performing a task or activity. [1]

Stage 1. Unconscious incompetence

At this ‘unconscious incompetence’ stage, a person is unaware of their inability to perform a particular task or activity.

Stage 2. Conscious incompetence

At the ‘conscious incompetence’ stage, a person becomes aware that they are unable to perform a particular task or activity.

Stage 3. Conscious competence

A person who is aware of their incompetence may decide it is worth their time and effort to learn how to perform the particular task or activity. At the ‘conscious competence’ stage, the person can perform the task but they will require concentration and may need to think about each step.

Stage 4. Unconscious competence

In the final stage of ‘unconscious competence’, performing the task becomes second nature because the person has learnt and practiced performing the task so much. They can perform the task easily and without thinking about it. As a result, the task may be performed while executing another task.

An experienced or skilled worker often performs many work tasks and activities without needing to think about how or what to do.

Becoming conscious again

As a trainer, we often have mastered skills and we are no longer conscious about how we perform work tasks or activities. We just do it.

But as a trainer, we must explain and demonstrate to others how to perform work tasks, and to do this we have to again become conscious about the steps required to perform the task. The development of a task breakdown is a method that helps us become conscious again about how to perform a work task or activity.

A task breakdown is a step-by-step description about how to perform a task. It may also be known as a procedure or work instructions.

What is a task breakdown?

A ‘task breakdown’ is a document that breaks down a task. It describes the steps in a logical sequence to perform the particular work task or activity.

I use three analogies to help people understand and appreciate the value of a task breakdown: Lego instructions, Ikea instructions, and a cooking recipe.

The following three examples have three common features:

• Steps are numbered
• Illustrations are used to visually communicate
• The outcome or desired result is shown.

Lego instructions

Ikea instructions

Cooking recipe

How can a task breakdown be developed?

Use the following six steps to develop a task breakdown.

Step 1. Create 3-column table

Open a new Microsoft Word document and create a 3-column table. For example:

Step 2. Add table headings and adjust column width

The following illustrates the recommended layout and format for the task breakdown. For example:

Note: Layout and format can be modified later, if required.

Step 3. Enter step numbers

Use the first column to enter step numbers. The numbers imply a logical sequence to followed, and repeatedly use the word ‘step’ because this reinforces that each row is a step to be performed. For example:

Note: Table rows can be added or removed later, as required.

Step 4. Write brief description for each step

Perform the task or observe someone performing the task. Write a brief description for each step. For example:

Step 5. Write explanations

Write a clear and concise explanation for each step. Use illustrations, diagrams, and photos to show examples. Do not decorate or use unnecessary visuals. For example:

Important note: Remember to highlight safety requirements for the task or for a particular step.

Step 6. Review and finalise task breakdown

Ask someone to use your draft task breakdown to perform the task. Observe the person performing the task, checking the sequence of steps, and seek feedback about readability.

Check your task breakdown for grammar and spelling. Add task title, version control, and page numbers before finalising the document. For example:

Download the finished product

You can download a PDF version of the finished task breakdown for creating a task breakdown from here.

When can a task breakdown be used?

The task breakdown has multiple uses:

• Used as a planning tool to identify required knowledge, required skills, safety requirements, and resources required to perform a task
• Used to explain the process or performance of the task
• Used to demonstrate the process or performance of the task
• Used by the learner to guide them when they learn or practice performing the task
• Used to check or assess the performance of the task
• Used as a diagnostic tool when a person is struggling with learning or performing the task (used to identify what step the learner is stuck on).

Training session plan

A task breakdown describes a task. A session plan is used to describe the training to be delivered. The task breakdown should not make reference to the training process. It should only describes the work task or activity.

A training session plan can be developed based on the steps described by the task breakdown.

Observation checklist

A task breakdown can be converted into an observation checklist. For example:

A procedure that can be used in the workplace

A task breakdown should be written to guide performance of a work task or activity. It should describe the step-by-step procedure to be followed in a training environment or in a real workplace. The task breakdown should be the procedure to follow, no matter where the task is performed.

In conclusion

Creating a task breakdown is a basic skill that all trainers and TAFE teachers must have. A task breakdown describes the steps in a logical sequence to perform a particular work task or activity.

A task breakdown has a multitude of uses:

• Used by the trainer as a planning tool
• Used by the trainer when explaining and demonstrating the performance of the task
• Used by the learner to guide them when they learn or practice performing the task
• Used by the trainer when checking or assessing the performance of the task
• Used by the trainer as a diagnostic tool.

Reference

[1] https://en.wikipedia.org/wiki/Four_stages_of_competence accessed 2 March 2023

Do you need help with your TAE studies?

Are you a doing the TAE40122 Certificate IV in Training and Assessment, and are you struggling with your studies? Do you want help with your TAE studies?

Ring Alan Maguire on 0493 065 396 to discuss.

Contact now!

Training trainers since 1986