This 90-odd page handbook from the European Training Foundation (ETF) explores how to monitor and evaluate work-based learning (WBL) in VET.
It is one in a “series of handbooks designed to provide policy makers, social partners and VET institutions with practice-oriented information and tools.” Some of these have already been published and are available. The handbook is a pretty dense read, so I have tried to distil its essence.
What is work-based learning?
The handbook defines work-based learning as involving knowledge acquisition and competence development in a genuine working environment. In Australia, the clearest VET examples are apprenticeships and traineeships. However, there are also a set of enterprise-based providers, or ERTOs, that are very much concerned with providing formal and other education and training in a workplace context.
The handbook suggests we should not discount simulated environments, though, as they are sometimes necessary to get the breadth of work-related training required efficiently, or because to do things for real can be just too dangerous!! However, for the sake of this handbook these forms of WBL are sort of borderline. Likewise, short courses can often be delivered effectively in the workplace and many employers like things that way, as another article in this issue tells us. Finally, WBL can be a good way for RTOs to get up close and personal with employers and develop constructive partnerships.
Monitoring and evaluating WBL
WBL programs focus around their inputs, processes, outputs and outcomes – which is a pretty standard way of thinking about things when you are evaluating or monitoring any sort of VET program or initiative. Each of these give rise to a set of indicators and tools for monitoring and measuring. ‘Inputs’ cover “all the arrangements preceding actual implementation, as well as the resources that feed into the process.” The ‘process’ factors are those that are directly related to the intervention and the production of the targeted goods or services. In simple terms it’s ‘what goes on to make things happen’!
Looking at outputs and outcomes, “the immediate results in terms of goods or services produced, for example, the knowledge, skills and competence imparted to the learners, constitute the ‘output’. Finally, the medium- and long-term effects achieved through the use of the output are referred to as the ‘outcome.’”
The handbook distinguishes between evaluation and monitoring. Evaluation, they suggest, “focuses on a specific problem and takes place over a limited period of time.” “Monitoring, by contrast, is a permanent scheme of regular and periodic assessments.” This might be seen to be closely tied to regulatory and quality management and improvement processes. Whether you monitor or evaluate, or do both, they play an important part in good institutional governance.
Chapter 3 looks at indicators and tools that can be used in evaluating and monitoring. Most of them are the traditional measures we use to judge the quality of learning, like surveys and statistical data such as completion and dropout rates. Time series and longitudinal studies may also be useful, and in Australia we tend to do more of the former than the latter – where we stay in touch with groups in the longer term. The only significant longitudinal work with a potential VET focus in Australia that comes to mind is the Longitudinal Survey of Australian Youth (LSAY).
Finally, the handbook gives some attention to cost-benefit analyses and return on investment.
Chapters 4 is concerned with inputs (e.g. student background), chapter 5 with implementation (process), 6 with outputs such as learning achievements, and 7 – the benefits.
As the handbook’s VOCEDPlus points out in its entry edited from the ETF’s website:
“For each of these four stages [inputs, process, outputs outcomes], examples of relevant indicators and tools from several countries around the globe are presented. These tools cover the following topics: support from companies and learners, financial resources, costs and benefits, curriculum design, quality of training staff, organisation of the learning process, use of digital technologies, assessment of learners, learning achievements, and labour-market outcomes.”
You may also be interested in practical guides such as this Canadian one which focuses on work integrated learning. Google will find you other resources. VOCEDPlus has an extensive set of practitioner focused resources on teaching, learning and assessment, including a couple specifically concerned with WBL.
