Using feedback from partners to improve the quality of VET
The Ferenc Hansági Vocational School in Szeged, Hungary – which awards qualifications in the catering and tourism sectors – has been monitoring the requirements of internal and external stakeholders – i.e. partners – and measuring the effectiveness of its training since 2000. Feedback from partners on their expectations, needs and satisfaction is collected on a regular basis and the information is used to improve the quality of the school’s professional work and activities.
As part of a nation-wide quality improvement initiative (launched in 2000) in the Hungarian VET system, schools have become more open and have started to develop/implement partner-focused operations. This focuses on developing an organisational culture where schools identify their partners and regularly assess their needs and demands; setting objectives and targets as well as continuously developing and improving their operations based on the analysis of these demands. By following this logic, schools incorporate the cycle of quality assurance and continuous quality improvement – the Plan, Do, Check, Act cycle – into their everyday work.
The partner-focused operation is a common approach to quality assurance in the Hungarian education and training sector, and is seen as one way to support quality improvement. Surveying partners’ needs and satisfaction has been an integral part of all the self-assessment models developed over the years, it is one of the four main components of the EQAVET compatible Common VET Quality Management Framework and part of the tool-kit used by the new re-introduced inspection system. It has a particularly important role in VET as providers are facing increasing demands to react to rapid changes; use new technologies; be more effective; respond to stakeholders’ and users’ needs; learn from others and be visible and competitive in the community. The partners’ needs and satisfaction survey helps VET providers to identify how they can respond to the demands facing them and balance the present and future needs of different stakeholders with the available resources.
How the EQAVET+ indicative descriptors are used
The process of conducting the partners’ needs and satisfaction survey is set out in the school’s Quality Management Programme. This explains the school’s approach to selecting respondents, setting targets, measuring the outcomes, and ensuring the results are valid, reliable and accurate. The procedures describe and regulate the following:
- Which partners are included in the survey?
Views are collected from the key – so called direct – partners identified by the school management team i.e. employees (teachers, trainers, support staff, other staff, the leadership team), students and their parents, the foundation which owns the school and organisations that provide apprenticeship places (owners and/or instructors of restaurants, hotels and patisseries)
- Areas of operation to be surveyed
Views are collected from each partner group about:
- the management/leadership, the management of the partnerships, the management of human resources, the operation of the school and the management of other resources;
- education – teaching – training;
- assessment and evaluation.
- Targets: satisfaction rates to be achieved in each area of operation and partner group
- Methods and instruments to be used: questionnaire-based surveys, on-line or paper-based. Since 2008 the views of students, staff, the school foundation and a sample of employers have been collected through on-line questionnaires targeted to each partner group. Feedback from parents continues to be collected using paper-based questionnaires. The questions cover the quality of leadership, the quality of teaching, the results achieved by students, the school environment and the effectiveness of the relationship with partners. The questionnaires were last revised in 2014 which led to a reduction in the number of questions
- Sampling and selection of respondents (30% of parents and 100% for every other group)
- Frequency of the surveys. The partners’ needs and satisfaction survey is completed yearly by parents, students and employees. For the school foundation and the external practical training sites a survey is completed every two years
The preparatory activities (production of questionnaires, providing information to the teachers and other colleagues) are carried out by the deputy head teacher responsible for the coordination of the quality assurance activities. The process is overseen by the school’s Quality Management Team whose analysis identifies trends, compares the results to the school’s targets and presents the results from each group of respondents. Based on the results, the Quality Management Team identifies strengths and suggests areas for improvement.
As part of the school’s organisational/quality culture (and following the completion of each survey of the partner’s needs and satisfaction) action plans are prepared and implemented. The focus for each plan is based on partners’ needs and their satisfaction with the current training programmes, training provision, school operation etc. Follow-up actions have included improvements in contact arrangements with parents, improvements in internal information flows and communication, improvements in cooperation between the school and the external practical training sites, improvements in individual care for students, improvements in school cleanness etc. Once the changes are implemented, the results are monitored and measured. All the school’s partners are informed of the outcomes from the surveys and the improvements.
The results from the partners’ needs and satisfaction surveys are used in subsequent self-evaluations to assess the efficiency and effectiveness of the internal processes. Some of the results are also used for the performance appraisal of teaching staff.
How is this practice linked to the EQAVET indicators?
This case study is linked to the following EQAVET indicators:
- Number 6: Utilisation of acquired skills at the workplace – this outcome indicator gives information on the occupation obtained by individuals after completing training and the satisfaction rate of individuals and employers with the acquired skills/competences
- Number 9: Mechanisms to identify training needs in the labour market – this context/input indicator gives information on mechanisms for identifying changing demands at different levels and the evidence of their effectiveness (qualitative information)
Each indicator can support policy priorities such as increasing the employability of VET learners, improving the responsiveness of VET to the changing demands in the labour market, improving the quality of training provision and supporting adapted training provision, particularly for disadvantaged groups.They may be used in the planning stage of the quality assurance cycle, and indicator number 6 can be used for assessing results and the effectiveness of VET. Indicator number 6 is based on the number of VET programme completers who, within 12-36 months of completion, find their training is relevant for their current occupation, and the number of employers in a sector who are satisfied with VET programme completers’ qualifications and their competence for the work place.
The largest difficulty to overcome is the reliability or the responses from partners. Initially it was hard to gauge the level of satisfaction from employers as the response rates to the questionnaire did not guarantee a representative sample. This was not the case with other groups such as parents where more communication channels could be used to increase the response rate. For employers additional communication methods, including the use of personal contacts, had to be adopted.
The paper-based questionnaire method is better for those groups of respondents who dislike using IT tools or need more time to consider their answers (e.g. parents). At the same time the production of the traditional questionnaire is costly, requires more people to be involved (for reproduction, distribution, collection and processing) and the assessment and evaluation process takes longer than using an online system.
The compilation of the questionnaires is time consuming and requires significant expertise. There needs to be one or more members of staff in the school, preferably from the teaching staff, with the competence to compile and prepare the questionnaires in a professionally appropriate manner. Time needs to be allocated for individuals to complete the questionnaires.
The frequent and regular use of action plans, based on employers’ and other partners’ satisfaction with provision, has improved the effectiveness of the school. For example, the individual development plans which improve the personal care for students have become an important element of the pedagogic programme. As a tangible impact of the improvements and due to changes (such as appointing a teacher responsible for media relations; the school’s presence on social media like Facebook; more targeted information and website editing; standard appearance and communication) the partners have a more favourable opinion of the school.
In addition using partner satisfaction measurements to inform self-evaluation and self-assessment have ensured the school and its managers stay focused on the needs of employers. To make this quality system work, there needs to be an effective approach to internal communications which is based on the commitment and willingness of all staff.
Knowledge and experience gained when compiling and processing the partners’ needs and satisfaction survey can be used on other occasions in the quality assurance process when there is a need for information collection e.g. when assessing and analysing particular issues, examining changes and taking stock of results.
The shift from the paper-based to the on-line method has resulted in a considerable change for students. Previously they forgot to return the questionnaires or brought them back late. Now all the students in a class can simultaneously allocate scores to the questions and the results can be obtained in a more economical way (in terms of time and human resources).
Our experience and practice confirms that the best time to complete the survey with school employees is at the end of the school year. In the case of the other partners (parents, students and entrepreneurs) April seems to be best as by this time the first year students have gained experience in the school, the graduates are still in the school and for the entrepreneurs their peak business season has not yet started.