Solution: Performance assessment in the age of ChatGPT
- Expand project-oriented teaching and extend practice-relevant tasks in order to promote student motivation and knowledge transfer.
- Give preference to "unique" and singular questions for written assignments and avoid assignments based solely on literature research and analysis.
- Increase the proportion of oral examinations at the expense of written assignments, despite concerns about having to increase examiner capacity.
- Make the responsible use of ChatGPT by learners a subject of teaching.
What does this mean for vocational school lessons?
In Germany, the teaching method in vocational school lessons is activity-oriented by law. This means that job-related activities are simulated in so-called learning situations. The students must carry out these activities as independently as possible, from planning to execution, including consideration of what could be improved. Depending on the complexity or the objective in terms of personal or social skills, this can be done individually or in groups:
Holistic circle of action orientation
The teacher (black person in the graphic above) is the creator and manager of this learning situation and assists the students in completing a real action product. At the end, grades are also determined together with the students.
Of course, an AI tool such as ChatGPT may be used, because it is helpful and available in the real professional situation and therefore the handling of it should also be learned.
What role could a tool like ChatGPT play here?
Even when the teacher is developing the learning situation, an AI-supported chatbot can certainly at least help them to come up with ideas for creating it.
Students also use the chatbot for research during the information, planning and decision phases, of course. Teachers have the opportunity to help their students learn how to use the machine correctly and responsibly with a real requirement. For example, the answers can be checked together and the dialog with the machine can be practiced and improved.
In the execution phase, previously planned action products are made by the students. Depending on the training occupation and learning situation, these action products can be, for example: configurations, installations, programs, plans, presentations, manuals, documentations or anything that could also be created in a real occupation. Here, an AI chatbot may be able to provide assistance, especially when problems arise and iteratively jump back one or two process steps. The arise of problems always provides a lot of learning opportunities. In
addition to expanding professional expertise, the quality of the
chatbot's answers could now be questioned again or compared with the
conventional research method using Bing, Google, etc.
If the documentation is the only action product, the problem or learning situation must be designed so individually that a chatbot can only be used to a limited extent. (A project diary, for example, could also be a product that represents the necessary individualization.) But when it comes to using a chatbot (as well as other sources), this is an opportunity to practice the correct citation and referencing style. In this case, give your students the opportunity to improve the deficiencies you have pointed out. Then their ability to act in this regard can be improved most effectively.
For the subsequent examination phase, it would of course be better if additional action products had to be created, such as an oral presentation followed by a technical discussion, through which the acquired specialist knowledge can actually be demonstrated.
In the final evaluation phase, the students look at the process itself together with the teacher and assess its quality for themselves. This critical self-reflection can also be used to determine grades.
In dialogue with the teacher, the benefits or pitfalls of the AI chatbot tool used can be discussed again, if that played a role.
Dr Wolfgang König, an expert in digital vocational training at the Bildungswerk der Wirtschaft, has created the following table based on corresponding ChatGPT self-assessment questions:
| Phase | Reliability of ChatGPT's answers (1 not good - 5 very good) |
Validation effort by aprentices (1 not high - 5 high) |
Necessary support for apprentices by trainers/teachers when learning with ChatGPT (1 not complex - 5 complex) |
|---|---|---|---|
| Information |
4 | 2 | 2 - Minimal support to ensure that apprentices find and understand the right information. |
| Planning |
4 | 4 | 3 - It may be necessary to help apprentices identify their individual needs and goals and create a suitable planning concept. |
| Decision | 4 | 3 | 4 - Apprentices may need support to make decisions and consider possible risks. |
| Execution | 4 | 4 | 5 - Apprentices may need support to ensure they are following the correct steps. |
| Examination |
4 | 4 | 4 - Apprentices may need support to ensure they are using the correct examination methods. |
| Evaluation | 4 | 3 | 3 - Support in analysing data and identifying opportunities for improvement. |
Therefore, when planning the learning situation, realise that the need for support can be high, especially in terms of time.
Reference list:
König, W. Dr. (2023, Jun 2023). Modell der vollständigen Handlung mit ChatGPT umsetzen?! Serie zu KI: Teil 4. Netzerk Q 4.0. Institut der deutschen Wirtschaft Köln e.V. https://netzwerkq40.de/blog/modell-der-vollstaendigen-handlung-mit-chatgpt-umsetzen/ (Accessed 6 May 2024)