Learning argumentation using feedback from artificial intelligence

Providing feedback on writing tasks such as argumentation is time-consuming and a challenge for teachers. As a result, students often receive insufficient or no feedback. An IPN project is investigating the extent to which feedback from artificial intelligence can provide a remedy and promote students' written argumentation.

Thorben Jansen

Imagine you have an idea that is so brilliant and new that you want to convince others of it. In most cases, this has to be done in writing, given that you want to reach many people. However, writing an essay is a complicated task. First you have to think about a line of argument, write it down and then accept that a well-thought-out argument first becomes a poorly written one. At this delicate point in the writing process, you can either stop and bury the good idea or - as experts do - revise the text until a convincing written argument has been created. True novices in argumentation, such as students, need particular help here, for example in the form of feedback. The challenge for schools and teachers is how to provide everyone with feedback that is conducive to learning when an entire class is writing a text at the same time. Artificial intelligence (AI) could help to overcome this challenge, at least according to the Standing Scientific Commission of the Standing Conference of the Ministers of Education and Cultural Affairs of Germany and UNESCO. We conducted three studies to investigate whether artificial intelligence can be used to multiply qualitative feedback so that all students benefit from it, each focusing on an important success factor for the use of AI in schools. In study 1 it was the teachers, in study 2 the students and in study 3 the AI itself.

The first study was about teachers' opinions on whether AI feedback is already good enough to be used by students to revise their argumentation. We showed 89 student-teachers argumentations from students, including feedback. Half of the feedback came from an AI, the other half from experienced teachers, the subjects were unaware of this. Although the feedback from the teachers was perceived as significantly better, the opinions about the feedback from the AI were still so positive that we concluded that the feedback was good enough to give to the students in class.

In the second study, we asked 459 students to write an essay and then gave them either AI feedback or - which is unfortunately the most common in school - no feedback. Students found it more motivating, enjoyable and useful to receive AI feedback than no feedback. The quality of text revision also improved, albeit not as much. Therefore, in Study 3, we wanted to find out what makes AI feedback efficient.

Our third study compared different types of feedback. We provided 345 students with a textbook chapter to help them with their revision. This textbook chapter was then supplemented with AI feedback. As feedback, we showed the assessment criteria, gave a score for the text or highlighted text passages in the textbook for the students to focus on when revising. The combination of all three types of feedback to individualize the textbook proved to be the most effective.

In summary, these three controlled, experimental studies indicate that AI-generated feedback can support students in revising their texts and thereby gaining argumentative expertise. The next step is to investigate how AI feedback can be implemented in the classroom and what effects it has in an authentic learning context over a longer period of time. Only if AI feedback proves its worth there can it help reduce the need for students to bury their good ideas and allow us to be persuaded by these ideas in the future.

About the author:

Dr. Thorben Jansen is a member of the Department of Educational Research and Educational Psychology at the IPN. Before joining the IPN, he studied psychology at Kiel University. At the IPN, he heads the junior research group Digital Argumentation Instruction for Science (DARIUS), which investigates how written scientific argumentation can be fostered in students with the help of automated formative assessments. The aim of the project, which is funded by the Telekom Foundation, is to develop a digital learning tool to help students learn and practice written scientific argumentation.