Measuring and fostering diagnostic skills in biology teaching degree programs
Using the SCRBio classroom simulation with and without chatbot support
Daniela Fiedler & Ute Harms
Teachers often neglect the subject-specific diagnosis of students' performance in the classroom despite its importance for successful learning processes. One reason may be the teacher's diagnostic skills. Little is known about the extent to which student teachers can correctly diagnose students' ideas, explanations or arguments based on their content and pedagogical content knowledge. The classroom simulation SCRBio (short for Simulated Classroom Biology) was developed and supplemented by a chatbot - as a prototype of an adaptive feedback provider to support student teachers - to measure and, above all, foster diagnostic skills in biology teacher training courses.
Digital learning environments have increasingly become the focus of educational research and university teaching, as they can create quasi-authentic learning situations in which the complexity of the real teaching situation is reduced and concentrated on a specific aspect of the teaching process, such as the diagnosis of student statements on a specific subject matter. Teachers must be able to quickly and accurately assess the performance of their students (e.g. their explanations or arguments on certain issues) in the classroom to support the further learning process, for example by providing direct feedback. However, in real lessons, teachers are usually so preoccupied with implementing their lesson plans and classroom management that the diagnosis and assessment of the students' disciplinary performance is often neglected.
Although teachers develop declarative, subject-specific and pedagogical-psychological knowledge - i.e. the foundations for their later professional work in the classroom - during their teacher training, this often remains "tacit". In other words, they do not apply their subject-specific knowledge in practical situations. This can lead to inadequate diagnoses of student performance and consequently to inadequate feedback and support in the classroom.
The SCRBio was developed to measure and, in particular, foster diagnostic skills during the biology teacher training course. In the SCRBio, biology student teachers take on the role of a teacher to diagnose the performance of virtual pupils (shown as portrait photos) in a simulated classroom environment during a teaching sequence. The digital teaching sequence follows a question-and-answer scheme in which questions from a predefined pool of questions can be addressed to the virtual class. The virtual students' answers are assigned to a specific level. The levels for the respective topics in the SCRBio were defined based in accordance with published competence models. The student teacher should diagnose each virtual student's answer during the teaching sequence (e.g., a misconception about a question on evolutionary biology) by assigning the answer to one of the predefined levels. After the teaching sequence, each student should in turn be assigned to the level that was shown most frequently in their answers throughout the entire teaching sequence. The degree of agreement between the assessments of the virtual class by the student teachers and the preset performance actually shown (measured at the respective level) provides a measure of the student teacher's diagnostic skills.
The first version of the SCRBio addressing the topic of evolution was developed in the project "ProSim: Developing procedural professional knowledge in the simulated classroom" (funded by the BMBF - Federal Ministry of Education and Research, in the years 2017-2020). In the subsequent project "FiSK: Effects of adaptive feedback bots in the simulated classroom on procedural professional knowledge" (also funded by the BMBF, in the years 2021-2024), the classroom simulation was expanded to include the topics of experimentation and argumentation and supplemented by a chatbot as a prototype for an adaptive feedback provider.
The diagnostic process of the student teachers (the users) in the SCRBio is supported during the teaching sequence by an integrated chatbot, which performs two functions in the digital learning environment: On the one hand, the user can actively ask questions to the chatbot (e.g. "What is a teleological misconception?") and receive a standardized answer to predefined terms or queries - similar to a digital dictionary. On the other hand, the chatbot reacts proactively to certain behavioral patterns of the user (e.g. selecting the same virtual student three times) by entering certain prompts or information (e.g. taking different students once for comparison).
A report module is available to the user after the teaching sequence and diagnosis of the virtual class, which again summarizes the completed teaching sequence. This feedback allows the user to check for themselves whether their diagnoses were correct.
Our initial studies with the SCRBio with chatbot show that biology students diagnose between 25% and 40% of the virtual students' answers correctly, i.e. they recognize all the technical elements or aspects stored in the respective answer. However, significant differences exist in the correct assessment of a student's level for the various topics of the SCRBio. For example, for the topic of evolution, the student teachers assigned an average of 67% of the virtual class to the correct level, while for experimentation the level was 47% and for argumentation only 24%. Although the focus on the diagnosis of individual answers represents a limitation compared to a complex classroom discussion, the SCRBio allows applying the declarative knowledge acquired during the course - e.g. common misconceptions about evolution - in a specific simulated action situation and thus to practice diagnosing relevant topics in biology lessons. As part of a recently launched project within the DigiProMIN project network (in the competence network learnen:digital, which is funded by the European Union - NextGenerationEU and the Federal Ministry of Education and Research), the SCRBio is presently also being integrated as a digital learning environment in training courses for second and third-phase biology teachers. The aim is to support their diagnostic skills in the competence areas defined for science education in the German educational standards.
About the authors:
Dr. Daniela Fiedler was until recently a research scientist in the Department of Biology Education at the IPN. She is currently a tenure-track assistant professor at the Institute of Science Education (IND) at the University of Copenhagen. She conducts research on the teaching and learning of evolution with a focus on competence development.
Prof. Dr. Ute Harms is a director at the IPN and Professor of Biology Education at Kiel University with teaching and learning of evolution across the lifespan as one of her main areas of research.
