Computer-aided diagnostics of learning processes concerning functional thinking

The rise of digital media and technological progress leads to cultural and social changes, which influence everyday, working, and school life. This impact is recognizable in newly developed methods of teaching and learning as well as in new possibilities and perspectives in social, economic, and scientific fields (Sekretariat der Kultusministerkonferenz [KMK], 2017, 2021).

One of these fields is the field of pedagogical diagnostics, which involves measuring teaching and learning success. Ingenkamp and Lissmann (2008, p. 69) describe a dilemma between the frequency and accuracy of diagnostic measurements. They claim that a high measurement frequency is necessary if insights into learning processes are to be obtained. As a result, only short questions, tasks, or observations can be used for diagnosis. Otherwise, it is possible that the diagnosis is less accurate due to frequent interruptions of the students in their learning process. In addition, the previously mentioned measurement methods are not as accurate as well-established tests or questionnaires. It seems that finding a balance between frequency and accuracy is necessary in order to diagnose all learning processes, e.g., in the context of functional thinking in mathematics (cf. Greefrath et al., 2016, p. 69).

But what if the diagnoses took place without the students noticing? Then the frequency of these diagnoses could be as high as possible to have a detailed look at the learning processes. Junco and Clem (2015, p. 54) describe learning analytics as “an application of big data and predictive analytics in educational settings” to measure, collect, and analyze learners’ data. Digital technologies measure and collect process data without involving direct input from students. The data are gathered in background processes. Often the digital tools that are used analyze and process the collected data in real-time. Hence, teachers can use the results to intervene in their students’ learning situations. For real-time analysis, an elaborated expert model is needed. A digital tool then compares the collected data with the expected data within the expert model to make an analysis.

In this research project, we want to evaluate how mathematical software is used to gather action-dependent process data to analyze individual learning processes in functional thinking contexts. With this objective in mind, the following research questions can be established.

What kind of mathematical tasks in the context of functional thinking can be used to collect action-dependent process data?

How are thinking processes expressed in actions with digital mathematics tools?

How must the acquisition of process data be designed so that thought processes can be inferred from given actions?

How to develop an expert model using thought and action processes so that student processing can be diagnosed in real time?

Does the automated action-diagnosis coincide with self-diagnoses?

Does the automated action-diagnosis coincide with expert diagnoses?

What process data acquisition capabilities does a digital mathematics tool need to enable learning process diagnostics?

We want to answer the first research question with a qualitative analysis of verbalized expert answers collected while experts are working on open assignments, for example, drawing a function graph based on a factual task. This will enable us to categorize the experts’ thinking processes. Afterwards, we will sort the categories based on their capability of being observed by an action made in digital mathematics tools. We aim to detect productive tasks and use them as a schema for an expert model, which is needed as a basis for learning process diagnostics. We will check the quality of the automated diagnosis by comparing it with self-diagnoses and expert diagnoses. To generalize the results, we will characterize the interfaces a mathematical software must provide to carry out learning process diagnosis.

References

Greefrath, G., Oldenburg, R., Siller, H.‑S., Ulm, V., & Weigand, H.‑G. (2016). Didaktik der Analysis: Aspekte und Grundvorstellungen zentraler Begriffe. Mathematik Primarstufe und Sekundarstufe I + II. Springer Spektrum.

Ingenkamp, K., & Lissmann, U. (2008). Lehrbuch der Pädagogischen Diagnostik (6. Aufl.). Beltz.

Junco, R., & Clem, C. (2015). Predicting course outcomes with digital textbook usage data. The Internet and Higher Education, 27, 54–63.

Sekretariat der Kultusministerkonferenz (Ed.). (2017). Bildung in der digitalen Welt: Strategie der Kultusministerkonferenz.

Sekretariat der Kultusministerkonferenz (Ed.). (2021). Lehren und Lernen in der digitalen Welt: Die ergänzende Empfehlung zur Strategie "Bildung in der digitalen Welt".