Fostering self-correcting reasoning with reflection systems (NSF)

This project is funded by the National Science Foundation (Cyberlearning and Future Learning Technologies, Award 1623419), September 2016 to August 2019.


In the 1980s, a team of 46 experts agreed that “self-examination” and “self-correction” are essential components of critical thinking. To correct thinking errors, gaps and weaknesses in reasoning, and the influence of biases, but also for being able to take new information and alternative points of view into account, students need to cultivate reflection and self-correcting reasoning skills.

The team’s long-term vision is to design, test, iteratively improve, and implement “reflection systems” on a broad scale, tailored to different subject matters, educational levels, and diversity in classes. Reflection systems are envisioned as a new genre of socio-technical systems in which software tools with reflection-promoting user guidance are integrated in a variety of specifically designed social interaction scripts, that is, structures that scaffold the activities of students in a sequence of steps to optimize learning.

In order to explore factors relevant for the design of reflection systems, the proposed project focuses on college students’ ability to assess and improve arguments they create to cope with “wicked problems” such as climate change or the increasing computerization of the workforce. These problems are characterized by the fact that it is impossible to define them and determine their boundaries because people look at them from varying, often conflicting points of view that are determined by different knowledge, methods, and information, but also by varying interests, world-views, and values. Being able to deal with wicked problems requires the skill to correct one’s own reasoning. Wicked problems therefore provide an excellent opportunity to test reflection systems. Through reflection systems, students will learn, among other things, how to construct and integrate arguments that can justify stakeholder positions on a wicked problem. Since these arguments represent their reasoning on the problem, the project focuses on the improvement of arguments as one important case of self-correcting reasoning.

There are two readily-available software tools for “computer-supported argument visualization” (CSAV) that focus explicitly on reflection, students’ ability to assess the quality of arguments, and the improvement of these arguments. Both tools realize complementary approaches to reflection, but these are insufficient to guide collaboration on wicked problems. The primary goal of the proposed project is to design, test, and iteratively improve the new argument-based reflection system Reflect! that combines the strengths of the two existing reflection tools, but opens up a new world of scripted collaboration on wicked problems. The design of the envisioned reflection system will be informed by analyses of how experts, using the two available tools, approach wicked problems, assess arguments, and improve them. The effects of specific features of the new reflection tool and interaction scripts on participants’ reflection and self-correcting reasoning will be assessed in a series of experiments. 

Intellectual merit. The analysis of how experts approach wicked problems, how they engage in reflection, and how they assess and improve the quality of their arguments will contribute to a better understanding of the strategies, techniques, knowledge needed, and procedural subgoals that a novice should become aware of and learn to use when performing these activities. Results of the experiments will contribute to an understanding of how reflective learning and self-correcting reasoning can be fostered by assessing specific features of reflection tools and interactions scripts. Research results will enrich debates about whether and how known obstacles to self-improvement such as students’ implicit assumptions about the nature and certainty of knowledge and myside bias can be addressed by educational interventions.

Broader impact. Reflection and self-correcting reasoning are a crucial foundation for critical thinking and self-regulated learning. They are at the core of scientific reasoning and a precondition for participating competently and confidently in civic deliberation. The ability to regulate one’s own learning process—as it is required both for life-long learning and for learning in unsupervised or less supervised learning environments such as problem-, project-, and inquiry-based learning—depends on students’ ability to identify weaknesses and biases in their own reasoning, and the ability to do something about it. The societal relevance of the proposed project lies in its contribution to improving reflection and self-correcting reasoning in education.

PI: Michael H.G. Hoffmann (Philosophy Program, School of Public Policy)

Co-PI: Richard Catrambone (School of Psychology)

Co-PI: Jeremy Lingle (Center for Education Integrating Science, Mathematics, and Computing)