Assessing Conceptual Representations
of Ill-defined Problems

Aytac Gogus, Tiffany A. Koszalka and J. Michael Spector

This paper presents research findings related to the “Dynamic Enhanced Evaluation of Problem Solving (DEEP)” (Spector & Koszalka, 2004) methodology for assessing how participants conceptualize ill-defined problems in biology using annotated concept maps. The methodology engages highly experienced (expert) and less experienced (novice) participants in creating annotated problem representations. This exploratory study addressed the lack of assessment methods to assess learning progress and relative level of expertise in complex, dynamic domains. This paper addresses (1) differences between experts and novices, (2) learning in complex domains, and (3) the rational for using annotated concept maps to assess learning in complex domains. Findings suggest that there are similarities in how experts think about ill-defined problems and these similarities are different than novices. These findings thus suggest that this methodology is useful in distinguishing relative levels of expertise in conceptualization of complex and challenging problems in a biology context.

Keywords: Annotated concept maps, complex cognitive skills, Ill-defined problems, problem solving, level of expertise, expert and novice differences, learning in complex domains, assessing learning progress.