Analyzing Individual Differences in Program Comprehension
Roman Badnarik, Niko Myller, Erkki Sutinen and Markku Tukiainen
Programming is a complex problem-solving domain often involving many dependent entities which may be even hidden or latent. Novice programmers have little knowledge about program execution and may see it as an abstract and non-deterministic process. To support novices, Jeliot was developed to visualize program execution, and thus help in specifying viable program development models. This paper reports on an empirical experiment in program comprehension where 16 subjects used Jeliot to comprehend two Java programs. The experiment focused on how the experience level and complexity of the program affected (a) the patterns of interaction with the tool, (b) the gaze behavior, (c) the use of visualization, and (d) the cognitive processes related to program comprehension. This was done by investigating the protocols obtained from an eye-tracker, interaction logging, and comprehension summaries. An interaction between experience and behavior was found. Experts read the whole code first, constructed a hypothesis, and tested it against the animation. Novice programmers did not read the code first. They animated the program directly, and replayed the animation several times focusing on the difficult sections. The results reveal the potentials of gaze as an additional modality in an adaptive tool for program visualization.