Visualizing and Assessing Knowledge from Unstructured Student Writing
Norma C. Ming and Vivienne L. Ming
We present a method to help faculty assess and visualize conceptual knowledge by applying topic modeling to unstructured student writing from online class discussion forums. To validate the technique against conventional assessment metrics, we evaluated its accuracy in predicting final grades in introductory undergraduate biology and graduate economics courses. Probabilistic latent semantic analysis (pLSA) and hierarchical latent Dirichlet allocation (hLDA) both produced significantly better than chance predictions, with hLDA yielding superior predictions. Upon projecting posts into a two-dimensional space and color-coding points by their temporal position in the course or by the author’s final grade, we captured patterns in students’ contributions that connect the topic modeling factors to more intuitively familiar characteristics. We consider how some possible qualitative features of the discussion may be represented in the topic space and outline future work to develop these tools further.
Keywords: Predictive assessment; automated assessment; learning analytics; educational data mining; artificial intelligence; text mining; topic modeling; online discourse; discussion facilitation; computer-supported collaborative learning