Development and Evaluation of Learning Analytics Dashboards to Support Online Discussion Activities

Online discussion plays an increasingly significant role in asynchronous online learning environments. While previous attempts have been made to develop learning analytics dashboards to facilitate such discussions, most of these dashboards have been designed without reference to data or visualization techniques that have been proven to make online discussions more effective. This study identified the difficulties and inconveniences experienced by learners in online discussion activities and generated a set of visual design guidelines for overcoming them. Applying these guidelines, a set of learning analytics dashboards were developed and evaluated. The study was conducted according to prototyping methodology, which yielded five prototype dashboards that display information on participation, interaction, discussion content keywords, discussion message types, and the distribution of debate opinions, respectively. The developed dashboards were then revised and refined in a three-step process: (1) expert validation to verify that the dashboards complied with the visual guidelines and provided learners with the information they needed; (2) tests to identify usability problems, collect qualitative and quantitative data, and determine participant satisfaction; and (3) user experience evaluations to determine how learners and instructors perceived their interactions with the dashboards. Practical and empirical discussions are provided based on the results, which offer a valuable base of user experience data that can be used in future studies.