Visual analytics for uncovering complex learning: Part II

Figure 1. Most students performed well throughout.

In an early article, I have introduced the idea of using graph theory to visualize learning. In this follow-up article, I will give an example based on our recent classroom study with our mixed-reality technology that engaged 65 high school students in three chemistry classes to learn the kinetic molecular theory through exploring gas laws. The primary task of the study is to investigate how a novel kind of mixed-reality technology that combines actions in and across real and virtual worlds can help students develop molecular reasoning skills to explain macroscopic phenomena with microscopic theories.

Figure 2. Less successful than Figure 1.

The key is to study students' interactions with the technology in great details in order to shed light on how the mixed-reality experience may help students make micro-macro connections. For most students, there is a huge gap between their macro perception and their micro conception. We want to fill the gap using this powerful real-time technology to provide just-in-time, integrated instruction.

Figure 3. Performed well initially and then went astray.

The challenge is to find a way to describe both the interaction process and the reasoning process. Our software logs everything students are doing behind the scenes (i.e., we can track students' hands). Meanwhile, through our embedded assessment using causality maps (a type of concept map), we can track students' minds. Our objective is to find any conceptual changes from the "mind-tracking" data and attribute them to the "hand-tracking" data. Only after we establish this association can we evaluate the worth of the mixed-reality technology in fostering science learning.

Both the "hand-tracking" and "mind-tracking" data are being studied using graph theory. In this article, we will show the analysis of the "mind-tracking" data first. The data were generated by students connecting 6-7 provided molecular concepts between a cause and an effect to compose an explanation. Some of the concepts are irrelevant and used as distractors. For example, increasing the number of molecules has nothing to do with the molecular mass.

Given the fact that there are 65 students and four causality maps, we need a way to quickly visualize student learning. Our solution is to construct graphs based on these data. It becomes immediately obvious that this visual analytics can provide extremely informative graphics that shows student learning on a statistical basis. The three images in this article show that: 1) Most students in Class E performed well throughout Graph 3; 2) Students are less successful in Class B in Graph 1; and 3) Students in Class D performed well in Graph 4 at the beginning and then went astray -- largely because this is a more difficult challenge that involves multiple reasoning paths. (The gray bands in the three images represent the correct reasoning chains in each case. The thickness of an edge represents the number of students who drew the link in the concept map.) These graphs can potentially show the weaknesses of the students at the scale of the whole class. I could not help thinking how useful this would be to the teacher if such informative feedback is provided to her.

This graph-based analytics is NOT just visual -- It is also interactive. You can examine all the student data we have collected at this page, programmed in Dart -- the latest Web programming language from Google. The page provides filters for you to examine and compare any number of students' work. You can also drag the nodes around for clarity. This kind of feedback tool is what teachers would want to have and what we should create for them. The world has spent billions of dollars building visual business analytics to assist business executives to make investment decisions day to day. How about spending a bit money to build an infographics system to assist teachers to make instructional decisions day to day?