Energy3D uses intelligent agents to create adaptive feedback based on analyzing the "DNA of design"

Fig. 1: A simple case of teaching thermal insulation.

Energy3D is a "smart" CAD tool because it can monitor the designer's behavior in real time, based on which it can generate feedback to the designer to regulate the design behavior. This capacity has tremendous implications to learning and teaching scientific inquiry and engineering design with open-ended nature that requires, ideally, one-to-one tutoring so intense that no teacher can easily provide in real classrooms.

The computational mechanism for generating feedback in Energy3D is based on intelligent agents, which consist of sensors and actuators (in very generic terms). In Energy3D, all the events are logged behind the scenes. The events provide the raw data stream from which various sensors produce signals based on subsets of the raw data. For instance, a sensor can be created to monitor any event related to solar panels of a house. An agent then uses a decision tree model to determine which actuators should be called to provide feedback to the user or direct Energy3D to change its state. For instance, if a solar panel is detected to be placed on the north-facing roof, the agent can remind the designer to rethink about the decision. Just like what a teacher may do, the agent can even suggest a comparative analysis between a solar panel on the north-facing roof and a solar panel on the south-facing or west-facing roof. Although this type of inquiry and design can be also taught using directly scaffolded instruction that guides students to explore step by step, in practice we have found the effect of this approach often diminishes because many students do not read instruction carefully enough and remember them long enough. It is also challenging for teachers to guide the whole class through this kind of long learning process as students often pace differently. Adaptive feedback provides a way to help students only when they need or just when a need is detected, thus providing a better chance to deliver effective instruction.

Let's look at a very simple example. Figure 1 shows a learning activity, the goal of which is to teach how the thermal property of a wall, called the U-value, affects the energy use of a house. Many students may walk away with a shallow understanding that the higher the U-value is, the more energy a house uses. The challenge is to help them deepen their understanding. For example, how can we make sure that students will collect enough data points to discover that the energy a house uses is proportional to the U-value? How can we support them to find out that the relationship is independent of seasonal change, wall orientation, and solar radiation (e.g., a lower U-value is good in both summer and winter, irrespective of whether or not the wall faces the sun). Helping students accomplish this level of understanding through inquiry-based activities is by no means a trivial task, even in this seemingly simple example. Let's explore what we may do in Energy3D now that we have a way to monitor students' interactions with it.

Fig. 2: An event sequence coded like a DNA sequence.

In nearly all software that support learning and teaching, the events during a process can be coded as a string with characters representing the events and ordered by their timestamps, such as Figure 2. In this case, A represents an analysis event in the Energy3D CAD tool, U represents an event of changing the U-value of a wall, C represents an event of changing the date for the energy simulation, a questionmark (?) represents an event of requesting help from the software, an underscore (_) represents an inactive time period longer than a certain threshold, and * is a wildcard that represents any other event "silenced" in this expression in order to reduce the dimensionality of the problem. For those who know a bit about bioinformatics, this resembles a DNA sequence. In the context of Energy3D, we may also call it as the DNA of a design, if that helps your imagination.

Now that we have converted the sequence of events into a string, we can use all sorts of techniques that have been developed to analyze strings to analyze these events, including those developed in bioinformatics such as sequence alignment or those developed in natural language processing. In this article, I am going to show how the widely-supported regular expressions (regex) can be used as a technique to detect whether a certain type of event or a certain combination of events occurred or how many times it occurred. I feel that regex, in our case, may be more accurate than edit distances such as the Levenshtein distance in matching the pattern. For example, a single substitution of event may represent a very different process despite the short edit distance.

Fig. 3: A sequence that shows high usage of feedback

We know that, a fundamental skill of inquiry is to keep everything else fixed but change only one variable at a time and then test how the system's output depends on that variable. Through this process of inquiry, we learn the meaning of that variable, as explained by Bruce Alberts, former president of the National Academy of Sciences and former Editor-in-Chief of the Science Magazine. In the example discussed here, that variable is the U-value of a selected wall of the house and the test is the simulation-based analysis. A pattern that has alternating U and A characters in the event string suggests a high probability of inquiry, which can be captured using a simple regex such as (U[_\\*\\?]*A)+. Between U and A, however, there may be other types of events that may or may not exist to weaken the probability or compromise the rigor. For example, changing the color of the wall between U and A may also result in an additional difference in energy use of the house that originates from the absorption of solar radiation by the external surface of the wall and has nothing to do with its U-value. In this case, changing multiple variables at a time appears to be a violation of the aforementioned inquiry principle that should be called out by the agent using another regex to analyze the substring between U and A.

An interesting feature in Energy3D is that feedback itself is also logged. Figure 3 shows a sequence that has an alternation pattern similar to that of Figure 2, but it records a type of behavior showing that the user may rely overly on feedback from the system to learn (the questionmarks in the string stand for feedback requests made by the user) and avoid deep thinking on their own. This may be a common problem in many intelligent tutors (sometimes this behavior is called "gaming the system").

The development of data mining and intelligent agents in Energy3D is opening interesting opportunities of research that will only grow more important in the era of artificial intelligence (AI). We are excited to be part of this wave of AI innovation.