Engineering engineering research: Understanding the fabric of engineering design

A house designed using our Energy3D CAD software.

Perhaps the most important change in the Next Generation Science Standards to be released in March 2013 is the elevation of engineering design to the same level of importance as of scientific inquiry (which was enshrined as a doctrine of science education in the 1996 science standards). But how much do we know about teaching engineering design in K-12 classrooms?

A house made using our Energy3D CAD software.

Surprisingly, our knowledge about students’ learning and ideation in engineering design is dismal. The Committee on Standards for K-12 Engineering Education assembled by the National Research Council in 2010 found “very little research by cognitive scientists that could inform the development of standards for engineering education in K–12.” Most educational engineering projects lacked data collection and analysis to provide reliable evidence of learning. Many simply replicated the “engineering science” model from higher education, which focuses on learning basic science for engineering rather than learning engineering design. Little was learned from these projects about students’ acquisition of design skills and development of design thinking. In the absence of in-depth knowledge about students’ design learning, it would be difficult to teach and assess engineering design.

In response to these problems, we have proposed a research initiative that will hopefully start to fill the gap. As in any scientific research, our approach is to first establish a theory of cognitive development for engineering design and then invent a variety of experimental techniques to verify research hypotheses based on the theory. This blog post introduces these ideas.

In order to study engineering design on a rigorous basis, we need a system that can automatically monitor student workflows to provide us all the fine-grain data we need to understand how they think and learn when they become expert designers from novice designers. This means we have no choice but to move the entire engineering design process onto the computer -- to be more exact, into computer-aided design (CAD) systems -- so that we can keep track of students’ workflows and extract information for inferring their learning. While some educators may be uncomfortable with the virtualization of engineering design, this actually complies with contemporary engineering practices that ubiquitously rely on CAD tools. If we have a CAD system, we can add some data mining mechanisms to turn it into a powerful experimental system for investigating student learning. Fortunately, we have created our own CAD software, Energy3D, from scratch (see the above images about it). So we can do anything we want with it. Since all the CAD tools are similar, the research results should be generalizable.

A cognitive theory of engineering design.

Next we need a cognitive theory of engineering design. Engineering design is interdisciplinary, dynamic, and complicated. It requires students to apply STEM knowledge to solve open-ended problems with a given set of criteria and constraints. It is such a complex process that I am almost certain that any cognitive theory will not be perfect. But without a cognitive theory our research would be aimless. So we must invent one.

Our cognitive theory assumes that engineering design is a process of “knitting” science and engineering. Inquiry and design are at the hearts of science and engineering practices. In an engineering project, both types of practices are needed. All engineering systems are tested during the development phase. A substantial part of engineering is to find problems through tests in order to build robust products. The diagnosis of a problem is, as a matter of fact, a process of scientific inquiry into an engineered system. The results of this inquiry process provide explanations of the problem, as well as feedback to revise the design and improve the system. The modified system with new designs is then put through further tests. Testing a new design can lead to more questions worth investigating, starting a new cycle of inquiry. This process of interwoven inquiry and design repeats itself until the system is determined to be a mature product. 

These elements in our cognitive theory all sound logical and necessary. Now the question is: If we agree on this theory, how are we going to make it happen in the classroom and how are we going to measure its degree of success? Formative assessment seems to be the key. So the next thing we need to invent is a method of formative assessment. But what should we assess in order not to miss the entire picture of learning? This requires us to develop a deep understanding of the fabric of engineering design.

A time series model of design assessment.

Engineering design is a complex process that involves multiple types of science and engineering tasks and subprocesses that occur iteratively. Along with the properties and attributes of the designed artifacts that can be calculated, the order, frequency, and duration learners handle the tasks provide invaluable insights into the fabric of engineering design. These data can be monitored and collected as time series. Formative assessment can then be viewed as the analysis of a set of time series, each representing an aspect of learning or performance. In other words, each time series logs a “fiber” of engineering design.

At first glance, the time series data may look stochastic, just like the Dow Jones index. But buried under the noisy data are students’ behavioral and cognitive patterns. Time series analysis, which has been widely used in signal processing and pattern recognition, will provide us the analytic power to detect learner behaviors from the seemingly random data and then generate adaptive feedback to steer learning to less arbitrary, more productive paths. For example, spectral or wavelet analysis can be used to calculate the frequency of using a design or test tool. Auto-correlation analysis can be used to find repeating patterns in a subprocess. Cross-correlation analysis can be used to examine if an activity or intervention in one subprocess has resulted in changes in another. Cross-correlation provides a potentially useful tool for tracking a designer’s activity with regard to knowledge integration and system thinking.

In the next six months, we will undertake this ambitious research project and post our findings in this blog as we move forward. Stay tuned!