Keywords: HMI, multimodal, design, brain, intelligence.

This paper reports a software tool called "Let's Construct" that has been designed to support handwriting and drawing recognition for structural engineers in the conceptual design phase. Structural engineers can for example sketch a cantilever, place forces on it and immediately receive the inner forces in the beam. Although this tool has many benefits as it aims to emulate pen and paper, it requires low-level explicit and consistent interaction and has very limited learning capabilities, which distracts attention from the actual design intent.

More advanced humanoid human-machine interfaces (HMI) that are loosely coupled to avoid protocol dependency can provide interoperability between humans and the HMI, between HMI mutually and between HMI and software systems. These HMI initially communicate at high-level but can agree on faster low-level protocols. In emergency cases they can fall back on high-level communication which results in a robust design. High-level communication will eventually be supported by most humanoid HMI systems as it is the main goal of such systems. Humans currently are the best HMI to other humans.

Computing power might become a problem in such demanding systems with vast amounts of interdependency. Possibly quantum computing will emerge in the near future and provide emulation instead of simulation of the human brain, in which case the HMI problem will be solved.

For the near future software systems can benefit from an architecture as found in the human brain. It starts with analogue preprocessed data that only gets digitized as needed (sensors). Specialized software that cooperates in an interconnected distributed network (cortex) produces output signals. A feedback mechanism (cerebellum) that has overall perception because it receives both input and output signals can interfere or fine-tune the process as required and learn from situations.

The main conclusions of this paper are:

1. The cortex-cerebellum system can help to understand the way people form a solution strategy and choose appropriate tools. The results will be useful for automated problem solving environments.
2. The human brain can also help to create a semantic model of reality without having exact measurements. Such a model will be useful to visualize preliminary designs in the conceptual design phase. It can tightly cooperate with the HMI to allow fast and natural design modifications.

purchase the full-text of this paper (price £20)

go to the previous paper
go to the next paper
return to the table of contents
return to the book description
purchase this book (price £62 +P&P)