Claims in connection with the supporting technology that will be deployed to engage the knowledge

1. The power of natural language interaction for knowledge acquisition and for collaborative problem solving can be increased by combining it with intelligent machine vision. The MIT AI Laboratory has developed a multi-sensory ``Intelligent Room'' in which active vision and language understanding technology can be used together. In many cases, the true ``natural'' language of interaction combines both visual and linguistic information.

   In particular, we believe that a natural way to explain a high level problem solving strategy is to draw box and arrow diagrams and to explain in natural language what each box is intended to accomplish. This need for mixed sensory expression is also true for many of the problem domains of interest to DARPA. The natural context for discussing air campaign planning is a map; we interact by pointing at the map, and discussing (in Air Force English, for example) how we want to go about achieving our strategic and tactical objectives. We describe the necessary sequencing of events by drawing PERT charts (boxes and arrows).

2. When used in an open environment for knowledge acquisition the system must somehow address the question of inconsistent information. We claim that this problem decomposes into two distinct cases.

   In the first case, we have experts with essentially the same perspective but with second order difference in their organization of knowledge. In this case, we can use consistency promoting techniques developed by Davis and colleagues to identify the differences between the experts, to present these differences to the experts in the most illuminating way, and to manage a collaborative process which moves towards consensus.

   In the second case, the differences are fundamental and represent distinct and equally powerful viewpoints on the domain. This is not a bug but a feature. Systems which maintain multiple perspectives with distinct representations may be able to switch viewpoints and make progress while single perspective systems cannot recover if they get stuck. Our system is not currently hampered by a need to have a single consistent viewpoint; it is aware when it has retrieved conflicting answers to a problem. We will extend our current representation with justification structures which can identify source of any conclusion thereby providing a ``pedigree'' for all answers.

3. Our natural language system can simultaneously and efficiently interface to a variety of external representations and reasoning capabilities (i.e. other HPKB components) using a methodology called the Protocol of Inference, first employed by Howard Shrobe and colleagues in the JOSHUA system. We will extend the reasoning capabilities of our natural language based system along the lines of a redesigned and more extensive protocol of inference, and demonstrate the ability to efficiently exploit multiple HPKB components.