Knowledge engineering

This article is about methods for developing expert systems. For application of knowledge based technology to the domain of manufacturing and CAD, see Knowledge based engineering.

Knowledge engineering (KE) refers to all technical, scientific and social aspects involved in building, maintaining and using knowledge-based systems.

Background

Expert systems

One of the first examples of an expert system was MYCIN, an application to perform medical diagnosis. In the MYCIN example, the domain experts were medical doctors and the knowledge represented was their expertise in diagnosis.

Expert systems were first developed in artificial intelligence laboratories as an attempt to understand complex human decision making. Based on positive results from these initial prototypes, the technology was adopted by the US business community (and later worldwide) in the 1980s. The Stanford heuristic programming projects led by Edward Feigenbaum was one of the leaders in defining and developing the first expert systems.

History

In the earliest days of expert systems there was little or no formal process for the creation of the software. Researchers just sat down with domain experts and started programming, often developing the required tools (e.g. inference engines) at the same time as the applications themselves. As expert systems moved from academic prototypes to deployed business systems it was realized that a methodology was required to bring predictability and control to the process of building the software. There were essentially two approaches that were attempted:

1. Use conventional software development methodologies
2. Develop special methodologies tuned to the requirements of building expert systems

Many of the early expert systems were developed by large consulting and system integration firms such as Andersen Consulting. These firms already had well tested conventional waterfall methodologies (e.g. Method/1 for Andersen) that they trained all their staff in and that were virtually always used to develop software for their clients. One trend in early expert systems development was to simply apply these waterfall methods to expert systems development.

Another issue with using conventional methods to develop expert systems was that due to the unprecedented nature of expert systems they were one of the first applications to adopt rapid application development methods that feature iteration and prototyping as well as or instead of detailed analysis and design. In the 1980s few conventional software methods supported this type of approach.

The final issue with using conventional methods to develop expert systems was the need for knowledge acquisition. Knowledge acquisition refers to the process of gathering expert knowledge and capturing it in the form of rules and ontologies. Knowledge acquisition has special requirements beyond the conventional specification process used to capture most business requirements.

These issues led to the second approach to knowledge engineering: development of custom methodologies specifically designed to build expert systems. One of the first and most popular of such methodologies custom designed for expert systems was the Knowledge Acquisition and Documentation Structuring (KADS) methodology developed in Europe. KADS had great success in Europe and was also used in the United States.

OK paradigm

Organic Knowledge Engineering

Organic Knowledge (OK) is a paradigm for OK systems engineering developed over the last two decades. Organic Knowledge (OK) system can be abstractly seen as advisory board of very different (even adversary) educators with mechanisms of cooperation that work together to suggest to the student the next game after constantly observing him.

In addition to using all the principles of SE, the organic SE adds another special layer of tools. It simulates and incorporates both the knowledge of the software engineer and the knowledge of the domain expert.

In a nutshell the organic approach is treating the problem and the solution process as a living organism (or ecology of organisms if more appropriate). The organic solution is non-algorithmic and evolving using feedback and data-to-knowledge mechanisms. It is as if the solution is like a child – in the beginning having no knowledge (except some basic mechanisms needed for evolution), and by process of feedback and Darwinian natural selection the solution gradually evolves into better and better reactions using the growing body of knowledge.

OK systems implementation

The computer implementation of such a system can be described in more precise terms. Organic Knowledge (OK) systems are ICT systems incorporating human expertise. One would be tempted to describe them as Expert Systems (ES) “on steroids” transforming them into Knowledge Systems (KS). If at all, they are an ecological system of many different and sometimes contradictory experts, called organs. But in reality the OK systems are so much more in almost every aspect that it is more correct to say they are the fulfillment of the ES vision. Other spiritual ancestors of OK system are Turing’s “child programs” and Minsky’s learning, evolving and non-algorithmic Society of Mind proposals.

OK system is:

• intelligent
• evolving
• learning
• organized
• distributed
• dialectical
• very big knowledge base

Each organ is simulating an independent expert, and includes:

• knowledge base (data, meta-data and procedures)
• feedback apparatus:
  • knowledge acquisition mechanism (interfaces and communication)
  • learning mechanism (inference of new knowledge and processing)
  • evolution mechanism (creating and changing organs in view of the new knowledge)
• interfaces:
  • environment (local)
    • subjective (user)
    • objective
  • communication (network)
    • with other organs (o2o)
    • with remote servers
    • with remote users (p2p)
    • with remote resources
• execution (proactive)

GML paradigm

General Outline

The Gestalt-Multiplex-Layering (GML) paradigm entails: gestalt – a deeper model of the expert knowledge and reasoning process; multiplicity – simultaneous use and cooperation of different and conflicting approaches; layering – use of a hierarchy of independent layers of control and processing, through which the input and intermediate results are propagated.

The independence of each layer enables implementation of different approaches at different layers. The hierarchical layering of control and abstraction of lower by upper layers enables the cooperation and solution of contradictions arising from the use of a variety of different approaches.

In very broad, plain terms, at each layer there is a small intelligent system controlling, generalizing and inducing the cooperation of different approaches in a larger intelligent system of the next layer.

Gestalt

Gestalt is the skeleton, the deeper model, the concept, the meta-model of the lower layer, abstracting, generalizing, controlling and interfacing it, and mediating between the lower layer, the upper layers and the user.

See also

• Knowledge level modeling
• Knowledge management
• Knowledge representation
• Knowledge retrieval
• Knowledge tagging
• Method engineering

External links

• Data & Knowledge Engineering – Elsevier Journal
• Knowledge Engineering Review, Cambridge Journal
• The International Journal of Software Engineering and Knowledge Engineering – World Scientific
• IEEE Transactions on Knowledge and Data Engineering
• Expert Systems: The Journal of Knowledge Engineering – Wiley-Blackwell