Reflections on Expert Systems
22/11/12 16:48 Filed in: Applications
About a week ago I was asked for the right to use a comment I made on expert systems on the I, Cringley Blog. I observed:
This thought developed over a number of years as I hoped between the disciplines of management and computer science and for the sake of completeness I thought to expand upon the idea here.Once again we confuse “Expert” with “Expert System”. These two things are almost diametrically opposed.
Experts understand exceptions, learn and focus on the unusual.
Expert Systems codify knowledge and focuses on the common.
In fact the economic incentives for expert systems reward handling the common cases first and returns tend to decrease as the more uncommon cases are added. IBM isn’t reproducing and storing real experts, just finding a way to stamp out repeatable workers and miss labelling them.
The first expert system I bumped into was used in diagnosing appendicitis and the documentary producer went to great pains to describe how hard it was for inexperienced doctors to get the right diagnosis but by gathering experts together and producing a computer program which asked questions of the inexperienced doctors they improved diagnosis. To give you an idea of how far back this system was they were proudly displaying their teletype printing terminal.
The technology shown was decision tree based and represented the earliest versions of expert systems. The computer was being used to operationalise a decision chart created by a group of experts - it probably could have been done on a piece of paper.
The earliest systems were quite inflexible in how they gathered and represented knowledge as they captured a tree like structure and were extremely painful to modify; slightly more modern systems allow more free form and less procedural structures as they can just capture predicates - this makes adding information easier - the computer does the book keeping required to present the rules, but does not change the fundamental nature of the system.
These systems are created by experts and allow them to create a representation of a procedure based on their knowledge. We get the usual benefits of moving from a cottage industry to a production line:
There are problems with this codification of knowledge:
This type of system isn’t an expert, does not generate knowledge and deskills a job - it does provide all the benefits of a production line which are considerable provided we can maintain and train true experts as well.
The technology shown was decision tree based and represented the earliest versions of expert systems. The computer was being used to operationalise a decision chart created by a group of experts - it probably could have been done on a piece of paper.
The earliest systems were quite inflexible in how they gathered and represented knowledge as they captured a tree like structure and were extremely painful to modify; slightly more modern systems allow more free form and less procedural structures as they can just capture predicates - this makes adding information easier - the computer does the book keeping required to present the rules, but does not change the fundamental nature of the system.
These systems are created by experts and allow them to create a representation of a procedure based on their knowledge. We get the usual benefits of moving from a cottage industry to a production line:
- uniform treatment
- easier distribution of practice improvement
- greater compliance with best practice
There are problems with this codification of knowledge:
- It is expensive and difficult to gather as it requires experts to decompose what they do into precise predicates
- The return is greatest for the most common cases i.e. covering the most common cases gives the greatest payback, and it rapidly becomes expensive to collect and identify the rare. Good systems gracefully refer users to an expert when the exceptional is discovered.
- The legal maxim “Hard cases make bad law” probably applies to expert systems - the rare and the unusual may distract from the correct handling of the average.
- How do we make new experts without seeing lots of cases - perhaps only seeing hard cases may make experts faster though I know of no evidence showing this.
This type of system isn’t an expert, does not generate knowledge and deskills a job - it does provide all the benefits of a production line which are considerable provided we can maintain and train true experts as well.