Yes, it does: it’s called the “Ironies of automation”, and was described by a british researcher:
Bainbridge, Lisanne: “Ironies of automation.” Analysis, Design and Evaluation of Man–Machine Systems 1982. 129-135.
The ironies of automation are that:
- when you add a control system to a process, you do so under the assumption that the control system will be faster, more reliable and accurate than a human operator. But the more advanced the control system is so the more crucial may be the contribution of the operator when something unexpected happens.
One irony is that errors of software and hardware developers can be a major source of operating problems.
- Thus the engineer leaves the operator to do tasks which the engineer cannot think how to automate. These include monitoring that the automation system is operating correctly.
But if it doesn’t then the operator may either have to take over manual control or to call a more experienced supervisor. However:
- Manual control requires manual control skills that degrade very rapidly if not repeated often. The higher the degree of automation and the less likely is that the operator does it.
- For the supervisor to be able to make effective decisions, the supervisor has to understand what was the past behavior of the system, what happened and why, and what could be possible course of actions. This needs to be done quickly and in a situation when the supervisor was not prepared to do it and usually these situations demand a very high level of cognitive load.
- Furthermore, the operator needs to know what the correct behavior of the process should be. Because the computerized system uses a lot of information to make its decisions, there’s no way that a human operator can check in real time that the system is following its rules correctly.
Even minor anomalies in complex and tightly coupled systems can increase the severity of potential consequences of operators actions. Operators’ vigilance degrades very quickly, and easy shortcuts are not effective, like asking operators to keep a log of what they observe and do.
Factors that positively influence operators’ subjective health and feeling of achievement are high coherence of process information, high process complexity, high process controllability, rich patterns of activities that operators can do. On the other hand, factors like fast process dynamics and low usability of the user interface have a negative influence.
A recent review shows that these problems are at the heart of many accidents that happened in the last 3 decades, in aerospace, in cruise vessels, in control of petroleum pipelines:
Strauch, B. (2017). Ironies of Automation: Still Unresolved After All These Years. IEEE Transactions on Human-Machine Systems.
