The National Transportation Safety Board release a draft report on the Asiana Airlines plane crash last year, which result in 3 death. Commenting on role of the autopilot auto throttle system, they said,
"in their efforts to compensate for the unreliability of human performance, the designers of automated control systems have unwittingly created opportunities for new error types that can be even more serious than those they were seeking to avoid."
I think this is a cautionary tale for the designers of self-driving car. When we have a system that is almost prefect but not completely infallible, it induce a new kind of risk because the operator becomes complacent and overly rely on automation.
2014.06.24 comments