How Complex Systems Fail
Complex systems run in degraded mode.
¶
Failure is ordinary
¶
Cook writes for medicine and operations, but software engineers recognize the pattern. Systems drift into unsafe states while appearing fine. Redundancy adds coupling. Postmortems find "human error" because the graph is too large to hold in one head.
¶
Degraded success
¶
Most outages are recovered, not prevented. Success comes from people compensating for brittle automation. The system "works" until it does not.
¶
Required reading for builders
¶
Short enough to read in one sitting. Pairs with any paper that promises reliable AI in production. Complexity does not negotiate.