This case study sits at the boundary of aviation and autonomous systems — included here because the failure modes it demonstrates are directly relevant to autonomous aviation, and because the safety architecture questions it raises are central to the certification of autonomous flight systems.
On 18 March 2018, an Uber self-driving Volvo XC90 test vehicle struck and killed Elaine Herzberg as she walked her bicycle across a four-lane road in Tempe, Arizona. The vehicle’s LIDAR system detected an object in the road. The classification algorithm assessed the object as ‘other’ — not a pedestrian, not a vehicle, not a category that required emergency braking. By the time the system resolved the classification to ‘bicycle,’ it was too late to stop. The safety driver was watching a video on her phone.
The vehicle saw Elaine Herzberg. It did not understand what it was seeing. And by the time it understood, she was dead.
The autonomous system detected the object 5.6 seconds before impact. It classified it incorrectly. It delayed emergency braking for 1.3 seconds after correct classification. The safety driver was not watching. Three independent safety systems failed simultaneously. This is the autonomous systems safety architecture problem in its starkest form.
Date | 18 March 2018 |
Flight | N/A — autonomous vehicle test operation |
Aircraft | Volvo XC90 (modified autonomous test vehicle) |
Operator | Uber Advanced Technologies Group |
Fatalities | 1 pedestrian |
Category | Autonomous Systems / Sensor Classification Error / Safety Driver / Human-Machine Interface |
Location | Tempe, Arizona, USA |
The Event
- 18 March 2018, 22:58 local: Uber ATG test vehicle operating in autonomous mode at approximately 43 mph
- The vehicle’s LIDAR detects Elaine Herzberg approximately 5.6 seconds before impact
- The object classification algorithm cycles through multiple classifications — other, vehicle, bicycle
- The system does not trigger emergency braking because it does not immediately classify the object as requiring avoidance
- 3 seconds before impact, the system determines emergency braking is needed but does not engage it — emergency braking had been disabled to reduce ‘erratic vehicle behaviour’
- The safety driver, Rafaela Vasquez, is watching a video on her phone; she looks up 0.5 seconds before impact
- The vehicle strikes Herzberg at approximately 43 mph; she dies of her injuries
- Uber suspended its autonomous testing program nationally
The NTSB investigation found that the Uber ATG safety management system had significant deficiencies. Emergency autonomous braking had been disabled to prevent unwanted braking events — a decision made for operational comfort that removed a critical safety function. The safety driver role had not been adequately defined or enforced. The system was not safe to operate on public roads at the time of the accident.
Systems Engineering Perspective
From a systems engineering perspective, the Uber crash demonstrates that sensor data is not situational awareness and that classification accuracy under real-world conditions is the critical safety parameter for any autonomous system operating in a public environment.
Detecting an object and understanding what it is are different problems. The Uber system solved the detection problem. It failed the classification problem — the problem that determines whether detected objects receive a safety response. Detection without reliable classification is not safe autonomous operation.
The Classification Problem — Seeing Without Understanding
The vehicle’s LIDAR accurately detected Elaine Herzberg. The electromagnetic return from the LIDAR confirmed an object in the road. The classification algorithm — which assigns the detected object to a category (pedestrian, vehicle, bicycle, other) — cycled through classifications for several seconds without arriving at a stable, confident result.
An autonomous safety system that detects an object but cannot classify it has seen something it does not understand. The question that safety architecture must answer is: what should an autonomous system do when it detects an object it cannot reliably classify? Uber’s system, at the time, did not have a conservative default — a fail-safe that responded to uncertain classification with caution rather than continuation.
‘I can see something but I don’t know what it is’ is not a safe operating condition for an autonomous vehicle in a public environment. Uncertain classification requires conservative default behaviour — not continued operation at speed.
Emergency Braking Disabled — Operational Comfort Over Safety
Uber ATG’s engineering team had disabled the emergency autonomous braking system to reduce incidents of unwanted emergency stops — events that were causing discomfort to test passengers and were affecting the vehicle’s perceived performance in development testing. The disabling of emergency braking was an explicit engineering decision made to optimise the development experience.
This decision removed a critical safety function from the operational system. When the classification algorithm eventually identified the need for emergency braking, the function was not available.
Disabling a safety function to improve operational comfort is not an engineering trade-off. It is the removal of a safety barrier with no compensating safety measure. The Uber ATG decision to disable emergency braking is the autonomous systems equivalent of the ValuJet cargo hold without fire suppression — a safety function absent from the operational system.
The Safety Driver Role — Defined But Not Enforced
The safety driver role existed to provide human oversight of the autonomous system — a backup for scenarios where the autonomous system failed or encountered conditions it could not handle. The safety driver was not engaged at the time of the collision. Uber ATG’s safety management system had not effectively defined, monitored, or enforced the safety driver’s required level of attention.
The safety driver role is the single-pilot equivalent of the co-pilot’s monitoring duty. When it is not performed, the redundancy it provides is theoretical.
Human Factors Perspective
The human factors analysis encompasses both the safety driver’s failure to monitor and the organisational culture that allowed safety management deficiencies to persist in a system operating on public roads with pedestrians.
The Safety Driver — Monitoring as a Safety System
The safety driver role in Uber ATG’s test operations was defined as a human backup for the autonomous system. The role required the driver to monitor the vehicle’s operation and intervene if necessary. Vasquez was watching a video on her phone for extended periods during the test drive.
The safety driver monitoring failure is not simply an individual discipline failure. It is a safety management failure: the organisation that deployed a human as a safety backstop did not ensure that the human was performing the safety function they had been deployed to perform.
A safety backup that is not performing its safety function provides no backup. The safety driver watching a video was, from a system perspective, absent — equivalent to a monitoring system with its power off.
Organisational Safety Culture
The NTSB found that Uber ATG’s safety management system was inadequate for the risk of operating autonomous test vehicles on public roads. The combination of disabled emergency braking, inadequately enforced safety driver procedures, and classification algorithm performance gaps reflected an organisational safety culture that had not adequately assessed and managed the risks of its operational program.
System Interaction Breakdown
1. Classification Failure — Detecting Without Understanding
The LIDAR detected the pedestrian. The classification algorithm failed to reliably identify the object as a pedestrian requiring avoidance in the available time.
2. Emergency Braking Disabled
The emergency braking function had been disabled for operational reasons, removing the primary automated response to the classification failure.
3. Safety Driver Not Monitoring
The human backup for autonomous system failures was not performing the monitoring function that was the basis of their deployment.
Three independent safety failures aligned simultaneously. The classification system failed. The automated braking was absent. The human backup was not watching. Every layer of the safety architecture failed at the same moment.
Significance in Aviation Risk
1. Autonomous Vehicle Safety Standards
The Uber crash accelerated the development of safety standards for autonomous vehicle testing on public roads — including requirements for emergency braking capability, safety driver engagement standards, and safety management system requirements.
2. Conservative Default for Uncertain Classification
The crash established the principle that autonomous systems operating in public environments must have conservative defaults for uncertain classification — not continuation at speed.
3. Relevance to Autonomous Aviation
The failure modes demonstrated in Tempe — sensor classification error, disabled safety function, unmonitored autonomous operation — are directly relevant to the certification of autonomous aviation systems, and are informing the development of safety standards for UAM, drones, and autonomous aircraft operations.
Related Aviation Risk Lab Content
Pillar Pages
Automation and Technology: Automation And Technology
Systems Engineering: Systems Engineering
Risk Assessment: Risk Assessment
Related Case Studies
Case Study: Air France 447 — When the Automation Stopped: Af 447
Case Study: Lion Air 610 — MCAS and the Single Point of Failure: Lion Air 610
Case Study: Qantas 72 — When the System Reacted Correctly to Something That Wasn’t Real: Qantas Flight 72
Closing Perspective
The Uber crash in Tempe is the most important autonomous systems safety case study available. It demonstrates, in a single event, the three fundamental failure modes of autonomous systems operating in complex environments: sensor data that is detected but not understood, safety functions disabled for operational convenience, and human oversight that is assigned but not performed.
For autonomous aviation — which faces the same challenges at higher speeds, in a more complex operational environment, with less ability to stop and assess — these failure modes are not hypothetical. They are the specific risks that certification frameworks for autonomous flight are designed to prevent.
Elaine Herzberg was detected 5.6 seconds before impact. She was not understood. The system that was supposed to understand her was not working. The human who was supposed to watch the system was not watching. Every layer of the safety architecture failed. Layers that are not functional provide no safety.
The Uber crash proved that sensor coverage is not safety. Understanding what the sensor data means — reliably, quickly, conservatively — is safety. Detection without understanding is surveillance, not protection.
Related Posts
