Transcript


Jackie Erickson:
You are listening to the Edge Case Research Self-Driving Car Safety Series, and in this episode, co-founder and CTO, Phil Koopman is going to address whether a driver’s test can prove a self-driving car is safe. You will hear Phil review how the aspects of a human driving test geared for 16 year olds changes when it’s a self-driving car. Now, over to Phil.

Phil Koopman:
This is Phil Koopman from Edge Case Research with a series on self-driving car safety. This time I’ll be talking about whether a classic driving test can be used to prove a self-driving car is safe. At some point, companies are done testing and they need to make a decision about whether it’s okay to let their vehicles actually be completely self-driving, driverless cars. The important change here is that there is no longer a human responsible for monitoring operational safety. That changes the complexion of safety, because now there’s no human driver to rely upon to take care of anything weird that might go wrong. That means you’ll need different metrics for safety when you deploy compared to those used during road testing.

One way people talk about knowing that it’s time to deploy is to give the car a road test, just like a human gets a driver’s test. Everyone wants something quick, cheap, and painless to decide whether they’re self-driving car’s ready to go, and a driver test has some intuitive appeal. Indeed, if a car can’t pass a driver test, then truly that’s a problem, but is it enough? Well, let’s go down the list of things that happen in a driver test.

For example, there’s a written test. Well, surely a self-driving car needs to know the rules of the road, just like a person, but for a self-driving car it’s a little more complicated, because it’s not just the rules of the road, but also what to do about conflicting rules or incomplete rules, or how do you handle justifiable rule breaking? For example, if there’s a big truck broken down in the travel lane on a two lane road, do you wait there until the truck is towed away, several hours perhaps, or do you go around it if there’s no traffic? Well, going over the double yellow line is clearly breaking the rule, but in everyday driving people do that sort of thing all the time.
Another thing you need is a vision test, and surely self-driving cars need to be able to see things on the road. For a person it’s typically just whether or not they have the right glasses on, but for a self-driving car it’s more complicated, because it isn’t just seen that something’s in the road, but rather figuring out what’s on the road and what might happen next. It isn’t just about reading road signs.

Another thing, the classic thing people have in mind is a road skills test, and surely a self driving car and needs to be able to maneuver the vehicle in traffic and account for all the things that happen, but again, it’s more complicated to know that a self-driving car is ready beyond what you see on a typical road test. Sure. A typical road test covers things like parallel parking and using turn signals. Yeah, but that’s easy stuff. Did your driver test cover spin-outs? Did it cover handling a blown out tire with an actual blown out tire. Did you have to deal with loss of brakes? Did you have another car run a red light during the driver test to see how you’d respond?

But even if you were to address all those types of things, there’s another important piece of a human driver test that people don’t think of as a test, and that’s actually proving you’re a human. You do that by showing your birth certificate. Oh, look. I’m a 16 year old human, and while I may not be the most mature person in the world, society’s determine I’m good enough to be able to handle a driver’s license. Now, what comes with that? It isn’t just that you’re 16. It’s that being 16, or whatever the age is where you are, is a proxy for things like being able to handle ambiguity and reason about consequences. It’s a proxy for knowing that a situation is unpredictable and becoming more cautious, because you’re not sure what happens next. Humans are moderately good at knowing when they’re not sure what’s going on, but things like machine learning are notoriously bad at knowing that they’re out of their element.

It’s also a proxy for handling unexpected events. While humans aren’t perfect, they do remarkably well when completely unexpected, unstructured events happen. They usually figure out something to do that’s reasonable. So, in general, being a 16 year old human is a proxy for some level of maturity and experience at handling the unexpected. So, what we do is we use a driver test to say, okay, this person has basic skills, and because they’re a reasonably mature human, they can probably handle all the weird stuff that happens and they’ll figure it out as they go. A big problem is it’s unclear how you figure out that a self-driving car has the level of judgment maturity of a 16 year old human. We’re not sure how to do that.

What we have known for decades is that you can’t prove a software based system is safe via testing alone. There are just too many complex situations, too many edge cases to be handled. No set of tests can cover everything. It’s just not possible. So, a driver test alone is never going to be enough to prove that a self-driving car is safe. Sure. Elements of a driving test are useful. You absolutely want the self-driving car to know the rules of the road, to be able to look down the right away and see what’s there, and to be able to maneuver in traffic for sure, but that is a minimum and insufficient set of things to prove it’s safe enough to responsibly deploy.

The point is you need more than a test. You need good engineering rigor. You need to know the car was designed properly, just like in every other safety critical computer based system. You don’t fly airplanes around to see if they fall out of this sky, and if they don’t, you say it’s safe. In fact, you do a lot of good engineering and you make sure the system is designed to do the right thing, and the testing is just there to make sure that the engineering process did what you thought it did. So, for self-driving cars, sure, a road test is helpful, but you’re going to need a good engineering process was executed as well. You need to know that the system is going to handle all the things that have to go right, as well as all the things that can possibly go wrong, and that goes far beyond any reasonable driver test.
Jackie Erickson:
That was Phil Koopman addressing a driver’s test as a metric for self-driving cars. The key takeaway from this episode: no single test will be enough to prove the safety of a self-driving car. Rather than one make or break driving test, like we do for 16 year olds, it’s about having a good engineering process to show the vehicle is designed to do the right thing. To get more perspective on this subject or to communicate with our Edge Case safety team, please visit our website www.ecr.ai, and from there you can email us and link to our social media. Thanks for listening, and we look forward to working with you on delivering the promise of autonomy.