Uber driverless car in fatal accident
Discussion
Programming safety critical hardware is an old discipline which was being taught when I was at university in the 90's (not to me, but to my friends reading Computer Science). I guess experts in that domain are a bit thin on the ground, though, compared to people accustomed to writing generic software.
DonkeyApple said:
But that’s not really the point. That’s obviously what happened and it will happen again and again. It’s part of the development process. It will happen to all the companies.
The fact that the more devout followers of autonomous driving appear to do readily want to separate out Uber now as having bad programmers is in itself interesting but it doesn’t redress the key issue but rather attempts to sweep it under the carpet.
And that, right there, is the problem. Perhaps Silicon Valley hubris extends to viewing individual people as expendable?The fact that the more devout followers of autonomous driving appear to do readily want to separate out Uber now as having bad programmers is in itself interesting but it doesn’t redress the key issue but rather attempts to sweep it under the carpet.
A grown-up approach would have 2 systems: the production intent one (tuned to test discrimination) and a much more conservative one as a failsafe. The first is analysed to see what it *would* have done, whilst the second *does* just hit the brakes for plastic bags.
Meanwhile, face-tracking camera pointed at "safety driver" would be a minimum to ensure their attention.
All in all this is amateur-hour failure for anyone who has ever worked on anything safety critical. Full disclosure: my engineering work was around aircraft engines - *nothing* is left to chance like that...
DonkeyApple said:
We know that it is flawed really because it is still in the development phase so there will be many flaws that are being worked through. It was more that while I look forward to having the option of autonomous driving I personally do not believe that we are anywhere near as close as more avid followers would like to believe and that this horrible incident has highlighted this rather clearly. One of the leading developers and arguably the key firm in pushing the boundaries and risk taking has a system which still struggles to plot what was a very large, moving obstacle/ threat as just that.
We're 3-5 years away from a viable solution imo. AI isn't all that 'I' at the moment. And I say that as a long term holder of NVDA - (bought 5 years ago specifically for their chipset for autonomous driving and partnerships with most mainstream manufacturers). I DO however think it will become reality however.For Uber, this is a key step forward for their business, at the moment they are just a logistics company.
juice said:
We're 3-5 years away from a viable solution imo
Rather more than that I suspect given where we were 5 years ago and the not particularly significant progress since then.As far as the Nvidia bits are concerned I doubt they've removed any of the fundamental stumbling blocks they've had to date with their architecture. It might be great as a demo/development toy/basic system but there are some things where there isn't even a proper theoretical concept of how to get it to work in an adequate way. If anything they've concentrated on trying to push their existing IP into a problem space without actually understanding what they need to do. 'Oooh look we've got GPUs and deep learning and trillions of operations per second' - shame about the I/O latency and determinism and stability and repeatability.
Their stuff for driving displays and infotainment isn't bad because it's the same old same old but safe real-time control systems are rather different.
I suspect a lot of the problems come down to there being only a tiny pool of people who understand how to develop systems for problems like this and there'll be plenty of people out there ploughing on not even realising they're getting the basics wrong; "it's mostly working a lot of the time so how hard can it be to get to the end?" **VERY**
skwdenyer said:
A grown-up approach would have 2 systems: the production intent one (tuned to test discrimination) and a much more conservative one as a failsafe. The first is analysed to see what it *would* have done, whilst the second *does* just hit the brakes for plastic bags.
Meanwhile, face-tracking camera pointed at "safety driver" would be a minimum to ensure their attention.
All in all this is amateur-hour failure for anyone who has ever worked on anything safety critical. Full disclosure: my engineering work was around aircraft engines - *nothing* is left to chance like that...
exactly - even in non-critical systems (my background is IT and web systems) this should be the approach - you run a new system in parallel, shadowing the live system, and then analyse the differences (mind you TSB don't seem to have realised that either in their recent debacle!)Meanwhile, face-tracking camera pointed at "safety driver" would be a minimum to ensure their attention.
All in all this is amateur-hour failure for anyone who has ever worked on anything safety critical. Full disclosure: my engineering work was around aircraft engines - *nothing* is left to chance like that...
The question is why are these companies not employing people from other critical safety industries to set up the testing programme - or are they doing so, but money speaks louder and they are being over-ruled?
juice said:
We're 3-5 years away from a viable solution imo. AI isn't all that 'I' at the moment. And I say that as a long term holder of NVDA - (bought 5 years ago specifically for their chipset for autonomous driving and partnerships with most mainstream manufacturers). I DO however think it will become reality however.
For Uber, this is a key step forward for their business, at the moment they are just a logistics company.
It is good to see the optimism around autonomous vehicles, but we are way off 3-5 years from robust systems that can sit within our current human rich / chaotic environments... I would agree that there will be a time when it does come together (but probably on a far more reduced scale than currently touted), however I do think that NVDA is probably a good investment in the short to medium term...For Uber, this is a key step forward for their business, at the moment they are just a logistics company.
However I think this might be more accurately seen as a step backwards for Uber - I can see why it might be argued to be needed as a key step forwards for them, however, sometimes there is a disconnect between what a business is - and what they claim to be / claim they will become - the fact that Uber is building corporate value based on claims for an autonomous future - could be a correct analysis, or it could be a self-promoted bubble, or perhaps a distraction from the issues with the business structure as it exists today - for me, I would not be investing in Uber, or the equivalent - they are the public facing brand who may succeed, but will also have the awkward moments such as this when they screw up in a very public manner - it is the infrastructure companies (such as Nvidia) who I think will be the winners...
Five years ago autonomous cars hadn't killed anyone, so some progress made there.RobDickinson said:
someone programmed it to look at a human with a bike and think it was a plastic bag.
that in my (admittedly limited experience of 26 years software development) is bad
Id probably chalk it up to an outright failure.
Pedantry incoming!that in my (admittedly limited experience of 26 years software development) is bad
Id probably chalk it up to an outright failure.
Its probably not bad programming, as this will mostly be machine learning to identify risks.
Its more likely (I think the register identified this) that its the desire for uber to offer a comfortable ride rather than braking heavily. Its a matter of the risk profile of the software seeing the hazard and taking the risk that its a plastic bag rather than a person pushing a bike.
This is probably a rather trivial fix to make the software risk adverse and slow down / stop for any potential hazard.
Their system couldn't tell if it was a plastic bag, a loose tarp, an empty box, a pallet of bricks or a person.
A large obstacle appeared in the forward detection zone and it ignored it. The most basic of features and it failed.
Meanwhile other people are leaning heavily on high resolution mapping and GPS location which also suggests systems that are OK for 'get me from A to B in a simple environment' but a lot less hot on the realtime environment detection thing. Otherwise the basic old mapping systems would be adequate as a crutch.
A large obstacle appeared in the forward detection zone and it ignored it. The most basic of features and it failed.
Meanwhile other people are leaning heavily on high resolution mapping and GPS location which also suggests systems that are OK for 'get me from A to B in a simple environment' but a lot less hot on the realtime environment detection thing. Otherwise the basic old mapping systems would be adequate as a crutch.
tight fart said:
Another death reported on Sky today, the Tesla hit a stationary fire truck at 60mph.
Sky quote "the Tesla's bonnet was entirely crushed"
No s
t Sherlock!
Do you have a link for this? The only story I can find involving a Tesla and a fire truck states that the driver broke their ankle.Sky quote "the Tesla's bonnet was entirely crushed"
No s
t Sherlock!Mr2Mike said:
tight fart said:
Another death reported on Sky today, the Tesla hit a stationary fire truck at 60mph.
Sky quote "the Tesla's bonnet was entirely crushed"
No st Sherlock!
Do you have a link for this? The only story I can find involving a Tesla and a fire truck states that the driver broke their ankle.Sky quote "the Tesla's bonnet was entirely crushed"
No s
t Sherlock!NTSB preliminary report:
https://www.ntsb.gov/news/press-releases/Pages/NR2...
Volvo safety systems turned off.
Uber computer identified that emergency braking was required.
Emergency braking was turned off to avoid "erratic" driving and was not set up to deliver any warnings to the driver.
System relied upon human operator to emergency brake.
Driver was monitoring the software (allegedly) not the road, did not start to brake until 1 second after impact.
Quite simply staggering. I *very* much hope such lax approaches are not allowed on UK roads?
https://www.ntsb.gov/news/press-releases/Pages/NR2...
Volvo safety systems turned off.
Uber computer identified that emergency braking was required.
Emergency braking was turned off to avoid "erratic" driving and was not set up to deliver any warnings to the driver.
System relied upon human operator to emergency brake.
Driver was monitoring the software (allegedly) not the road, did not start to brake until 1 second after impact.
Quite simply staggering. I *very* much hope such lax approaches are not allowed on UK roads?
Does it not look ok?
https://www.ntsb.gov/news/press-releases/Pages/NR2...
Its registered there is a bike and if you look at the green line the car has altered its path, assuming the bike is following the lane marking
It hasnt realised the bike is travelling at right angles until 1 second before impact
Would a human have seen it or done any different?
https://www.ntsb.gov/news/press-releases/Pages/NR2...
Its registered there is a bike and if you look at the green line the car has altered its path, assuming the bike is following the lane marking
It hasnt realised the bike is travelling at right angles until 1 second before impact
Would a human have seen it or done any different?
skwdenyer said:
NTSB preliminary report:
https://www.ntsb.gov/news/press-releases/Pages/NR2...
Volvo safety systems turned off.
Uber computer identified that emergency braking was required.
Emergency braking was turned off to avoid "erratic" driving and was not set up to deliver any warnings to the driver.
System relied upon human operator to emergency brake.
Driver was monitoring the software (allegedly) not the road, did not start to brake until 1 second after impact.
Quite simply staggering. I *very* much hope such lax approaches are not allowed on UK roads?
Is the driver gonna get fhttps://www.ntsb.gov/news/press-releases/Pages/NR2...
Volvo safety systems turned off.
Uber computer identified that emergency braking was required.
Emergency braking was turned off to avoid "erratic" driving and was not set up to deliver any warnings to the driver.
System relied upon human operator to emergency brake.
Driver was monitoring the software (allegedly) not the road, did not start to brake until 1 second after impact.
Quite simply staggering. I *very* much hope such lax approaches are not allowed on UK roads?
ked for this 
simoid said:
Is the driver gonna get fked for this
In my opinion, these test vehicles which are using humans as the fail-safe emergency backup system should have something similar to the Cadillac Super Cruise driver monitoring system.ked for this It detects where the driver's eyes are looking and so crucially it ensures a driver is at least looking at the road ahead. It also doesn't require the person in the driver's seat to hold the steering wheel.
https://www.youtube.com/watch?v=RxeK0F-D3gg
simoid said:
Is the driver gonna get fked for this
He can handle jail if it comes to it.ked for this Seeing as he is a twice convicted felon already... (benefit fraud and then armed robbery).
Not to mention a string of traffic violations, you know the usual nothing to worry about, driving whilst suspended, going through a red light, speeding...
Uber on the other hand will need to demonstrate how their vetting passed him as a fit and proper person for this role, I'm sure it was more than the fact he would work for 1 cent an hour. Right????
http://www.dailymail.co.uk/news/article-5527575/Op...
Edited by hyphen on Thursday 24th May 22:22
cptsideways said:
100% at fault in my book
Driver is on their phone obviously & not paying attention, 10% attention level tops.
A driver would have seen into what appears to be shadows in the vid easily if looking
The standard Volvo with safety systems would have reacted imho, are these disabled?
The system fitted obviously does not work, or anywhere near as well at it should do.
On observant good driver "would have seen more than in the video" & at least reacted.
I think this is a very good example of what drivers will do in semi autonomous vehicles let alone fully, as we have been saying all along.
This video shows what a 2015 xc90 see's & does (though it looks a bit fakey to me) https://www.youtube.com/watch?v=xfgWE5nIyG0
Looks like I was bang on the moneyDriver is on their phone obviously & not paying attention, 10% attention level tops.
A driver would have seen into what appears to be shadows in the vid easily if looking
The standard Volvo with safety systems would have reacted imho, are these disabled?
The system fitted obviously does not work, or anywhere near as well at it should do.
On observant good driver "would have seen more than in the video" & at least reacted.
I think this is a very good example of what drivers will do in semi autonomous vehicles let alone fully, as we have been saying all along.
This video shows what a 2015 xc90 see's & does (though it looks a bit fakey to me) https://www.youtube.com/watch?v=xfgWE5nIyG0
Gassing Station | News, Politics & Economics | Top of Page | What's New | My Stuff