Classic mistakes in language design that have to be fixed later.
- "We don't need any attributes", like "const" or "mut". This eventually gets retrofitted, as it was to C, but by then there is too much code without attributes in use. Defaulting to the less restrictive option gives trouble for decades.
- "We don't need a Boolean type". Just use integers. This tends to give trouble if the language has either implicit conversion or type inference. Also, people write "|" instead of "||", and it almost works. C and Python both retrofitted "bool". When the retrofit comes, you find that programs have "True", "true", and "TRUE", all user-defined.
Then there's the whole area around Null, Nil, nil, and Option. Does NULL == NULL? It doesn't in SQL.
That's what's nice about coarse-grained feature options like Rust's editions or Haskell's "languages", you can opt in to better default behavior and retain compatibility with libraries coded to older standards.
The "null vs null" problem is commonly described as a problem with the concept of "null" or optional values; I think of it as a problem with how the language represents "references", whether via pointers or some opaque higher-level concept. Hoare's billion-dollar mistake was disallowing references which are guaranteed to be non-null; i.e. ones that refer to a value which exists.
Caltrain, from SF to SJ, is part of the California high speed rail system, and you can ride it right now. It's now electrified at 25KV, welded rail, concrete ties, and compatible with high speed rail. The Stadler trains are capable of 125MPH but are run slower because there are so many stations.
Self-driving vehicles need aircraft-type maintenance. Yet there's nothing like the FAA to enforce a minimum equipment list, maintenance intervals, or signoffs by approved mechanics.
Is there a scratch or chip in the scanner dome? Are both the primary and backup steering actuators working? Is there any damage to the vehicle fender sensors? Is dispatch allowed with some redundant components not working? If so, for how long?
Here's the FAA's Minimum Equipment List for single-engine aircraft.[1] For each item, you can see if it has to be working to take off, and, if not, how long is allowed to fix it.
There's nothing like that for self-driving land vehicles.
What's the fleet going to look like at 8 years of wear and tear?
> Self-driving vehicles need aircraft-type maintenance.
That's a hyperbolic false equivalence.
Aircraft typically carry hundreds of people and can crash to the ground. As long as a self-driving car can detect when it is degraded, it can just stop with the blinkers on. Usually with 0 - 2 people inside.
The question is how broken can a car be when dispatched. What's the safe floor? See the other article today about a Tesla getting into an accident because of undetected sensor degradation.
> Aircraft typically carry hundreds of people and can crash to the ground.
Cars are more numerous and could spontaneously either plow into pedestrians, or rear-end someone, causing chain damage and, quite often, a spillage of toxic chemicals (e.g., a cistern carrying acid/fuel/pesticide).
Plus, you have a problem of hostile actors having easier access to cars compared to planes.
The web sites seized are listed in the DOJ release.[1]
One of them is archived. It seems to be primarily about Albania.[2][3] Albania and Iran are not currently getting along; they severed diplomatic relations in 2022. This seems to be from an anti-Albania organization. Looks like an ordinary extremist site. Unclear why the US DOJ is involved with that one. Doesn't seem to involve the US directly.
One of the other sites is associated with the Stryker cyber attack, so that shutdown makes sense.
"In the crashes that ODI has reviewed, the system did not detect common
roadway conditions that impaired camera visibility and/or provide alerts when camera performance had deteriorated until immediately before the crash occurred."
Does it not detect them at all, or fail to deal with detected sensor degradation adequately?
Does "Full Self Driving (assisted)" slow down under conditions of poor visibility?
Does Tesla even look for the road surface? One big advantage of those up-top LIDAR units is that you have a good scan of the pavement ahead. If you're not sensing flat pavement ahead, don't go there. That's basic. Vision-only systems, going back to Mobileye, have been overly dependent on looking for known kinds of obstacles. Original Mobileye could only detect car rear ends.
I own two Tesla’s. When conditions are adverse, i.e. fog, heavy rain, the system simply shuts off and reverts back to manual driving. Elon has said several times that humans can drive with two eyes and Tesla should be able to drive with X number of cameras. however, it suffers from the same problems humans do: if it can’t see it can’t drive and ironically that’s when it reverts back to human control.
I definitely agree that in principle a computer can drive with cameras alone. I don't know whether it's a useful statement. Like a human can determine the genre of a movie merely by watching it. I wouldn't suggest to blockbuster in 1990 that they should collect no genre metadata for movies because the database server should automatically sort it out on its own. (Nowadays somewhat feasible with ML of course, but 20+ years later.) What sensors/data you need is a question of where computers are now or will shortly be, and it seems that for now they need the extra structure of LIDAR for best effectiveness.
>I definitely agree that in principle a computer can drive with cameras alone.
Obvious things first, cameras have way worse contrast and low light sensitivity than human eyes.
Humans have much more evolved logical thinking capacity, even the stupid ones can figure stuff out that modern AI struggles with.
Humans have other sensors, too that they use to plausibility check the picture they see. I.e. one of the best sensor fusion systems on the planet.
When in doubt humans can figure out whether it's a lens occlusion or a some other artifact in their vision by virtue of moving their head around.
There's probably other things I'm not thinking of. In any case to make full self driving work we should first start by using all available tech to make it safe. When you have safe tech you can slowly start removing individual sensors while verifying that safety remains high. As the experience and system evolves there will be optimization potential.
And until we have that low light thing and high contrast figured out, camera alone doesn't cut it.
Right, but if these things are so rare that we all only know the one viral example, I feel like that lends credence to the models basically generally not having this problem.
Researchers built the Winnograd Schema Challenge more than a decade ago to assess common sense reasoning, and LLMs beat that challenge task around GPT 4.
They're not so rare. Hallucinations have been spotted everywhere, but the "driving a car to the car wash" is an amusing one that's been recently publicised. Developers aren't going to point out every time an LLM hallucinates an entire library.
I'd add to this, any moderately involved logical or numerical problem causes hallucinations for me on all frontier models.
If you ask them in isolation they may write a script to solve it "properly", but I guess this is because they added enough of these to the training set. But this workaround doesn't scale.
As soon as I give the LLM a proper problem and a small part of it requires numeric reasoning, it almost always hallucinates something and doesn't solve it with a script.
If the logic/math is part of a larger problem the miss rate is near 100%.
LLMs have massive amounts of knowledge, encoded in verbal intelligence, but their logic intelligence is well below even average human intelligence.
If you look at how they work (tokenization and embeddings) it's clear that transformers will not solve the issue. The escape hatches only work very unreliably.
If you ask this of any current day AI it will answer exactly how you would expect. Telling you to drive, and acknowledging the comedic nature of the question.
That's because AI labs keep stamping out the widely known failures. I assume without actually retraining the main model, but with some small classifier that detects the known meme questions and injects correct answer in the context.
But try asking your favorite LLM what happens if you're holding a pen with two hands (one at each end) and let go of one end.
Not unlikely that you're talking to a lot of AI-based AI boosters. It's easier to create astroturfed comments with chatbots than fixing the inherent problems.
Nice. My test was always a blond bald guy. It always adds hair. If you ask for bald you get a dark haired bald guy, if you add blond, you can't get bald because I guess saying the hair color implies hair (on the head), while you may just want blonde eyebrows and/or blond stubble.
Well that, but Elon is also downplaying the quality of the human vision system compared to the cameras Tesla's have.
They're just not that good - nowhere near human vision performance. And a human in a car has a surprisingly good view of the road and a very fast pan tilt system to look around.
Tesla's do not actually have 360 degree full binocular vision coverage - nor the ability for a camera to lean left or right to improve an ambiguous sensor picture.
So while I fully believe that vision only self driving could work, within the constraints of automobile platforms and particularly the Tesla and it's current camera deployments, it is not remotely similar enough to human visual fidelity for that to solve a valid argument.
Tesla’s actually have zero binocular vision coverage because the cameras have different focal lengths and are too close even if they did have the same focal lengths.
They are also below minimum vision requirements for driving in many states.
> When conditions are adverse, i.e. fog, heavy rain, the system simply shuts off and reverts back to manual driving.
I also own a Tesla, and there is no indication shown to the user that FSD's vision is degraded. They need to add this in.
For example, numerous times I have been driving my Tesla with FSD activated with ostensibly a clean and clear windshield when suddenly the car will do the "clean the windshield in front of the camera routine" without any indication that the car's camera is degraded. If people haven't seen this "clean the windshield routine", the wiper fluid is dispensed and the wiper will vigorously wipe in front of the camera only -- the rest of the windshield only gets a cursory wipe.
This indicates to me that the camera has poor visibility and I am not informed or aware of this as a driver, which is concerning. I am often curious if there is a thin occluding film on the windshield in the camera box in front of the camera, or something that has degraded FSD's vision, but they do not give you the ability to view the camera feed, nor do they notify you that the vision is degraded. I think a "thin occluding film" may be in the camera box because my normal windshield outside of the camera box started to show a thin chemical film after a couple of months, which apparently (according to a Google search) happens when a new car off-gasses, adding a thin film of chemical byproduct to the windshield. This is my first new car so I've no idea if this is normal or not.
> yes it does, and it's annoying as all hell. Dirt, sun, etc all pop an alert about degraded performance
As with all things FSD, it does sometimes and not others. I've driving my parents' Tesla with FSD engaged and it did complain when the windshield got dirty but didn't say anything when it drove into fog. (I took over manually.)
Out of curiosity, was the camera view compromised? I would probably take over too, but like the poster above, I get the warning in all kinds of conditions.
Absolutely could be a clouded windshield on the inside (where it's really hard for normal people to clean). I brought this up when I got my last Model Y that it was foggy and they said it was "fine". Took it into service over a year ago and noticed they cleaned it. Clearly it's a problem but they're not being too transparent about it. I suspect they don't want to because it's not the easiest thing to remove the cover for normal people to clean.
Recent Tesla updates will detect dirty glass inside the camera enclosure and offer to schedule (one!) free glass cleaning. You can do it yourself if you have a trim tool. (A thin plastic prybar) https://www.notateslaapp.com/news/3327/tesla-now-offers-free...
I've always hated this argument. Why should I want a system that can drive "just as any human driver can"? I want it to be much much much better than the best driver out there, like 100x or 1000x better.
That argument is dumber and dumber any time I think about it. And we haven't even gotten into the fact that human eyes and its partner in crime the brain work much different than a camera.
That’s how Musk works. He waves his hands and uses words like “orders of magnitude” and “first principles” and then you end up with 250-meter long tunnels under your city with Teslas driving back and forth in it and fanboys forget this thing called subways ever existed.
> humans can drive with two eyes and Tesla should be able to drive with X number of cameras
Systems built from cameras that are only nearly as capable as human eyes and software that is only nearly as capable as the human brain will fall short overall. To match or surpass human performance, the individual components need to exceed human abilities where possible--and that's where LiDAR provides an advantage.
> the system simply shuts off and reverts back to manual driving.
That's not good enough. Too many accidents at manual takeover. The new standard, which Mercedes has demonstrated and China is mandating, is that the system must be able to pull the vehicle over and stop safely when there are problems.
My Lexus does this too. I rarely get it due to weather however it’s how I know I’m past due for a car wash (dust on sensors)
In any case, it seems reasonable to me that the human should be making the decisions once conditions become adverse. It’s a simple liability issue for the car company but also I’d rather trust my own judgment if it’s only 80% certain it’s not driving me off a cliff.
One of the coolest features I saw like this was on a Jaguar XJL I had recently, that had an air particulate sensor and would automatically switch to recirc cabin air when that count was too high (i.e. dusty / smoky conditions).
yea, when it rains the world
stops and we all sit home and wait for the Sun to make an appearance. coolest part is that some places in the US get like 200+ rainy days and you get to stay home cause you have no choice, schools closed etc :)
> There isn't some kind of god riven right to transportation, it is always conditions permitting.
If the condition is a little fog and little rain and little snow/sleet I hate to break it to you but those are very permitting. In most of the continental US the number of days where driving conditions for an (below)average human and such that it is wiser not to get on the road is very small. If the "robo"taxi technology you posses cannot match that of a (below)average human you got nothing but vaporware you've been pitching as "done deal" for more than a decade.
> Well, even robotaxi's can't beat the laws of physics. There isn't some kind of god riven right to transportation, it is always conditions permitting.
> Elon has said several times that humans can drive with two eyes and Tesla should be able...
And this is an amazingly stupid statement. Humans drive with most of their senses, not just vision. In fact our proprioception plays an important role in driving.
Even Tesla's use of cameras is poor because they're monocular and fixed in place. Most humans have binocular vision and those visual sensors have multiple degrees of freedom and the ability to adjust focus.
Even if you wanted to only use vision for navigation it's irresponsible to not use binocular configurations that get more reliable depth sensing.
>> "Elon has said several times that humans can drive with two eyes and Tesla should be able to drive with X number of cameras"
This must be one of the most stupid takes that gets repeated non stop by Tesla fans.
I just don't get it. Humans also have emotions and other biological senses that Computers don't have. You just cannot compare both.
What makes human so good at driving is that they can react relatively well to unknown new events. Teslas cannot do that, and with the current hardware never will.
Elon is deeply involved in engineering decisions in his companies, and has by all accounts deep knowledge in those areas.
And yet randos on the web keep asserting he's not an engineer. Is there any factual basis for this? Is it just that he doesn't have a degree with that word in the title?
Being an engineer is neither having a degree in engineering which he doesn't have nor managing them and it certainly isn't owning a company that employs them. It's working as an engineer.
He continually says dumb things that aren't true or reasonable and has never worked in the field he's a rich boy who bought things with daddy's aparteid money.
> Does "Full Self Driving (assisted)" slow down under conditions of poor visibility?
Under conditions of poor camera visibility?
Humans drivers can see under conditions that cameras cannot, and people will otherwise misinterpret “visibility” as referring unpredictably (and with personal biases) towards either human sight and/or camera processing and/or lidar processing.
I think vision-only can certainly work for 99.9% of driving.
But it's that 0.1% of situations where the results will be catastrophic. Sure, you can detect vehicles, traffic cones, bikes (both bicycles and motorcycles), people, mopeds, traffic lights, lane markings, everything you'd expect on a road.
But what about the mattress that fell out of someone's truck? If the car doesn't know what a mattress is and what it looks like, it can't really adequately determine its size based on the monocular vision that Tesla has. Sure, maybe it could use motion vectors between video frames to make a guess, but I'm not convinced that's going to work well, especially relative to LIDAR.
Steering back to the subject at hand...
> "In the crashes that ODI has reviewed, the system did not detect common roadway conditions that impaired camera visibility and/or provide alerts when camera performance had deteriorated until immediately before the crash occurred."
I don't think I've ever had my Tesla disable Autopilot based on road conditions, though maybe it's because when conditions are bad, I've just taken manual control preemptively. I've let it go through construction areas where cones are guiding traffic outside the painted lines, and surprisingly, it's handled it fine, though I've only done this at low speeds (~20 mph).
Camera visibility is another story. In heavy rain at night, I've had it not allow me to enable AP, though I've never had it disable AP and tell me to take control. However, it HAS limited the cruise speed based on visibility.
All this to say...
...anybody buying Tesla's FSD is being swindled, as far as I'm concerned. "FSD (Supervised)" is a scam. If you have to supervise it, it's not self-driving. It's just a party trick that you have to watch to make sure nothing goes wrong.
99.9%? I'm not an expert on climate, but I would guess that at minimum 50% of the world faces snow or fog or heavy rain while driving at times. In some places, 30%+ of all driving year-round could be in snow-inclusive conditions.
99.9% of driving of sea-level, non-rainy, near-the-coast California/Austin weather, maybe. I would guess it's a no-go in the inland foggy conditions in CA, for example, or freezing rain in TX.
I once saw a presentation I think from one of the Argo ai guys with a greatest hits reel of the long tail of driving. One of them was a stake bed truck with a bunch of pigs in the back, and the back gate opened up and the pigs were falling onto the highway and running around injured. Most people will experience something at this level of unusualness at least a few times in their lives, so you have to be prepared to handle it.
> vision-only can certainly work for 99.9% of driving
Human vision only? Sure. Cameras only? I'm sceptical of the quality of these cameras. Particularly given, unlike the human eye, they don't do the rapid involuntary movements that make human eyes ridiculously robust.
The difference between HW4 and older Teslas (before 2023) is night and day. HW4 absolutely can handle 99.9% or more of driving now whereas the older models drive like an idiot. I've never had to intervene on HW4 yet. It did well avoiding debris and animals, even seeing them heading towards the road. Worked well in heavy rain and snowstorms, even when lanes were barely visible. Handled moving and parked emergency vehicles with lights on correctly. Navigated through roads undergoing work (lanes shifted, closed off, pylons everywhere). Automatic parking actually works but not all the time yet.
No software update is going to give working FSD to all those older models on the road. I don't think they can even have HW4 retrofitted either. Ideally FSD would be disabled on those cars because it's confirmed never going to work properly but people paid for lifetime FSD on them (newer cars are subscription only).
Even if 99.9% is true (and there’s no reason to think it is), that’s nowhere near reliable enough to match Waymo or do truly autonomous driving. The average human is much better than 99.9%.
That's just me being pessimistic because I've seen how terrible FSD is on older Teslas, and only have personal experience using it in HW4 for ~2 months.
It's really unfortunate that FSD is judged as a singular feature when it is vastly different depending on the car. HW3 cars are getting cut down versions of FSD software, which is useless garbage because it drives like an idiot. HW2 and earlier are probably not getting FSD updates at all anymore. The majority of Teslas on the road are HW3 or earlier, run significantly worse (unsafe) FSD versions, and yet Tesla still allows it to be turned on.
Yes. They use an occupancy network which segments the environment into drivable and non-drivable space. This has been shown in patents and company presentations.
As for road surface, yeah, that is how it should be (as you describe). Higher end Audi model/trims do road surface scanning and will adjust ride height, vehicle camber, even (to the greatest extent possible) help you avoid potholes while maintaining lane positioning. This is - or should be - table stakes.
For some problems, it is. Web front-end development, for example. If you specify what everything has to look like and what it does, that's close to code.
But there are classes of problems where the thing is easy to specify, but hard to do correctly, or fast, or reliably. Much low-level software is like that. Databases, file systems, even operating system kernels. Networking up to the transport layer. Garbage collection. Eventually-consistent systems. Parallel computation getting the same answer as serial computation. Those problems yield, with difficulty, to machine checked formalism.
In those areas, systems where AI components struggle to get code that will pass machine-checked proofs have potential.
VPN is like SSL some time ago (and there were times when a browser would come without SSL, and you'd have to explicitly download it yourself) - it quickly becomes a basic necessity even in civilized societies, let alone say Russia, Iran and the likes.
So you mean I can trust an American corporation that ships its software with telemetry on by default and has a history of data-mining its users more than my standard ISP?
Ladybird(alpha) cannot come soon enough.
Mass production of engineered structural lumber.[1]
[1] https://www.youtube.com/watch?v=RCYn3xQ0yS8
reply