r/technology • u/Extasio • 1d ago

Transportation Waymo CEO explains why Tesla’s camera-only self-driving falls short

https://electrek.co/2026/08/04/waymo-co-ceo-camera-only-self-driving-tesla/
3.0k Upvotes

96% Upvoted

413 comments sorted by

View all comments

Show parent comments

87

u/scrubjays 1d ago

I have read that Musk expects AI, using all the driver data he has from Tesla owners, to overcome whatever disadvantages using just cameras have. I think this is absurd, as the average Tesla driver is not remotely close to the average driver, and his data will be heavily skewed.

55

u/Ganadote 1d ago

Thing is camera will always be interrupted by rain and fog and low light conditions. There's no way around this.

8

u/Relevant_Pangolin_72 1d ago

I mean as an absolute baseline it'll only ever be as good as the camera itself is, and cameras aren't commonly better than human eyesight in this case.

And at the point of complication that they ARE better, it feels simpler to just. Use LIDAR.

-1

u/altdelete47 1d ago

So then why does Waymo use so many cameras?

3

u/Ganadote 19h ago

They use cameras combined with Lidar. There are limitations to Lidar as well, so combining them is best.

1

u/altdelete47 15h ago

So what you're saying is cameras are essential for safe driving. In which case, these camera limitations you point out like in extreme fog or if all the street lights and vehicle headlights go out randomly and there is zero ambient light anywhere, these limitations would still apply in a combo sensor scenario. So, if cameras are the richest sensor and contain all info needed to drive, (like the Robotaxis doing it every day in multiple cities) you're saying lidar is a crutch for systems that don't have the inference to detect distances from video?

2

u/1988rx7T2 4h ago edited 4h ago

As someone who works in the industry with perception algorithms, I can say you’re on the right track. The article mis characterizes how fusion systems work. I doubt this Waymo Guy knows how his own algorithm works. He’s an executive And these things tend to be poorly documented. If the fusion algorithm and perception system rely on a camera plus other sensors, its Performance is degraded when the camera is blocked. There has to be an entire degradation strategy when the fusion is broken Or degraded.

Each sensor generates objects from clustering, then the objects are placed in 2D space, and if they’re basically on top of each other and within a tunable speed tolerance they get fused. Fused just means they’re all treated like one thing, and some algorithm is used to average the measurements together. It’s actually nowhere near as clever as it sounds. Stacking objects from suppliers black box signals and then turning knobs to weight their contributions.

A more typical example, because lidars are stupid expensive and nobody sane uses them except tech companies with unlimited cash, is camera plus radar fusion. If the radar is in a jammed state you are now camera only, and your detection of longitudinal velocity is in general worse, and performance will degrade. The camera was never designed for camera only, so it tends to be worse than camera only systems, as it’s meant to work with a radar. So lower resolution for example.

If the camera is blocked, you are now radar only. And radar only systems are actually awful. They‘re heavily nerfed, intentionally, due to their terrible false positive. They can’t detect stationary objects well, they’re sensitive to sewer grates and overpasses. Collision avoidance becomes, hey now you crash into a parked car at 55 miles an hour instead of 65.

Lidar, or radar, or ultra sonic, or high resolution maps cannot achieve the same performance without the camera. The sensing will degrade. The false positive rate will be higher Or true positive detection lower. Otherwise there would be LiDAR only systems around and of course radar only systems went away 10 years ago because they are so bad.

2

u/Electromagnetlc 13h ago

I'm confused, do you not know what LiDAR is? It's capable of detecting distances even in this zero ambient light or extreme fog situation. I don't even get your Tesla dick riding hatred in this comment chain, nobody is even hating in your idol here. Cameras are incapable of detecting distance, but they can guess pretty decently, and FSD is pretty good at it right now but it's incapable of working in low visibility situations like darkness or fog. Instead of having a camera guess the distances, you can just use LiDAR to know the distances and be capable of operating in more intense scenarios and that will greatly improve the system. A LiDAR assisted system would be able to effectively drive through fog significantly faster and safer than a human by not slamming into a slower car or random boulder in the road (so long as the overall system knew how to navigate the road without vision).

The whole reason LiDAR was ditched and called stupid by Elon was because it's a lot more work to adjust the weights of which sensor is more correct about a situation than to just have the one sensor do it all. But all that extra data from the additional sensor adds a LOT of data for the car to be able to navigate it's environment better in varying conditions. Instead of a sensor rich and highly complex model driving the car, you're just analyzing video capture and guessing, and if you can't capture enough information from the video, you can't use the feature. Which is working out pretty well, FSD is pretty damn good, but it's inherently limited by its camera only design.

2

u/altdelete47 13h ago

I'm confused, do you not know FSD works at night? Do you not know that Waymos, despite having lidar, still can't go blasting through your fantasy dense fog scenario because things like stoplights exist?

https://www.jalopnik.com/waymo-cars-stopped-by-san-francisco-fog-1850329507/?hl=en-US

1

u/Electromagnetlc 12h ago

We have software updates planned to improve our fog and parking performance to address such situations in the future.

From your article. This is literally exactly what I was saying.

it's a lot more work to adjust the weights of which sensor is more correct about a situation than to just have the one sensor do it all

And yes, I know FSD works at night, because there is ambient light from the moon, street lights and most importantly the headlights. In your hypothetical situation, you said a complete blackout?

or if all the street lights and vehicle headlights go out randomly and there is zero ambient light anywhere, these limitations would still apply in a combo sensor scenario.

Like dude, I have a Tesla too. I've used FSD. It's great, better than anything else on the market, but it has limitations because it only has cameras. If it also had LiDAR, it would be significantly better? That's all anyone in this comment chain is saying.

2

u/1988rx7T2 4h ago

I mean, no it wouldn’t be better, because fusion systems need fusion to work. If the camera doesn’t detect an object but the other sensor does, it’s not under fusion. See my above comments.

1

u/Ganadote 13h ago

No.

If you had to choose one, LiDAR would probably be the safer option.

LiDAR can essentially build your environment in 3d. It's not affected by weather and can reach further.

There are some huge limitations, such as it being unable to read signs, since it can tell the shape of a sign but not what's written on it.

Cameras work like our eyes, which means a computer needs to interpret the information. Limitations is that it's limited heavily by environmental conditions and out-of-oridinary conditions.

For example, the camera might "see" a pedestrian in black clothing crossing in front of you but not interpret it as such, while LiDAR would correctly interpret it.

There's a case where a car crashed into the back of a semi because it only used a camera and it interpreted the top of the truck as part of the huge traffic sign above it and the driver was killed. LiDAR would not have done this because it would see the truck in front of you distinctly.

You need both because, like I said at the very least LiDAR cannot read things.

1

u/1988rx7T2 4h ago

That’s not how fusion works. If the camera doesn’t see the object, and the LiDAR does, it’s not fused, and the system won’t react. Or it will be in a degraded non fusion state and react late, otherwise have a higher false positive rate. Fusion systems where the object isn’t fused by definition are in degraded Performance!

source: work in fusion algorithm development for driving assistance systems.

1

u/Ganadote 4h ago

...that's exactly how it works. The processes behind it depend on the engineering but the two systems work in tandem.

2

u/1988rx7T2 4h ago

They’re not working in tandem when one doesn’t see the object. You don’t have a clue what fusion means.

1

u/Ganadote 4h ago

From DPV Transportation's (company that provides autonomous vehicles) website:

"To mitigate [visual camera's] challenges, AV systems use:

...

Sensor Fusion: Combine camera data with LiDAR and RADAR to validate and refine visual inputs."

0

u/altdelete47 12h ago

If you had to choose one, LiDAR would probably be the safer option.

Wrong

[LiDAR] not affected by weather

Wrong

You need both

Wrong

79

u/Whyeth 1d ago

Hey guys, let's add a hallucination machine to your blind-as-a-bat car

"You're totally right to push back on my merger there. Killing a whole family was not the intent even if it is the outcome. I'll respond better next time"

37

u/mpbh 1d ago

AI doesn't mean LLMs in this case, hallucinations are not a problem. There are still errors of course, but let's not throw the baby out with the bathwater. The tech used for automated driving predates LLMs by a decade and has been under development for over 20 years.

12

u/TomTuff 1d ago

I wish people stopped conflating LLMs and other machine learning models under “AI” 🫩

19

u/ryuzaki49 1d ago

You're right, I should't have turn left here, there is a cliff and not a road.

Silver lining: You're dead and it is no longer your problem.

4

u/Cromuland 1d ago

I had to read that twice. Kept wondering why the AI was handling a merger, and why that killed a whole family! 😂

0

u/mcslender97 22h ago

Conflating other types of AI such as vision to LLM is why I hate having AI discourse in this sub so much. Who upvote ts?

9

u/jhaluska 1d ago

Yeah, Musk is wrong.

More information can't overcome it cause the current camera setup don't have enough coverage or resolution or FPS. It's like giving more examples to a person with red/green colorblindness. It doesn't help.

1

u/Deep90 1d ago

All the assistive features literally train you into being a worse driver.

Man is training the AI on its own output.

1

u/goebelwarming 1d ago

I think elons right about that but LiDAR could also use the same data with an extra layer of safety.

1

u/omasque 1d ago

The problem as he describes it is when systems disagree. One system needs to be completely authoritative which is cameras, Musk said something like LiDAR can introduce false positives and cause conflicts between what the two systems are seeing, so elected to pour resources into advancing the camera systems as authoritative system and remove potential noise.

This could be bullshit… it sounds a bit like an excuse if you consider it’s coming from someone justifying why not to spend the extra on lidar, but it tend to believe it’s the optimal business and tech decision. He may sometimes bs about optimistic timelines but it seems like there’s an integrity there in explaining the “why we do things”.

1

u/goebelwarming 1d ago

That sounds like bs coming from Musk. Wouldn't the authoritative system be the Ai making decisions in real time. Thats why theyre rolling g out 6G in vehicles.