I use FSD every day and it has driven easily 98% of the miles on my model 3. I would never let it drive unsupervised. I honestly have no idea how they think they're ready for robotaxis. FSD is an incredible driver assistance system. It's actually a joy to use, but it's simply not capable of reliable unsupervised performance. A big reason, it struggles exactly where you think it would based on a vision only system. It needs a more robust mechanism of building it's world model.
A simple example. I was coming out of a business driveway, turning left onto a two lane road. It was dark out with no nearby street lights. There was a car approaching from the left. FSD could see that a car was coming. However, from the view of a camera, it was just a ball of light. There was no reasonable way the camera could discern the distance given the brightness of the headlights. I suspected this was the case and was prepared to intervene, but left FSD on to see how it would respond. Predictably, it attempted to pull out in front of the car and risked a collision.
That kind of thing simply can not be allowed to happen with a truly autonomous vehicle and would never happen with lidar.
Hell, just this morning on my way to work FSD was going run a flashing red light. It's probably 95% accurate with flashing reds, but that needs to be 100%. That being said, my understanding is the current model being trained has better temporal understanding such that flashing lights will be more comprehensible to the system. We'll see.
Your report matches many other real world reports I've read. I'm pretty good at day dreaming or thinking while driving, so having to keep my hands ready to take over while being completely alert that FSD might error would be a big downgrade in my driving experience. I'd rather drive myself where my subconscious muscle memory does the driving so my conscious mind can think about other things. Having to pay attention to what FSD was doing would be a drag and prevent me from relaxing.
I've used FSD a lot. Supervising it is a skill that you actively develop and can get very good at. Some argue that if you have to supervise it, there's no point, but I disagree. I still use it for much of my daily commute even though I have to supervise it and occasionally intervene. It's still a significant net positive addition to the driving experience for me overall. I would legitimately consider using it a skill that improves with practice; there's a threshold of skill where it becomes a huge positive, but below that threshold it can be a negative.
A simple example. I was coming out of a business driveway, turning left onto a two lane road. It was dark out with no nearby street lights. There was a car approaching from the left. FSD could see that a car was coming. However, from the view of a camera, it was just a ball of light. There was no reasonable way the camera could discern the distance given the brightness of the headlights. I suspected this was the case and was prepared to intervene, but left FSD on to see how it would respond. Predictably, it attempted to pull out in front of the car and risked a collision.
That kind of thing simply can not be allowed to happen with a truly autonomous vehicle and would never happen with lidar.
Hell, just this morning on my way to work FSD was going run a flashing red light. It's probably 95% accurate with flashing reds, but that needs to be 100%. That being said, my understanding is the current model being trained has better temporal understanding such that flashing lights will be more comprehensible to the system. We'll see.