A Friday night cruise with Elon Musk has shown the world just how far Tesla’s Full Self-Driving has advanced and gave viewers a preview of real-world problems. To take a line from Rob Zombie, the latest iteration of FSD may be "More Human Than Human," which could be an issue.
"Nothing but net!" proclaimed Musk as he live-streamed to X.com, a video with more than 42 million views. The Tesla CEO referred to neural nets, the technology underpinning Version 12 of FSD. Tesla is removing 300,000-plus lines of code running the current FSD and is now relying on the car's neural networks to drive the car in V12.
Driving through Palo Alto, California, Musk, with Ashok Elluswamy, Tesla’s Director of Autopilot Software, sitting shotgun, picked random places for the Model S to drive to, and it did so smoothly and almost perfectly.
Technology Behind FSD
The test drive started with a construction zone, which was easily handled, but interestingly, at the end of the cones, the car wasn’t jerking over to find a lane immediately. Instead, like a confident human driver, it easily made its way to the lane with no oncoming traffic. We also watched as it effortlessly navigated speed bumps, roundabouts, pedestrians, bicyclists, and, of course - traffic.
For those who have had successful drives with FSD, you may be thinking, what's the big deal? Musk underscored that this version of FSD does not rely on hard-coded programming or a constant internet connection to function. All necessary inference occurs locally, a significant departure from previous versions.
“This is all nets, baby, nothing but net,” Musk exclaimed, emphasizing the vehicle’s dependency on machine learning rather than programmed instructions. The software operates at full frame rate, taking eight cameras at 36 frames per second to decide what to do. No line of code tells the car what to do at an intersection, on the highway or around other vehicles.
The software was impressive to watch, with its countless improvements over the current V11 version of FSD. Musk's vehicle kept a reasonable speed throughout the drive, even though Autopilot was set to a maximum speed of 85 mph.
When Musk reached his destination, the vehicle also automatically pulled over to the side of the road to park itself instead of just stopping in the middle of the road like it does today.
The Real-World, Real Person Challenge
Instead, Tesla has shown the car countless videos of real-world driving so that it will learn what to do in those situations. But that has caused a problem. As the car noticeably did not come to full stops, Musk explained the challenge posed by the fact that less than 0.5% of drivers come to a complete stop at stop signs.
This makes it incredibly difficult to program the vehicle with such sparse data on people obeying the law. This lack of data highlights Tesla's obstacles in training its AI systems to operate in real-world scenarios where human drivers frequently do not adhere to the rules.
Musk's FSD V12 Stream in 10 Minutes
Watch the highlights of Musk's 45-minute video stream below.
Regulatory Hurdles
This appeared in a few other instances where the car behaved like a confident driver, not endangering anyone but technically breaking the rules of the road. FSD suddenly went from the nervous teenager learning to drive to a good driver. But we know how the NHTSA feels about Tesla not following the rules.
In February, the watchdog issued a recall stating that Tesla was acting too human – I mean not following the rules. It cited FSD Beta software:
Traveling or turning through certain intersections during a stale yellow traffic light;
The perceived duration of the vehicle’s static position at certain intersections with a stop sign, particularly when the intersection is clear of any other road users;
Adjusting vehicle speed while traveling through certain variable speed zones, based on detected speed limit signage and/or the vehicle’s speed offset setting that is adjusted by the driver.
Yes, I’m sure we all stop at every yellow light, come to a complete stop at every stop sign, and never go over the speed limit.
Kudos to Tesla and Musk for this real-world demonstration, and if there was any speculation that it was rigged, Musk did have to intervene at an advanced left-hand signal. This small glitch, Musk reassured, could be addressed with further training and data collection. There will be a lot of training announced Tesla’s commitment to invest $4 billion over the next two years in training computing, signaling the company's dedication to perfecting this groundbreaking technology.
