Tesla has released vision Park Assist with Tesla update 2023.6.9
@EVBaymax & @ManZoneBeer
Tesla has unveiled the Vision Park Assist feature with its new software update, version 2023.6.9 for non-FSD Beta vehicles. This cutting-edge feature employs the car's cameras to measure distances to nearby objects, offering users valuable parking assistance.
When Tesla removed ultrasonic sensors (USS) from their vehicles six months ago, some owners expressed concerns regarding the loss of parking assistance. In response, Tesla embarked on the transition towards a vision-based solution, culminating in the introduction of Vision Park Assist.
Accuracy of Park Assist
Twitter user @EVBaymax couldn’t wait till morning to test out the new Vision Park Assist feature. Equipped with his Model 3 and a measuring tape, @EVBaymax put the new technology to the test and shared it all on Twitter, providing some valuable insight into its performance. In one video, he said, “super-impressive what Tesla has been able to do. This is… Wow! I’m impressed.” @EVBaymax was showing the car within an inch or two of what the reading said inside the car.
However, he did spot something less impressive. When shifting into drive or reverse after being parked for a few minutes, a message pops up that says: Park Assist is Loading. That load took 6-8 seconds as the system recalls what was around it before it was parked. The time is quite a lag compared to the USS-enabled systems. @EVBaymax is hopeful this is addressed. However, the vehicle did eventually load the data it had before it was turned off, showing the same distance to the curb that was in front of it, even though the curb was out of view of the cameras.
Several online videos show a significant difference in readings between USS and Vision. USS mostly displays smooth readings with straight edges, but vision does not display many straight lines. When backing up to a curb, @EVBaymax notes that the line representing the curb is “squiggly and is moving.”
Availability
Although Park Assist was initially included in FSD Beta 11.3.2 and limited to North American markets, Tesla is rolling out Park Assist to additional markets with update 2023.6.9.
Currently, the Vision Park Assist feature is compatible with Model 3 and Model Y vehicles. Users also have the option to turn off Park Assist if they prefer, just like owners with USS. This innovative technology offers 360-degree detection, instead of just front and rear, as highlighted in our previous article.
Park Assist Detecting a Curb
One of the advantages of vision-based Park Assist is the ability to detect objects on the side of the vehicle. @EVBaymax does a great job illustrating that in this video below.
At this time, it appears that vehicles with ultrasonic sensors still offer a higher level of accuracy, however that could depend on the height of the object and the type of object itself.
Vision Park Assist does not currently apply to vehicles with ultrasonic sensors. However, since Vision Park Assist does provide some advantages over its hardware-based version, it'll be interesting to see if Tesla incorporates it into all vehicles in the future as the feature matures.
As more Tesla owners install and utilize Vision Park Assist, the feature is expected to improve. The company will use the collected data to enhance distance estimates, aiming for accuracy on par with sensor-based systems.
Tesla's Vision Park Assist offers visual and auditory alerts for objects in the vehicle's surroundings, utilizing the occupancy network to generate high-definition object outlines. However, it is essential to remember that this feature should be treated as guidance, not as a substitute for an attentive driver.
Tesla launched two FSD updates simultaneously on Saturday night, and what’s most interesting is that they arrived on the same software version. We’ll dig into that a little later, but for now, there’s good news for everyone. For Hardware 3 owners, FSD V12.6.1 is launching to all vehicles, including the Model 3 and Model Y. For AI4 owners, FSD V13.2.4 is launching, starting with the Cybertruck.
FSD V13.2.4
A new V13 build is now rolling out to the Cybertruck and is expected to arrive for the rest of the AI4 fleet soon. However, this build seems to be focused on bug fixes. There are no changes to the release notes for the Cybertruck with this release, and it’s unlikely to feature any changes when it arrives on other vehicles.
FSD V12.6.1 builds upon V12.6, which is the latest FSD version for HW3 vehicles. While FSD V12.6 was only released for the redesigned Model S and Model X with HW3, FSD V12.6.1 is adding support for the Model 3 and Model Y.
While this is only a bug-fix release for users coming from FSD V12.6, it includes massive improvements for anyone coming from an older FSD version. Two of the biggest changes are the new end-to-end highway stack that now utilizes FSD V12 for highway driving and a redesigned controller that allows FSD to drive “V13” smooth.
It also adds speed profiles, earlier lane changes, and more. You can read our in-depth look at all the changes in FSD V12.6.
Same Update, Multiple FSD Builds
What’s interesting about this software version is that it “includes" two FSD updates, V12.6.1 for HW3 and V13.2.4 for HW4 vehicles. While this is interesting, it’s less special when you understand what’s happening under the hood.
The vehicle’s firmware and Autopilot firmware are actually completely separate. While a vehicle downloading a firmware update may look like a singular process, it’s actually performing several functions during this period. First, it downloads the vehicle’s firmware. Upon unpacking the update, it’s instructed which Autopilot/FSD firmware should be downloaded.
While the FSD firmware is separate, the vehicle can’t download any FSD update. The FSD version is hard-coded in the vehicle’s firmware that was just downloaded. This helps Tesla keep the infotainment and Autopilot firmware tightly coupled, leading to fewer issues.
What we’re seeing here is that HW3 vehicles are being told to download one FSD version, while HW4 vehicles are being told to download a different version.
While this is the first time Tesla has had two FSD versions tied to the same vehicle software version, the process hasn’t actually changed, and what we’re seeing won’t lead to faster FSD updates or the ability to download FSD separately. What we’re seeing is the direct result of the divergence of HW3 and HW4.
While HW3/4 remained basically on the same FSD version until recently, it is now necessary to deploy different versions for the two platforms. We expect this to be the norm going forward, where HW3 will be on a much different version of FSD than HW4. While each update may not include two different FSD versions going forward, we may see it occasionally, depending on which features Autopilot is dependent on.
Thanks to Greentheonly for helping us understand what happened with this release and for the insight into Tesla’s processes.
At the 2025 Consumer Electronics Show, Nvidia showed off its new consumer graphics cards, home-scale compute machines, and commercial AI offerings. One of these offerings included the new Nvidia Cosmos training system.
Nvidia is a close partner of Tesla - in fact, they produce and supply the GPUs that Tesla uses to train FSD - the H100s and soon-to-be H200s, located at the new Cortex Supercomputing Cluster at Giga Texas. Nvidia will also challenge Tesla’s lead in developing and deploying synthetic training data for an autonomous driving system - something Tesla is already doing.
However, this is far more important for other manufacturers. We’re going to take a look at what Nvidia is offering and how it compares to what Tesla is already doing. We’ve done a few deep dives into how Tesla’s FSD works, how Tesla streamlines FSD, and, more recently, how they optimize FSD. If you want to get familiar with a bit of the lingo and the background knowledge, we recommend reading those articles before continuing, but we’ll do our best to explain how all this synthetic data works.
Nvidia Cosmos
Nvidia’s Cosmos is a generative AI model created to accelerate the development of physical AI systems, including robots and autonomous vehicles. Remember - Tesla’s FSD is also the same software that powers their humanoid robot, Optimus. Nvidia is aiming to tackle physical, real-world deployments of AI anywhere from your home, your street, or your workplace, just like Tesla.
Cosmos is a physics-aware engine that learns from real-world video and builds simulated video inputs. It tokenizes data to help AI systems learn quicker, all based on the video that is input into the system. Sound familiar? That’s exactly how FSD learns as well.
Cosmos also has the capability to do sensor-fused simulations. That means it can take multiple input sources - video, LiDAR, audio, or whatever else the user intends, and fuse them together into a single-world simulation for your AI model to learn from. This helps train, test, and validate autonomous vehicle behavior in a safe, synthetic format while also providing a massive breadth of data.
Data Scaling
Of course, Cosmos itself still requires video input - the more video you feed it, the more simulations it can generate and run. Data scaling is a necessity for AI applications, as you’ll need to feed it an infinite amount of data to build an infinite amount of scenarios for it to train itself on.
Synthetic data also has a problem - is it real? Can it predict real-world situations? In early 2024, Elon Musk commented on this problem, noting that data scales infinitely both in the real world and in simulated data. A better way to gather testing data is through real-world data. After all, no AI can predict the real world just yet - in fact, that’s an excellent quantum computing problem that the brightest minds are working on.
Yun-Ta Tsai, an engineer at Tesla’s AI team, also mentioned that writing code or generating scenarios doesn’t cover what even the wildest AI hallucinations might come up with. There are lots of optical phenomena and real-world situations that don’t necessarily make sense in the rigid training sets that AI would develop, so real-world data is absolutely essential to build a system that can actually train a useful real-world AI.
Tesla has billions of miles of real-world video that can be used for training, according to Tesla’s Social Media Team Lead Viv. This much data is essential because even today, FSD encounters “edge cases” that can confuse it, slow it down, or render it incapable of continuing, throwing up the dreaded red hands telling the user to take over.
Cosmos was trained on approximately 20 million hours of footage, including human activities like walking and manipulating objects. On the other hand, Tesla’s fleet gathers approximately 2,380 recorded minutes of real-world video per minute. Every 140 hours - just shy of 6 days - Tesla’s fleet gathers 20 million hours of footage. That was a little bit of back-of-the-napkin math, calculated at 60 mph as the average speed.
Generative Worlds
Both Tesla’s FSD and Nvidia’s Cosmos can generate highly realistic, physics-based worlds. These worlds are life-like environments and simulate the movement of people and traffic and the real-life position of obstacles and objects, including curbs, fences, buildings, and other objects.
Tesla uses a combination of real-world data and synthetic data, but the combination of data is heavily weighted to real-world data. Meanwhile, companies who use Cosmos will be weighting their data heavily towards synthetically created situations, drastically limiting what kind of cases they may see in their training datasets.
As such, while generative worlds may be useful to validate an AI quickly, we would argue that these worlds aren’t as useful as real-world data to do the training of an AI.
Overall, Cosmos is an exciting step - others are clearly following in Tesla’s footsteps, but they’re extremely far behind in real-world data. Tesla has built a massive first-mover advantage in AI and autonomy, and others are now playing catch-up.
We’re excited to see how Tesla’s future deployment of its Dojo Supercomputer for Data Labelling adds to its pre-existing lead, and how Cortex will be able to expand, as well as what competitors are going to be bringing to the table. After all, competition breeds innovation - and that’s how Tesla innovated in the EV space to begin with.
![Elon talks about Twitter and job cuts at Bloomberg forum [video]](https://www.notateslaapp.com/images/news/2022/elon-musk-qatar-forum_300w.jpg)
