Tesla uses a combination of map data and vision data for FSD visualizations
@ArthurFromX
Tesla’s FSD visualizations are admittedly one of the coolest parts of owning a Tesla. Seeing vehicles all around you on the screen, and watching the car make decisions can be mesmerizing.
However, have you ever wondered exactly how the car generates the visualizations? Is it real-time, or does it rely on map data that is downloaded separately from the vehicle’s firmware?
Map Data
Tesla distributes map data to vehicles worldwide separately from its vehicle firmware versions. This means that the map data updates can have a different tempo than the firmware updates. For instance, some cars on 2024.8.9 (an FSD V11 version from ~3mo ago), as well as cars on 2024.14.7 (an FSD V12 version from just a few days ago) just started receiving the NA-2024.8-14924 map update last week. These updates ensure that vehicles can get the most current information about road layouts, traffic patterns, and other critical driving data.
While FSD can usually work with different map versions, how much does FSD and the in-car visualizations rely on the predefined map data? It obviously uses it for some things such as speed limit, but does it use it more than that? Thanks to user, ArthurFromX on X, we now have a better idea of how much relies on map data.
FSD Visualizations
Tesla’s FSD visualizations rely on a combination of pre-mapped data and real-time information gathered by the cameras. This approach provides both background context (pre-mapped data), as well as real-time context via computer vision.
It all comes together to provide an accurate representation of what the car sees around it, taking into account the width, height, and length of nearby vehicles, curbs, and other objects, such as garbage bins or traffic cones.
A post on X highlighted that FSD visualizations might be more impacted by background data than initially thought. The post mentioned that construction to a local roundabout rendered the visualizations inaccurate, although the car was correctly navigating the lane and making its exit.
This suggests that while real-time data is crucial, the accuracy of pre-mapped data still plays a significant role in the overall effectiveness of FSD visualizations, and perhaps a lesser role in the actual decision-making process of FSD.
FSD Navigation
Of course, these visualizations impact more than just what you see on screen. The mapping data and real-time data provide information on how the vehicle plans its path, and how it makes driving decisions, such as moving into right, or left-turn lanes as required.
Mapping data most likely provides background information, enabling the onboard hardware to process and work on driving the vehicle with an initial idea of where it is and what the road looks like.
In short, map data plays a backseat role to the actual driving of FSD, but does play a bigger role in route decision, as well as providing context and predictions to what the vehicle sees. It seems at the moment that visualizations are based on a combination of map data and camera data the vehicle gathers.
During Tesla’s shareholder meeting, Elon Musk was asked if FSD transfers could return for existing owners. He hemmed and hawed, but eventually gave in when an audience member asked if they could return for just one more quarter.
Final Chance?
This could be the last chance for FSD owners to transfer their FSD capability to a new vehicle. Elon Musk mentioned that it removes revenue from upcoming vehicle sales. Every time Tesla offers FSD transfers, it impacts Tesla’s potential earnings.
On the flip side, it does incentivize current owners to upgrade to newer vehicles, especially if it’s a limited-time offer. Tesla previously offered FSD transfers in Q3 of 2023. FSD transfers are expected to be limited to North America where FSD software is currently available.
When Will Tesla Offer FSD Transfers
Musk nor Tesla mentioned a timeline for when FSD transfers would begin. Tesla owners looking to upgrade their vehicles would likely wait for this offer to be available before buying their new Tesla. Since this would reduce some vehicle sales, it seems that Tesla wouldn’t want to wait too long before making FSD transfers available again.
FSD Transfers in the Future
While Elon Musk didn’t seem to be too enthused with the idea of continuing transfers but gave in to “just one more quarter”, there could be more down the road.
Musk has said that Tesla has various demand “levers” they can use to increase or slow demand as needed. FSD transfers and Tesla’s referral program are prime examples of those levers. Although Musk says Tesla will only offer FSD transfers one more time, Tesla won’t hesitate to offer it again if they need to increase sales. Tesla faced this situation when the new Model 3 was unveiled in China, but it took several months for it to be offered in North America.
Additional FSD Trials
Elon Musk confirmed that two free weeks of FSD would be given to all vehicle owners whenever a major FSD update hits. We don’t expect another trial of FSD with the upcoming update of v12.4, but perhaps with v12.5 or 12.6, especially if Tesla adds new features such as ‘Actually Smart Summon,’ which will be the first time Smart Summon is available on a vehicle without ultrasonic sensors.
Did you miss the Shareholder Meeting? Want to know what Elon Musk said?
We’ve got an easy-to-read recap for you below, split by sections. We’ll be writing more in-depth about many of these topics over the upcoming days as we unpack all the awesome news!
Robotaxi
Fleet monetization planning is ongoing.
Owners will be able to send their cars to the Robotaxi fleet and summon them back when they need them.
Tesla will take a margin, but most of the income will go to owners.
Tesla has plans to solve driverless interventions in the future, but expects them to be edge cases of edge cases.
FSD
Headed towards Unsupervised FSD!
New subversions are essentially new full versions; 12.4 and 12.5 are massive updates.
Expect a 2-5x increase in performance between versions.
Major versions will include a 2-week free trial.
10 thousand miles between interventions for some internal builds.
Hardware 4 training will begin later this year, Hardware 3 will continue to be developed.
Hardware 5 is 18 months from retail release.
Optimus
Optimus will be able to do many simple and difficult tasks:
Teaching
Companionship
Babysitting
Factory Work
Tesla expects 2 humanoid robots per person in the future.
Tesla expects 1000-2000 robots working in its factories by next year, with a limited production run.
22 degrees of freedom in the next generation hand, one new major hardware revision coming this year.
Cybertruck
China and Europe-compliant Cybertruck coming after production is scaled and costs are reduced.
Foundation series ending soon, non-foundation series starting next quarter.
Semi
Semi is in low-volume production, high-volume production starting soon.
Companies are actively choosing Semi due to economic advantages.
Massive potential impact on carbon emissions
Big impact on the profitability.
Other New Products
New products are coming soon, and Tesla is working hard on them.
Possible 12-passenger van, added to the “To-do” list.
Supercharger
58K global sites, and continuing to grow.
More superchargers deployed this year than the rest of the industry combined.
Tesla is providing adapters to other car companies and providing them with support for Supercharger access.
Tesla is moving to fill gaps and relieve congestion where possible.
Battery Production and Stationary Storage
New 4680 innovations, best cell from a manufacturing and efficiency standpoint.
4680 will be a continuing focus to improve over time.
Tesla has made their battery supply chain stronger and is working on further vertical integration.
2023 had 15GWh of Megapack production, 2024 is looking closer to 40GWh.
100-200% growth rate in the energy storage market.
Tesla is constrained by Megapack and Powerwall 3 production, can’t keep up with demand, even as they scale rapidly.
AI and Compute
AI growth is on a weekly basis, with Tesla taking in video and training on decision-making faster than ever.
Tesla is not compute-limited for FSD, but scenario-limited.
There is an AWS-style opportunity for parked vehicles to run distributed inference and compute.
Tesla intends to introduce LLMs with customizable learning personalities to Optimus and its vehicle lineup.
Investor Facts
Elon Musk expects $5-10 Trillion valuation for autonomy, and $20 Trillion for Optimus.
Musk mentioned that Ark Invest has the closest scenario – with Robotaxi releasing in 2025 or 2026, for a 110x valuation increase.
Elon’s stock award isn’t cash, and he has no intent to cut-and-run and is with Tesla for the long haul to accomplish the mission.
Other Items
Tesla Home HVAC, with a built-in HEPA filter, humidity control, super-efficient, super-quiet, with Powerwall integration – one day.
Kids Factory Tour Event – for children under 12!
If you’d like to watch the shareholder meeting, you can view it here, along with all the stockholder vote results.
TeslaFi logs your drives and charging sessions, letting you keep a log of your vehice's activity. We highly recommend checking them out if you use your car for business trips and would like to keep track of reimbursements, if you like to see how much you spend on charging or if you just love statistics. Visit their site and see everything they have to offer!
Tesla Android Project enables you to run Android apps in your Tesla. The platform is Open Source and you can deploy it on your own Raspberry Pi 4. Consider supporting the initiative by donating or purchasing the Compute Module 4 Bundle that delivers the best experience. Get $20 off by using the code: NotATeslaApp
The official Tesla app only notifies you if your car is broken into. By installing Sentry Pro on your phone, you will be notified for all Sentry Mode events. Stay connected and avoid potential surprises by receiving notifications. Stop constantly checking the cameras to ensure safety. Check only when necessary, save battery and get peace of mind. Get a 7 day free trial here!
Stay organized, connected, and stylish with Cyberbackpack — the #1 backpack, luggage & travel accessories for Tesla owners. We highly recommend checking them out if you commute or travel and would like a backpack or luggage that is sleek, secure and durable. View their products page and see everything they have to offer!
![Andrej Karpathy considers returning to Tesla to work on Optimus [video]](https://www.notateslaapp.com/images/news/2022/karpathy-podcast_300w.jpg)
