We previously dived into how FSD works based on Tesla’s patents back in November, and Tesla has recently filed for additional patents regarding its training of FSD.
This particular patent is titled “Predicting Three-Dimensional Features for Autonomous Driving” - and it’s all about using Tesla Vision to establish a ground truth - which enables the rest of FSD to make decisions and navigate through the environment.
This patent essentially explains how FSD can generate a model of the environment around it and then analyze that information to create predictions.
Time Series
Creating a sequence of data over time - a Time Series - is the basis for how FSD understands the environment. Tesla Vision, in combination with the internal vehicle sensors (for speed, acceleration, position, etc.,) establishes data points over time. These data points come together to create the time series.
By analyzing that time series, the system establishes a “ground truth” - a highly accurate and precise representation of the road, its features, and what is around the vehicle. For example, FSD may observe a lane line from multiple angles and distances as the vehicle moves through time, allowing it to determine the line’s precise 3D shape in the world. This system helps FSD to maintain a coherent truth as it moves forward - and allows it to establish the location of things in space around it, even if they were initially hidden or unclear.
Author’s Note
Interestingly, Tesla’s patent actually mentions the use of sensors other than Tesla Vision. It goes on to mention radar, LiDAR, and ultrasonic sensors. While Tesla doesn’t use radar (despite HD radars being on the current Model S and Model X) or ultrasonic sensors anymore, it does use LiDAR for training.
However, this LIDAR use is for establishing accurate sensor data for FSD - for training purposes. No Tesla vehicle is actually shipped with any LiDAR sensors. You can read about Tesla’s use for its LIDAR training rigs here.
Associating the Ground Truth
Once the ground truth is established, it is linked to specific points in time within the time series - usually a single image or the amalgamation of a set of images. This association is critical - it allows the system to predict the complete 3D structure of the environment from just that single snapshot. In addition, they also serve as a learning tool to help FSD understand the environment around it.
Imagine FSD has figured out the exact curve of a lane line using data from the time series. Next, it connects this knowledge to the particular image in the sequence where the lane line was visible. Next, it applies what it has learned - the exact curve, and the image sequence and data - to predict the 3D shape of the line going forward - even if it may not know for sure what the line may look like in the future.
Author’s Note
This isn’t part of the patent, but when you combine that predictive knowledge with precise and effective map data, that means that FSD can better understand the lay of the road and plan its maneuvers ahead of time. We do know that FSD takes into account mapping information. However, live road information from the ground truth is taken as the priority - mapping is just context, after all.
That is why when roads are incorrectly mapped, such as the installation of a roundabout in a location where a 4-way stop previously existed, FSD is still capable of traversing the intersection.
Three Dimensional Features
Representing features that the system picks up in 3D is essential, too. This means that the lane lines, to continue our previous example, must move up and down, left and right, and through time. This 3D understanding is vital for accurate navigation and path planning, especially on roads with curves, hills, or any varying terrain.
Automated Training Data Generation
One of the major advantages of this entire 3D system is that it generates training data automatically. As the vehicle drives, it collects sensor data and creates time series associated with ground truths.
Tesla does exactly this when it uploads data from your vehicle and analyzes it with its supercomputers. The machine learning model uses all the information it gets to better improve its prediction capabilities. This is now becoming a more automated process, as Tesla is moving away from the need to manually label data and is instead automatically labeling data with AI.
Semantic Labelling
The patent also discusses the use of semantic labeling - a topic covered in our AI Labelling Patent. However, a quick nitty-gritty is that Tesla labels lane lines as “left lane” or “right lane,” depending on the 3D environment that is generated through the time series.
On top of that, vehicles and other objects can also be labelled, such as “merging” or “cutting in.” All of these automatically applied labels help FSD to prioritize how it will analyze information and what it expects the environment around it to do.
How and When Tesla Uploads Data
Tesla’s data upload isn’t just everything they may catch - even though they did draw an absolutely astounding 1.28 TB from the author’s Cybertruck once it received FSD V13.2.2. It is based on transmitting selective sensor information based on triggers. These triggers can include incorrect predictions, user interventions, or failures to correctly conduct path planning.
Tesla can also request all data from certain vehicles based on the vehicle type and the location - hence the request for the absurd 1.28 TB coming from one of the first Canadian Cybertrucks. This allows Tesla to collect data from specific driving scenarios - which it needs to help build better models that are more adaptable to more circumstances while also keeping data collection focused, thereby making training more efficient.
How It Works
To wrap that all up, the model applies predictions to better navigate through the environment. It uses data collected through time and then encapsulated in a 3D environment around the vehicle. Using that 3D environment, Tesla’s FSD formulates predictions on what the environment ahead of it will look like.
This process provides a good portion of the context that is needed for FSD to actually make decisions. But there are quite a few more layers to the onion that is FSD.
Adding in Other Layers
The rest of the decision-making process lies in understanding moving and static objects on the road, as well as identifying and reducing risk to vulnerable road users. Tesla’s 3D mapping also identifies and predicts the pathing of other moving objects, which enables it to conduct its path planning. While this isn’t part of this particular patent per-say, it is still an essential element to the entire system.
If all that technical information is interesting to you, we recommend you check out the rest of our series on Tesla’s patents:
We’ll continue to dive deep into Tesla’s patents, as they provide a unique and interesting source to explain how FSD actually works behind the curtains. It’s an excellent chance to get a peek behind the silicon brains that make the decisions in your car, as well as a chance to see how Tesla’s engineers actually structure FSD.
Subscribe
Subscribe to our newsletter to stay up to date on the latest Tesla news, upcoming features and software updates.
Elon Musk has always been a bit of a celebrity and not always one that has been well received. With his many different companies - including Tesla, SpaceX, Boring, Neuralink, X, and xAI, he’s spread throughout many high-tech fields and has a massive say in the way technology impacts the lives of many.
Most recently, Elon has taken to the fore of the US government under the Trump Administration and is running the Department of Government Efficiency (DOGE). Here, he’s been tasked with reducing government spending.
These moves, in addition to what looked like an inappropriate hand gesture at a Republican rally last year, have led to him being labeled a Nazi - and those who own his cars or support his companies - as Nazi sympathizers. This is particularly true for Cybertruck owners.
We’re not calling for Elon to step down or stop - that’s beyond the scope of this opinion piece. Instead, we’re hoping that we make a point - harming real people and their property is a step too far, especially when they haven’t done anything other than spend their money or choose their place of employment.
Boycotts and Protests
Few CEOs are as active in politics as Elon is - and that’s made him and his companies the targets of protestors who don't agree with the changes and policies. This has led to boycotts of Tesla, protests outside factories, Supercharger sites, and showrooms.
Interestingly, some protestors at several locations throughout the United States have admitted they are being paid to protest. This isn’t surprising or new - political activist organizations sometimes bring in paid protestors to pad crowd numbers or guide the protestors toward their goals. While we can’t say whether this is true or not at this time, Elon appears to believe so, as per a post on X on the 8th of March.
Some of the protests have descended into violence, which has likely been escalated by people in a crowd mentality. An example of this is the incident at Tesla’s NYC Meatpacking showroom - where the crowd attempted to break down the doors and enter the showroom, putting the lives of innocent employees and customers at risk.
While protesting is a right, harming others and harming private property is not, regardless of personal beliefs or political opinions. This is what we’d like to emphasize here - harming or threatening Tesla’s employees and their customers isn’t making for a better world - it is hurting real people - members of your community. Some of them may have stood in solidarity with you just mere months ago at the voting booths, but now the simple ownership of a Tesla vehicle is considered extreme.
Vandalism and Who It Impacts
The protests are really just the tip of the iceberg. There have been ongoing and escalating incidents of vandalism - both at Tesla’s showrooms and Superchargers, as well as of private vehicles.
Vandalism is a crime - full stop. Attacking others and their private property with the intent to harm them isn’t an effective means to portray a message - it is political violence. Harming others is never the correct option - and is the worst way to push your message.
Vandalism of a private vehicle doesn't hurt Tesla. That vehicle will be repaired with parts sold by Tesla, paid for by the owner’s insurance. Those insurance costs will be passed down to customers everywhere - whether they’re Tesla-specific or not. Tesla will have made money replacing parts or replacing a vehicle because nearly 70% of Tesla’s customers continue to buy Tesla.
The vandalism isn’t focused on just private vehicles either - much of it has hit Tesla’s showrooms - which have been shot at in the US - and one was even burned down overnight in France. These vandals have also been targeting Tesla’s lots - and damaging cars on their way to customers. Once again, Tesla is insured against all of this - and will only be making replacement vehicles and fixing the damage - just like they did with the arson incident at a Supercharger site - where they replaced the Supercharger in under 48 hours.
Superchargers are public charging infrastructure. Damaging them doesn’t just affect Tesla owners - it impacts all EV drivers. Many Supercharger sites are open to other electric vehicles and serve as a vital charging option, especially in remote areas where charging options may be limited.
Additionally, some Supercharger stations are sometimes partially or fully funded by government grants aimed at expanding EV infrastructure. Acts of vandalism only harm local communities by undermining efforts to build essential public resources.
Sales and Stock
Tesla’s stock and sales have also been hit hard by the recent boycotts and response to Elon’s political activity.
Several analysts are predicting that Tesla’s deliveries in Q1 2025 are expected to be the lowest in the last three years - which will likely be a big hit. On the flipside, Elon is still optimistic about the stock - and has mentioned that the long-term benefits of holding Tesla far outweigh the short-term tensions.
However, Tesla’s stock also swings heavily with the market and is one of the most actively traded stocks by retail investors - both regular stocks and options included. The US markets have been on a relatively downward trend, with the Trump administration telling people to tighten their belts and prepare for a potential recession in the coming months.
These indicators aren’t very clear - and the ramp of the refreshed Model Y will have an impact on sales as Tesla won’t be able to deliver as many vehicles as they normally do.
Musk Owns Only 12% of Tesla
At the end of the day, these protests and all this vandalism only serve one purpose. It hurts hardworking citizens of the countries these events are happening in - whether the United States, Canada, Germany, France, or others.
They don’t impact Elon as much because he owns a small percentage of Tesla. Elon owns 12.8% of Tesla as of February 28, 2025. There are several other large stakeholders who own nearly as much as he does - and hundreds of thousands of retail shareholders. His fortune is so large that even if he lost his entire stake in Tesla today, he’d still be worth over $200 billion.
This amount of money is almost unimaginable. If you were to be given $400,000 every single day since 0 BC - until today - you still wouldn’t be worth as much as Elon Musk. That amount of money is absolutely insane and ridiculous—when you consider how little it affects him and how much real harm it causes to everyday people, it’s clear it’s not the correct approach.
Tesla employs about 130,000 people worldwide. That’s 129,999 people who aren’t Elon Musk that are directly harmed by these actions, all for choosing to work at one of the most innovative tech companies - whose mission is to electrify the world.
Putting all that together - the protests, boycotts, and vandalism aren’t hurting Elon as much as they’re hurting everyone else. They’re hurting regular people. And these people who have been threatened or hurt are never going to look in the same direction as these misguided activists.
Instead, these would-be vandals would be better off redirecting their efforts to reach out to the political leaders in their regions and working on getting better representation - and ensuring that people come to the stands to vote next time there is an election. There is a proper, legal, and moral way to change someone’s mind - with discussion. Threats to damage their property and hurt them are not the way.
Wrapping it all up - harming people and their property is a crime - and it isn’t the way to change people’s minds in a democratic society. It does nothing to harm Elon Musk, and all it hurts are your neighbors and friends.
Following Tesla’s Q1 2025 All-Hands Meeting, the company quietly updated the Cybertruck’s webpage. When we first checked it during our Live Space, nothing immediately stood out. At first glance, the changes seemed to be mostly stylistic—aligning the page’s format with the rest of Tesla’s site while adding a Cyber-inspired twist with unique fonts and boxed media elements.
But this morning, we took a closer look—because if there’s one thing Tesla loves, it’s hiding details in plain sight. And sure enough, we found an interesting change around Sentry Mode — playing loud music.
Sentry Mode Music
Tesla quietly updated the Sentry Mode section on the Cybertruck’s about page, adding new text that reveals Sentry Mode will soon be able to play loud music when it’s triggered. The text on Tesla’s site states:
“Enable Sentry Mode to monitor your unattended vehicle or trailer, and automatically activate the alarm, increase the touchscreen brightness, and play music at max volume if a threat is detected.”
This latest change appears to build on that, leveraging the Cybertruck’s Superhorn and external speaker, which can already play music when parked via the Boombox feature. We suspect this feature won’t rely on the internal speakers, as the music would have to be extremely loud in order to be heard outside the car.
And given how loud the Superhorn and external speakers are—powerful enough to function as a proper horn—this could be an effective deterrent for vandals or intruders.
Other Models Too?
While this feature was spotted on the Cybertruck’s page, it’s unlikely that it’ll be exclusive to the stainless steel beast. We did a quick scan of Tesla’s other vehicle pages, but the change hasn’t made it over to other pages yet. While Tesla’s other vehicles don’t include the Superhorn, most are equipped with an external speaker for the Pedestrian Warning System (PWS)—which also powers the Boombox feature.
If your Tesla supports Boombox, it’s likely that Sentry Mode Music will roll out across the lineup once the update is released. However, legacy vehicles without a PWS will likely miss out on this feature.
When Will It Be Released?
To answer everyone’s favorite question, we know it’s not in Tesla’s 2025.8 update, which has recently started rolling out. Given the small 2025 releases so far (2025.2 and 2025.8), Tesla may be preparing a larger 2025 Spring Update similar to last year’s.
We’re about a month away from the arrival of Tesla’s Spring Update, and this could be one of the included features.
The real question is: Will Tesla allow custom sounds or music via USB? If so, the possibilities just got a whole lot more entertaining.
![Tesla Updates Energy App in Update 2025.8.3: Adds Reset and Distance Filter [VIDEO]](https://www.notateslaapp.com/img/containers/article_images/2025/updated_energy_app.jpg/979943fb17df5ae4e9fa668b8cd61fca/updated_energy_app.jpg)
