How Tesla Uses Simulated Data to Improve FSD

By Karan Singh
Not a Tesla App

Tesla recently launched FSD in China, which has led many people to wonder exactly how they did it so quickly. Tesla isn't allowed to send training data out of China, meaning that it can’t leverage the capacity of the new Cortex Supercomputer Cluster at Giga Texas.

Instead, Tesla is using their generalized model, in combination with Synthetic Training Data, to train FSD for China. Of course, Tesla also uses this same synthetic data to supplement training for North America and for training for Europe. With European FSD on the horizon, we’ll likely see more and more use of synthetic training data for a sure-fire means to handle edge cases.

Simulated Content

Tesla officially refers to the synthetic training data as “Simulated Content” throughout their patent, which is titled “Vision-Based System Training with Synthetic Content.” Let’s break it down into easier-to-understand chunks.

Vision-Only Training

As you may well know, Tesla’s approach to autonomy focuses on using Tesla Vision. That means cameras providing visual data are the primary - and really only - means of acquiring data from outside of the vehicle. They no longer use radar and only use LiDAR to ensure vision sensor accuracy during training.

Capturing all the information from around the car builds a 3D environment that the vehicle uses to plan its path and conduct its decision-making. All that data is processed to build a fairly comprehensive view of what is around the vehicle and what is predicted to be around the vehicle in the future. All of that is also tagged and characterized to help the system prioritize various decisions.

Supervised Learning Model

Tesla’s FSD training is done through a supervised learning model. That means that the training model is fed data that is already labeled, either by humans or by Tesla’s unique AI model. The objects in the images that are being fed are identified and also tagged with position, velocity, and acceleration. This information acts as a ground truth for the AI model to learn from, allowing it to recognize and interpret similar objects and situations when encountered in real-world driving.

Ground Truth Label Data

The ground truth label data is a critical portion of this supervised learning process. The labeled data provides the model with accurate information about objects and their characteristics in the images. This enables Tesla to develop FSD’s robust understanding of the environment around it while it's driving. This data is typically collected from real-world driving scenarios and is either manually or automatically annotated with data.

Generating Simulated Content

Supplementing the real-world ground truth label data, Tesla employs a simulated content system to generate synthetic training data - which is really the key portion of this patent. This system generates synthetic training data that closely resembles the labeled ground truth data from above. 

Content Model Attributes and Contextual Labeling

The generation of that simulated content is guided by what Tesla calls “content model attributes,” which are essentially the key characteristics or features that are extracted from the ground truth label data. These could include things like road edges, lane lines, stationary objects, or even dynamic objects like vehicles or pedestrians.

By varying these attributes, the system can create a wide array of simulated scenarios - which means that FSD’s training program is exposed to as many unique and normal situations as possible.

In addition to the attributes, the system also incorporates contextual labeling - which involves adding labels to the simulated content to help refine it with even more detail. These labels can include things like weather conditions, time of day, or even the type of road or environment the vehicle is driving in. All this information is useful context to help develop FSD’s understanding of driving environments.

Training Data Generation

Tesla’s simulated content system generates vast amounts of training data by creating variations of the content models. These variations generally involve making tweaks to the attributes of the objects in the scene - thereby changing environmental conditions, or introducing new types of driving scenarios, like heavy traffic or construction. 

Training FSD

Wrapping it all up - the combined dataset of both real-world data and simulated data is then used to train FSD. By continuously providing new sets of both types of input, Tesla can continue to refine and improve FSD further.

Why Use Simulated Content?

It might seem counterintuitive that Tesla utilizes simulated content for training their autonomous driving system when their vehicles already collect vast amounts of real-world driving data. Their vehicles drive hundreds of millions of miles a month, all across the globe - providing them access to an unfathomable amount of unique data. Well, there are a few reasons to do so.

Not a Tesla App

Cost Reduction

One of the primary advantages of using simulated content is cost reduction. By not having to collect, transmit, sort, label, and process the incoming data from the real world, Tesla can instead just create data locally.

That cuts costs for data transmission, data storage, and all the processing and labeling - whether by human or machine. That can be a fairly significant amount when you think about just how much data goes through Tesla’s servers every single day from vehicles all around the world.

Simulating Challenging Conditions

Simulated content allows Tesla to train FSD in a wide range of environmental conditions that might be rare, difficult, or even dangerous to encounter consistently in real-world driving. This can include challenging conditions like heavy rain, fog, or snow - or even nighttime driving in those conditions. 

By training the system on this type of content without trying to pull it from real vehicles, Tesla can ensure that FSD remains operable and fairly robust even in more difficult scenarios in the real world.

Edge Cases & Safety

Another crucial benefit of simulated content is the ability to train FSD on edge cases. While we sometimes jokingly refer to edge cases as things like stopping for a school bus, there are real edge cases that may not be frequently encountered in real-world driving scenarios but can pose real safety risks for drivers, occupants, pedestrians, or other road users. Think of things that you could see happening but have never actually seen, like a car falling off a transport trailer or a highway sign falling down.

As such, Tesla simulates many unique edge cases, including sudden pedestrian crossing, unexpected obstacles in the road, or even erratic behavior from other drivers. All these unique simulations are fairly hard to capture regularly in the real world, which means simulating and training on them is essential to ensure safety.

Efficient and Continuous Optimization

Finally, the vast amount of diverse training data that can be generated by Tesla on demand means that they can quickly and efficiently iterate on FSD without needing to wait for real-world data. This means they can keep a continuous learning process going, ensuring that FSD is always improving bit by bit.

If you’re interested in reading more about the guts that make FSD tick, check out our entire series on FSD-related patents from Tesla here.

We’d also recommend our deep dive into Nvidia’s Cosmos - which is a training system for autonomous vehicles that primarily uses synthetic data to train machine models. It's a different take on Tesla’s FSD training cycle that primarily relies on real data, but it does have some similarities to this particular means of using simulated content.

Tesla Updates Robotaxi App: Adds Adjustable Pick Up Locations, Shows Wait Time and More [VIDEO]

By Karan Singh
Nic Cruz Patane

Tesla is rolling out a fairly big update for its iOS and early-access-only Robotaxi app, delivering a suite of improvements that address user feedback from the initial launch last month. The update improves the user experience with increased flexibility, more information, and overall design polish.

The most prominent feature in this update is that Tesla now allows you to adjust your pickup location. Once a Robotaxi arrives at your pickup location, you have 15 minutes to start the ride. The app will now display the remaining time your Robotaxi will wait for you, counting down from 15:00. The wait time is also shown in the iOS Live Activity if your phone is on the lock screen.

How Adjustable Pickups Work

We previously speculated that Tesla had predetermined pickup locations, as the pickup location wasn’t always where the user was. Now, with the ability to adjust the pickup location, we can clearly see that Tesla has specific locations where users can be picked up.

Rather than allowing users to drop a pin anywhere on the map, the new feature works by having the user drag the map to their desired area. The app then presents a list of nearby, predetermined locations to choose from. Once a user selects a spot from this curated list, they hit “Confirm.” The pickup site can also be changed while the vehicle is en route.

This specific implementation raises an interesting question: Why limit users to predetermined spots? The answer likely lies in how Tesla utilizes fleet data to improve its service.

Release Notes

While the app is still only available on iOS through Apple’s TestFlight program, invited users can download and update the app.

Tesla included these release notes in update 25.7.0 of the Robotaxi app:

  • You can now adjust pickup location

  • Display the remaining wait time at pickup in the app and Live Activity

  • Design improvements

  • Bug fixes and stability improvements

Nic Cruz Patane

Why Predetermined Pick Up Spots?

The use of predetermined pickup points is less of a limitation and more of a feature. These curated locations are almost certainly spots that Tesla’s fleet data has identified as optimal and safe for an autonomous vehicle to perform a pickup or drop-off.

This suggests that Tesla is methodically “mapping” its service area not just for calibration and validation of FSD builds but also to help perform the first and last 50-foot interactions that are critical to a safe and smooth ride-hailing experience.

An optimal pickup point likely has several key characteristics identified by the fleet, including:

  • A safe and clear pull-away area away from traffic

  • Good visibility for cameras, free of obstructions

  • Easy entry and exit paths for an autonomous vehicle

This change to pick-up locations reveals how Tesla’s Robotaxi Network is more than just Unsupervised FSD. There are a lot of moving parts, many of which Tesla recently implemented, and others that likely still need to be implemented, such as automated charging.

Frequent Updates

This latest update delivers a much-needed feature for adjusting pickup locations, but it also gives us a view into exactly what Tesla is doing with all the data it is collecting with its validation vehicles rolling around Austin, alongside its Robotaxi fleet.

Tesla is quickly iterating on its app and presumably the vehicle’s software to build a reliable and predictable network, using data to perfect every aspect of the experience, from the moment you hail the ride to the moment you step out of the car.

Tesla Will Face $2 Billion in Lost Profit as 'Big Beautiful Bill' Kills EV Credits

By Karan Singh
Not a Tesla App

The massive legislative effort titled the "Big Beautiful Bill" is taking direct aim at what has become one of Tesla’s most critical and profitable revenue streams: the sale of US regulatory credits. The bill could eliminate billions of dollars from Tesla’s bottom line each year and will slow down the transition to electric vehicles in the US.

The financial stakes for Tesla are absolutely immense. In 2024, Tesla generated $2.76 billion from selling these credits. This high-margin revenue was the sole reason Tesla posted a profit in Q1 2025; without the $595 million from regulatory credits, Tesla’s reported $409 million in profit would have been a $189 million loss.

How the ZEV Credit System Works

Zero-Emission Vehicle (ZEV) credits are part of state-level programs, led by California, designed to accelerate the adoption of electric vehicles. Each year, automakers are required to hold a certain number of ZEV credits, with the amount based on their total vehicle sales within that state. Under this system, automakers that fail to sell a certain percentage of zero-emission vehicles must either pay a significant fine or purchase credits from a company that exceeds the mandate.

Automakers who fail to sell enough EVs to meet their quota have a deficit and face two choices: pay a hefty fine to the state government for each missing credit (for example, $5,000 per credit in California) or buy credits from a company with a surplus.

As an all-EV company, Tesla generates a massive surplus of these credits. It can then turn around and sell them to legacy automakers at prices cheaper than the fine, creating a win-win scenario: the legacy automaker avoids a larger penalty, and Tesla gains a lucrative, near-pure-profit revenue stream. 

This new bill will dismantle this by eliminating the financial penalties for non-compliance, which would effectively make Tesla’s credits worthless. While the ZEV program is a state law, the Big Beautiful Bill will fully eliminate the penalties at a federal level.

A Multi-Billion Dollar Impact

The removal of US ZEGV credits would be a severe blow to Tesla’s financials. One JPMorgan analyst estimated that the move could reduce Tesla’s earnings by over 50%, representing a potential annual loss of $2 billion. While Tesla also earns similar credits in Europe and China, analysts suggest that 80-90% of its credit revenue in Q1 2025 came from US programs. 

Why the Program Exists

While the impact on Tesla would be direct and immediate, the credit system has a wider purpose. It creates a strong financial incentive for legacy automakers to develop and accelerate their zero-emission vehicle programs, whether it’s hydrogen, electric, or another alternative.

Eliminating the need for these credits would remove that financial pressure. This could allow traditional automakers to slow their EV transition in the US without the fear of a financial penalty, potentially leading to fewer EV choices for consumers and a slower path to vehicle electrification in the country.

Big, But Not Beautiful

On Sunday Morning TV, Elon Musk was asked his thoughts on the Big Beautiful Bill. They were pretty simple. A bill could be big, or it could be beautiful - I don’t know if it can be both, Musk stated.

The bill poses a threat to Tesla’s bottom line and to the adoption of EVs in the US market, where automakers will no longer have a financial incentive to transition to cleaner vehicles, a market they’ve regularly struggled in when competing against Tesla.

Tesla will have to work carefully in the future to cut expenses to remain profitable after the elimination of these regulatory credits.

Latest Tesla Update

Confirmed by Elon

Take a look at features that Elon Musk has said will be coming soon.

More Tesla News

Tesla Videos

Latest Tesla Update

Confirmed by Elon

Take a look at features that Elon Musk has said will be coming soon.

Subscribe

Subscribe to our weekly newsletter