Tesla FSD will be pure vision and not rely on radar use

May 10, 2021

By Nuno Cristovao

Elon tweeted that v9 of the FSD beta would remove its reliance on radar completely and instead determine decisions based purely on vision. Humans don’t have radar after all, so it seems like a logical solution and tells us Tesla is feeling much more confident in their vision AI.

Tesla Vision AI

Radar and vision each have their advantages, but radar has thus far been much more reliable in detecting objects and determining speed. If you’ve ever noticed your Tesla being able to detect two vehicles in front of you when you can only see the one directly ahead of you, that’s radar at work.

In this situation the radio waves from the radar sensor are bouncing underneath the car in front of you and are able to continue traveling and detect that there is another object ahead even though it could never “see” it.

It really is one of those wow moments where you can feel the future and the ability for AI-powered cars to drive better than humans one day. It’s baby steps and slowly we’ll see more and more of these situations where the vehicle simply sees or does something we could never do.

There’s no doubting that more sensors could provide a more reliable and accurate interpretation of the real world as they each have their own advantages. In an ideal world a vehicle with radar, lidar, vision, ultrasonic sensors and even audio processing would provide the best solution. However, more sensors and systems come at a price, resulting in increased vehicle cost and system complexity.

After all humans are relatively safe drivers with two “cameras” and vision alone. If Tesla can completely solve vision, they’ll easily be able to achieve superhuman driving capabilities. Teslas have eight cameras, facing in all directions. They’re able to analyze all of them concurrently and make much more accurate interpretations then we ever could in the same amount of time.

Tristan on Twitter recently had some great insight into Tesla vision AI and how they’re going to replace radar. Here’s what Tristan had to say:

"We recently got some insight into how Tesla is going to replace radar in the recent firmware updates + some nifty ML model techniques

From the binaries we can see that they've added velocity and acceleration outputs. These predictions in addition to the existing xyz outputs give much of the same information that radar traditionally provides (distance + velocity + acceleration).

For autosteer on city streets, you need to know the velocity and acceleration of cars in all directions but radar is only pointing forward. If it's accurate enough to make a left turn, radar is probably unnecessary for the most part.

How can a neural network figure out velocity and acceleration from static images you ask?

They can't!

They've recently switched to something that appears to be styled on an Recurrent Neural Network.

Net structure is unknown (LSTM?) but they're providing the net with a queue of the 15 most recent hidden states. Seems quite a bit easier to train than normal RNNs which need to learn to encode historical data and can have issues like vanishing gradients for longer time windows.

The velocity and acceleration predictions is new, by giving the last 15 frames (~1s) of data I'd expect you can train a highly accurate net to predict velocity + acceleration based off of the learned time series.

They've already been using these queue based RNNs with the normal position nets for a few months presumably to improve stability of the predictions.

This matches with the recent public statements from Tesla about new models training on video instead of static images.

To evaluate the performance compared to radar, I bet Tesla has run some feature importance techniques on the models and radar importance has probably dropped quite a bit with the new nets. See tools like https://captum.ai for more info.

I still think that radar is going to stick around for quite a while for highway usage since the current camera performance in rain and snow isn't great.

NoA often disables in mild rain. City streets might behave better since the relative rain speed is lower.

One other nifty trick they've recently added is a task to rectify the images before feeding them into the neural nets.

This is a common in classical CV applications so surprised it only popped up in the last couple of months.

This makes a lot of sense since it means that the nets don't need to learn the lens distortion. It also likely makes it a lot easier for the nets to correlate objects across multiple cameras since the movement is now much more linear.

For more background on LSTMs (Long Short-Term Memory) see https://towardsdatascience.com/illustrated-guide-to-lstms-and-gru-s-a-step-by-step-explanation-44e9eb85bf21

They're tricky to train because they need to encode history which is fed into future runs. The more times you pass the state, the more the earlier frames is diluted hence "vanishing gradients".

Tesla’s FSD beta v9 will be a big improvement forward from what FSD beta users have been using where the system was still relying on radar. And it’ll be an even bigger leap from what non-beta testers currently have access to. We can’t wait. Now where’s that button?

Almost ready with FSD Beta V9.0. Step change improvement is massive, especially for weird corner cases & bad weather. Pure vision, no radar.
— Elon Musk (@elonmusk) April 9, 2021

Tesla Building Cortex 2.0 Supercomputer at Giga Texas to Power FSD

May 14, 2025

By Karan Singh

Not a Tesla App

FSD’s insatiable appetite for AI compute is taking shape once again at Giga Texas, where Tesla is building out Cortex 2.0 — its second large-scale GPU supercomputer cluster — on the facility’s north side.

This confirmation comes thanks to the detailed research and findings by Giga Texas drone pilot Joe Tegtmeyer, who shared his findings on X.

For months, there has been some speculation on what this new site would be - new storage, the Optimus production line, preparation for the Cybercab’s unboxed assembly process, and more. Now, thanks to permits filed by Tesla, we know that this new area is destined to become the second supercomputer destined for FSD training.

Cortex 2.0

There are some key details we can unpack from what we’ve learned about Cortex 2.0. The new facility will be on the north side of Giga Texas, opposite Cortex 1.0, which is on the south side. The permits explicitly tie the usage of this expansion to Cortex 2.0, namely through the fact that it is a data center.

This is actually a change from Tesla’s original plans - the northern section was intended to be used as a central campus support facility, with three smaller facilities and water storage tanks. Now, it has been redesignated as Cortex 2.0 and is one large structure.

The permits that have been filed have already been approved and cover the foundations, underground water mains, and building itself. The steel structure is actively being put together on the concrete foundations, and about 50% of the roof decking is already complete, bringing the exterior shell closer to completion.

Fueling FSD’s Brain

The deployment of Cortex 2.0 is the latest in Tesla’s massive and ongoing investment in the computational power required to train FSD. While this primarily covers FSD for cars, this will also eventually apply to Optimus, which is powered by the same AI4 computer and also runs a variant of FSD.

This isn’t Tesla’s first foray into large-scale AI infrastructure and won’t be their last. Alongside Dojo, their home-grown AI supercomputer, Tesla has been partnering closely with Nvidia to ensure they have the GPUs necessary to do all the hard work. Cortex 2.0 is the next iteration, and once it's fully online, we can expect that Tesla’s work on FSD will accelerate even further.

Especially since Elon is planning something truly outrageous once again:

Been thinking about the fastest way to bring a terawatt of compute online.

That is roughly equivalent to all electrical power produced in America today.
— gorklon rust (@elonmusk) May 12, 2025

You can check out Joe’s full video below:

Tesla FSD in Europe: Highway Approval Expected to Arrive in September

May 14, 2025

By Karan Singh

Not a Tesla App

With European Tesla owners eagerly awaiting any news on FSD in Europe, we’ve seen Tesla tease FSD, and also point out exactly what the barrier has been - regulatory approval. Now, following the latest meeting of UNECE, new regulatory amendments are set to unlock “System-Initiated Maneuvers” (SIM) on highways across participating European nations.

This development, highlighted by Kees Roelandschap on X, notes that the latest documents from the United Nations Economic Commission for Europe (UNECE) center on amendments to UNECE regulation 171. These changes were formally adopted into UNECE during the WP29 World Forum in March 2025.

Now, the amendment that will enable SIM will come into force on September 26, 2025. This six-month period after approval is standard UNECE procedure and allows objections from party states that would halt the implementation. Objections to UNECE amendments are genuinely rare, with 95% of amendments passing without objection once the World Forum adopts them.

What This Means for FSD in Europe

System-initiated maneuvers will allow your Tesla to perform actions autonomously, such as changing lanes on a highway, while the driver remains fully responsible for supervision. This is a substantial step up from current regulatory standards that only allow for suggested maneuvers or require explicit driver initiation for every automated maneuver.

This inches towards the more normal “hands-off, eyes-on” approach that Tesla has taken with FSD Supervised in North America and China. This regulatory change will help to provide the legal frameworks needed to deploy more advanced autonomy capabilities in Europe, at least matching what’s available on highways in North America.

Tesla is waiting for EU approval (sigh)
— gorklon rust (@elonmusk) May 12, 2025

Highways Only

The UNECE regulation will apply to all countries that adopt its standards, including the European Union, Japan, and South Korea - unless they specifically block it. This is a relatively positive development, but there are some hiccups.

This regulation only allows for system-initiated maneuvers on highways, not low-speed roadways. That means the city streets portion of FSD’s capabilities - including handling ‘Start FSD from Park` and reaching your destination’s parking spot - still won’t be available in Europe.

In addition, the UNECE framework has stricter requirements for driver monitoring and attentiveness - which means that some nags, including the dreaded wheel nag, are likely to remain a key portion of the experience.

Progress on autonomous driving regulations in Europe has been fraught with indecision and caution, which has been a source of frustration for many who are watching the everyday progress of FSD in North America, and more recently, in China. Even with individual countries recently approving testing of FSD on public roads, harmonized UNECE regulations are the key to wider, consistent deployment, which will allow everyday customers to use it as well.

While the path to full parity with North America may be long and involve even more regulatory machinations, the upcoming implementation of System-Initiated Maneuvers is a big step forward for Tesla owners in Europe. It is a key piece of the puzzle that will help Tesla bring some more features of FSD to Europe, assuming the September timeline holds.