Tesla FSD will be pure vision and not rely on radar use

Elon tweeted that v9 of the FSD beta would remove its reliance on radar completely and instead determine decisions based purely on vision. Humans don’t have radar after all, so it seems like a logical solution and tells us Tesla is feeling much more confident in their vision AI.

Radar and vision each have their advantages, but radar has thus far been much more reliable in detecting objects and determining speed. If you’ve ever noticed your Tesla being able to detect two vehicles in front of you when you can only see the one directly ahead of you, that’s radar at work.

In this situation the radio waves from the radar sensor are bouncing underneath the car in front of you and are able to continue traveling and detect that there is another object ahead even though it could never “see” it.

It really is one of those wow moments where you can feel the future and the ability for AI-powered cars to drive better than humans one day. It’s baby steps and slowly we’ll see more and more of these situations where the vehicle simply sees or does something we could never do.

There’s no doubting that more sensors could provide a more reliable and accurate interpretation of the real world as they each have their own advantages. In an ideal world a vehicle with radar, lidar, vision, ultrasonic sensors and even audio processing would provide the best solution. However, more sensors and systems come at a price, resulting in increased vehicle cost and system complexity.

After all humans are relatively safe drivers with two “cameras” and vision alone. If Tesla can completely solve vision, they’ll easily be able to achieve superhuman driving capabilities. Teslas have eight cameras, facing in all directions. They’re able to analyze all of them concurrently and make much more accurate interpretations then we ever could in the same amount of time.

Tristan on Twitter recently had some great insight into Tesla vision AI and how they’re going to replace radar. Here’s what Tristan had to say:

"We recently got some insight into how Tesla is going to replace radar in the recent firmware updates + some nifty ML model techniques

From the binaries we can see that they've added velocity and acceleration outputs. These predictions in addition to the existing xyz outputs give much of the same information that radar traditionally provides (distance + velocity + acceleration).

For autosteer on city streets, you need to know the velocity and acceleration of cars in all directions but radar is only pointing forward. If it's accurate enough to make a left turn, radar is probably unnecessary for the most part.

How can a neural network figure out velocity and acceleration from static images you ask?

They can't!

They've recently switched to something that appears to be styled on an Recurrent Neural Network.

Net structure is unknown (LSTM?) but they're providing the net with a queue of the 15 most recent hidden states. Seems quite a bit easier to train than normal RNNs which need to learn to encode historical data and can have issues like vanishing gradients for longer time windows.

The velocity and acceleration predictions is new, by giving the last 15 frames (~1s) of data I'd expect you can train a highly accurate net to predict velocity + acceleration based off of the learned time series.

They've already been using these queue based RNNs with the normal position nets for a few months presumably to improve stability of the predictions.

This matches with the recent public statements from Tesla about new models training on video instead of static images.

To evaluate the performance compared to radar, I bet Tesla has run some feature importance techniques on the models and radar importance has probably dropped quite a bit with the new nets. See tools like https://captum.ai for more info.

I still think that radar is going to stick around for quite a while for highway usage since the current camera performance in rain and snow isn't great.

NoA often disables in mild rain. City streets might behave better since the relative rain speed is lower.

One other nifty trick they've recently added is a task to rectify the images before feeding them into the neural nets.

This is a common in classical CV applications so surprised it only popped up in the last couple of months.

This makes a lot of sense since it means that the nets don't need to learn the lens distortion. It also likely makes it a lot easier for the nets to correlate objects across multiple cameras since the movement is now much more linear.

For more background on LSTMs (Long Short-Term Memory) see https://towardsdatascience.com/illustrated-guide-to-lstms-and-gru-s-a-step-by-step-explanation-44e9eb85bf21

They're tricky to train because they need to encode history which is fed into future runs. The more times you pass the state, the more the earlier frames is diluted hence "vanishing gradients".

Tesla’s FSD beta v9 will be a big improvement forward from what FSD beta users have been using where the system was still relying on radar. And it’ll be an even bigger leap from what non-beta testers currently have access to. We can’t wait. Now where’s that button?

Almost ready with FSD Beta V9.0. Step change improvement is massive, especially for weird corner cases & bad weather. Pure vision, no radar.

— Elon Musk (@elonmusk) April 9, 2021

Follow

Follow

Follow

Tesla Updates Santa Mode & Rainbow Road to Use Ambient Lighting and Updates SteamOS [VIDEO]

Flo / Roberto Bonato

While Tesla’s 2024 Holiday Update brought many new features, including Apple Watch Support, Cross Traffic Alerts, Weather Overlays, Search Along Route, and others, there are also several smaller items in this update that didn’t get much attention.

Ambient Lighting Now Affects Easter Eggs

The 2024+ Model 3 and Cybertruck both come equipped with ambient lighting, and Tesla is now taking advantage of this custom lighting feature by incorporating it into several easter eggs.

Rainbow Road: If you head over to Apps > ToyBox > Rainbow Road, you’ll find a new option available that allows the ambient lighting in the vehicle to also change shades while Rainbow Road is active. The Rainbow Road easter egg is a tribute to the iconic course course in Nintendo’s Mario Kart.

Check out the feature in action below:

🇮🇹 Con l’aggiornamento di Natale, quando attiviamo la pista arcobaleno, anche i LED interni diventeranno arcobaleno. Questa è una novità non documentata, scovata da un utente di TESLERS (chiaramente riguarda le Model 3 Highland)

🇺🇸 UNDOCUMENTED CHANGE: With the Holiday update,… pic.twitter.com/xyocOIrWfX

— 𝗧𝗘𝗦𝗟𝗘𝗥𝗦 (@TeslersOfficial) December 8, 2024

Santa Mode: Just in time for Christmas, Tesla has improved Santa Mode by cycling the ambient lighting between Christmas colors. Similar to Rainbow Road, the use of ambient lighting for the easter egg is optional and can be enabled under Apps > ToyBox > Santa Mode.

When using Santa Mode with ambient lighting turned on, the lights will cycle just between Christmas colors, red, green and white, instead of the full rainbow. There’s a video of the feature below:

Yup and in Santa Mode it cycles between red white and green pic.twitter.com/HVQUlfUyze

— Roberto Bonato (@BonatoRoberto) December 9, 2024

Update: It looks like it affects other Easter eggs as well, such as turning orange when using Romance mode.

SteamOS (Beta) Update

The Model S and Model X vehicles that support Steam gaming are receiving a Steam app update in the holiday update as well. This update could end up being very interesting, but for now Tesla says that the SteamOS, which also runs on Valve’s Steam Deck has been updated to version 3.6.

Valve outlines a wide range of fixes and improvements in their release notes for version 3.6, but Tesla simply states that the OS update includes performance enhancements and improved game compatibility.

The SteamOS hasn’t been updated on the Model S and X for quite a while, so we believe there may be more to this besides performance improvements and fixes. We’ll be looking to share some insights into this in the near future.

So, there you have it—three new features in the Holiday update that have mostly gone under the radar. While they may not apply to everyone, there are plenty of other cool features in the Holiday Update, like the arrival of Dynamic Road Closure Routing and Nearby Parking, that are coming to every vehicle, so go check out the full release notes.

Follow

Follow

Follow

New Report Offers Specs & Price for Tesla's Upcoming Next-Gen Model

A concept of Tesla's 'Model Q'

@DominicBRNKMN on X

At the 2023 Q3 Earnings Call, Tesla provided an update on its next-generation model, what many are referring to as the “Model 2” or “Model Q.” Although the actual model name is yet to be determined, it is known internally as Redwood.

During the 2024 Shareholder Meeting, Musk again mentioned the upcoming model, saying that Tesla is working on new products, including a more affordable vehicle and a larger SUV.

While we know the next-gen vehicle is aimed at the $25k to $30k USD market, we now have some new information thanks to a new report out of China.

Launch & Price

According to the Chinese outlet CNMO Technology News (Sina Finance), Travis Axelrod, Tesla’s Head of Investor Relations, announced plans to launch a new model. Axelrod shared this update while addressing senior corporate investors at an investor conference hosted by Deutsche Bank on December 5, 2024.

The goal of Project Redwood, internally dubbed “Model Q” by Deutsche Bank, is to achieve a post-subsidy price of under $30,000 USD. With the Federal EV Rebate potentially expiring next year, this suggests that the Model Q’s starting price will be around $30,000 USD. For comparison, the Model 3 currently starts at $42,490 in the U.S.

We previously reported that Tesla has already been speaking to suppliers and is looking to begin volume production of the vehicle sometime around June 2025. Of course, prototyping is still necessary, but we haven’t seen any signs of Tesla testing a new vehicle since spotting the Cybercab on the streets ahead of We, Robot.

Specs

The Model Q will supposedly be about 15% smaller and 30% lighter than the Model 3, with an approximate length of 3,988mm, or 157 inches. For scale, the Mini is 157 inches, while the Model 3 is 185 inches, so it’s fairly compact.

The battery will also be smaller to match the vehicle’s smaller size as Tesla tries to reduce the vehicle’s price while maintaining a similar range as its other vehicles. Tesla is expected to offer two models—a single-motor RWD model and a Long-Range Dual-Motor AWD variant.

The lower trim level will include a 53 kWh lithium iron phosphate (LFP) battery, while the AWD version is expected to include a 75 kWh LFP battery. The report says that the vehicle will be able to travel up to 500 km (310 miles) on a single charge, but it’s not clear whether this will be for the more efficient RWD version or the AWD variant.

Given the vehicle’s weight reduction compared to the Model 3, this type of range is likely for the lower trim level.

There’s no news yet on the price of the AWD version, but we expect it to be about $5,000 more expensive than the RWD variant.

Once the vehicle ramps up in production, Tesla intends to produce about 10,000 of these per week globally or about 500,000 per year. As Tesla mentioned during one of their previous earnings calls, this next-gen vehicle will be built using Tesla’s current assembly lines with minimal changes. This should allow for a smoother and more predictable production ramp. While Tesla still plans to use its new unboxed assembly process, it’s now saving it for the robotaxi.

Project Juniper

One of the items that came up during the investor conference was Tesla’s Project Juniper—the Model Y refresh. Juniper has already had a few leaks, including some images and information that mention a return to the 7-seat, 3-row format—at least in China. 

We’re expecting Juniper to launch early next year, as production has supposedly already begun at Giga Shanghai, at least on a limited prototyping basis. If the rollout matches the Model 3 Highland refresh, we’ll see Juniper arrive in China first and then make its way to North America and Europe by the end of 2025. The performance variants should launch in early 2026, after the standard models.

If you’re excited about Juniper, so are we! Check out our wishlist of features for Tesla’s Model Y refresh, and see everything we know so far.

Follow

Follow

Follow