Tesla FSD Beta 11.4.7.1 Released to Employees With a Massive Amount of New Features

By Not a Tesla App Staff
Tesla adds a huge amount of features to its latest FSD Beta
Tesla adds a huge amount of features to its latest FSD Beta
Not a Tesla App

Tesla has released FSD Beta 11.4.7.1 to Tesla employees for internal testing. Although there are improvements to FSD Beta in the update, the big news is that this update comes in as version 2023.27.5, which means that it includes a massive number of features that have so far been missing in Tesla's FSD Beta.

Many owners are on Tesla's latest public FSD Beta v11.4.7 (update 2023.7.30), which includes non-FSD features up through update 2023.6. With this latest beta, Tesla includes features from three major updates, including update 2023.12, 2023.20 and 2023.26.

This amounts to a monumental release that includes over 50 new features across Tesla's models. Some of the features include the ability to customizable the scroll wheel, the new Spotify interface, support for Bluetooth game controllers, and Tesla's Charge on Solar feature. But there's so much more.

Below, we'll go through all of new, non-FSD features that are included in this massive release.

Features Included in Update 2023.12

Refresh Your Data Sharing Settings

Tesla added 'Refresh Your Data Sharing Settings' in update 2023.12
Tesla added 'Refresh Your Data Sharing Settings' in update 2023.12

Enable or disable data sharing and learn more about how Tesla uses anonymous data to improve existing safety and convenience features and create new ones.

Go to Controls > Software > Data Sharing. For more information on how we protect your data, go to www.tesla.com/privacy.

Track Mode

Tesla added 'Track Mode' in update 2023.12 for the new Model S
Tesla added 'Track Mode' in update 2023.12 for the new Model S

With Track Mode, you can now:

- Reach a top speed of 200 mph when the optional Track Pack is installed.

- Monitor vehicle health at a glance with updated visuals on the instrument panel and touchscreen.

- Extract maximum track performance using adaptive suspension, torque vectoring and traction control systems, which have been updated to improve consistency and driver confidence.

Please refer to the Owner's Manual for more details.

Phone Call Controls

Tesla added 'Phone Call Controls' in update 2023.12 for the new S/X and the Model 3/Y
Tesla added 'Phone Call Controls' in update 2023.12 for the new S/X and the Model 3/Y

Control your phone calls from the steering wheel. Use the left scroll button to answer or decline incoming calls. While you're on a call, use it to mute/ unmute yourself or hang up.

Text Size

Tesla added 'Text Size' in update 2023.12 for the Model 3/Y
Tesla added 'Text Size' in update 2023.12 for the Model 3/Y

Choose Standard or Large text size for the touchscreen.

Go to Control > Display > Text Size

Scroll Wheel Customization

Tesla added 'Scroll Wheel Customization' in update 2023.12 for the Model 3/Y
Tesla added 'Scroll Wheel Customization' in update 2023.12 for the Model 3/Y

Do more with the left scroll button on the steering wheel. You can adjust settings like brightness and Acceleration Mode, or perform actions like toggling the Camera App, opening the glovebox, and saving Dashcam footage.

Long press the left scroll button to bring up a list of functions and scroll through the list. To choose which function comes up by default, go to Controls > Display > Scroll Wheel Function.

Controls Search

Tesla added 'Controls Search' in update 2023.12
Tesla added 'Controls Search' in update 2023.12

Use the 'Search' function for quicker access to controls and settings.

Go to Controls > Search and enter a search term. Make changes directly from the result or tap the link to jump to that panel in Controls.

Gear Chimes

Tesla added 'Gear Chimes' in update 2023.12 for the Model 3/Y
Tesla added 'Gear Chimes' in update 2023.12 for the Model 3/Y

A chime now sounds when you shift gears. To toggle gear chimes, go to Controls > Safety > Gear Chimes.

Passenger Seat Controls

Tesla added 'Passenger Seat Controls' in update 2023.12 for the new Model S/X
Tesla added 'Passenger Seat Controls' in update 2023.12 for the new Model S/X

Adjust the passenger seat from the rear touchscreen to make it easier to enter or exit the rear seat. To access the passenger seat controls, tap the seat icon on the rear touchscreen. Hold one of the arrows to move the passenger seat forward or backward.

The controls are available only when the vehicle is not moving. To access the same controls from the front touchscreen, tap the App Launcher and go to the rear screen remote control app.

Points of Interest

Tesla added 'Points of Interest' in update 2023.12
Tesla added 'Points of Interest' in update 2023.12

See photos and reviews when you select a point of interest or Supercharger location.

Voice Recognition Language

Tesla added 'Voice Recognition Language' in update 2023.12
Tesla added 'Voice Recognition Language' in update 2023.12

British English is now available as a voice recognition language.

To update your voice recognition language, go to Controls > Display > Voice Recognition Language and choose an option from the list.

Get to Know Your Tesla

Tesla added 'Get to Know Your Tesla' in update 2023.12
Tesla added 'Get to Know Your Tesla' in update 2023.12

Learn basics like creating Driver Profiles, using Phone Key and regenerative braking from the 'Get to Know Your Tesla' educational experience within the new 'Manual' app. Each topic includes a visual and, if applicable, a link to the relevant feature.

The 'Manual' app also provides easy access to resources for learning much more about your Tesla, such as the on-screen Owner's Manual and online tutorial videos. Go to App Launcher > Manual.

Rename Vehicle in App

Tesla added 'Rename Vehicle in App' in update 2023.12
Tesla added 'Rename Vehicle in App' in update 2023.12
Walgermo/X

You can now set your vehicle's name through the Tesla app. To set your vehicle's name, simply tap the current name of your Tesla at the top of the app. You can then rename your vehicle and press 'OK.'

Parked Visualization

Tesla added 'Parked Visualization' in update 2023.12 for the Model 3/Y
Tesla added 'Parked Visualization' in update 2023.12 for the Model 3/Y

The vehicle visualization that appears while your vehicle is parked and contains labels to lock/unlock your car, as well as open your trunk/frunk can now be rotated to view the vehicle from different angles.

Speedometer Font

Tesla added 'Speedometer Font' in update 2023.12 for the Model 3/Y
Tesla added 'Speedometer Font' in update 2023.12 for the Model 3/Y
KyleCoolky

The font for the speedometer has been made bolder to be easier to see.

Nav Improvements

Tesla added 'Nav Improvements' in update 2023.12
Tesla added 'Nav Improvements' in update 2023.12

When searching for destinations in your vehicle’s navigation system, each destination will now display the distance in miles/km from your current location.

In addition, the map pin now looks more like a button, which brings up the destination information on the map, instead of navigating to it.

Favoriting Songs

Tesla added 'Favoriting Songs' in update 2023.12
Tesla added 'Favoriting Songs' in update 2023.12
StianWalgermo

While the functionality hasn't changed, your car will now display a plus sign (+) next to music controls to favorite a song. The plus sign replaces the heart icon in the music player.

Adjust Wiper Speed

Tesla added 'Adjust Wiper Speed' in update 2023.12 for the new Model S/X and Model 3/Y
Tesla added 'Adjust Wiper Speed' in update 2023.12 for the new Model S/X and Model 3/Y
LexBarker

You can now adjust the windshield wiper speed by using the scroll wheel on the steering wheel. First, press the button at the end of the left stalk to activate the windshield wipers and have the controls appear on the screen.

You can then tilt the left scroll wheel left/right to cycle through the various speeds of the windshield wipers.

Air Suspension

Tesla added 'Air Suspension' in update 2023.12 for the Model S/X
Tesla added 'Air Suspension' in update 2023.12 for the Model S/X
LuizFl

You now have more control over your air suspension. In the suspension menu, you now have toggles for enabling or disabling a vehicle's height or speed at a specific location.

The ride height is now also expressed in inches in the U.S.

Tesla Theater Icons

Tesla added 'Tesla Theater Icons' in update 2023.12
Tesla added 'Tesla Theater Icons' in update 2023.12
StianWalgermo

Tesla Theater now features square, app-like icons, instead of larger, horizontal logos.

App Drawer

Tesla added 'App Drawer' in update 2023.12
Tesla added 'App Drawer' in update 2023.12

All available options will now be displayed when you open the app drawer. Previously, options such as Frost Defrost and Wipers only showed up when you were customizing the launcher.

There is now a Customize button at the top of the app drawer as well.

New Labels

Tesla added 'New Labels' in update 2023.12
Tesla added 'New Labels' in update 2023.12

Tesla will now display a "NEW" badge next to new features or options. This makes new features more noticeable when scrolling through menus.

Charging Screen

Tesla added 'Charging Screen' in update 2023.12
Tesla added 'Charging Screen' in update 2023.12
Drill/TFF

The charging menu in the car has been updated to remove the car visualization and make the charging slider larger.

Features Included in Update 2023.20

Tire Service

Tesla added 'Tire Service' in update 2023.20 for the new Model S/X
Tesla added 'Tire Service' in update 2023.20 for the new Model S/X

Go to Controls > Service to see how many miles it's been since your last tire service.

When you get your tires rotated, replaced, or swapped, tap 'Reset'.

You can also go to Controls > Service > Wheel and Tire > Tire to input your tire configuration. This tells your car to reset the learned tire settings to improve your driving experience.

Tire Service Mileage

Tesla added 'Tire Service Mileage' in update 2023.20 for the Model 3/Y
Tesla added 'Tire Service Mileage' in update 2023.20 for the Model 3/Y

Go to Controls > Service to see how many miles it's been since your last tire service.

When you get your tires rotated, replaced, or swapped, tap 'Reset' to reset the counter.

Text Size

Tesla added 'Text Size' in update 2023.20 for the Model 3/Y
Tesla added 'Text Size' in update 2023.20 for the Model 3/Y

Large text size feature is now available in all supported languages.

To update your settings, go to Controls > Display > Text Size.

Windshield Wiper Control

Tesla added 'Windshield Wiper Control' in update 2023.20 for the Model 3/Y
Tesla added 'Windshield Wiper Control' in update 2023.20 for the Model 3/Y

Windshield wiper controls have now been added to the list of options that you can control in Scroll Wheel Function.

The steering wheel customization feature lets you control a variety of vehicle settings directly from the steering wheel. You simply long press the left scroll wheel and you can scroll through various settings and options such as HVAC fan speed, rear camera and more.

You can customize the list of options shown, or add the windshield wiper functionality to the list by navigating to Controls > Display > Scroll Wheel Function.

Phone Key

Tesla added 'Phone Key' in update 2023.20 for the new Model S/X and the Model 3/Y
Tesla added 'Phone Key' in update 2023.20 for the new Model S/X and the Model 3/Y

If you use your phone as a key for your Tesla, the device name that shows up in Bluetooth settings now has a more intuitive name.

The device name used to be a string of 18 characters without the word Tesla anywhere in the name, leaving some users wondering what the mystery may be.

As of this update, the vehicle key will now be displayed as "Tesla" followed by the last six characters of the vehicle's VIN.

Navigation Improvements

Tesla added 'Navigation Improvements' in update 2023.20
Tesla added 'Navigation Improvements' in update 2023.20
Garth Schanock

When searching for a destination in the vehicle's navigation system, the search results will now display which destinations are open and which are closed.

Set PIN to Drive

Tesla added 'Set PIN to Drive' in update 2023.20
Tesla added 'Set PIN to Drive' in update 2023.20

You can now enable the 'PIN to Drive' feature directly from the Tesla app. Simply navigate to Security & Drivers, toggle it on and set your PIN.

Having the ability to enable PIN to Drive in the app is useful for owners who may normally not use it, but may want to remotely enable the feature if they lose their key fob or phone.

This feature requires app update 4.21.5 or higher.

View All Cameras

Tesla added 'View All Cameras' in update 2023.20 for the new Model S/X and Model 3/Y
Tesla added 'View All Cameras' in update 2023.20 for the new Model S/X and Model 3/Y

You can now preview more than just the cabin camera in your Tesla. By navigating to Controls > Service, you can now preview all nine (eight on hardware 4.0) of Tesla's cameras. This includes being able to view all the front-facing cameras and B-pillar cameras for the first time.

You can change cameras by tapping the name of the camera at the top of the live video.

Note: These camera views are only here to give customers a better understanding of what their vehicle can see and to confirm that the cameras are working correctly. There is no additional functionality gained from this feature, at this time.

Current Trip Odometer

Tesla added 'Current Trip Odometer' in update 2023.20
Tesla added 'Current Trip Odometer' in update 2023.20

The 'Current Trip' odometer metrics have been updated with this update. Instead of displaying kWh, the Current Trip odometer will now display the number of minutes elapsed since the vehicle was put into drive.

The other odometers, such as 'Since Last Charge', 'Trip A' and 'Trip B' still display kWh.

Passenger Climate Temperature

Tesla added 'Passenger Climate Temperature' in update 2023.20
Tesla added 'Passenger Climate Temperature' in update 2023.20

The app launcher at the bottom of the screen will now only display the driver's climate setting if there is no one sitting in the passenger seat.

If someone sits in the passenger seat, the passenger temperature setting will automatically appear.

Features Included in Update 2023.26

Blind Spot Camera

Tesla added 'Blind Spot Camera' in update 2023.26 for the new Model S/X
Tesla added 'Blind Spot Camera' in update 2023.26 for the new Model S/X

Drag your blind spot camera to another location on your touchscreen. After you move the camera window, it will appear in that same position whenever your turn signal is on.

Go to Controls > Autopilot > Automatic Blind Spot Camera.

Customize the Scroll Wheel

Tesla added 'Customize the Scroll Wheel' in update 2023.26 for the new Model S/X
Tesla added 'Customize the Scroll Wheel' in update 2023.26 for the new Model S/X

Do more with the left scroll button on the steering wheel. You can adjust settings like brightness and acceleration mode, or perform actions like toggle the camera, raise suspension, open the glovebox, or save Dashcam footage.

Long press the left scroll button to bring up a list of functions and scroll through the list. To choose which function comes up by default, go to Controls > Display > Scroll Wheel Function.

Destination Closing Soon

Tesla added 'Destination Closing Soon' in update 2023.26
Tesla added 'Destination Closing Soon' in update 2023.26.10

Navigation now shows if your destination is closing soon or may be closed by the time you arrive.

Spotify Refresh

Tesla added 'Spotify Refresh' in update 2023.26
Tesla added 'Spotify Refresh' in update 2023.26.10

Play your Spotify music, playlists, podcasts, and audiobooks with a more familiar look and feel.

Go to App Launcher > Spotify. Requires Premium Connectivity.

Bluetooth Game Controllers

TTesla added 'Bluetooth Game Controllers' in update 2023.26 to the legacy Model S/X and the Model 3/Y
Tesla added 'Bluetooth Game Controllers' in update 2023.26 to the legacy Model S/X and the Model 3/Y

Use Bluetooth controllers to play games in Arcade.

To pair a controller, open the Bluetooth panel and follow the instructions.

This feature works best with PS5 controllers. Refer to the Owner's Manual for more information.

Warmer Display Colors

Tesla added 'Warmer Display Colors' in update 2023.26 to the new Model S/X and the Model 3/Y
Tesla added 'Warmer Display Colors' in update 2023.26 to the new Model S/X and the Model 3/Y

Set your touchscreen to automatically adjust to warmer colors at night.

Go to Controls > Display > Reduce Blue Light.

Automatic Headlights

Tesla added 'Automatic Headlights' in update 2023.26
Tesla added 'Automatic Headlights' in update 2023.26.10

Your lights turn on automatically when windshield wipers are activated and headlights are set to Auto in Controls > Lights > Headlights.

Charge On Solar

Tesla added 'Charge On Solar' in update 2023.26 to the new Model S/X and the Model 3/Y
Tesla added 'Charge On Solar' in update 2023.26 to the new Model S/X and the Model 3/Y

Charge your Tesla with solar energy if you have a Powerwall. You can choose how much charging comes from solar versus other sources.

To set it up, open the Tesla app for your vehicle, tap the “Charge on Solar” message, and follow the instructions. Requires Powerwall v23.12.10 and Tesla app v4.22.5.

Camera View on Tesla App

Tesla added 'Camera View on Tesla App' in update 2023.26 to the new Model S/X and the Model 3/Y
Tesla added 'Camera View on Tesla App' in update 2023.26 to the new Model S/X and the Model 3/Y

You can now see multiple cameras at once when checking your vehicle's surroundings.

To start using this feature, go to your vehicle settings in Controls > Safety > View Live Camera via Mobile App.

This feature requires Mobile App v4.22.5 and Premium Connectivity.

Vampire Survivors Game

Tesla added 'Vampire Survivors Game' in update 2023.26 to the new Model S/X and the Model 3/Y with MCU 3
Tesla added 'Vampire Survivors Game' in update 2023.26 to the new Model S/X and the Model 3/Y with MCU 3

Defend yourself against an onslaught of night creatures to survive until dawn.

To play, shift to Park (P) and go to the App Launcher > Arcade > Vampire Survivors.

Other Updates

Tesla added 'Other Updates' in update 2023.26
Tesla added 'Other Updates' in update 2023.26.10

Recents & Favorites are now in separate tabs in the Media Player.

The sliders to adjust volume, temperature, and charge limit are more precise

Sentry Mode records if someone tries to open a door or trunk when the vehicle is locked.

Access the Owner's Manual and Release Notes more quickly with the new Manual app. Read up on everything from regenerative braking to phone key or maximizing range.

Automatic Navigation shows more details about your destination.

Swiss French keyboard is available.

Availability varies by model and location.

Improved Phone Calls

Tesla added 'Improved Phone Calls' in update 2023.26
Tesla added 'Improved Phone Calls' in update 2023.26.10

When you have your climate system set to 'Auto,' the HVAC fan speed may be lowered automatically to reduce cabin noise while you're on a phone call.

Service Mode Improvements

Tesla added 'Service Mode Improvements' in update 2023.26
Tesla added 'Service Mode Improvements' in update 2023.26.10

Several new diagnostic screens have been added to Service Mode. They include the ability to test your steering wheel stalks, scroll wheels, horn and parking brake. You can now view the status and details of your airbags, low-voltage battery and the pitch of your vehicle's cameras.

Service Mode is a mode that's typically reserved for service technicians, although it can be accessed by anyone. It is used to view information and test various aspects of a vehicle.

Supercharging Details

Tesla added 'Supercharging Details' in update 2023.26.10
Tesla added 'Supercharging Details' in update 2023.26.10
Zer

When charging your vehicle at a Tesla Supercharger, your vehicle will now display more details about the Supercharging session.

On the Charging screen (Controls > Charging), your Tesla will now display the price per kWh you're being charged, as well as the total energy charged.

This is in addition to the Supercharger location and the total cost for the Supercharger session, both of which were added in prior updates.

Automatic Navigation Improvements

Tesla added 'Automatic Navigation Improvements' in update 2023.26.10
Tesla added 'Automatic Navigation Improvements' in update 2023.26.10
BabyTesla/X

If you have Automatic Navigation enabled in your Tesla, you'll now be presented with your suggested destination when you enter your vehicle.

The popup on the screen will display your destination, your ETA, battery upon arrival, as well as a photo of your destination if available.

You can begin driving to automatically start the route, or you can cancel the suggested destination.

Note: You can turn on Automatic Navigation by going to Controls > Navigation and enabling Automatic Navigation.

Updated Apps Badges

Tesla added 'Updated Apps Badge' in update 2023.26
Tesla added 'Updated Apps Badge' in update 2023.26.10

Similar to the "New" badges that Tesla added in update 2023.12 that signify which features are new in menus, Tesla will now display a blue dot underneath each app that is new or has been updated.

Daily Charge Limit

Tesla added 'Daily Charge Limit' in update 2023.26
Tesla added 'Daily Charge Limit' in update 2023.26.10

The recommended daily charge level has been updated for some vehicles. Depending on your vehicle, the types of batteries it uses and where it was manufactured, Tesla may have lowered the recommended daily charge limit.

Updated Manual App

Tesla added 'Updated Manual App' in update 2023.26.10
Tesla added 'Updated Manual App' in update 2023.26.10
Tico/X

Software Release Notes are now part of the Manual App, which already houses the owner's manual and tutorials. The app's icon has also been updated slightly.

You can tab between the three sections of the app by tapping on the names near the top.

The Release Notes' left column has been simplified and now only lists the name of the new feature, instead of the category such as "Climate Control Improvements."

Tesla Vehicle Data Report: How to Request It and What's Included

By Karan Singh
Not a Tesla App

If you’re ever involved in a collision with your Tesla, there’s a good chance you can retrieve dashcam footage of the incident—provided the USB drive and glovebox survive. However, sometimes they don’t, and even dashcam footage may not be sufficient.

In such cases, you can request a full recording from Tesla, along with a detailed data report showing comprehensive information about your vehicle’s performance during the collision. Shortly after any impact, your Tesla automatically uploads its crash log and related data to Tesla’s servers when possible, even using the low-voltage battery if the high-voltage pyro fuses are triggered.

Hopefully, you’ll never need this, but here’s how to request a Tesla Vehicle Data Report and what you can expect in the report.

How to Request a Report

Tesla has a simple, automated process for owners to request a Vehicle Data Report. To do so, simply go to Tesla’s Data Association Page and log into your Tesla Account.

From there, you’ll see a form that contains several options. Under “Regarding,” you’ll choose “Data Privacy Request,” and in the next selection, choose “Obtain a Copy of My Data.”

Tesla will then ask you to choose a vehicle that’s attached to your account and a range of dates for which you want data.

Tesla will provide details for the whole date range you specify, so it’s best to keep it small. Once you hit Submit, Tesla will start processing the request. Collision data is kept for an extended period of time, enabling people to go back and find data as required.

The summary page for the Vehicle Data Report
The summary page for the Vehicle Data Report
@bilalsattar on X

Within 30 days, and often much sooner, Tesla should e-mail you back with the report in PDF format and a CSV sheet containing all the raw information related to the request. This information is available for any country in which Tesla sells cars. If your country is outside of Tesla’s regular sales zones, you can try reaching out anyway, but we’re not sure if they will retain your data or not.

Tesla can also send you footage and data even if the incident wasn’t recorded as a collision – they’ll send you whatever is available – for any specific timeline you request.

Tesla Vehicle Data Report

The report is several pages long and comes in a nicely formatted PDF package. It breaks down the incident into various sections that help highlight what happened during the event. The first page summarizes the incident by highlighting key events and metrics like Autopilot use and speed.

Summary & Event Information

The Summary and Event Information sections are on the left side of the first page. The summary section is a text version of what happened during the event, reporting the time of the incident, speed, and whether seat belts were used, among other details.

The event information section includes some high-level information, such as the location of the incident, whether dashcam recordings are available, and the date and time.

Driver Log Data Overview

The Driver Log Data Overview section focuses on a few Tesla features and shows whether they were enabled at the five-second mark before the incident, one second before the incident occurred, or at the time of the incident. Tesla will show whether Autosteer / FSD, Driver Monitoring, Cruise Control, and Manual Brakes were used at all of these points. They will also show the status of the driver’s seat belt. Later in the report, Tesla will show graphs for each of these features so that you can see if they changed over the course of the incident.

Speed and Collision

The Vehicle Data Report's formatting
The Vehicle Data Report's formatting
@bilalsattar on X

The Speed and Collision section shows a timeline-based graph of the vehicle’s speed at the time of collision, four seconds before the collision, and four seconds after. The vertical line on the graph represents the collision, giving you a better understanding of what happened before and after the incident. However, in the following pages of the report, Tesla provides numerous time-based graphs that highlight many other metrics, including brake pedal use, accelerator, steering wheel torque and more.

Area of Detected Impact

This area of the report shows the vehicle from the top-down view and which areas detected an impact.

Time-Based Graphs

The rest of the report includes detailed graphs showing various vehicle metrics before, during, and after the incident.

Some of these graphs include the vehicle’s speed, steering wheel torque, steering wheel angle (how much the wheel was being turned and in which direction), accelerator and brake pedal usage, and pressure of the brake master cylinder. Tesla will even show whether any doors were opened during the seconds leading up to or following the incident.

If Tesla has a video of the incident, that will also be provided as a link for you to download. All the other information - the charts and graphs - raw information is also provided in a CSV file, which you can open in software like Microsoft Excel or Google Sheets.

Not a Tesla App

To wrap up, Tesla is currently the only car manufacturer in the world that can provide this information to its customers at the drop of a hat. This information is immensely valuable, and it could be the difference between someone paying an insurance claim or being charged with a crime.

Tesla produces some of the safest vehicles on the planet, and their commitment to safety and reporting is spectacular. We’re happy to see Tesla continue to take steps to better help their customers.

Thanks to Nic Cruze Patane for sharing the report. We hope you never need to use this, but it’s good to know that it’s available.

Inside Tesla’s FSD: Patent Explains How FSD Works

By Karan Singh
Not a Tesla App

Thanks to a Tesla patent published last year, we have a great look into how FSD operates and the various systems it uses. SETI Park, who examines and writes about patents, also highlighted this one on X.

This patent breaks down the core technology used in Tesla’s FSD and gives us a great understanding of how FSD processes and analyzes data.

To make this easily understandable, we’ll divide it up into sections and break down how each section impacts FSD.

Vision-Based

First, this patent describes a vision-only system—just like Tesla’s goal—to enable vehicles to see, understand, and interact with the world around them. The system describes multiple cameras, some with overlapping coverage, that capture a 360-degree view around the vehicle, mimicking but bettering the human equivalent.

What’s most interesting is that the system quickly and rapidly adapts to the various focal lengths and perspectives of the different cameras around the vehicle. It then combines all this to build a cohesive picture—but we’ll get to that part shortly.

Branching

The system is divided into two parts - one for Vulnerable Road Users, or VRUs, and the other for everything else that doesn’t fall into that category. That’s a pretty simple divide - VRUs are defined as pedestrians, cyclists, baby carriages, skateboarders, animals, essentially anything that can get hurt. The non-VRU branch focuses on everything else, so cars, emergency vehicles, traffic cones, debris, etc. 

Splitting it into two branches enables FSD to look for, analyze, and then prioritize certain things. Essentially, VRUs are prioritized over other objects throughout the Virtual Camera system.

The many data streams and how they're processed.
The many data streams and how they're processed.
Not a Tesla App

Virtual Camera

Tesla processes all of that raw imagery, feeds it into the VRU and non-VRU branches, and picks out only the key and essential information, which is used for object detection and classification.

The system then draws these objects on a 3D plane and creates “virtual cameras” at varying heights. Think of a virtual camera as a real camera you’d use to shoot a movie. It allows you to see the scene from a certain perspective.

The VRU branch uses its virtual camera at human height, which enables a better understanding of VRU behavior. This is probably due to the fact that there’s a lot more data at human height than from above or any other angle. Meanwhile, the non-VRU branch raises it above that height, enabling it to see over and around obstacles, thereby allowing for a wider view of traffic.

This effectively provides two forms of input for FSD to analyze—one at the pedestrian level and one from a wider view of the road around it.

3D Mapping

Now, all this data has to be combined. These two virtual cameras are synced - and all their information and understanding are fed back into the system to keep an accurate 3D map of what’s happening around the vehicle. 

And it's not just the cameras. The Virtual Camera system and 3D mapping work together with the car’s other sensors to incorporate movement data—speed and acceleration—into the analysis and production of the 3D map.

This system is best understood by the FSD visualization displayed on the screen. It picks up and tracks many moving cars and pedestrians at once, but what we see is only a fraction of all the information it’s tracking. Think of each object as having a list of properties that isn’t displayed on the screen. For example, a pedestrian may have properties that can be accessed by the system that state how far away it is, which direction it’s moving, and how fast it’s going.

Other moving objects, such as vehicles, may have additional properties, such as their width, height, speed, direction, planned path, and more. Even non-VRU objects will contain properties, such as the road, which would have its width, speed limit, and more determined based on AI and map data.

The vehicle itself has its own set of properties, such as speed, width, length, planned path, etc. When you combine everything, you end up with a great understanding of the surrounding environment and how best to navigate it.

The Virtual Mapping of the VRU branch.
The Virtual Mapping of the VRU branch.
Not a Tesla App

Temporal Indexing

Tesla calls this feature Temporal Indexing. In layman’s terms, this is how the vision system analyzes images over time and then keeps track of them. This means that things aren’t a single temporal snapshot but a series of them that allow FSD to understand how objects are moving. This enables object path prediction and also allows FSD to understand where vehicles or objects might be, even if it doesn’t have a direct vision of them.

This temporal indexing is done through “Video Modules”, which are the actual “brains” that analyze the sequences of images, tracking them over time and estimating their velocities and future paths.

Once again, heavy traffic and the FSD visualization, which keeps track of many vehicles in lanes around you—even those not in your direct line of sight—are excellent examples.

End-to-End

Finally, the patent also mentions that the entire system, from front to back, can be - and is - trained together. This training approach, which now includes end-to-end AI, optimizes overall system performance by letting each individual component learn how to interact with other components in the system.

How everything comes together.
How everything comes together.
Not a Tesla App

Summary

Essentially, Tesla sees FSD as a brain, and the cameras are its eyes. It has a memory, and that memory enables it to categorize and analyze what it sees. It can keep track of a wide array of objects and properties to predict their movements and determine a path around them. This is a lot like how humans operate, except FSD can track unlimited objects and determine their properties like speed and size much more accurately. On top of that, it can do it faster than a human and in all directions at once.

FSD and its vision-based camera system essentially create a 3D live map of the road that is constantly and consistently updated and used to make decisions.

Latest Tesla Update

Confirmed by Elon

Take a look at features that Elon Musk has said will be coming soon.

More Tesla News

Tesla Videos

Latest Tesla Update

Confirmed by Elon

Take a look at features that Elon Musk has said will be coming soon.

Subscribe

Subscribe to our weekly newsletter