All Tesla FSD Visualizations and What They Mean

By Nuno Cristovao

Updated: September 21st, 2022 as of FSD Beta 10.69.2.2

Tesla has slowly added more visualizations to the car display, showing what the car can detect and respond to in its environment. Tesla initially showed just road markings and some vehicles, but then slowly added more vehicle types, pedestrians and traffic cones.

However, with the release of FSD Beta version 9, Tesla has drastically increased the amount of objects the car can visualize and interact with.

The visualizations in the car aren't tied one-to-one with what the car is capable of detecting and using to make decisions. However, Tesla keeps visualizations and object detection closely coupled so that drivers have a good understanding of what the car can see.

Unidentified Objects

FSD Beta 10.69.2 will now display objects it can not recognize
FSD Beta 10.69.2 will now display objects it can not recognize
Not a Tesla App

As of FSD Beta 10.69.2 the vehicle visualizations will now display unidentified objects or debris. The shape of the visualization will not match the actual object, but it will be displayed as a pile of debris.

This lets the driver know the vehicle is aware of an object in its path or surroundings, but that it can not yet idenfiy what the object is.

Vector-based Lanes

Vector-based lanes
Vector-based lanes
Chazman/Twitter

As of FSD Beta 10.11, your Tesla will now display vector-based lanes. This means that the lanes are not just raster-based lines drawn in a 3D environment, but that Tesla is actually building the lane markings with vector-based graphics.

This gives Tesla additional freedom when choosing how to display the lanes on the car's display.

With this update, Tesla now fills in the lane in blue when your vehicle is changing lanes. This was something Tesla couldn't do before since the lane markings were just drawn lines.

The new vector-based graphics will scale well to any size and reduce blurring and pixelation you normally see on the display with lane markings and road edges.

Since lane markings are now vectors, hopefully, Tesla will soon apply the same process to other markings, such as the road edges.

Scalable Vehicle Models

All vehicles that Tesla displayed were pre-defined static-sized assets. However, in the 10.10.2 FSD update, we are now seeing Tesla scale individual vehicle models so that they represent the calculated size of surrounding vehicles. Contextually this could be helpful in better understanding our car's situation in the world.

Tesla now shrinks or stretches the 3D vehicle models in each dimension so that the 3D model matches the calculated dimensions for each vehicle. This is especially apparent in longer vehicles such as buses, trucks, and tractor-trailers, where the vehicle lengths are more likely to vary, but you can also see it scale other vehicle models such as very small cars.

In this example below, you'll see that Tesla is now able to accurately represent buses of different sizes. Tesla only has a model for a full length bus, but in this case, Tesla detected that the length of one of the buses is considerably shorter than the vehicle model so it chose to reduce the length of the bus to the length Autopilot had calculated. In the image below you can see how the same bus model is shown in two different sizes.

Tesla adds scalable vehicle models to the latest FSD Beta
Tesla adds scalable vehicle models to the latest FSD Beta

You can also read more details or see additional examples of Tesla dynamically scaling vehicle models.

Road Users

Vulnerable road users or VRUs as Tesla calls them are pedestrians and other users of the road that the car must be especially careful with.

Tesla already does a good job displaying some of these, but there is room for improvement here since it’s such a critical area. We hope that Tesla will add additional animals and sidewalk detection in future updates.

Pedestrians

Pedestrians are one of the few objects that feature an animation. If the pedestrian is walking, you'll see them animated on the screen.

Tesla FSD visualizations
FSD Beta visualizations start representing the real world
DirtyTesla/YouTube

Bicycles

Bicycles are visualized separately from motorcycles.

Dogs

Tesla recently added a visualization for a dog which shows up for dogs and other similarly sized animals. It's likely Tesla will add more animals in the future such as squirrels, deer and other common animals seen on roadways.

Tesla FSD visualization of a dog

Objects

Garbage/Recyling Bins

This is an object that's also displayed outside of the FSD Beta.

Traffic Cones

Traffic cones are also displayed outside of the FSD Beta and are displayed in orange. The car will display them for cones, construction barrels or sometimes mailboxes.

Speed Bumps

The car has been reacting to speed bumps for the last few betas, but they're now visualized on the screen with small arrows on them, which is a nice improvement.

Tesla visualization speed bumps
As of beta 10.4, speed bumps now appear in the visualization
Frenchie/YouTube

Poles

These poles can often be seen on the side of some highways and are displayed as short gray sticks at the edge of the road.

Tesla visualization poles
FSD Beta shows a visualization for small poles
Frenchie/YouTube

Vehicles

Tesla displays the most common vehicle types. Obviously, Tesla will need to expand this list once they expand to some other countries where other vehicle types are popular. Elon has said in the past that Tesla will visualize other Teslas specifically in the car. Recent car data shows that Tesla is already identifying whether a car is a Tesla and what model it is. This is a nice addition, but really would likely have little benefit to FSD. Tesla may provide car-to-car communication if it detects there's another Tesla nearby, which could make some situations easier, such as the right way of situations.

Another vehicle type I'd like Tesla to react better to and visualize are trailers. There are many types of trailers and they often have segments you can see through, so this may be a little tougher, but it's an important vehicle type to add.

New Vehicle Models

Vehicle models are now much more detailed
Vehicle models are now much more detailed
Not a Tesla App

FSD Beta 10.12 introduces new vehicle models for almost every vehicle type.

Some of the vehicles have been completely redesigned and are more detailed and realistic looking than the previous models.

For example, a sedan now has wheels, windows and a glass roof, instead of the previous simplistic look that resembled a Model S keyfob.

Vehicle models are now much more detailed
Vehicle models are now much more detailed
Tesla_Raj/Twitter

Tesla currently shows seven different vehicle types, which include:

  • motorcycles
  • sedans
  • minivans/SUVs
  • pickup trucks
  • small trucks
  • tractor trailers
  • buses

Own Vehicle Attributes

When Autopilot is enabled you'll see a single line that determines the path the car is going to take. This is similar to using Navigate on Autopilot on the highway, except in this case the visualization is dotted and changes color.

Creep Visualization

A new visualization was introduced in FSD Beta 10.69 that allows you to visually see how far your vehicle is expected to creep forward. Prior to this visualization it was impossible to determine how far your vehicle was going to creep, which could make the driver feel uncomfortable in busy areas.

You can now visually see until where your vehicle will creep forward
You can now visually see until where your vehicle will creep forward
Chuck Cook/YouTube

Vehicle Path

The vehicle's intended path
The vehicle's intended path
DirtyTesla/YouTube

The line in front of the vehicle denotes the path the car is planning to take. The color of the path will vary, letting you know whether the car will be accelerating or braking at the given location.

As of FSD Beta 10.8, the line is now continous and blue. The darker blue portion of the line denotes that the vehicle will accelerate until it gets to the faded blue portion of the path. The faded blue segment represents when the vehicle will stop.

This is extremely helpful as it lets you know when the vehicle is planning to stop.

Prior to FSD Beta 10.8, the vehicle's path was shown as a dotted gray and teal line. When the dot is teal it shows that the vehicle is planning to continue moving at that location.

Tesla visualization displaying teal dots and what it means
The teal path represents when the car will accelerate
DirtyTesla/YouTube

In contrast to the teal path, the gray line denotes that the vehicle will not be accelerating at that location.

Ultrasonic Sensor Arcs

If the vehicle detects an object through its ultrasonic sensors, it will display an arc on the display in the direction of the object. The arc will change color depending on the distance of the object. The colors range from gray (furthest away), to yellow and red (closer object).

Road markings

There are various markings on the ground that help aid drivers and pedestrians.

Stopping Line

When coming to a red traffic light or a stop sign, you'll see the stopping line displayed. You may see it displayed in the car even if it's not on the road itself.

Crosswalks

Crosswalks have been displayed in FSD Betas for a while, but recently their visualization has been changed to a solid gray area inside of a pair of parallel lines. I'd like to see a texture added on top of these to include the often used crossing stripes on crosswalks.

Tesla displaying crosswalks

Arrows on Road

Arrows on the road which are often used to display which direction you can go in the given lane are displayed in the visualization.

Tesla visualization displaying road arrows

Images on the Road

Tesla FSD railroad crossing
Tesla FSD railroad crossing

Certain roads may contain images directly on the pavement to indicate a special use case. The car can correctly identify bicycle lanes, railroad crossings and handicap parking spots.

Tesla FSD can identify handicap parking spots
Tesla FSD can identify handicap parking spots

Words on the Road

Sometimes words are painted directly on the road. The car will only display a small subset of words that are seen while driving. You may see Stop and some others, but it's important to note that the car is not able to decipher each letter on the road and put together a word. The text shown in visualizations is predetermined.

Road Chevrons

You may occasionally see chevrons displayed in your lane. This visualization is not mimicking a real world object, but is instead used to let you know the car is slowing down because your lane is moving much faster than adjacent lanes.

Object Colors and Attributes

Sometimes you may see an object in the visualization change color. There are various visualization that change color to represent a special meaning.

Red Pedestrian

Similar to when a vehicle turns red, you may see a pedestrian flash red if your car is approaching a pedestrian too quickly.

Tesla warning you about a pedestrian
Tesla warning you about a pedestrian

Red Stopping Line

You'll sometimes see a stopping line turn red if you're approaching a stop sign or red light.

Blue Vehicle

If a vehicle turns blue in the visualization it means that this vehicle is in or will be in the direct path that your vehicle is planning to take. It will return to it's normal shade of gray once it passes.

Tesla visualization displaying blue cars and what it means

Red Vehicle

A vehicle will turn red when immediate action is required, such as in a Forward Collision Warning. The car has deeme that its rate of speed is too high based on the distance and speed of the vehicle in front of you.

Dark Gray Vehicle

A vehicle will turn a darker shade of gray if it's considered a lead vehicle. Your car uses a lead vehicle to help determine what to do. If your car is headed in the same direction as the lead vehicle it will follow it and use the lead vehicle to help determine the path your car should take.

A dark gray vehicle represents a lead vehicle
A dark gray vehicle represents a lead vehicle
DirtyTesla/YouTube

Brake lights

Brake lights are now displayed on other vehicles. This helps give the car another cue of when to slow down instead of basing it on the distance of the vehicle alone.

Elon has already said that Tesla will be expanding this beyond just brake lights and we'll see the car detect and react to turn signals, hazards, hand gestures and more.

The brake lights visualization has been improved
The brake lights visualization has been improved
@FrenchieEAP/Twitter

Traffic control

Traffic Lights

Tesla FSD displays traffic lights with arrows
Tesla FSD displays traffic lights with arrows

Traffic lights are displayed in collections of three and will display solid or flashing colors. The car will also display red, yellow or green arrows. You may see traffic lights displayed in other areas that use flashing lights such as "slow" signs or railroad crossings.

The visualization in the car is currently only capable of displaying the traditional three traffic lights, so regardless of how many lights there are in the object your car is detecting, it'll always display a 3-light traffic light.

Tesla visualization traffic lights

Speed Limit Signs

The car will detect and display various speed limit signs. It's not able to detect arbitrary signs, such as 23 MPH, but it will display predetermined speed limit signs.

Stop Signs

Stop signs are also displayed in red. As Tesla continues to develop FSD, I'd love to see Tesla augment real world data with map data. If the car is at an intersection that it knows there are stop signs in all directions, that's useful data when deciding when to go. It would also be a useful aid when someone is not in Autopilot, warning drivers that the crossing street does not stop.

Lines & Curbs

The car will display major lane markings and display them based on the road width and curvature. The car determines the road edges using only vision and it does not rely on map data to determine where roads are located or configured.

Single/Double Yellow Lines

Tesla visualization red lines showing a curb

The car will display continuous yellow lines and display them in yellow on the display.

Continuous/Dashed White Lines

Continuous white lines are also very accurately displayed

Red Lines

Red lines in the visualization determine the road edge and determine the driveable area for the vehicle. The red line may be at the same location as a yellow or white line if there is no space before the curb. You may also often see a large gap between a white or yellow line and the red line, which can be due to an emergency lane or large space before the curb.

Future Visualizations

Tesla continues to add more objects that the car is able to detect, react to and visualize in every update.

The latest update added speed bumps and improved sidewalks. The car can already detect emergency vehicles, but they're not displayed on the screen yet, but we know they're coming as models have already been found in recent Tesla firmware.

Teslas will soon be able to identify various emergency vehicles, including motocycles. They will also display sirens on the vehicles.

Tesla has come a long way in a short period of time with how many objects they're able to detect, but obviously when you compare the environment the car sees to the real world, there is still a lot missing.

In the short-term we'll likely see a bunch more objects visualized. We'll likely see other common objects added, especially if they appear on the road, such as trailers and gates.

In the future, I think we'll see Tesla display a rich, fuller 3D environment that will display static objects that the car will want to avoid in case of an accident, such as buildings, walls, trees, sidewalks and more.

Have a visualization we missed? Let us know and we'll add it.

Tesla is updating its 'Automatic Window Reversal System' to fix issue

By Kevin Armstrong
Tesla notified the NHTSA that it will update its automatic widnow system
Tesla notified the NHTSA that it will update its automatic widnow system

Tesla will soon send an over-the-air update to address the Automatic Window Reversal System. During product testing in mid-August, the company's technicians detected the system varied in response to obstructions in the window path. After more testing, Tesla concluded that it would notify the National Highway Traffic Safety Administration (NHTSA) of a voluntary recall to address the issue.

The majority of Tesla's recalls are much different than traditional automotive recalls. Tesla owners don't have to schedule an appointment and bring the vehicle to a service department. Instead, the company will update nearly 1.1 million Teslas using an over-the-air update. However, several news organizations picked up the recall, some linking it to the lower stock price when the market closed. Elon Musk tweeted: The terminology is outdated & inaccurate. This is a tiny over-the-air software update. To the best of our knowledge, there have been no injuries.

The recall and Musk's tweet generated a lot of discussions on social media and in comments below articles on news websites. What Tesla is doing is not a recall in traditional terms. However, it is the only course of action available to the company in the current system to notify the government and owners of required changes.

With more of the automotive industry following Tesla's lead, the recall term will be misleading as most fixes to electric vehicles should be conducted through updates, not physically bringing the car to a service center to change parts. A more fitting term that should replace recall in this new EV world should be: update. The headlines would read, Tesla is Updating its Automatic Window Reversal System. That is much more accurate to what is happening. It should surprise no one that Tesla does not fit the archaic terminology used by the industry.

In a report filed with the NHTSA, Tesla states that it "is not aware of any warranty claims, field reports, crashed, injuries or deaths related to this condition." However, starting September 13, the company started updating vehicles' firmware in pre-delivery and one production line. This change came one day after notifying the federal authorities about the voluntary recall. The over-the-air update will go to Model Xs, and Model Ss built between 2021 and 2022; Model Ys produced from 2020 to 2022 and Model 3s dating back to when the car first hit the market in 2017.

Tesla update 2022.36 features new Energy App, Supercharger popular times

By Nuno Cristovao
New Energy app
New Energy app
Not a Tesla App

Tesla's next update will be update 2022.36 instead of 2022.32. Tesla's version naming is based on the year and the week number. So update 2022.36 would have been started around the 36th week of 2022.

Tesla traditionally releases an update every four weeks, but sometimes updates can take longer to develop, test, or fix bugs, causing the version number to fall behind the calendar.

Although it doesn't happen too often, Tesla has skipped update version numbers in the past.

Given that we're now in the 39th week of the year, Tesla has chosen to skip version 2022.32 and jump directly to 2022.36.

Update 2022.36 is right around the corner and we're now having our first look at the features included in this update.

New Language

Tesla is adding support for a new language bringing its total to well beyond 20 now. Update 2022.36 introduces Lithuanian as a supported language for the vehicle's on-screen controls.

More Notifications

About two years ago Tesla added the ability to notify you via your mobile phone if the car doors, trunk or windows were left open.

With 2022.32 they're adding on to this feature and will now notify you if the car has been left unlocked.

If you have the walk-away lock feature turned on your vehicle automatically locks when you walk away if you're using a phone key or key fob. You also have the option to exclude locking when at home.

Supercharger Details

When tapping on a Supercharger icon on the vehicle's navigation system will reveal a pop-up that displays additional information about that Supercharger location.

Tesla already shows you the number of stalls available, the price and available amenities at the location. 

With 2022.36 Tesla has redesigned the pop-up and will display historical usage for each Supercharger location.

It appears that Tesla will display typical usage for the location by providing an hourly analysis of how busy the Supercharger typically is at certain times of the day.

This feature should let us easily view the busiest times at Superchargers, letting us plan our charging stops a little better.

New Energy Graph

The biggest feature of 2022.36 is by far a new energy graph.

Tesla has made tremendous improvements around its energy predictions in the last few updates and although they haven't changed the UI of the energy graph, it has resulted in much more accurate predictions that now take wind, humidity, the number of occupants, phone charging and more into account when predicting energy usage.

With 2022.36 Tesla delivers a brand new energy app that shows much more detail on how the vehicle is using energy.

The new Model S and Model X haven't had the energy graph available and now this may explain why. Tesla has likely been working on this new energy graph for a while and didn't want to spend time adapting the old app to fit the new vehicles. The new vehicles may finally receive an energy app with this update.

The new energy graph will not only let you view the energy the vehicle is using while driving but also the amount of energy that was used while the vehicle was parked.

Whether the vehicle is driving or parked, you'll now be able to see a complete breakdown of all the different systems in the vehicle and how much energy they've used. Unfortunately, Tesla doesn't display energy used in kW, but in percentage.

The vehicle will also provide suggestions on how to improve efficiency.

At the top of the energy graph where you normally have Consumption and Trip, you'll now have a new option called Park which displays energy used while parked.

The consumption screen looks similar to the way it did before. It displays the vehicle's energy usage on a graph over a chosen distance.

However, the Park and Trip views have been completely revamped.

On the Park screen, the vehicle will display various vehicle components and their energy usage. For example, the vehicle will break down its energy usage over several categories including the vehicle's screen, vehicle pre-conditioning, cabin overheat protection, Sentry Mode and mobile app usage and more.

Next to each category, the vehicle will display the percentage of the energy used for the feature and how it compares to the rest of the fleet. This could be helpful to let you know whether the amount of energy used for that feature is in line with expectations.

You'll also be able to change the timeframe since the last departure or other periods.

Similar to the Park screen, the new Trip screen also displays a breakdown of energy used across different vehicle systems, although it also adds a graph at the top, similar to what's available in the vehicle today.

The graph will automatically change colors, not based on the battery state of charge, but dependent on whether the vehicle used more or less energy in the given segment of the route when compared to its predictions.

Underneath the graph, Tesla will display the amount of energy used for driving, climate, battery conditioning, altitude, and other systems.

In addition to displaying the percent of the energy used for each system, Tesla will also compare your usage to its prediction and the percentage difference above or below the prediction.

There could be other features in the 2022.36 update that aren't mentioned here and we'll have to wait a little longer to see whether there are any additional features.

The biggest update in this release appears to be the new energy app and many owners will be thrilled with its release.

We may start seeing 2022.36 going out to the public in the next couple of weeks. You can also view the release notes for 2022.36.

Upcoming Release

View the release notes for the upcoming version 2022.36.

Tesla News

Confirmed by Elon

Take a look at features that Elon Musk has said will be coming soon.

Subscribe

Subscribe to our weekly newsletter.

Sponsors

Find out how to become a sponsor and have your site listed here.

Although we share official Tesla release notes, we are not affiliated with Tesla Motors. We are Tesla fans and supporters.

Tesla News

Upcoming Release

View the release notes for the upcoming version 2022.36.

Confirmed by Elon

Take a look at features that Elon Musk has said will be coming soon.

Subscribe

Subscribe to our weekly newsletter.