Tesla's early FSD Betas included driving visualizations that used simple wireframe boxes to represent vehicles and lane markings were made up of individual dots.
Tesla adds scalable vehicle models to the latest FSD Beta
The visualizations were a great look into some of the information that is provided to Autopilot, but even then only a fraction of the information Autopilot uses was actually displayed onscreen.
In reality, Autopilot is creating a 3D representation of every object it tracks. Each object detected then has various attributes. For example, a detected vehicle will have attributes for how fast it's going, how far away it is, the type of vehicle, its predicted path, and more.
Tesla's visualizations in early FSD Betas
The car visualizations are an important part of FSD because they help us better understand what the car is capable of seeing and reacting to. However, the information and visualization Autopilot needs is drastically different than what humans need.
In order for Tesla to achieve FSD, they essentially need to be able to build a highly accurate video game that represents the real world, in real-time.
The car wants access to as much information about each object as possible. Meanwhile, humans want a visualization that closely resembles the real world.
With the introduction of FSD Betas 9.x, Tesla released a more human consumable visualization. One that included proper 3D models of general vehicle types, road pylons, and solid lane markers.
The road edges and lane markings are more distinguished lines, 3D models have working brake lights, and other objects such as speed bumps, bike lanes, and crosswalks are depicted using visualizations that match the real world.
In order for Tesla to achieve FSD, they essentially need to be able to build a highly accurate video game that represents the real world, in real-time.
However, something that has been missing is visualizations is dynamic vehicle sizing. The 3D vehicle models that Tesla has been using have a static size. When the vehicle sees a bus, it calculates its length, width, and height in addition to a bunch of other metrics. However, the 3D model that is shown onscreen is a predefined size, meaning that it does not actually match what the vehicle saw.
This is why you may have seen a tractor-trailer shift forward and backward or you may have seen two vehicles on top of each other. One is signifying the start of the vehicle and since the vehicle is so much longer than the model, it's adding another vehicle to the end to signify the end of the vehicle.
Scalable Vehicle Models
However, in the latest 10.10.2 FSD update, we are now seeing Tesla scale individual vehicle models so that they represent the calculated size of surrounding vehicles. Contextually this could be helpful in better understanding our car’s situation in the world.
In 10.10.2, the car shrinks or stretches the 3D vehicle models in each dimension so that the 3D model matches the calculated dimensions for each vehicle. This is especially apparent in longer vehicles such as buses, trucks, and tractor-trailers, where the vehicle lengths are more likely to vary, but you can also see it scale other vehicle models such as very small cars.
In this example below, you'll see that Tesla is now able to accurately represent buses of different sizes. Tesla only has a model for a full length bus, but in this case, Tesla detected that the length of one of the buses is considerably shorter than the vehicle model so it chose to reduce the length of the bus to the length Autopilot had calculated. In the image below you can see how the same bus model is shown in two different sizes.
Tesla can now accurate render vehicles of different sizes
It's important to realize the difference between the visualizations and what Autopilot uses. The visualizations are there merely to help us better understand what Autopilot can see. The FSD computer itself has always been taking note of the size of surrounding objects and various other data points. Trajectory, approach velocity, proximity, and so forth have also been a part of this, but this update helps Tesla achieve visualizations that provide a more accurate representation of reality.
It's not only buses and trucks that are scaled up or down. Tesla resized a bobcat down to a vehicle that is about half the length of its normal sedan model.
Tesla can now accurate render vehicles of different sizes
Models are adjusted in all three dimensions. We witnessed some truck models that were stretched to become taller while also having their length reduced. It's not perfect because you're scaling all components of the truck at the same rate, but it produces a much more accurate representation of the vehicle and the amount of space it takes up.
Vehicles are resized in three dimensions to better match the vehicle's length, width and height
Tesla has come a long way in a short period with how many objects they're able to detect, but obviously, when you compare the environment the car sees today, there is still a lot missing.
In the short term, we'd like to see more objects visualized. Objects that are commonly encountered while driving, such as trailers and gates.
We'd also like to see other common objects added, such as additional traffic light configurations, crosswalks, mailboxes, and maybe even a generic object that lets us know the vehicle sees something it needs to maneuver around, but it may not know exactly what it is.
In the future, I think we'll see Tesla display a rich, fuller 3D environment that will display static and moving objects that are important for the vehicle to avoid, objects such as barriers, buildings, trees, sidewalks, and more. Today Tesla is one step closer to achieving this goal.
During Tesla’s Q1 earnings call, Tesla talked about its plan for its upcoming robotaxi network and even showed off a design for the app that would allow customers to request vehicles, much like Uber and Lyft.
While Tesla plans to unveil the robotaxi, officially now called CyberCab this August, a true autonomous taxi is at the very least, a couple of years away. So why is Tesla so eager to show off an app and start offering a Tesla taxi service?
in 2023 Uber had a revenue of 37.28 billion, while Lyft had a much smaller revenue of 4.4 billion. For comparison, Tesla’s revenue last year was 96.77 billion. A taxi service, even one operated by humans can be incredibly lucrative. While Tesla’s ultimate goal may be an automated taxi service, they may be itching to get into the space.
Tesla’s robotaxi was initially supposed to be a taxi service owners would lend their vehicles to. While that’s still the plan, Tesla wants their own vehicles to be a part of the service as well.
Will the Robotaxi Service Start With Tesla Drivers?
Tesla comparing themselves to Airbnb and Uber during their earnings call is intriguing. While Tesla can start producing robotaxis almost whenever they want, it’d likely be smart to wait until autonomy is solved so they’re not limiting themselves to current FSD hardware.
While full autonomy is likely several years away, Tesla seems to be eager to make this push toward a Tesla-owned service. Is Tesla thinking about operating their own Uber-like service? Tesla could be thinking about releasing their future robotaxi app and service in “beta,” letting current Tesla owners operate their own vehicles on the service.
This could result in several benefits for Tesla, not only letting them test their service but also opening up another revenue stream. This would allow Tesla to start operating their robotaxi network as soon as this year, and then slowly replace drivers and owner vehicles with Tesla-owned robotaxis.
Tesla Shows Off Robotaxi App
Tesla showed off it's robotaxi app
At the earnings call, Tesla also showed off a design of their robotaxi app. While it feels early to design an app for a service that could be years away, Tesla appears to be preparing itself for the future and getting customers excited about the prospect.
Tesla showed off five screens of the app, essentially showing how you’d request a robotaxi, how you can view its progress, and control certain features of the vehicle. Through the various screens, you can see most of Tesla’s Autopilot features coming to fruition in terms of autonomy, such as summon, self-driving and eventually Autopark.
Summon - Much like Tesla has re-thought the interior of a car and so many features, you can see the same mechanics applied to the app. To request a vehicle, you simply hold down a summon button and you’ll be presented with an estimate of when a vehicle will arrive.
Set Preferences and Destination - On the second screen, you can see the vehicle’s current location on a 3D map, possibly alluding to future FSD visualizations. It also lets you set your destination and set the climate temperature to your preference.
Trip Progress - While you’re traveling in the vehicle, you’ll be able to view trip information as well as set entertainment options.
Recap
Tesla had a lot to talk about during their earnings call and specifically about the Robotaxi — more information was revealed than ever before.
While there are various things at play to make a service like this come together, we can easily separate them out into separate components.
There’s the robotaxi itself, which Musk recently said would be similar to Tesla’s next-gen vehicle without a steering wheel. However, during this earnings call, he revealed that Tesla will save its new “unboxed” manufacturing process for the robotaxi and use a more traditional method for their next vehicle.
Then there’s FSD itself, while it’s crucial to operating a driverless robotaxi network, it’s not necessary to start a Tesla taxi service.
The last piece is the ride-hailing component itself and how it’s managed, and Tesla was happy to show this off, which makes us believe that it may be closer to reality than many think. While Tesla needs all three of these components to come together to operate a true robotaxi network, they piece them together separately, much like they’ve done with Autopilot. Initially, Tesla only released auto-steer then slowly added on summon, Autopark and city driving.
When we look back at Tesla five years from now, we may very well look back to this earnings call and say this was the pivotal moment when Tesla started transitioning to a services company.
Tesla has invested billions of dollars over the years toward vehicle autonomy. The mission continues as Elon Musk and Tesla now prepare to unveil their ride-hailing product, Robotaxi this August. Or, as Musk called it on Tuesday, Tesla CyberCab.
Early Days
Five years ago, during Tesla’s Autonomy Investor Day in April 2019, Elon Musk said he felt “very confident predicting autonomous robotaxis for Tesla in the following year [2020]”. At the time, Musk added a bolder claim, predicting that Tesla wouldn’t even make cars with steering wheels or pedals by 2022. While timeliness may not be Musk’s strong suit, he has a track record for getting things done that others were unwilling to try or thought were impossible. Musk later admitted he can be overly optimistic and said “sometimes I am not on time, but I get it done.”
Now, 5 years later, we have the robotaxi unveiling scheduled for August 8th. After the release of FSD v12, it’s clear that we’re much closer to autonomy than we were in 2019, although FSD v12 is still a far cry from full autonomy.
While Tesla still has the robotaxi unveiling scheduled for August, Tesla announced yesterday that it’d be prioritizing a simpler “next-gen” model that could be released by early 2025.
Latest Updates
On the conference call, Musk added that Tesla now has over 300 million miles driven with FSD v12 since it was launched just last month. He added that it's becoming “very clear that the vision-based approach with end-to-end neural networks is the right solution for scalable autonomy”.
Tesla said it has been investing in the hardware and software ecosystems necessary to achieve vehicle autonomy and a ride-hailing service. The company is confident that it can establish a scalable and profitable autonomous driving business by employing a vision-only architecture.
think of it [Tesla] as combination of Airbnb and Uber meaning that there will be some number of cars that Tesla owns itself and operates
Tesla = Uber + AirBnb
Later on, Elon Musk unveiled that the new service will operate and result in a mix between Uber and Airbnb where the Tesla driver decides if and when he wants his Tesla to be used and by whom. Tesla stated:
“We believe the Tesla software experience is best-in-class across all our products, and plan to seamlessly layer ride-hailing into the Tesla App.”
Tesla’s CEO clarified that the owners will be able to add or remove their car from the fleet “whenever they want” adding that it will be up to them to decide if they want to only let the car be used “by friends and family or only by five-star users or by anyone at any time”. The flexible program will, just like Airbnb, allow the owners to take the car out of the market when they want.
The upcoming ride-hailing service will enable users to easily request a Tesla vehicle, control the car's temperature, monitor its real-time location, and adjust the audio system. The only question is when.
TeslaFi logs your drives and charging sessions, letting you keep a log of your vehice's activity. We highly recommend checking them out if you use your car for business trips and would like to keep track of reimbursements, if you like to see how much you spend on charging or if you just love statistics. Visit their site and see everything they have to offer!
Tesla Android Project enables you to run Android apps in your Tesla. The platform is Open Source and you can deploy it on your own Raspberry Pi 4. Consider supporting the initiative by donating or purchasing the Compute Module 4 Bundle that delivers the best experience. Get $20 off by using the code: NotATeslaApp
The official Tesla app only notifies you if your car is broken into. By installing Sentry Pro on your phone, you will be notified for all Sentry Mode events. Stay connected and avoid potential surprises by receiving notifications. Stop constantly checking the cameras to ensure safety. Check only when necessary, save battery and get peace of mind. Get a 7 day free trial here!
Stay organized, connected, and stylish with Cyberbackpack — the #1 backpack, luggage & travel accessories for Tesla owners. We highly recommend checking them out if you commute or travel and would like a backpack or luggage that is sleek, secure and durable. View their products page and see everything they have to offer!
