Elon tweeted that v9 of the FSD beta would remove its reliance on radar completely and instead determine decisions based purely on vision. Humans don’t have radar after all, so it seems like a logical solution and tells us Tesla is feeling much more confident in their vision AI.
Radar and vision each have their advantages, but radar has thus far been much more reliable in detecting objects and determining speed. If you’ve ever noticed your Tesla being able to detect two vehicles in front of you when you can only see the one directly ahead of you, that’s radar at work.
In this situation the radio waves from the radar sensor are bouncing underneath the car in front of you and are able to continue traveling and detect that there is another object ahead even though it could never “see” it.
It really is one of those wow moments where you can feel the future and the ability for AI-powered cars to drive better than humans one day. It’s baby steps and slowly we’ll see more and more of these situations where the vehicle simply sees or does something we could never do.
There’s no doubting that more sensors could provide a more reliable and accurate interpretation of the real world as they each have their own advantages. In an ideal world a vehicle with radar, lidar, vision, ultrasonic sensors and even audio processing would provide the best solution. However, more sensors and systems come at a price, resulting in increased vehicle cost and system complexity.
After all humans are relatively safe drivers with two “cameras” and vision alone. If Tesla can completely solve vision, they’ll easily be able to achieve superhuman driving capabilities. Teslas have eight cameras, facing in all directions. They’re able to analyze all of them concurrently and make much more accurate interpretations then we ever could in the same amount of time.
Tristan on Twitter recently had some great insight into Tesla vision AI and how they’re going to replace radar. Here’s what Tristan had to say:
"We recently got some insight into how Tesla is going to replace radar in the recent firmware updates + some nifty ML model techniques
From the binaries we can see that they've added velocity and acceleration outputs. These predictions in addition to the existing xyz outputs give much of the same information that radar traditionally provides
(distance + velocity + acceleration).
For autosteer on city streets, you need to know the velocity and acceleration of cars in all directions but radar is only pointing forward. If it's accurate enough to make a left turn, radar is probably unnecessary for the most part.
How can a neural network figure out velocity and acceleration from static images you ask?
They can't!
They've recently switched to something that appears to be styled on an Recurrent Neural Network.
Net structure is unknown (LSTM?) but they're providing the net with a queue of the 15 most recent hidden states. Seems quite a bit easier to train than normal RNNs which need to learn to encode historical data and can have issues like vanishing gradients for longer time windows.
The velocity and acceleration predictions is new, by giving the last 15 frames (~1s) of data I'd expect you can train a highly accurate net to predict velocity + acceleration based off of the learned time series.
They've already been using these queue based RNNs with the normal position nets for a few months presumably to improve stability of the predictions.
This matches with the recent public statements from Tesla about new models training on video instead of static images.
To evaluate the performance compared to radar, I bet Tesla has run some feature importance techniques on the models and radar importance has probably dropped quite a bit with the new nets. See tools like https://captum.ai for more info.
I still think that radar is going to stick around for quite a while for highway usage since the current camera performance in rain and snow isn't great.
NoA often disables in mild rain. City streets might behave better since the relative rain speed is lower.
One other nifty trick they've recently added is a task to rectify the images before feeding them into the neural nets.
This is a common in classical CV applications so surprised it only popped up in the last couple of months.
This makes a lot of sense since it means that the nets don't need to learn the lens distortion. It also likely makes it a lot easier for the nets to correlate objects across multiple cameras since the movement is now much more linear.
For more background on LSTMs (Long Short-Term Memory) see https://towardsdatascience.com/illustrated-guide-to-lstms-and-gru-s-a-step-by-step-explanation-44e9eb85bf21
They're tricky to train because they need to encode history which is fed into future runs. The more times you pass the state, the more the earlier frames is diluted hence "vanishing gradients".
Tesla’s FSD beta v9 will be a big improvement forward from what FSD beta users have been using where the system was still relying on radar. And it’ll be an even bigger leap from what non-beta testers currently have access to. We can’t wait. Now where’s that button?
Almost ready with FSD Beta V9.0. Step change improvement is massive, especially for weird corner cases & bad weather. Pure vision, no radar.
A sketch of a two-door Cybertruck can be found at the Peterson Automotive Museum
MissJilianne/Twitter
The Tesla crowd of enthusiasts has eyes everywhere. Twitter user @MissJilianne caused quite a stir after visiting the Peterson Automotive Museum in Los Angeles, which recently opened a new Tesla exhibit. The Model S Plaid owner and FSD Beta tester posted a few pictures of her visit, including some artwork at the Cybertruck display.
The artistic design shows the Cybertruck in a two-door version. Miss Jilianne asks: Are we not going to talk about the beautiful artwork of a two door Cybertruck displayed at the @Petersen_Museum Tesla exhibit?
Her tweet generated discussion on social platforms and Tesla blogs, and it had mixed reactions. @KounisTou tweeted: It's not only that, but it has a totally different, dare I say better looking? A-pillar design with a wraparound windshield like early Semi prototype. Looks great. Except maybe that it looks a bit too much like a Lamborghini.
That may be the biggest clue to this spectacular piece of art. Franz von Holzhausen, Tesla's chief designer, is a huge fan of the “wedge” design. He grew up with Lamborghini Countach posters on his wall. Other responses to the picture believe it is the origin of the piece of art. It is likely an early sketch of what the Cybertruck could look like, and there were likely several. Given von Holzhausen’s love for the Countach, this is quite likely an early concept he was sketching out.
Elon Musk has publicly stated his support for a smaller truck. On November 24, 2019, he tweeted: Long term, it probably makes sense to build a smaller Cybertruck too.
However, the four-door version is due in 2023. CNBC interviewed von Holzhausen right in front of the Cybertruck. He stated, “Cybertruck will look, for all intents and purposes, just like the one behind us, maybe slightly smaller, a few percentages, but in general, this is what the truck will look like.”
Still, the designer did say something else during that interview that leaves the door open for a possible two-door Cybertruck. The creative mind behind the Model S, 3, X, Y, Semi and Cybertruck was asked to pick a favorite. His response was, "My favorite one is the one that is coming. I can't talk about (it).”
For a man that loves sports cars, it was believed that he was referring to the second-generation Roadster. Maybe he is referring to a smaller Cybertruck that looks a lot like those wedge cars that inspired him to become a car designer.
Tesla has a history of providing sketches to early reservation holders of their vehicles. It waits to be seen whether Tesla will do the same for the Cybertruck when deliveries start next year.
Below you'll find other sketches that Tesla provided to Model S, 3 and X reservation holders.
Model S Sketch
An early sketch of the Model S that was given to owners
Electrek
Model 3 Sketch
A sketch of the Model 3 was given to owners who reserved the vehicle on the first night
Electrek
Model X Sketch
An early sketch of the Model X that was initially given to owners of the Model X
Tesla deliveries the first semi trucks to Pepsi Co
Tesla
The first Tesla Semis have been delivered, and so has the promise to adhere to the company’s mission: accelerate the advent of sustainable transportation. Yes, it took five years from the first peak at the Semi and three years longer than initially projected, but the Semi is set to change the trucking industry the same way the Model S, 3, X and Y changed the automotive industry.
While Tesla has been increasing the number of passenger vehicles it produces worldwide; these large trucks are not needed at the same level. The company states that 15 million passenger vehicles are sold in the U.S. every year, but only a couple hundred thousand big rigs. So why would Tesla put so much time and effort into something that doesn’t sell as many units?
Musk’s response, “it seems like a small percentage, but it (semi-trucks) is actually 20 percent of U.S. vehicle emissions. Because you’ve got a huge vehicle, and it’s being driven all the time.” Musk said, “Over a third of all the articulate emissions (are from semi-trucks). So, from a health standpoint, particularly in cities, this is a huge impact, like gigantic — that’s why we are doing it.”
The Semi is a Recruiting Tool for Truckers
Back in 2017, Elon Musk said he could drive a Tesla Semi. That was quite a statement, given that trucks this size usually come with anywhere from 10 to 18 gears. However, the CEO did just that five years later. He drove the newest member of the Tesla lineup to the event to deliver the Semi to Pepsi Co. Musk said, “It’s like driving a Tesla, literally.” That is because it’s built the same way with a single stalk to put the vehicle in drive, reverse, neutral or park. “It looks sick. I mean, you want to drive that! That thing looks like it came from the future… It’s fun. It looks awesome, there is a big shortage of drivers, so if you are a truck driver and want the most bad ass rig on the road - this is it.”
The Semi is known as the Beast
Senior Manager of Tesla’s Semi Engineering, Dan Priestley, explained why many people refer to the Semi as the beast. “(It has) three times the power than any diesel truck on the road right now. So you’ve got all the power you need to get the job done, but the other reason that it’s a beast is because it is efficient, you can go 500 miles on a single charge on one of these things. It’s the mix of those two - this is a game-changer. What’s awesome is both of those are enabled by our 1000-volt powertrain. This is the first vehicle we are doing with that.”
That max power combined with hyper-efficiency is due to a tri-motor power train system. One motor is for highway driving and will always be engaged, while the other two are for torque and acceleration. Those motors use the system straight out of the Model S Plaid powertrain. Priestly said, “These are clutched automatically. No driver input is needed. It is seamless.”
Musk commented on the Semi’s performance, “It looks crazy. It looks like an elephant moving like a cheetah.” He was also astonished at the size of the motor powering the Semi. It’s described as the size of a football, “I find it amazing that this enormous thing can be pulled by something that you can carry in your hands — it’s like, wow, that is power density.”
500-mile Range
The Semi has been tested in all conditions in the real world, in a lab and virtually. One of those real-world tests was proving the naysayers wrong by taking the Semi for a 500-mile drive, fully loaded, from Fremont to San Diego on just a single charge.
Elon Musk later revealed on Twitter that the Tesla Semi's current efficiency is about 1.7kWh/mile, but that there is a path for Tesla to make it even more efficient. It's not clear whether the additional efficiency could be achieved through software or if it would require hardware changes. The quoted 1.7kWh/mile efficiency is a pleasant surprise, as many were expecting the truck to have an efficiency of about 2kWh per mile.
V4 Superchargers
Speaking of charging, the new V4 Superchargers were also revealed. They produce 1 megawatt of energy and will be used for the Semi and the Cybertruck.
Tesla's Semi Delivery Event
Musk finished the event with this message: This is going to revolutionize the road, make the world a better place in a meaningful way.
TeslaFi is a service that logs your drives and charging sessions so that you can later refer back to them. We highly recommend checking them out if you use your car for business trips and would like to keep track of reimbursements, if you like to see how much you spend on charging or if you just love statistics. View their about us page and see everything they have to offer!
DIMO is building a web3, user-owned network dedicated to supporting the next generation of mobility infrastructure. As a user, you can start today by accessing the best connected vehicle experience via the DIMO Mobile App. It works for nearly any vehicle and across any OEM; users are in control of their data and their DIMO wallet is a conduit to other apps and services, saving time and money. Learn more
Find out how to become a sponsor and have your site listed here.
Although we share official Tesla release notes, we are not affiliated with Tesla Motors. We are Tesla fans and supporters.
