Tesla Full Self-Driving to Recognize Hand Gestures in Upcoming Update, According to Employee

By Karan Singh
Tesla has been improving its FSD features since introducing it in 2016
Tesla has been improving its FSD features since introducing it in 2016
Tesla

Previously, the Tesla X account shared a video where FSD V12 appeared to consider a pedestrian’s hand gestures when navigating. Elon Musk also confirmed on X that hand gesture recognition would be improved in V12.4, and even better by V12.5.

Tesla has had plans for gesture recognition for many years, with Musk mentioning hand gesture recognition in 2021. FSD is supposed to be able to adapt to new and unique circumstances in a safe manner, whether following instructions from a police officer, or understanding the hand signals from a cyclist.

Boris Johnson and FSD

Boris Johnson, the ex-Prime Minister of the UK, recently got to experience FSD V12 in the crowded streets of LA, with his wife and child in the backseat. Tesla provided a vehicle and assistant for his self-driven experience through 5-road intersections, heavy traffic, and pedestrians.

At the end of about 45 minutes I feel like a driving test examiner – except that I want to tell the car that it has passed, with flying colours. -Boris Johnson

Gestures coming soon

Johnson had a single experience with hand gestures in his drive – being waved at a hotel. While the car didn’t recognize the gesture – even on FSD V12.3.6 – the Tesla official in the vehicle confirmed that “‘It doesn’t yet recognise that gesture, but we are fixing it for the next iteration. It should be done next month.”.

It’s not immediately clear whether the Tesla employee is referring to FSD v12.4, which is expected to start rolling out to the public later this week, or Tesla’s next major FSD release, V12.5. According to Musk, V12.5 is expected to handle much more complicated situations and even start vehicle-to-fleet communication.

Solving FSD

When you’re tackling a problem like FSD, which has never been solved before, you sometimes have to pivot and realize there’s a better approach. That’s what Tesla has done with FSD and AI. Just a few years ago where people thought cars would need to be able to read signs to achieve autonomy, but with the latest approach, there’s no need to. As long as the vehicle has seen a specific sign before and has enough examples, it’s already trained on what humans do when such a sign is found. The same will be true for hand gestures, the vehicle won’t need to know what each hand gesture means, it’ll just recognize that when a current gesture is made, this is how the vehicle should react.

Tesla does this by feeding it millions of examples of very specific situations and AI recognizes patterns. This is an example where Tesla leverages fleet data. They can easily capture millions of examples of hand gestures and how humans reacted, and feed that to their AI training model.

As Tesla processes more video, FSD will continue to improve. Tesla is expected to spend $10 billion on AI this year alone, most of it going toward improving FSD.

Optimus - What We Learned About Tesla's Robotic Future

By Karan Singh
Optimus Gen 2
Optimus Gen 2
Tesla

Optimus was a major point of coverage at the 2024 Tesla Shareholder meeting, and we’ll help break down some of the key points for those interested in Tesla’s future humanoid robots.

What Is It?

Optimus is Tesla’s humanoid robot, built entirely in-house, from the batteries to the motors and actuators in the arms, legs, and hands. Tesla has taken a unique design approach to Optimus and intends to have it replace humans in mundane or risky tasks.

It is a bipedal robot, built around the same aspect as the human body. Optimus was originally unveiled in August 2021 and has since seen several major design iterations. And those aren’t the only ones, Optimus is scheduled to undergo at least one more major design revision this year, as well as one more major design revision for its hands – which will feature 22 degrees of freedom.

In comparison, the human hand has 27 degrees of freedom – Tesla is quite close to replicating the complexity of a hand in its custom-designed hands. Musk mentioned that with the 22 degrees of freedom, Optimus is capable of learning and playing music on a piano – an intricate task that many humans find difficult today.

Best of all, they’ve placed the immense learning prowess of FSD behind its brains – each Optimus unit runs similar hardware and software as Tesla cars . It can also navigate autonomously, using the same object recognition and learning that Tesla’s cars use every day. Optimus learns from watching humans do things or can be taught how to do something by a remote operator. Elon Musk also mentioned that it will eventually be able to watch a video and learn how to do a task.

What Can It Do?

Elon Musk has mentioned that Optimus’ primary goal is to replace humans in certain tasks, especially those that could put a human at risk. This could be anything from being a humanoid companion or caretaker, a construction worker, or even working in factories. Of course, it has a focus on high-precision tasks, owing to its intricately designed hands, and is intended to replace human workers doing everyday precision work that robots today cannot do.

The primary goal is to have Optimus robots begin working in factories, and to this end, two have been deployed to one of Tesla’s factories, and are working on the battery cell assembly lines in a prototype and testing deployment. Today, these two units are moving battery cells off the production line and into shipping containers.

2:1 Robot to Human Ratio

There are some ambitious plans for Optimus – Elon Musk envisions that there will be 2 humanoid robots for every human on the planet in the future. This is alongside an eye-watering build rate of 1 billion humanoid robots a year – of which Tesla intends to build at least 100 million per year or more.

With these numbers, Tesla sees the market cap for Optimus as double that of FSD – approximately $20 trillion, with an expected profit of $1 trillion per year at scale. That’s an expected profit of $10,000 per unit, which will be quite the achievement.

When’s It Coming?

Given the fact that Tesla still has design revisions planned, scale production isn’t starting anytime soon. However, Elon Musk did mention that Tesla currently plans to have approximately 1000 to 2000 Optimus units deployed for internal use in Tesla factories by the end of next year. This limited production run will be the start of Tesla’s larger Optimus deployments and will serve to help them refine the FSD stack that runs Optimus, helping teach it the many tasks it could do in a factory.

Costs

The next big question is what it will cost. Musk has mentioned that it will cost less than a car – with an expected cost of $20,000 USD, once large-scale production kicks off. Just like the Cybertruck, that means initial adopters will be faced with fairly high adoption costs for the initial production runs. Economies of scale will eventually lower the cost as more units are produced.

One of Tesla’s significant challenges will be scaling to reduce these costs. Currently, each unit is hand-built in Tesla’s Optimus labs. Eventually, this will have to scaled up to a proper production line, which will require a factory. Optimus also uses 4680 cells, which means some production of the newer 4680 batteries will be required to produce Optimus.

So perhaps, someday soon, there will be an Optimus knocking on your door, delivering itself to help you take care of your home. Definitely a bright future to look forward to.

A Better Routeplanner 5.0 Launches; Adds EV Charger Ratings Using Rivian Data

By Karan Singh
Not a Tesla App

A Better Routeplanner 5.0 launched yesterday, and there are some pretty awesome features coming to all EV owners courtesy of Rivian. Rivian purchased ABRP last year and has made good on its promises to continue its improvement and ensure it remains open to all EV owners.

Charger Scoring

Rivian recently added a feature that would rate any chargers compatible with Rivian vehicles. The list of chargers includes Rivian Adventure Network (RAN) chargers, Tesla Superchargers and any other compatible third-party chargers. The charger score is automatically calculated based on the station's average top speed and reliability.

With the launch of ABRP 5.0, Rivian is integrating its charger scores directly into the free tier of ABRP so that all EV owners can benefit. ABRP users will now be able to see charger scores, and ABRP will automatically route users to chargers with higher scores if they are available on your route.

Google Automotive

Another cool feature for ABRP is that it will now be available as an app to install and use directly in vehicles that support Google Automotive. Any EV that uses Google Automotive, including Volvo,  Polestar, Ford, and GM will support the in-system experience, which will also provide data for charger scoring and routing.

This will be an excellent way to hold third-party networks accountable, which have commonly suffered from uptime or speed issues.

Tesla’s Implementation

Tesla previously implemented a “Qualified Third-Party Charger” program, that would allow highly-rated third-party chargers that meet a strict set of requirements to be displayed directly in the vehicle. However, this is currently limited to Europe and parts of the Middle East. Within North America, Tesla only displays third-party Tesla destination chargers in addition to Superchargers.

While Tesla doesn’t directly show charger scores, they clearly are tracking charge data, and are providing the cream of the crop of third-party chargers for navigation where the program is available. We’d hope that this implementation of qualified third-party chargers also comes to North America, as NACS is becoming the de facto standard for charging.

If Tesla does expand the display of third-party chargers to other regions, it’ll likely be similar to what we see in Europe today, and won’t be as open as Rivian’s implementation in ABRP.

Latest Tesla Update

Confirmed by Elon

Take a look at features that Elon Musk has said will be coming soon.

More Tesla News

Subscribe

Subscribe to our weekly newsletter.

Sponsors

Find out how to become a sponsor and have your site listed here.

Although we share official Tesla release notes, we are not affiliated with Tesla Motors. We are Tesla fans and supporters.

Tesla Videos

Latest Tesla Update

Confirmed by Elon

Take a look at features that Elon Musk has said will be coming soon.

Subscribe

Subscribe to our weekly newsletter.