Why Full Self-Driving is Harder Than We Think

April 30, 2021

By Barry Shell

It’s now been over a year since Elon Musk started saying Full Self Driving (FSD) is coming “next month.” If you are wondering what’s taking so long maybe check out this recent paper “Why AI is Harder Than We Think” by Melanie Mitchell. She is the Davis Professor of Complexity at the Santa Fe Institute, and Professor of Computer Science (currently on leave) at Portland State University. Her research focuses on conceptual abstraction, analogy-making, and visual recognition in artificial intelligence systems. Given that Tesla FSD relies on visual AI it would seem that she may be qualified to explain the delay in FSD rollout. Her paper does not mention Tesla or FSD at all, but I think her observations apply to the current status of FSD.

Mitchell points to four classic fallacies in the predictions made by AI developers:

1. Narrow intelligence is on a continuum with general intelligence

It’s easy to assume that if you make some incremental progress in an AI problem that it’s just a matter of time before you solve the whole thing. I.e. just a few more months to FSD. But Mitchell says that’s like claiming that the first monkey that climbed a tree was making progress towards landing on the moon. Ain’t gonna happen. Plus there’s this unexpected obstacle in the assumed continuum of AI progress. “The problem of common sense,” she says, which humans have subconsciously but AI systems lack completely. Nobody knows how to code for common sense, which comes in handy when you’re driving a car.

Tesla FSD

2. Easy things are easy and hard things are hard

In fact easy things for us are hard for computers. She quotes Hans Moravec the computer scientist who came up with one of the first algorithms for computer vision. He once wrote, “It is comparatively easy to make computers exhibit adult level performance on intelligence tests or playing checkers, yet difficult or impossible to give them the skills of a one-year-old when it comes to perception and mobility.” Unfortunately FSD is all about perception and mobility. We simply don’t appreciate the complexity of our own thought processes and we overestimate how easy it is to give these abilities to a computer. Mitchell says Moravec put it this way: “Encoded in the large, highly evolved sensory and motor portions of the human brain is a billion years of experience about the nature of the world and how to survive in it.” FSD doesn’t have any of this. Mitchell then quotes the grandfather of AI Marvin Minsky who said, “In general, we’re least aware of what our minds do best.”

3. The lure of wishful mnemonics or metaphors

The computational technique underlying FSD is a neural network, a metaphor loosely inspired by the brain, but with major differences. Mitchell says, “Machine learning or deep learning methods do not really resemble learning in humans (or in non-human animals). Indeed, if a machine has learned something in the human sense of learn, we would expect that it would be able use what it has learned in different contexts. However, it turns out that this is often not the case.” Computer scientist Drew McDermott first used the term “wishful mnemonics” in 1976, pointing out that by labeling some computer code “Full Self Driving,” for instance, we are imbuing it with the “wish” that it will actually do what it says. He said a better idea would be to label it “G0034” and then see if the programmers can convince themselves or anyone else that G0034 implements some part of self driving. We all seem to be caught in a sort of wishful FSD state at the moment. But wishing it won’t make it happen.

4. Intelligence is all in the brain

This fallacy assumes intelligence is disembodied and lives only in the brain, the so called “information processing model of mind.” It’s the old idea that if you had enough computing power you could “upload” a mind into a machine. But a growing number of cognitive scientists now believe in a sort of “embodied cognition.” Mitchell says, “Nothing in our knowledge of psychology or neuroscience supports the possibility that “pure rationality” is separable from the emotions and cultural biases that shape our cognition and our objectives. Instead, what we’ve learned from research in embodied cognition is that human intelligence seems to be a strongly integrated system with closely interconnected attributes, including emotions, desires, a strong sense of selfhood and autonomy, and a common sense understanding of the world. It’s not at all clear that these attributes can be separated.” While one could argue that Tesla’s FSD also embodies the car via sensors and cameras it may take more than a few weeks before Tesla’s programmers pull off this trick that took Nature a billion years to integrate.

Ultimately Mitchell uses these four fallacies to explain the cyclic nature of AI research since its inception in the 1970s. It tends to blossom in Spring time with magnificent overconfident predictions at first, but then when the scale of the challenge is realized, a sort of AI Winter descends and progress can stall for up to a decade.

Obviously we all want Tesla to be successful and pull off Full Self Driving next week, next month, or even next year. And Elon Musk has pulled a rabbit out of a hat more than once. Who can forget when those two returning Falcon Heavy booster rockets landed perfectly and simultaneously in 2018? So fingers crossed. But if you’re ever craving an explanation for why FSD is taking so long devote 20 minutes to Melanie Mitchell’s paper.

Is Tesla Planning to Add Steam Support to All Vehicles?

December 11, 2024

By Karan Singh

Not a Tesla App

Yesterday, we reported that Tesla updated their Steam integration on Model S and Model X vehicles. The update was part of their 2024 Holiday Update, but it looks like there may be more to this than a simple update.

Steam, a video game library app, makes it easy for users to buy or launch games on their computers. However, a couple of years ago, Valve, who created Steam, launched their own standalone device, the Steam Deck. The Steam Deck runs a custom OS based on Linux.

Steam Launch

When Tesla launched the redesigned Model S and Model X, Tesla introduced a dedicated gaming GPU with 16GB of RAM and touted the ability to play top-tier PC games in Tesla vehicles.

In 2022, Tesla finally launched the Steam app for the Model S and Model X as part of its 2022 Holiday Update. The Steam app runs Steam OS, the same OS as the Steam Deck in a virtual environment.

However, earlier this year, Tesla stopped including the GPU and Steam (Beta) in their vehicles, and we haven’t seen any updates to the Steam in quite some time. In fact, we thought Tesla was axing their gaming-on-the-go dreams.

SteamOS Update

The Steam app, which is still in Beta, is getting an interesting update for the Model S and Model X vehicles with the discrete GPU.

Those vehicles received an update to SteamOS 3.6 - the same version of SteamOS that runs on the Steam Deck. While nothing has visually changed, there’s a long list of performance optimizations under the hood to get things running smoother.

Comparing Steam Deck to Tesla Vehicles

Let’s take a look at the Steam Deck - according to Valve, its onboard Zen4 CPU and GPU combined push a total of 2 TFlops of data, which is fairly respectable, but much lower than today’s home consoles. The Steam Deck is capable of 720p gaming fairly seamlessly on low-to-medium settings on the go and is also built on the AMD platform.

AMD-equipped Teslas, including the Model 3 and Model Y, are packing an older Zen+ (Zen 1.5) APU (processor with a combined CPU and GPU). AMD claims that the V1000 - the same embedded chip as on AMD Tesla vehicles (YE1807C3T4MFB), brings up to 3.6 TFLops of processing power with it, including 4K encoding and decoding with the integrated GPU on board.

While that’s not enough for 4K gaming or comparable to a full-blown console or desktop GPU, that’s enough raw horsepower for light gaming and is currently more powerful than the Steam Deck.

The Model S and Model X’s GPU brings that up to about 10TFlops of power - comparable to modern consoles like the Xbox Series X at 12 TFlops.

Steam Gaming for All Vehicles?

The fact that Tesla is updating SteamOS even though the feature is no longer available in any new vehicles could indicate that Tesla is not only bringing Steam back to Teslas but that it’s going to play a much bigger role.

While SteamOS is run in a virtual environment on top of Tesla’s own OS, we could see Tesla bring SteamOS to all of its current vehicles, including the Model 3, Model Y, and Cybertruck. Steam in these vehicles would likely support any game that’s capable of running on the Steam Deck.

We think this Steam update, which includes performance improvements and a variety of fixes, has quietly passed under most people’s radars. This could be a very exciting update for those who enjoy gaming, especially for those who love to do it in their Tesla.

Tesla Holiday Update Weather Features: All the Small Details

December 11, 2024

By Karan Singh

Robert Rosenfeld / YouTube

As part of Tesla’s 2024 Holiday Update, Tesla included two awesome new features - Weather at Destination and the long-awaited Weather Radar Overlay. These two features are big upgrades built upon the weather feature that was added in update 2024.26. The original weather feature added an hourly forecast, as well as the chance of precipitation, UV index, Air Quality Index, and other data.

However, this update also added some smaller weather touches, such as the vehicle alerting you if the weather at the destination will be drastically different from the current weather.

Not a Tesla App

Weather At Destination

When you’re navigating to a destination and viewing the full navigation direction list, the text under the arrival time will show you the expected weather next to your destination. You can also tap this, and the full weather pop-up will show up, showing your destination's full set of weather information.

Note the weather under the arrival time

Not a Tesla App

You can also tap the weather icon at the top of the interface at any time and tap Destination to switch between the weather at your current location and the weather at your destination.

You’re probably considering that the weather at your destination doesn’t matter when you’re three hours away - but that’s all taken into account by the trip planner. It will add in both charge time and travel time and show you the weather at your destination at your expected arrival time.

And if the weather is drastically different or inclement, such as rain or snow, while you’ve got sunshine and rainbows - the weather will be shown above the destination ETA for a few moments before it tucks itself away.

Tesla also recently introduced a new voice command. Asking, “What’s the weather?” or something similar will now bring up Tesla’s weather popup.

The weather pop-up above the ETA

Not a Tesla App

One limitation, though—if you’re planning a long road trip that is more than a day of driving, the weather at destination feature won’t be available until you get closer.

Weather Radar Overlay

As part of the improvements to weather, Tesla has also added a radar overlay for precipitation. You can access the new radar overlay by tapping the map and then tapping the weather icon on the right side of the map. It’ll bring up a radar overlay centered on your vehicle. It’ll animate through the radar data over the last 3 hours so that you can see the direction of the storm, but you can also pause it at any point.

You’re able to scroll around in this view and see the weather anywhere, even if you zoom out. It also works while you’re driving, although it can be a little confusing if you’re trying to pay attention to the navigation system. If you like to have Points of Interest enabled on your map, the weather overlay will hide POIs except for Charging POIs.

Requirements / Data

Unfortunately, you’ll need Premium Connectivity for any of the weather features to work, and being on WiFi or using a hotspot will not be enough to get the data to show up. The data, including the weather radar, is provided by The Weather Channel.

As for supported models, weather and weather at destination are available on all vehicles except for the 2012-2020 Model S and Model X. The weather radar has more strict requirements and requires the newer AMD Ryzen-powered infotainment center available on the 2021+ Model S and Model X and more recent Model 3 and Model Y vehicles.