Tesla's FSD hardware 4.0 to use cameras with LED flicker mitigation

January 13, 2022

By Nuno Cristovao

At Tesla's AI Day event last summer, Elon answered some questions during the Q&A portion of the event that revealed Tesla's plans for FSD hardware.

Elon informally announced that Tesla is already researching the next major hardware revision for full self-driving.

Tesla's camera housing

Elon said that he expects the current hardware in today's vehicles to be capable of "super-human" driving, but Tesla will continue to update their cars to have the most powerful hardware available.

Elon guessed that Tesla may release FSD hardware 4.0 with the introduction of the Cybertruck. Since the Cybertruck won't be released until the end of this year at the earliest, we may even see new hardware before its arrival, although this is unlikely.

In another question related to new hardware, Elon also said that Tesla is looking into new cameras. He was clear that the current cameras aren't a hindrance, but if Tesla can do better, then they will.

Elon didn't go into details about whether the new cameras would be tied to the Cybertruck or hardware 4.0. He also didn't mention specifically whether the new cameras would be higher resolution, contain larger sensors, or possibly even be placed in different locations.

Tesla currently utilizes three separate front-facing cameras in the housing at the top of the windshield. The camera unit contains wide-angle, standard, and narrow view lenses with three separate CMOS sensors.

Each sensor has a resolution of 1280x960 or about 1.2 megapixels.

New Camera

According to Chris Zheng on Twitter, who has connections with some Tesla suppliers, Tesla is planning to use Sony's new IMX490 automotive sensor in their upcoming hardware 4.0 revision.

Wide-angle camera comparison

This new sensor offers many benefits over the current sensors that Tesla uses in their vehicles.

The new sensors support a much higher resolution of 2896x1876. That's 5.4 megapixels compared to the 1.2 megapixels that Tesla's current front-facing cameras support.

At more than four times the resolution of current sensors, these new sensors will allow Tesla to more accurately detect objects that are required for FSD.

Detecting road edges and lane markings that are further away are especially difficult. Due to the angle of the road and the current image resolution, Tesla often has to make assumptions based on just a few pixels.

The increased resolution will offer a more detailed image and should improve the accuracy in these situations.

The new wide-angle sensor may also allow Tesla to drop the number of cameras used in the front-facing module. Due to the higher resolution in these sensors, Tesla may be able to use this wide-angle sensor to interpret objects that previously required the use of the standard or telephoto lenses before. This may allow Tesla to reduce the number of cameras that are required in the front of the vehicle from three down to two or even a single lens.

The new camera sensor also supports HDR shooting at 10 bit and 40 FPS, offering better contrast and richer colors. It's not clear whether Tesla would take advantage of HDR as it would require more power to process, but it's something Tesla may consider depending on the compute power available.

LED flicker mitigation

More importantly, the new sensor will also support LED flicker mitigation. Camera sensors often have a hard time capturing the light emitted from LED lights in a constant fashion. Since LEDs are widely used in electronic signs and traffic lights this will be an important feature and should make it easier for Tesla to interpret such signs.

Tesla is working closely with Samsung to produce their next-generation chips that will be used in FSD hardware 4.0. Tesla will likely launch updated cameras at the same time they introduce a new FSD computer.

It makes sense for Tesla to introduce this new hardware in a new model first before retrofitting new camera modules into existing models.

Tesla Is Now Offering Cybertruck Test Drives [List of Locations]

July 26, 2024

By Karan Singh

@brandonhd

Tesla has begun to reach out to customers in the United States and is offering demo drives of the Cybertruck on an invite-only basis. @brandonhd on X shared the first invite image, and it looks like it's going out quite quickly to multiple locations that have Cybertrucks on display.

Up until now, display vehicles have been for just that – looking, but no touching. Some Tesla delivery centers and showrooms have also allowed viewers to take a seat inside and open it up, but most are still locked away behind the velvet rope.

It’s exciting to see that Tesla has begun to demo them – and it can only be so long until demo drives open up to everyone.

Advisor-led Demo

This Demo Drive is led by a Tesla Advisor – while you’re driving, the advisor will be in the passenger seat, walking you through the experience. They’ll explain steer-by-wire, off-road controls, and will introduce new-to-Tesla buyers to all the features that are currently available.

Once FSD V12.5 comes to the Cybertruck, we’re sure they’ll be demoing its capabilities FSD capabilities as well.

Locations

The list of locations so far for demo drives appears to be fairly limited, but we will hopefully see demo drives roll out throughout the United States, and eventually Canada too. Here is the current list of city/state locations that have seen demo drive invites.

Manhasset, New York
West Covina, California
Los Angeles, California
Houston, Texas
Orlando, Florida
Westmont, Illinois
Tucson, Arizona
Phoenix, Arizona
Delaware

Musk Confirms 'Actually Smart Summon' is Coming in a Revision of FSD V12.5

July 26, 2024

By Karan Singh

Not a Tesla App

Actually Smart Summon, the successor to Tesla’s Smart Summon feature, has been confirmed to be coming as soon as next month as per Elon Musk. Colloquially known as “ASS”, Actually Smart Summon is supposed to improve on the regular Smart Summon capabilities, which have historically used ultrasonic sensors (USS) to navigate parking lots.

We’re looking forward to Actually Smart Summon, and we’re quite excited about what other features will be unlocked once it finally shows up at our doorstep.

Vision-based Improvements

ASS is supposed to bring Smart Summon to vehicles that don’t have USS – using Tesla Vision. Most recently, Tesla has brought Autopark to vehicles without USS, and Vision-based Autopark has been a huge success. It is faster, more reliable, and smoother than the older USS-based solution. It can also park and maneuver in tighter locations.

Just recently, Tesla returned the Model X’s self-presenting doors, using Tesla Vision. Vision has seen some fantastic improvements, and many people are waiting for ASS to bring forth these improvements to Smart Summon.

Smart Summon Today

Today, Smart Summon is sometimes useful at best, usually a party trick, or downright dangerous at worst. It’s excellent and best used in straight lines, where you have a clear line of sight of the vehicle, and can stop it if doesn’t see an obstacle.

I used Smart Summon today, just before writing this article – my 2022 Model Y (with USS) – did manage to show up just fine – in a straight-line location from where I was parked, after exiting the parking spot fairly cleanly. However, I’ve previously used Smart Summon and have had the vehicle turn towards obstacles, such as cart returns or curbs. Overall, it’s a sometimes-useful feature that has a lot of drawbacks right now.

Smart Summon is also restricted to a certain distance – and it's pretty small. The max distance is shorter than most parking lots, being only about 215 ft (65m). If you’re lucky enough to park close, it's very useful to bring your car right to you.

Actually Smart Summon

ASS will bring FSD V12-like capabilities to Smart Summon – the vehicle will primarily use Vision to drive toward you (or the location you’ve indicated). ASS has been supposed to be coming since about September 2022 and has seen many delays – we’re almost at the 2-year point today.

We’re looking forward to Actually Smart Summon being able to bring the vehicle – safely – right to you. ASS is also one of the key steps towards bringing some other cool features that Elon Musk has previously mentioned – Park Seek and Banish Autopark.

We’re also hopeful that Actually Smart Summon will increase the maximum range a vehicle can be summoned, because the current range is fairly small. Alongside Park Seek and Banish Autopark, we’re closing in on a Robotaxi-based future.

Park Seek and Banish Autopark

When the FSD v12.4 showed up, Elon also mentioned that going forward, vehicles would automatically park themselves when arriving in a parking lot – Park Seek – and that if you got out of the car at the entrance to a business, you could get it to automatically park itself in the parking lot – Banish Autopark. These are two features that are needed for Robotaxi to work – which itself is delayed to October 10^th of this year. We’re looking quite forward to them – as in combination with Actually Smart Summon, Tesla has reached the cusp of vehicle autonomy.

A vehicle that arrives, driverless, at the location you’re at, then drives somewhere, parks itself there, and waits for you – is the definition of a Robotaxi.

While we’re likely to have to wait for Banish Autopark and Park Seek just a bit longer, Actually Smart Summon being around the corner is exciting! Gone will be the days of having to run out to your car when you forgot your umbrella – your car will just come right to you instead.