A detailed look at Tesla's FSD Beta using 10.10.2

On February 22nd, 2022, I took my 2017 Model X 100D, which has Autopilot 2.5 cameras, for an extended test drive on FSD Beta 10.10.2. The path being driven was:

Using A Better Route Planner I plotted the following destinations for the vehicle to navigate in the City of Plant City:

Plant City, so named after Henry B. Plant, who brought railroads to Florida, is a historical town with a mix of modern and historic roads, as well as being rife with railway crossings. The decision to have FSD Beta navigate from school to school was a blend of these being routes that would be common for a self-driving vehicle, as well as the destinations being spaced out enough to demonstrate the vehicle navigating from point to point, while also avoiding repeatedly hitting common highway roads.

As a preface to the write-up, understand that the trip was originally plotting using Tesla’s waypoints, however, my intention was to simply do “drive bys” of the destinations, not enter them. The Waypoints wanted to enter the destinations, which being that they were schools, was not something I wanted to do after hours, or ever. After noticing this when reaching the first destination I canceled the navigation and then manually selected each destination, aborting them as needed when I got close and selecting the next destination, while FSD Beta is engaged, and required to replot the trip.

In the recordings you’ll notice I don’t hit the snapshot buttons, that’s entirely on me. I was focused on recording and simply forgot to do my part by hitting the button. Normally I’m all over the button, I was just focused on the video.

The drive was also done close to the end of rush hour so that I would not have to sit in traffic.

The drive starts with the vehicle heading north on James L Redman Parkway. The drive is largely uneventful until it is required to turn right, heading east on to Reynolds St. Reynolds St is two one-way lanes heading east. A semi-truck ahead of us was making a wide turn appeared to confuse the Model X, causing it to enter the opposing traffic lane to attempt to make the right turn, a diagram of this playing out is below, the red line indicates the path the vehicle should have taken, while the yellow line indicates what it tried to do.

Upon completing the right turn the semi-truck was on top of the center line, which again, appeared to confuse FSD Beta as it was unable to pick a lane, requiring me to disengage and get it into a lane.

You can see the behavior here: here.

While traveling east on Reynolds St the Model X then had to make a left turn, heading north, on North Michigan Avenue. This turn was taken reasonably well. You can see that here.

The vehicle then had to cross Baker St, which is two one-way lanes heading west. The Model X, as shown in the diagram below, took a substantial time to cross the intersection. It crawled its way into the middle of the two-lane road, then accelerated out of it. The yellow line indicates when the vehicle determined it was clear to cross.

You can also see that here.

This theme repeats any time I must cross Reynolds or Baker St. Seems like an issue with handling one-way two-lane roads.

Upon arriving at Andrew Jackson Elementary school, I realized the Tesla Waypoint system wasn’t going to work in my favor, so I disengaged FSD Beta, drove past the school entrance, and plotted for the next destination.

The left turn from North Michigan Avenue to East Calhoun St went well, with no issues, you can see that here.

As noted earlier, Plant City is rife with railroad crossings, and the FSD Beta doesn’t seem to handle them very well. While traveling west on Calhoun St the vehicle drove over a train crossing without a care:

Resulting in some unpleasant bumps, leading straight into an uneven historic brick road, which made for further unpleasant driving.

You can see that here.

The Model X then had to make a right turn from Calhoun St to heading north on Wheeler Street. The turn was taken comfortably, you can see that here.

A couple intersections up the Model X then had to turn left on to Tever Street, I had to apply the accelerator to get it to cross quickly as there was a car coming, as indicated in the diagram below. The red arrow indicates the Model X’s direction of travel, and the yellow line is when I applied the accelerator, while the orange line was the rough position of the oncoming vehicle.

You can see here: here.

While traveling west on Tever Street the Model X encountered a bump that was missed by FSD Beta and myself, resulting in some unpleasant driving.

You can see that here.

Prior to the turn onto Franklin St I almost hit a squirrel, causing me to disengage FSD Beta, you can see that here.

The vehicle then turned north on to Franklin Street making a wider than desired turn.

The next turn the vehicle had to navigate was a right turn from Franklin St to Dixie St. Unfortunately, while passing Woodrow Wilson elementary school the vehicle encountered issues. The area next to the school, where I assume parents pick their kids up, was a big dirt patch. The vehicle assumed this was a turn lane and quickly got into it to take the turn ahead, as indicated in the diagram below, the red arrow is the expected path of the vehicle, while the yellow line is what it tried to do.

I was required to take control and manually make the turn, you can see that here.

While traveling east on Dixie St the vehicle had no lane markings in the middle of the road, causing it to drive in the middle of the road. Once lane markings were seen prior to the right turn required to get back on to Wheeler the car got back into its lane and made the turn with no issues, you can see that here.

While traveling south on Wheeler St the vehicle had to make a left turn, heading east on Calhoun St, the turn was made with no issues, however, again, the uneven historic road made the ride unpleasant, you can see that here.

The vehicle then had to make a right turn from Calhoun St to heading south on Gordon Street. The turn itself was unremarkable, however I did have concerns regarding poor visibility at the intersection as the corners were busy with obstructions. You can see that here.

The vehicle then had to make a left turn from Gordon St to Baker St. The intersection in question is the fork where the two one-way two-lane roads we crossed separately merge, or split apart, depending on your direction of travel. The vehicle took the intersection without issues, while waiting for traffic to pass.

You can see that play out here: here.

The vehicle then had to merge into the right lane and make a right turn from eastbound Baker St to south bound on Maryland Avenue. Shortly after the turn the vehicle is required to pass over another railway crossing, which it zoomed over at a high rate of speed, causing an uncomfortable right. You can see that play out here: here.

I'm uncertain if it was something I did, however, the car kept the right turn signal on the entire distance on Maryland Avenue until I got to Renfroe St. The vehicle made the right turn wide, and when I noticed the turn signal wasn’t disengaging, I turned it off manually. You can see the turn in question here: here.

Driving down Renfroe St provided some interesting navigational obstacles, which the vehicle handled well. Several vehicles were parked in various manners along the road, some with pedestrians at the side of car talking to the occupants inside. The Model X navigated the gauntlet with ease, the image below indicates the vehicle’s direction of travel with the orange lines being the position of the vehicles in question:

You can see the gauntlet here: here.

However, there was one point where it came really close to letting the side view mirrors hit the mailboxes on the side of the road. As shown in the image below:

And watch it below:

Impact was likely missed by a matter of centimeters. The mirrors did not fold in, which I suspect was due to the ultrasonic sensors at the base of the vehicle sensing more distance than at the top of the vehicle, where the impact would’ve occurred. This is actually a common complaint of mine regarding FSD, and autopilot, behavior. If the obstacle is above the wheel base, it puts the vehicle at risk, such as a pick pickup truck with mirrors that stick out far.

The vehicle then had to make a right turn from Renfroe St to heading north on Warnell Street. The turn itself has some obstructions on the right, however, the vehicle was able to make the turn with no issues. You can see that here.

While heading north on Warnell Street the vehicle again encountered uneven historic brick roads, and another railroad crossing that, this time, I applied the brakes for as I was getting tired of flying over them. You can see that here.

Once past the railroad tracks the vehicle had to cross Reynolds Street and turn left, heading west on Baker St. Again, while crossing the two-lane one-way road the car pulled out about half-way into the road before continuing on, the yellow line in the diagram indicating how far the vehicle got before getting going again.

Once at the turn for Baker Street the car scooted forward, but a car turning left cut it pretty close to my bumper. It’s my perception that FSD Beta has some issues mathing out how to handle turns on one-way roads. You can see that all play out here.

Interestingly enough, the car gets into the right most lane immediately, versus getting into the left most lane. So I think there’s some tuning which needs to occur in handling one way roads.

Further along Baker St I had to make a left turn onto Woodrow Wilson Street. The car took this turn just fine, you can see it here: here.

Next up is actually an interesting one. Plant City is also home to the Strawberry Festival, which is two weeks in March where tourists flock to the town for fresh strawberries and make traffic unbearable for locals. Part of setting up the Strawberry festival though is closing roads and redirecting traffic a bit to try and make traffic bearable for people. On a normal day you can drive down the roads of the fairgrounds, however, as the fairgrounds ramp up for the festival, they start closing off roads. This occurred during this drive. In the diagram below I have boxed the fairground in a green boundary, and indicated, in red, what the desired path was. The orange line denotes a barrier which closes the road needing to be traveled. FSD Beta moved into the middle of the intersection before deciding it had to go left, which it did. The turn itself was handled badly, and ultimately there was a disengagement, however, that was more my fault than the car as it was wiggling the wheel around a lot and I rubbed it too hard and took control.

After making the left the car replotted the trip and turned right, the left to get back on course. When trying to make the left turn from Lemon Street to Reynolds Street, however, a vehicle obstructing the view from the left caused me to have to disengage because a car was coming that couldn’t be seen, similar to what’s in this image here. You can see the entire sequence play out here: here.

When traveling east on Reynold St we came across a merge where Baker St merges with Reynolds St to become Baker St. The vehicle was unable to proceed on the stop sign, despite having sufficient time. As indicated in the diagram below, it choked at the yellow line, requiring me to hit the go pedal and get us moving again.

You can also see that play out here: here.

The vehicle then proceeded east on Reynolds St until it had to make a southbound turn onto Wheeler St, the light was red, and it took a bit for the vehicle to confirm the way was clear, then it proceeded. The vehicle, however, hung a bit to the right over the lane markings, as shown in the diagram below, with the red line intending to be the path, and the yellow line being what it did:

You can watch it here: here.

The vehicle then proceeded south on Wheeler St until it got to Alabama St. As indicated in the diagram below the vehicle was expected to make a left turn onto Alabama, then an immediate right turn onto Evers St. Unfortunately, this distance proved to be too difficult for the Tesla to handle and it choked where indicated by the yellow X at the end of the yellow line.

You can watch that here.

Once heading south on Evers St the vehicle had to make a left onto Merrick St. The left turn itself was handled properly, though it was nerve wracking because it came closer to hitting the wall at the intersection Which resulted in my accidentally disengaging FSD Beta by holding onto the wheel too tightly. You can watch that here.

The vehicle then made an unremarkable right turn to south on Collins St, you can see that here., and then another further up the road when turning from James L Redman Parkway to Maki Rd, which you can also see here: here.

While on Maki Rd the vehicle had to make a 90 degree turn. The vehicle turned the left turn signal on, despite that being unnecessary.

In reviewing the image above I can understand why the logic might move forward with that, however, the oncoming lane is more of a driveway, and if you weren’t turning you’d still have the right of way. To me signaling to go left here is unnecessary. You can see that here.

The next intersection required the vehicle to turn left on to Alexander St. The vehicle did not handle the intersection with grace. In the diagram below we can see how far back the stop sign is from the actual intersection. The vehicle stopped at the stop line, indicated in black, and began to signal right. A normal driver would have scooted forward to close the gap, however, the Model X remained at the stop line until traffic from the left cleared up and it could move forward. Once it could move forward it crawled forward, eventually getting me honked at, and getting me to hit the accelerator pedal to get moving faster. On doing this the vehicle was meant to follow the red line, however, it followed the yellow one instead, requiring me to intervene and get it back on the road again.

You can see that all here: here.

While traveling on Alexander St a car pulled out from an adjacent road on the left, the Model X began to slow down as there was a perception that it was going to cut into my lane of travel. You can see that here. No real traffic behind me, I let the scenario play out.

The next remarkable intersection was when traveling north on Alexander St and making a left turn to head west on Baker St. The end goal is that once on Baker street the car would be making a right turn from Baker St to Thonotosassa Rd. Baker Steet’s right lane at this point is a dedicated lane that must turn right and go into another dedicated lane. The optimum maneuver here is for the vehicle to remain in the right most left turn lane when turning left. Unfortunately, as the diagram below shows, the vehicle went into the left most turn lane to turn left, as indicated by the yellow arrow, when it should have tried to stay in the right lane that turns left, as indicated by the red arrow.

This resulted in the vehicle needing to make a right lane change to get into the right lane. In this case the lead vehicle in the right lane floored it out of there, and my vehicle accelerated into the pocket they created. You can see that here.

At the intersection of Baker St and Thonotosassa Rd I then stayed in the right lane to go right, however, as the diagram below shows the vehicle choked when it got halfway through the turn at the point with the yellow line. The light was red and I suspect it was waiting for an oncoming vehicle to clear, however, the lane is a dedicated lane, so no stopping should be necessary here.

You can see that here.

While traveling north on Thonotosassa the Model X appeared to misjudge the light timing, requiring me to hit the accelerator to “make the light” versus braking hard, you can see that here.

Traveling north on Thonotosassa Rd I made a questionable U-turn to return back down south on Thonotosassa Rd until I had to make a right turn onto Woodrow Wilson St, the turn was a less than 90 degrees, and the vehicle took it like a champ.

You can see that here.

While traveling south on Woodrow Wilson St the vehicle then had to cross Baker St, as indicated in the diagram below this was not successful, it choked at the yellow line, requiring me to complete the maneuver myself.

You can watch that here.

Continuing south on Woodrow Wilson St meant another interaction with the closed road for the Strawberry Festival grounds. This time the vehicle was able to make the unplanned route correction as desired. Turning left on to Oak Avenue, right onto Lemon St, then right onto Reynolds St before turning left onto Woodrow Wilson St again, as shown in the diagram below:

You can see the above play out again here: here.

The left back onto Woodrow Wilson Dr was a little hairy because the left lane was closed, the vehicle was able to make the turn, though it did so poorly. You can see that here.

Traveling south on Woodrow Wilson St the road turns into Sydney Rd, which then passes a number of distribution centers, where the vehicle does appear to intermittent apply the brakes for no expected reason. Nothing super significant, but enough to be noticeable. You can see an example here: here. and here: here.

After continuing on Sydney Rd the vehicle then came to Turkey Creek road and had to make a left to turn south on Turkey Creek Dr. As shown in the diagram below it’s a pretty simple turn. This light, however, is a fairly short light and the vehicle was not able to figure out how to turn left in time. When the 2nd cycle of the light came around I hit the gas pedal and scurried it across.

You can see that play out here: here.

After continuing down Turkey Creek Rd the vehicle then made a left to go east onto Trapnell Rd . It was remarkable only that I hit the accelerator to go because the car couldn’t see the vehicle ahead of me had a turn signal on, which meant it was going to wait for the car ahead to turn before committing to anything, as seen here: here.

Continuing on down Trapnell Rd the vehicle had to make a left onto Timberlane Dr. The Model X made the turn perfectly, even stopping for an oncoming vehicle, you can see that here.

While traveling north on Timberlane Rd the vehicle did hard braking at the spot shown by the yellow line below:

I suspect it was related to a vehicle that was leaving the community, and the car got confused. You can see yourself here: here.

Continuing north on Timberlane Dr the vehicle had to make a left onto Griffin Boulevard. As shown in the diagram below there was a car already trying to make the left, though further up than me, indicated by the first orange arrow, and a vehicle approaching, indicated by the second orange arrow. The Model X was following the red line, and I applied the brakes at the yellow line.

You can see that here.

Traveling west on Griffin Boulevard the vehicle then had to make a right onto Barrett Avenue. This turn, while successful, was concerning. As indicated in the diagram below a golf cart was approaching the intersection I had to turn at, indicated by the orange line. The golf cart had a stop sign, however, it was a little nerve wracking because I wasn’t sure what level of analyzing the Tesla was doing on the Golf cart. I did allow the vehicle to cross in front of the golf cart once I saw the stop sign, and no issues occurred.

You can see that play out here: here.

While driving along Barrett Avenue the vehicle would recognize the various stone mailboxes on either side of the road as garbage cans and would periodically jerk the wheel left/right away from the “cans” as needed, in some cases quite badly. There were no disengagements, however, it was quite unpleasant. You can see examples of it here: here. and here: here. and another here: here.

Once to the end of Barrett Avenue the vehicle makes an unremarkable right turn onto Clubhouse Dr, which you can see here: here., then looped back around to make another unremarkable right turn to head south on Timberlane Dr. You can see that here.

While heading south on Timberlane Dr the vehicle had to make a left turn onto Griffin Boulevard again, however, as indicated in the diagram below the vehicle failed to judge the distance of an oncoming vehicle requiring me to apply the brakes and stop at the yellow line.

You can watch that here.

Continuing on Griffin Boulevard the vehicle then made an unremarkable right turn onto Mud Lake Rd, you can watch it here: here. When coming to the intersection of Mud Lake Rd and Trapnell Rd the vehicle had to make an unremarkable left turn to go east on Trapnell Rd, you can see it here: here.

While continuing east on Trapnell Rd the vehicle had to make a left turn onto James L Redman parkway. This left turn was remarkable in that the two vehicles ahead of me did sloppy jobs at getting through, while FSD Beta handled it properly.

You can see that here.

The final leg of the trip was going north on James L Redman Parkway where it had to make a right turn to go east on Alexander St. Since FSD Beta 10.2 the vehicle has not been able to do this reliable. As indicated in the diagram below the vehicle will approach Alexander St but try to turn right too early, following the yellow arrow instead of the red one, which is the desired path. There was one instance in v10.9 where it DID navigate properly, after a wobble, but never again after that.

You can watch that here.

After making the right turn onto Alexander St the vehicle was then expected to make a left turn into a local set of businesses. As shown in the diagram below the vehicle failed to do it with sufficient time. The vehicle should follow the red arrow, but instead follows the yellow one, ultimately missing the turn and trying for a U-turn up the road.

And you can see that here.

Follow

Follow

Follow

Tesla’s Optimus Robot Learns to Walk Without Vision [VIDEO]

Optimus Falls - but catches itself!

Not a Tesla App

Tesla recently showed off a demo of Optimus, its humanoid robot, walking around in moderately challenging terrain—not on a flat surface but on dirt and slopes. These things can be difficult for a humanoid robot, especially during the training cycle.

Daily walks help clear your mind pic.twitter.com/dUsW58trS6

— Tesla Optimus (@Tesla_Optimus) December 9, 2024

A Look Behind the Curtain

Most interestingly, Milan Kovac, VP of Engineering for Optimus, clarified what it takes to get Optimus to this stage. Let’s break down what he said.

Optimus is Blind

Optimus is getting seriously good at walking now - it can keep its balance over uneven ground - even while walking blind. Tesla is currently using just the sensors, all powered by a neural net running on the embedded computer. 

Essentially, Tesla is building Optimus from the ground up, relying on as much additional data as possible while it trains vision. This is similar to how they train FSD on vehicles, using LiDAR rigs to validate the vision system’s accuracy. While Optimus doesn’t have LiDAR, it relies on all those other sensors on board, many of which will likely become simplified as vision takes over as the primary sensor.

Today, Optimus is walking blind, but it’s able to react almost instantly to changes in the terrain underneath it, even if it falls or slips. 

What’s Next?

Next up, Tesla AI will be adding vision to Optimus - helping complete the neural net. Remember, Optimus runs on the same overall AI stack as FSD - in fact, Optimus uses an FSD computer and an offshoot of the FSD stack for vision-based tasks.

Milan mentions they’re planning on adding vision to help the robot plan ahead and improve its walking gait. While the zombie shuffle is iconic and a little bit amusing, getting humanoid robots to walk like humans is actually difficult.

There’s plenty more, too - including better responsiveness to velocity and direction commands and learning to fall and stand back up. Falling while protecting yourself to minimize damage is something natural to humans - but not exactly natural to something like a robot. Training it to do so is essential in keeping the robot, the environment around it, and the people it is interacting with safe.

We’re excited to see what’s coming with Optimus next because it is already getting started in some fashion in Tesla’s factories.

Follow

Follow

Follow

Is Tesla Close to Licensing FSD? GM Quits Cruise, BMW Praises Tesla

Not a Tesla App

In a relatively surprising move, GM announced that it is realigning its autonomy strategy and prioritizing advanced driver assistance systems (ADAS) over fully autonomous vehicles.

GM is effectively closing Cruise (autonomous) and focusing on its Super Cruise (ADAS) feature. The engineering teams at Cruise will join the GM teams working on Super Cruise, effectively shuttering the fully autonomous vehicle business.

End of Cruise

GM cites that “an increasingly competitive robotaxi market” and “considerable time and resources” are required for scaling the business to a profitable level. Essentially - they’re unable to keep up with competitors at current funding and research levels, putting them further and further behind.

Cruise has been offering driverless rides in several cities, using HD mapping of cities alongside vehicles equipped with a dazzling array of over 40 sensors. That means that each cruise vehicle is essentially a massive investment and does not turn a profit while collecting data to work towards Autonomy.

Cruise has definitely been on the back burner for a while, and a quick glance at their website - since it's still up for now - shows the last time they officially released any sort of major news packet was back in 2019. 

Competition is Killer

Their current direct competitor - Waymo, is funded by Google, which maintains a direct interest in ensuring they have a play in the AI and autonomy space.

Interestingly, this news comes just a month after Tesla’s We, Robot event, where they showed off the Cybercab and the Robotaxi network, as well as plans to begin deployment of the network and Unsupervised FSD sometime in 2025. Tesla is already in talks with some cities in California and Texas to launch Robotaxi in 2025.

GM Admits Tesla Has the Right Strategy

As part of the business call following the announcement, GM admitted that Tesla’s end-to-end and Vision-based approach towards autonomy is the right strategy. While they say Cruise started down that path, they’re putting aside their goals towards fully autonomous vehicles for now and focusing on introducing that tech in Super Cruise instead.

NEWS: GM just admitted that @Tesla’s end-to-end approach to autonomy is the right strategy.

“That’s where the industry is pivoting. Cruise had already started making headway down that path. We are moving to a foundation model and end-to-end approach going forward.” pic.twitter.com/ACs5SFKUc3

— Sawyer Merritt (@SawyerMerritt) December 10, 2024

With GM now focusing on Super Cruise, they’ll put aside autonomy and instead focus solely on ADAS features to relieve driver stress and improve safety. While those are positive goals that will benefit all road users, full autonomy is really the key to removing the massive impact that vehicle accidents have on society today.

In addition, Super Cruise is extremely limited, cannot brake for traffic controls, and doesn’t work in adverse conditions - even rain. It can only function when lane markings are clear, there are no construction zones, and there is a functional web connection. 

The final key to the picture is that the vehicle has to be on an HD-mapped and compatible highway - essentially locking Super Cruise to wherever GM has time to spend mapping, rather than being functional anywhere in a general sense, like FSD or Autopilot.

Others Impressed - Licensing FSD

Interestingly, some other manufacturers have also weighed into the demise of Cruise. BMW, in a now-deleted post, said that a demo of Tesla’s FSD is “very impressive.” There’s a distinct chance that BMW and other manufacturers are looking to see what Tesla does next. 

BMW chimes in on a now-deleted post. The Internet is forever, BMW!

Not a Tesla App

It seems that FSD has caught their eyes after We, Robot - and that the demonstrations of FSD V13.2 online seem to be the pivot point. At the 2024 Shareholder Meeting earlier in the year, Elon shared the fact that several manufacturers had reached out, looking to understand what was required to license FSD from Tesla.

There is a good chance 2025 will be the year we’ll see announcements of the adoption of FSD by legacy manufacturers - similar to how we saw the surprise announcements of the adoption of the NACS charging standard.

Follow

Follow

Follow