Tesla still resists releasing its Full Self-Driving data, but early crowdsourced data shows that improvements are much more limited than CEO Elon Musk told us.
He is now lowering expectations.
Over the last few months, Musk has hyped up Tesla’s two latest Full Self-Driving (FSD) software update, 12.4 and 12.5. For 12.4, the CEO said that it will come without steering wheel nag and it will be able to drive “5 to 10x more miles per intervention“.
Unfortunately, Tesla ran into some issues in internal and limited external testing and decided not to do a wide-release of FSD 12.4.
Regardless, we shared our concerns about Musk’s claims in the first place since Tesla undoubtedly has FSD intervention data and it always refused to share it. Therefore, Tesla’s lack of transparency regarding the data combined with Musk clearly guessing the improvement in intervention with a massive 5 to 10x margin doesn’t add up to much credibility.
As we previously reported, the best FSD data we have right now is crowdsourced.
With Tesla now rolling out FSD v12.5, the crowdsourced data is starting to come in, but it’s still extremely limited. Nonetheless, and despite Tesla’s resistance to releasing official data, Musk has commented on the crowdsource data, which many Tesla fans are already framing as positive:
The CEO is talking about “doubling average miles between interventions” with 12.5 this month when he was talking about a 5 to 10x increase with 12.4 last month.
The data they are referring to is from Tesla FSD Tracker and it is great data, but again, it is extremely limited with a total of 150,000 miles versus the 1.6 billion miles of data Tesla claims to have about FSD.
Since the release of v12, the best data we have is about v12.3.6, which was the update that made it to the wider fleet:
When I say best data, it is still only limited to 24,000 miles, so take it with a grain of salt, but it does show only 28 miles between disengagement and 181 miles between “critical disengagement.”
The difference is that sometimes FSD drivers will disengage out of frustration or discomfort rather than a necessity for safety.
Again, FSD 12.4’s release was very limited and therefore, we barely have any data on it:
Now, as for the v12.5 data that some Tesla fans are already raving about, it’s also super limited with 1,591 miles reported at the time of publishing:
We are talking about only 44 miles between disengagement and 331 miles between critical disengagement.
For comparison, Waymo, which, to be fair, uses a geo-fenced system, travels on average 95,000 miles between interventions.
Electrek’s Take
The truth is that it’s way too early to be valuable data, but it’s still enough to tell that Elon’s claim of 5 to 10x improvement is likely nonsense.
He is himself now talking about only doubling miles by the end of the month.
Now, I don’t want to dismiss the improvement either. If Tesla ends up doubling the miles between interventions with 12.5, it’s impressive, but it needs to do that another 20x before it can reach the promised unsupervised level.
At this pace, we are still years away even though Elon again said this year or next year just last month.
Finally, the fact that Elon comments on limited crowdsourced data without releasing Tesla’s actual data on FSD doesn’t help the credibility.
Tesla should back what it is saying with intervention data, and the only reason that I can think of why they are not showing it is that it looks bad.