Sentry & Saved has just 2 files
Opened this issue · 66 comments
Describe the problem
Till February 23, all Sentry & Saved filed were synced using rclone. 2/23 at the end of the day updated to latest version 2024.2.7 and now every single day there are just 2 files named event.json and thumb.png files are the only 2 that gets uploaded. What else should I look into. Will check another car where 2024.2.7 version got updated few days later.
Today (March3 ) updated teslausb using self update and still cannot get older saved and sentry files are uploaded.
Tried to check to best of my understanding and found sentry_found_files file in tmp folder that has below logged
SentryClips/2024-02-23_12-12-30/2024-02-23_12-01-03-front.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-01-03-left_repeater.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-01-03-right_repeater.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-02-04-back.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-02-04-front.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-02-04-left_repeater.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-02-04-right_repeater.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-03-04-back.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-03-04-front.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-03-04-left_repeater.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-03-04-right_repeater.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-04-05-back.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-04-05-front.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-04-05-left_repeater.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-04-05-right_repeater.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-05-05-back.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-05-05-front.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-05-05-left_repeater.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-05-05-right_repeater.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-06-06-back.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-06-06-front.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-06-06-left_repeater.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-06-06-right_repeater.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-07-06-back.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-07-06-front.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-07-06-left_repeater.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-07-06-right_repeater.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-08-07-back.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-08-07-front.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-08-07-left_repeater.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-08-07-right_repeater.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-09-07-back.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-09-07-front.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-09-07-left_repeater.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-09-07-right_repeater.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-11-08-back.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-11-08-front.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-11-08-left_repeater.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-11-08-right_repeater.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-12-09-back.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-12-09-front.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-12-09-left_repeater.mp4
SentryClips/2024-02-23_12-12-30/2024-02-23_12-12-09-right_repeater.mp4
SentryClips/2024-02-23_12-12-30/event.json
SentryClips/2024-02-23_12-12-30/thumb.png
SentryClips/2024-02-24_10-50-05/event.json
SentryClips/2024-02-24_10-50-05/thumb.png
SentryClips/2024-02-24_11-11-09/event.json
SentryClips/2024-02-24_11-11-09/thumb.png
SentryClips/2024-02-26_11-52-07/event.json
SentryClips/2024-02-26_11-52-07/thumb.png
SentryClips/2024-02-26_18-36-41/event.json
SentryClips/2024-02-26_18-36-41/thumb.png
SentryClips/2024-02-26_19-13-32/event.json
SentryClips/2024-02-26_19-13-32/thumb.png
SentryClips/2024-02-28_13-24-50/event.json
SentryClips/2024-02-28_13-24-50/thumb.png
SentryClips/2024-02-28_15-11-59/event.json
SentryClips/2024-02-28_15-11-59/thumb.png
SentryClips/2024-02-28_18-00-21/event.json
SentryClips/2024-02-28_18-00-21/thumb.png
SentryClips/2024-02-28_19-18-50/event.json
SentryClips/2024-02-28_19-18-50/thumb.png
SentryClips/2024-03-02_19-35-31/event.json
SentryClips/2024-03-02_19-35-31/thumb.png
SentryClips/2024-03-02_20-09-48/event.json
SentryClips/2024-03-02_20-09-48/thumb.png
Below is what shows up in archiveloop.log right after rebooting
==============================================
Sun 3 Mar 21:20:02 EST 2024: Starting archiveloop at 43.21 seconds uptime...
Sun 3 Mar 21:20:03 EST 2024: Running fsck on /backingfiles/cam_disk.bin...
Sun 3 Mar 21:20:04 EST 2024: | fsck from util-linux 2.33.1
Sun 3 Mar 21:20:08 EST 2024: | fsck.fat 4.1 (2017-01-24)
Sun 3 Mar 21:20:08 EST 2024: | Long filename fragment "2023-10-20_20" found outside a LFN sequence.
Sun 3 Mar 21:20:08 EST 2024: | (Maybe the start bit is missing on the last fragment)
Sun 3 Mar 21:20:08 EST 2024: | Not auto-correcting this.
Sun 3 Mar 21:20:08 EST 2024: | Long filename fragment "2023-05-31_16" found outside a LFN sequence.
Sun 3 Mar 21:20:08 EST 2024: | (Maybe the start bit is missing on the last fragment)
Sun 3 Mar 21:20:08 EST 2024: | Not auto-correcting this.
Sun 3 Mar 21:20:08 EST 2024: | Long filename fragment "2023-04-16_14" found outside a LFN sequence.
Sun 3 Mar 21:20:08 EST 2024: | (Maybe the start bit is missing on the last fragment)
Sun 3 Mar 21:20:08 EST 2024: | Not auto-correcting this.
Sun 3 Mar 21:20:08 EST 2024: | Long filename fragment "2023-04-09_18" found outside a LFN sequence.
Sun 3 Mar 21:20:08 EST 2024: | (Maybe the start bit is missing on the last fragment)
Sun 3 Mar 21:20:08 EST 2024: | Not auto-correcting this.
Sun 3 Mar 21:20:08 EST 2024: | Wrong checksum for long file name "2024-02-23_15-12-44-back.mp4".
Sun 3 Mar 21:20:08 EST 2024: | (Short name 209B50~1.MP4 may have changed without updating the long name)
Sun 3 Mar 21:20:08 EST 2024: | Not auto-correcting this.
Sun 3 Mar 21:20:08 EST 2024: | /dev/loop0p1: 423 files, 742871/1965055 clusters
Sun 3 Mar 21:20:08 EST 2024: Finished fsck on /backingfiles/cam_disk.bin.
Sun 3 Mar 21:20:08 EST 2024: taking snapshot of cam disk in /backingfiles/snapshots/snap-003216
Sun 3 Mar 21:20:09 EST 2024: waiting for autofs to be active
Sun 3 Mar 21:20:10 EST 2024: waiting for autofs to be active
Sun 3 Mar 21:20:12 EST 2024: waiting for autofs to be active
Sun 3 Mar 21:20:13 EST 2024: waiting for autofs to be active
Sun 3 Mar 21:20:14 EST 2024: waiting for autofs to be active
Sun 3 Mar 21:20:16 EST 2024: took snapshot
Sun 3 Mar 21:20:19 EST 2024: comparing new snapshot with /backingfiles/snapshots/snap-003215/snap.bin
Sun 3 Mar 21:20:19 EST 2024: new snapshot is identical to previous one, discarding
Sun 3 Mar 21:20:31 EST 2024: Trying to set time...
Sun 3 Mar 21:21:21 EST 2024: Time adjusted by 50.088968 seconds after 0.190000 seconds
Sun 3 Mar 21:21:21 EST 2024: Not keeping car awake.
Sun 3 Mar 21:21:21 EST 2024: Archiving...
Sun 3 Mar 21:21:22 EST 2024: Checking saved folder count...
Sun 3 Mar 21:21:22 EST 2024: There are 0 event folder(s) with 0 file(s) and 0 track mode file(s) to move. 0 short recording(s) will be skipped.
Sun 3 Mar 21:21:23 EST 2024: Ensuring cam file is mounted...
Sun 3 Mar 21:21:23 EST 2024: Disconnecting usb from host...
Sun 3 Mar 21:21:23 EST 2024: Disconnected usb from host.
Sun 3 Mar 21:21:23 EST 2024: Mounting /mnt/cam...
Sun 3 Mar 21:21:23 EST 2024: Mounted /mnt/cam.
Sun 3 Mar 21:21:23 EST 2024: Ensured cam file is mounted.
Sun 3 Mar 21:21:23 EST 2024: cleaning cam mount
Sun 3 Mar 21:21:24 EST 2024: done cleaning cam mount
Sun 3 Mar 21:21:25 EST 2024: Trimming free space in /mnt/cam, which has 1682 extents
Sun 3 Mar 21:21:25 EST 2024: Trim complete, image now has 1682 extents
Sun 3 Mar 21:21:25 EST 2024: Unmounting /mnt/cam...
Sun 3 Mar 21:21:25 EST 2024: Unmounted /mnt/cam.
Sun 3 Mar 21:21:25 EST 2024: Finished archiving.
Sun 3 Mar 21:21:25 EST 2024: Music archive not configured or unreachable
Sun 3 Mar 21:21:25 EST 2024: Connecting usb to host...
Sun 3 Mar 21:21:26 EST 2024: Connected usb to host.
Sun 3 Mar 21:21:31 EST 2024: Waiting for archive to be unreachable...
### Device
Raspberry Pi Zero W
### OS Image
Prebuilt TeslaUSB image
### Car Model
Model Y
### USB connection
Center console
### Logs
_No response_
### Additional information
_No response_
I have seen this issue too, as have others. Since it only started happening recently I suspect it's due to a Tesla software update, but it's unclear what exactly the trigger is. If I clear out the "cam" drive, it starts working properly again.
Thanks for confirming @marcone. I checked 2018 Model 3 and that does exactly the same.
When you say clearing out “cam” drive is it the Teslacam folder on SD card? I will definitely give that a try once you confirm to see if it works again like you observed.
Connecting the Pi to a PC and deleting the content of the TeslaCam folder in the "CAM" drive is what I did. Don't delete the TeslaCam folder itself.
Or if you want to do it without taking the Pi out of the car:
sudo -i
bin/disable_gadget.sh
mount /mnt/cam
rm -rf /mnt/cam/TeslaCam/*
umount /mnt/cam
bin/enable_gadget.sh
This probably isn't a permanent fix, but it made it work again for me for now.
Connecting the Pi to a PC and deleting the content of the TeslaCam folder in the "CAM" drive is what I did. Don't delete the TeslaCam folder itself. Or if you want to do it without taking the Pi out of the car:
sudo -i bin/disable_gadget.sh mount /mnt/cam rm -rf /mnt/cam/TeslaCam/* umount /mnt/cam bin/enable_gadget.shThis probably isn't a permanent fix, but it made it work again for me for now.
I believe this fixed it for me, will verify when I get home from work.
Having the same issue here. I gave the above series of "in-vehicle" commands a shot. There is one modification needed. I got an error that I couldn't rm on a read-only file. The correct sequence, I think, is:
sudo -i
bin/disable_gadget.sh
mount -o rw /mnt/cam
rm -rf /mnt/cam/TeslaCam/*
umount /mnt/cam
bin/enable_gadget.sh
I'll monitor and report back if this fixes it.
you shouldn't need to add that -o rw to the mount command. It should be mounted read-write by default. If it's not, there's probably some kind of filesystem corruption.
Connecting the Pi to a PC and deleting the content of the TeslaCam folder in the "CAM" drive is what I did. Don't delete the TeslaCam folder itself. Or if you want to do it without taking the Pi out of the car:
sudo -i bin/disable_gadget.sh mount /mnt/cam rm -rf /mnt/cam/TeslaCam/* umount /mnt/cam bin/enable_gadget.shThis probably isn't a permanent fix, but it made it work again for me for now.
Thanks @marcone - it works for now on 2 separate instances.
I'm having a similar issue. TeslaUSB was only backing up 2 files. I ran the commands @marcone suggested. I am using TeslaFi to keep the car awake, and the archive log never gets past "Starting background task to keep car awake.". I verified through the TeslaFi API page, the call is going through and is received by the vehicle. I re-rolled the API token for good measure and updated the config, restarted, aswell as ran the update script to make sure I was on the latest TeslaUSB version. The Diagnostics page (attached) does not immediately jump out at me.
This worked for me for a few days and now it’s exhibiting the behavior again. Notably on one new vehicle (2024) which got a new and freshly-provisioned Pi Zero W teslausb installation. Now I’m also seeing the same on an existing installation on a 2020 vehicle where teslausb had been working flawlessly for years.
Seems related to a recent vehicle SW update, I guess? Both are running v11.1 (2024.2.7 d522c44937f7).
If it's not, there's probably some kind of filesystem corruption.
What’s the recommended way to check for/repair this for these mounts?
Broke after few days again on 2 cars. Just 2 files.
@Gizmotoy - this seems to be Tesla firmware update, both vehicles are on 2024.2.7 - 2018 RWD 3 and 2020 Y.
Maybe wait till next Tesla update.
Mine are also again broken.
this seems to be Tesla firmware update
That's the big concern, I think. It doesn't seem super likely to me that Tesla will make fixes for this use case (or even notice they've broken it).
@marcone , is there anything we could do to help provide some insight into what is actually going on here? Run a diagnostic build? Capture logs? Happy to help investigate.
Same here, broke after a while. Going back to manual USB until this is resolved, if it can be. I'll format it and see if theres any obvious directory changes. I'll be happy to help as well. What exactly has changed? How the video files are saved? Naming?
@marcone , is there anything we could do to help provide some insight into what is actually going on here? Run a diagnostic build? Capture logs? Happy to help investigate.
I don't think there is anything that can be done on the TeslaUSB side. I think this is just a bug on Tesla's side, since it started recently with no notable TeslaUSB changes, and I'm seeing problems even with a regular USB drive.
@marcone , is there anything we could do to help provide some insight into what is actually going on here? Run a diagnostic build? Capture logs? Happy to help investigate.
I don't think there is anything that can be done on the TeslaUSB side. I think this is just a bug on Tesla's side, since it started recently with no notable TeslaUSB changes, and I'm seeing problems even with a regular USB drive.
So you think this problem is widespread to all Tesla dashcam users regardless if they are using teslausb or not? What problems do you see with the regular usb drive?
I was wondering the same. I flipped my 2024 back to a USB drive and it seems to be working fine so far, while the 2020 continues to fail after clearing the TeslaCam folder every few days. I can't say I see any complaints of this on the various Tesla forums despite scouring pretty hard, but it doesn't mean it's not happening.
I was seeing this problem, and tried every fix in this thread. I still was only seeing 2 files per "event" and nothing thereafter.
I've gone back to using an SSD plugged directly into the car (2022 M3) and its recording just fine.
@Gizmotoy are you using the center console or glove box? I've personally seen it on a Model Y with center console USB, and @cleanev also reported using the center console.
With my Model Y, the problem started out as only having those 2 files in the sentry folder as described above, but has since evolved to the car simply not seeing the USB drive at all until the car is reset, at which point it'll work again for a few days.
I have one of each: my 2020 doesn’t have a glove box port at all. I still have that TeslaUSB Pi Zero W device installed and it fails every few days and I do the process above to resolve it temporarily.
My 2024 has the glove box port and TeslaUSB was installed there. Subjectively, the behavior was worse in the 2024, lasting only a day or two max, so I replaced it with the original USB drive. It’s been solid again and I’ve encountered no issues on USB at all.
I have a 2023 Model 3 RWD with the USB plugged into the glovebox and have had no problems using a default USB stick. Same behavior after a couple days with the fix above on teslausb.
The consensus reading these comments is Tesla made a change affecting TeslaUSB in all models, regardless of where you plug in while allow plain vanilla USB key to work.
Hope is that if one of us gets the latest version 2024.8.4 that is currently being deployed, we will get to know if TeslaUSB works or not.
My 2020 just received 2024.8.4. Installing now. It has TeslaUSB still installed. Hope to understand if this is just a thing they've fixed within the next few days.
This happened twice for me. What I've notice is that the issue occurred after a software update. I updated to 2024.8.4 and the issue happened again. I'll keep an eye out but that is something I notice for me.
Not sure if anyone other than @mondagoar has had any success. Model Y-2020 with 2024.3.10, Model 3 2018 with 2024.2.7 still has same issue where after removing the files it lasts for a few drives with and then 2 files again. Something that Tesla did or does reinstalling take care of this?
I tried reinstalling and was still affected. Running 2024.3.10 and 2024.8.7.
I switched back to flash drives until this is sorted. I was almost hit and had to swerve off the road last week and teslausb didn’t capture anything but the two meta files. Flash is safer for now, I guess.
@Gizmotoy Have you verified that the camera footage is actually present on the USB drive? Before finding this issue I tried switching to a USB flash drive and it did not have the content on it I expected. Is it possible the camera footage is saved to internal storage on the Tesla system?
I really hope this issue is resolved soon as really don't like not having camera footage available if there was an accident.
I did pull the Raspberry Pi to confirm and no, the video files were indeed missing.
I’ve had no problems on flash drives on either vehicle, so switched back to those for now.
I did pull the Raspberry Pi to confirm and no, the video files were indeed missing.
Were there RecentClips though? What I noticed when this issue first started happening, RecentClips were still being recorded and snapshotted, and so it was still possible to see Sentry events in the RecentClips timeline in the web interface. It was like the car just stopped bothering to move Sentry recordings to their own folder.
Then with a later software update the car stopped recording altogether. Or rather it stopped seeing the drive, including plain USB drives.
@marcone That is inline with what I saw, plain USB drives had the same issue. @Gizmotoy You mention checking the RPi and seeing the same thing as others but what about the USB drive you are now putting your trust in? I would seriously suggest you remove your USB drive and check it on the PC too.
My gut feel is an update to the Tesla software has stop saving video to external storage in favor of internal storage. If that is the case then there is no fix that can be done on the RPi and we just have to wait for Tesla to re-enable external storage.
USB drive has been 100% flawless for me. We discussed this pretty extensively above over the past month or two. The behavior has followed through 3 or 4 different vehicle software versions too. Whatever it is is only affecting Teslausb for me.
There might have been RecentClips. I’ve done the posted recovery process dozens of times so I didn’t think too much of it when I had to do it again, but then I thought about it a bit more and just put the flash drive back in instead. I guess a mistake.
I encountered this issue starting on 2/18/24, which persisted until I recently recreated my TeslaUSB. Before re-imaging, I noticed an error while FSCKing the cam backing image: 'Long filename fragment "2024-03-24_19" found outside a LFN sequence.' Although I attempted manual repair and it seemed successful, the problem persisted. While I can't confirm if this is the root cause, it's possible that simply recreating the backing image for the cam drive would have resolved the issue.
It appears that the cam drive can become corrupted in a way that leads to this problem over time. Ideally, we could identify a solution for resolving it when it occurs, allowing TeslaUSB to fix it automatically.
I did pull the Raspberry Pi to confirm and no, the video files were indeed missing.
Were there RecentClips though? What I noticed when this issue first started happening, RecentClips were still being recorded and snapshotted, and so it was still possible to see Sentry events in the RecentClips timeline in the web interface. It was like the car just stopped bothering to move Sentry recordings to their own folder.
Then with a later software update the car stopped recording altogether. Or rather it stopped seeing the drive, including plain USB drives.
@marcone - yes RecentClips does have associated clips for specific time - say the folder has date-time then I can find all the files around this time and many more from earlier & later in RecentClips folder. As you stated this may be something that the car stopped copying over all MP4 files to its respective folders while still copying thumbs.db & events files.
reading @tomrund post, I see 4 items when FSCKing related to LFN. Also the snapshot number was up to 3578 since I built this Pi.
What I did next is below & while it seems to work please
DO NOT ATTEMPT if you are not comfortable as this is not a tried solution
I got a bit brave and removed cam_disk.bin from backingfiles folder while logged in as sudo -i.
Ended up with recurring errors in achiveloop.log.
To remediate this I ran setup-teslausb from /root/bin/ folder with upgrade option.
Somehow doing what I did there are files that are no longer accessible in /var/www/html/RecentFiles or /SavedFiles or SentryFiles folder, like so when I list directory. New directories since yesterday created using test script have all the files. Will need to figure out how to clean up so they are no longer showing in browser - if clicked on folder in browser, 500 Internal Server error pops up.
ls: cannot access '2024-04-12/2024-04-12_23-41-09-back.mp4': No such file or directory
ls: cannot access '2024-04-12/2024-04-12_23-41-09-front.mp4': No such file or directory
ls: cannot access '2024-04-12/2024-04-12_23-41-09-left_repeater.mp4': No such file or directory
ls: cannot access '2024-04-12/2024-04-12_23-41-09-right_repeater.mp4': No such file or directory
ls: cannot access '2024-04-12/2024-04-12_23-42-09-back.mp4': No such file or directory
ls: cannot access '2024-04-12/2024-04-12_23-42-09-front.mp4': No such file or directory
Above cleared out all the snapshots & now started with 0000. Also cleared archiveloop.log as it had grown to 4MB, not that it matters in 256GB microSD.
Used @marcone createtestfiles script and adapted to create sentry and saved files.
Seems to be working well & has worked for 20+ I created test files; so am hopeful that it will work when plugged in the car, but only time will tell. I will report later once I see success or lack thereof.
Best
Anyone having this problem using exFAT? It's been good for a few days since I switched, could just be luck though. On FAT32 mine was also recording RecentClips fine, but failed to move the files into the SavedClips/SentryClips folders.
Is your backingfiles formatted as ExFAT? My installation is XFS and all others are shown below on PiZeroW
Filesystem Type Size Used Avail Use% Mounted on
/dev/root ext4 2.0G 1.4G 462M 75% /
devtmpfs devtmpfs 183M 0 183M 0% /dev
tmpfs tmpfs 216M 0 216M 0% /dev/shm
tmpfs tmpfs 216M 5.8M 210M 3% /run
tmpfs tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs tmpfs 216M 0 216M 0% /sys/fs/cgroup
tmpfs tmpfs 216M 0 216M 0% /var/lib/nginx
tmpfs tmpfs 216M 0 216M 0% /var/spool
tmpfs tmpfs 216M 0 216M 0% /var/tmp
tmpfs tmpfs 216M 364K 215M 1% /tmp
tmpfs tmpfs 216M 4.0K 216M 1% /var/lib/ntp
tmpfs tmpfs 216M 0 216M 0% /var/log/nginx
/dev/mmcblk0p3 xfs 237G 8.9G 228G 4% /backingfiles
/dev/mmcblk0p1 vfat 247M 51M 197M 21% /boot
/dev/mmcblk0p4 ext4 93M 3.0M 83M 4% /mutable
tmpfs tmpfs 43M 0 43M 0% /run/user/1000
No, I believe that's still XFS. All I do is uncomment export USE_EXFAT=true in teslausb_setup_variables.conf
which, if I'm interpreting things correctly, is for the internal file system of the
bin files (such as cam_disk.bin). Formatting the 'drive' from the Tesla menu may have the same effect, haven't tried that though.An interesting lead. The two flash drives I've been using that have been 100% reliable are both ExFAT. It looks like my teslausb instances are both Fat32.
I wonder if this would explain why we're seeing different reports on the behavior of USB flash drives. Were those of you who were also having problems with flash drives using Fat32-formatted flash drives?
Edit:
Per the above, the full config setting
# Uncomment to use ExFAT filesystem instead of FAT32.
# ExFAT is the filesystem used by Tesla when the drive is formatted in the car.
# Using ExFAT is not recommended for the TeslaUSB prebuilt image, as it is
# based on an older kernel that has no TRIM support for ExFAT.
export USE_EXFAT=true
Could be rough to lose TRIM for a use case like this. According to Wikipedia this feature was added in 5.13. The prebuilt image appears to use 5.10.63+, based on my install, so the comment appears to still be applicable. This configuration would lack TRIM support.
I replaced my teslausb FAT32 formatted drive with an exFAT one despite the loss of TRIM just to see if this works to fix the problem.
The pre-release Bullseye image is on 6.1 and supports TRIM for exFAT, although it does have an annoying issue with my Pi Zero 2 W where the wifi driver crashes when it leaves the range of my AP.
I tried the pre-release image in my vehicles. No issues yet with exFAT storing sentry mode clips properly. I do have some trouble getting the drive recognized, though. It seems like the car doesn't recognize the drive every third time or so (works fine on my computer, and works in the car after I reboot the center display).
Hey, I don’t Alan’s to rain on anyone’s parade. But I was using the default file system for years before it acted up. So my assumption would be that any reformat, regardless of the chosen format, would seem to resolve the issue.
Also if the GitHub gui wasn’t trash on mobile, I’d happily fix my typo from above.
Are you saying that formatting and reinstalling the TeslaUSB image would fix this issue? I've kept my current install in the hopes that there was an eventual tweak or fix for this.
Yes, I did a fresh install, but I believe just removing and recreating the cam drive backing file would likely also be sufficient. After my fresh install things have been working normally again for a couple weeks without changing any settings or hardware.
Yes, I did a fresh install, but I believe just removing and recreating the cam drive backing file would likely also be sufficient. After my fresh install things have been working normally again for a couple weeks without changing any settings or hardware.
Sweet, just reformatted the drive and everything is working again. I have no idea why it did that, but I'm happy it's working again. Appreciate it, tomrund!
The pre-release Bullseye image is on 6.1 and supports TRIM for exFAT, although it does have an annoying issue with my Pi Zero 2 W where the wifi driver crashes when it leaves the range of my AP.
I had this issue with WiFi crapping out on my RP4 with the Bullseye image, I found this comment that so far, seems to work for me:
Add "modules-load=brcmfmac roamoff=1 feature_disable=0x82000" into /boot/cmdline.txt
See here for more info:
https://retropie.org.uk/forum/topic/32462/can-not-connect-to-wifi/5
and here:
https://forums.raspberrypi.com/viewtopic.php?t=349845
Ultimately I had to reinstall to solve this issue (sentry folders have no videos). Funny enough the car seems to see them just fine. I di have a growing list of folders with weird names that seemed to correspond to the number of sentry events, but I didn't look into it that much. So while recent clips would grow, I could command the car to erase the video files or reformat the drive and when I checked it again, all the files were still there. So I did a reinstall.
I made the drives ExFAT just in case and it is working. Then the issue above happened a few minutes after I got home and it started syncing. Now it stays up, so that's good. The only remaining issue is the script won't keep the car awake unless I manually clear the cache and do the dance mentioned in another issue.
Connecting the Pi to a PC and deleting the content of the TeslaCam folder in the "CAM" drive is what I did. Don't delete the TeslaCam folder itself. Or if you want to do it without taking the Pi out of the car:
sudo -i bin/disable_gadget.sh mount /mnt/cam rm -rf /mnt/cam/TeslaCam/* umount /mnt/cam bin/enable_gadget.shThis probably isn't a permanent fix, but it made it work again for me for now.
This method doesn't work for me tried couple times. Still has only two files in folders like these:
SentryClips/2024-03-29_20-06-48/event.json
SentryClips/2024-03-29_20-06-48/thumb.png
That process solves it pretty routinely for me, but the fix is temporary and returns.
Here’s the things I have tried that work temporarily, but the issue eventually returns:
- The series of commands above to delete the corrupted files
- Formatting the TeslaCam drive from the Tesla UI
- Formatting the TeslaCam drive from a computer
- Full wipe/reinstallation
Here’s what I know works:
- Tesla’s provided flash drive (ExFAT)
Here’s what I’m uncertain works:
- Formatting the cam drive as ExFAT via TeslaUSB config
- Wipe/reinstallation using the Bullseye pre-release image AND an ExFAT drive via config
On my vehicle, now running #2 from this list for about 2.5 weeks, I haven’t seen the issue recur. However, it introduces another issue for about 20% of trips where the drive is not recognized in the vehicle until resetting the MCU.
This method doesn't work for me tried couple times. Still has only two files in folders like these:
SentryClips/2024-03-29_20-06-48/event.json
SentryClips/2024-03-29_20-06-48/thumb.png
Were you expecting these steps to fix existing Sentry events from over a month ago?
This method doesn't work for me tried couple times. Still has only two files in folders like these:
SentryClips/2024-03-29_20-06-48/event.json
SentryClips/2024-03-29_20-06-48/thumb.pngWere you expecting these steps to fix existing Sentry events from over a month ago?
No i tried recently. Same for the files from 2024-04-29 and 2024-04-30. That method is not working.
On my vehicle, now running #2 from this list for about 2.5 weeks, I haven’t seen the issue recur. However, it introduces another issue for about 20% of trips where the drive is not recognized in the vehicle until resetting the MCU.
I also had the drive not recognized issue a few weeks ago. I thought it had something to do with the Rock Pi 4C+. Every time I plugged that in I'd have to reset the MCU to even get the stock flash drive to be recognized. Issue went away after a software update, had it once today right after the 2024.3.25 update installed. Haven't had the missing video files issue since switching to exFAT.
What I find odd about this is how there are only two files (and no videos) in the directories, yet while the TeslaUSB is plugged into the car, it can see and play the sentry and dashcam clips just fine?
Mine wasn't able to play the clips at all. The clips are (or were) saved to the drive in RecentClips, it seems to be an issue with moving them into the event folder.
For those running a RPi 4C+ what benefit(s) does it offer over a RPi 4 as it relates to Teslausb?
I use a RockPi 4C+, the benefit for me is USB 3.0 OTG so you dont have to see the slow USB connected notification all the time. My only issue with this SBC is I cant get the HotSpot feature to work. Everything else has been fine.
I use a RockPi 4C+, the benefit for me is USB 3.0 OTG so you dont have to see the slow USB connected notification all the time. My only issue with this SBC is I cant get the HotSpot feature to work. Everything else has been fine.
@ravibowman is it confirmed hat the USB 3.0 OTG is much faster than the RPi 4? While I reliably use a RPi 4, I've come to ignore the slow USB messages entirely yet there's never been a speed issue, at least thus far.
Just to add another datapoint, after experiencing this issue for the first time, I re-imaged my TeslaUSB instance and had my first successful Sentry recording afterwards at 2024-04-02_22-34-18 and it worked all the way until 2024-04-24_19-08-52. At some point between that recording and a recording on 2024-04-26_13-48-31, things went awry and I started to experience this issue again.
These dates do not coincide with any car software updates (I received 2024.3.25 on Apr 29, 2024 and 2024.3.15 on Apr 12, 2024), I am using the default filesystem and haven't tried switching to exFAT, although I'm tempted to.
This time, rather than re-image, I just removed the cam drive backing file as I previously suggested and re-ran setup and things are working normally once again.
sudo -i && bin/disable_gadget.sh && rm -rf /backingfiles/cam_disk.bin && bin/setup-teslausb upgrade
I was prompted to Delete snapshots and recreate recording and music drives? (yes/cancel), I said yes, setup completed, the device rebooted, and I was back in action again.
I ended up in this state again. I saw some discussions in the Discord where users were reporting fsck recovered files after mounting their cam_disk.bin, and after removing these files their issue went away but this hasn't been my experience. When I mount my cam_disk.bin there are no fsck recovered files anywhere in the filesystem.
I'm once again getting the Long filename fragment found outside a LFN sequence errors from fsck as I did previously when I encountered this issue, and as the original reporter of this issue was experiencing. I previously tried to repair these errors in a way that seemed to fix my cam_disk.bin but was unsuccessful, I'll keep looking for a solution.
I believe this issue has been resolved with this commit. I'd recommend updating with sudo -i && bin/setup-teslausb upgrade when you have a chance to receive the fix.
I believe this issue has been resolved with this commit. I'd recommend updating with
sudo -i && bin/setup-teslausb upgradewhen you have a chance to receive the fix.
Will this fix it if it stops working? Mine was working when I reinstalled everything from scratch, but now it reverted to saving only two small files. I did push the dashcam button on the screen to capture an event manually today.. I wonder if that triggers this problem?
@nickastaldo I did an update then checked the change in the commit has been applied. It hadn't, so I simply manually edited the file in question to match the commit. Only limited testing since then but appears to be working so far. Maybe that commit has not been merged in the main branch yet? I didn't bother digging further to see why the update didn't work.
Hopefully google picks up on this. This fixed my missing video clip issues with my teslacam sentry mode and dashcam mode. More SEO following.
Missing video clip teslacam sentry mode and dashcam using teslausb but json and folder present.
Switching to EXFAT FS worked out well and all event files are being archived (including newest event.mp4 that is short file from exact time of sentry event.
However, it seems that fstrim does not work as indicated in log snippet below:
Mon 17 Jun 21:59:45 EDT 2024: Ensuring cam file is mounted...
Mon 17 Jun 21:59:45 EDT 2024: Disconnecting usb from host...
Mon 17 Jun 21:59:46 EDT 2024: Running fsck on /backingfiles/cam_disk.bin...
Mon 17 Jun 21:59:46 EDT 2024: | fsck from util-linux 2.33.1
Mon 17 Jun 21:59:46 EDT 2024: | exfatprogs version : 1.0.4
Mon 17 Jun 21:59:46 EDT 2024: | /dev/loop2p1: clean. directories 10, files 350
Mon 17 Jun 21:59:46 EDT 2024: Finished fsck on /backingfiles/cam_disk.bin.
Mon 17 Jun 21:59:46 EDT 2024: Disconnected usb from host.
Mon 17 Jun 21:59:46 EDT 2024: Mounting /mnt/cam...
Mon 17 Jun 21:59:47 EDT 2024: Mounted /mnt/cam.
Mon 17 Jun 21:59:47 EDT 2024: Ensured cam file is mounted.
Mon 17 Jun 21:59:47 EDT 2024: cleaning cam mount
Mon 17 Jun 21:59:48 EDT 2024: done cleaning cam mount
**Mon 17 Jun 21:59:48 EDT 2024: Trimming free space in /mnt/cam, which has 4011 extents
fstrim: /mnt/cam: the discard operation is not supported
Mon 17 Jun 21:59:48 EDT 2024: Trimming free space in /mnt/cam failed.
Mon 17 Jun 21:59:48 EDT 2024: Unmounting /mnt/cam...**
Mon 17 Jun 21:59:48 EDT 2024: Unmounted /mnt/cam.
Mon 17 Jun 21:59:48 EDT 2024: Finished archiving.
Mon 17 Jun 21:59:48 EDT 2024: Music archive not configured or unreachable
Mon 17 Jun 21:59:48 EDT 2024: Truncating log...
Mon 17 Jun 21:59:49 EDT 2024: Connecting usb to host...
Mon 17 Jun 21:59:49 EDT 2024: Connected usb to host.
Mon 17 Jun 21:59:54 EDT 2024: Waiting for archive to be unreachable...
Mon 17 Jun 21:59:54 EDT 2024: Simulating archive being unreachable.
Mon 17 Jun 21:59:54 EDT 2024: Waiting for archive to be reachable...
Mon 17 Jun 21:59:54 EDT 2024: Archive is reachable.
Mon 17 Jun 21:59:54 EDT 2024: Trying to set time...
Mon 17 Jun 21:59:55 EDT 2024: Time adjusted by 0.091861 seconds after 0.270000 seconds
Mon 17 Jun 21:59:55 EDT 2024: Not keeping car awake.```
@marcone - is there anything that needs to be handled manually at this time or can continue using and files will get removed as it runs out of space? I'm using 256GB microsd and teslausb shows below values for df -hT
/dev/mmcblk0p3 xfs 237G 215G 22G 91% /backingfiles
@cleanev if I remember correctly exfat fstrim support was added with Linux kernel 5.13. Are you using a kernel older than that?
Thanks @marcone. I was under the impression that I was using bullseye image.
Here’s the version.
Kernel: Linux 5.10.103+
Architecture: arm```
Time to recreate using bullseye image of yours. Appreciate your quick response.
Oh man, I find myself in a situation where I really need the Saved video files and don't have them. I didn't realize I had this problem until I went to review the video from an incident. I don't see the files in the mounted directory. Am I correct in assuming they're not retrievable in any way? (I made the recording by pressing the camera button)
Oh man, I find myself in a situation where I really need the Saved video files and don't have them. I didn't realize I had this problem until I went to review the video from an incident. I don't see the files in the mounted directory. Am I correct in assuming they're not retrievable in any way? (I made the recording by pressing the camera button)
Check the live, always running recording in recent, top of the list, it may still have the video. If so pull the pi out and hook it up to a computer to pull the files of it.
Oh man, I find myself in a situation where I really need the Saved video files and don't have them. I didn't realize I had this problem until I went to review the video from an incident. I don't see the files in the mounted directory. Am I correct in assuming they're not retrievable in any way? (I made the recording by pressing the camera button)
Check the live, always running recording in recent, top of the list, it may still have the video. If so pull the pi out and hook it up to a computer to pull the files of it.
It wasn't in Recents. I was on a two-week road trip and the event I'm interested in was a week ago.