delete_source not deleting files
kdvlr opened this issue · 11 comments
Is delete_source designed to be used in conjunction with monitor or monitor_once ? When I use that on my NAS, it does not delete the files that have been merged.
It can be used without monitor or monitor_once.
delete_source would delete the front, left,right camera files and the folder when it was able to create the merged file for the folder successfully.
By default the intermediate files (these are the video files created that have the merged cameras on 1 minute intervals) are deleted from the moment the merged file for the folder was created. Using --keep-intermediate those files will be kept.
So, this is what I am using
set DIRNAME=%DATE:~-4%-%DATE:~-7,2%-%DATE:~-10,2%
mkdir E:\Tesla\TeslaCam\Merged\%DIRNAME%
E:\Tesla\tesla_dashcam\tesla_dashcam.exe --merge --compression ultrafast --encoding x265 --quality HIGH --delete_source --keep-intermediate --output E:\Tesla\TeslaCam\Merged\%DIRNAME%\ E:\Tesla\TeslaCam\SavedClips
Without keep-intermediate, I don't get a single merged file. This is understandable because the per-folder intermediate files are deleted (which is used to create the complete merged file)
However, at the end of the process, I have my original front, right and left files, the intermediate file and the merged file.
All I want is the merged file. Is there a way of achieving this ?
Here's the output from the script
E:\Tesla\tesla_dashcam>E:\Tesla\tesla_dashcam\tesla_dashcam.exe --merge --compression ultrafast --encoding x265 --quality HIGH --delete_source --keep-intermediate --output E:\Tesla\TeslaCam\Merged\2019-13-05\ E:\Tesla\TeslaCam\SavedClips
Discovered 2 folders with 2 clips to process.
Processing 1 clips in folder E:\Tesla\TeslaCam\SavedClips\2019-04-24_21-16 (1/2)
Processing clip 1/1 from 04/24/19 21:16:27 and 59 seconds long.
Creating movie E:\Tesla\TeslaCam\Merged\2019-13-05\2019-04-24_21-16.mp4, please be patient.
Movie E:\Tesla\TeslaCam\Merged\2019-13-05\2019-04-24_21-16.mp4 for folder E:\Tesla\TeslaCam\SavedClips\2019-04-24_21-16 is ready.
Processing 1 clips in folder E:\Tesla\TeslaCam\SavedClips\2019-04-24_21-19 (2/2)
Processing clip 1/1 from 04/24/19 21:19:29 and 59 seconds long.
Creating movie E:\Tesla\TeslaCam\Merged\2019-13-05\2019-04-24_21-19.mp4, please be patient.
Movie E:\Tesla\TeslaCam\Merged\2019-13-05\2019-04-24_21-19.mp4 for folder E:\Tesla\TeslaCam\SavedClips\2019-04-24_21-19 is ready.
Creating movie E:\Tesla\TeslaCam\Merged\2019-13-05\2019-13-05.mp4, please be patient.
Movie E:\Tesla\TeslaCam\Merged\2019-13-05\2019-13-05.mp4 has been created, enjoy.
Total processing time: 0:08:42
I just checked the code, --delete_source indeed only removes the source file using --monitor or --monitor_once; will change this in next release.
Not specifying --keep_intermediate should result in the merged minute files that are created to be removed. They are then removed when the 10-minute clip (all 1-minute clips from the folder) is created.
The final merged file (with --merge) uses the 10-minute clip ones and not the 1-minute clips.
The merged files on the folder level (normally the 10 minute long clip then) currently would always be there as there is nothing in the code to remove that.
Ran similar like yours, running it without --keep_intermediate does result in the final merged file being created for me. The 10-minute clip files are still there but the 1-minute clip files are deleted. Hence not sure why you are not getting the final merged file then. Can you run it without --keep_intermediate and provide me the output?
Here's the output
E:\Tesla\tesla_dashcam>merge_files.bat
E:\Tesla\tesla_dashcam>set DIRNAME=2019-16-05
E:\Tesla\tesla_dashcam>mkdir E:\Tesla\TeslaCam\Merged\2019-16-05
E:\Tesla\tesla_dashcam>E:\Tesla\tesla_dashcam\tesla_dashcam.exe --merge --compression ultrafast --encoding x265 --quality HIGH --delete_source --output E:\Tesla\TeslaCam\Merged\2019-16-05\ E:\Tesla\TeslaCam\SavedClips
Discovered 2 folders with 2 clips to process.
Processing 1 clips in folder E:\Tesla\TeslaCam\SavedClips\2019-04-24_21-16 (1/2)
Processing clip 1/1 from 04/24/19 21:16:27 and 59 seconds long.
Creating movie E:\Tesla\TeslaCam\Merged\2019-16-05\2019-04-24_21-16.mp4, please be patient.
Movie E:\Tesla\TeslaCam\Merged\2019-16-05\2019-04-24_21-16.mp4 for folder E:\Tesla\TeslaCam\SavedClips\2019-04-24_21-16 is ready.
Processing 1 clips in folder E:\Tesla\TeslaCam\SavedClips\2019-04-24_21-19 (2/2)
Processing clip 1/1 from 04/24/19 21:19:29 and 59 seconds long.
Creating movie E:\Tesla\TeslaCam\Merged\2019-16-05\2019-04-24_21-19.mp4, please be patient.
Movie E:\Tesla\TeslaCam\Merged\2019-16-05\2019-04-24_21-19.mp4 for folder E:\Tesla\TeslaCam\SavedClips\2019-04-24_21-19 is ready.
Creating movie E:\Tesla\TeslaCam\Merged\2019-16-05\2019-16-05.mp4, please be patient.
File E:\Tesla\TeslaCam\Merged\2019-16-05\2019-04-24_21-16.mp4 does not exist anymore, skipping.
File E:\Tesla\TeslaCam\Merged\2019-16-05\2019-04-24_21-19.mp4 does not exist anymore, skipping.
Error: No valid clips to merge found.
All folders have been processed, resulting movie files are located in E:\Tesla\TeslaCam\Merged\2019-16-05\
Total processing time: 0:04:22
Here's the resulting directory structure.
E:\Tesla\tesla_dashcam>dir /s E:\Tesla\TeslaCam
Volume in drive E is NAS
Volume Serial Number is 8091-F15A
Directory of E:\Tesla\TeslaCam
05/11/2019 10:55 PM <DIR> .
05/11/2019 10:55 PM <DIR> ..
05/16/2019 11:55 AM <DIR> Merged
05/15/2019 04:36 AM <DIR> SavedClips
0 File(s) 0 bytes
Directory of E:\Tesla\TeslaCam\Merged
05/16/2019 11:55 AM <DIR> .
05/16/2019 11:55 AM <DIR> ..
05/16/2019 12:00 PM <DIR> 2019-16-05
0 File(s) 0 bytes
Directory of E:\Tesla\TeslaCam\Merged\2019-16-05
05/16/2019 12:00 PM <DIR> .
05/16/2019 12:00 PM <DIR> ..
0 File(s) 0 bytes
Directory of E:\Tesla\TeslaCam\SavedClips
05/15/2019 04:36 AM <DIR> .
05/15/2019 04:36 AM <DIR> ..
05/13/2019 10:11 PM <DIR> 2019-04-24_21-16
05/13/2019 10:31 PM <DIR> 2019-04-24_21-19
0 File(s) 0 bytes
Directory of E:\Tesla\TeslaCam\SavedClips\2019-04-24_21-16
05/13/2019 10:11 PM <DIR> .
05/13/2019 10:11 PM <DIR> ..
04/24/2019 09:17 PM 29,972,998 2019-04-24_21-16-front.mp4
04/24/2019 09:17 PM 29,966,497 2019-04-24_21-16-left_repeater.mp4
04/24/2019 09:17 PM 29,952,832 2019-04-24_21-16-right_repeater.mp4
3 File(s) 89,892,327 bytes
Directory of E:\Tesla\TeslaCam\SavedClips\2019-04-24_21-19
05/13/2019 10:31 PM <DIR> .
05/13/2019 10:31 PM <DIR> ..
04/24/2019 09:20 PM 30,024,504 2019-04-24_21-19-front.mp4
04/24/2019 09:20 PM 30,027,339 2019-04-24_21-19-left_repeater.mp4
04/24/2019 09:20 PM 29,987,456 2019-04-24_21-19-right_repeater.mp4
3 File(s) 90,039,299 bytes
Total Files Listed:
6 File(s) 179,931,626 bytes
17 Dir(s) 4,675,108,347,904 bytes free
FYI, not ignoring this. Just haven't had time with end of school stuff for the kids etc.. :-)
I will be looking into this and ensuring it is fixed in the next release however.
No worries. It's a long weekend; Have fun!
I have just created a quick batch file to help me sort it all out. It's a quick hack, but just putting it here in case it helps someone.
@echo off
SETLOCAL ENABLEDELAYEDEXPANSION
set YEAR=%DATE:~-4%
set MONTH=%DATE:~-10,2%
set DAY=%DATE:~-7,2%
for /L %%i IN (%DAY%,-1,1) DO (
set DIRNAME=%YEAR%-%MONTH%-%%i
echo !DIRNAME!
for /r "E:\Tesla\TeslaCam\SavedClips\" %%x in (*!DIRNAME!*) do (
mkdir "E:\Tesla\TeslaCam\Merged\!DIRNAME!"
move "%%x" "E:\Tesla\TeslaCam\Merged\!DIRNAME!\"
)
)
for /f "delims=" %%d in ('dir /s /b /ad E:\Tesla\TeslaCam\SavedClips\ ^| sort /r') do rd "%%d"
for /f "usebackq tokens=*" %%D in (`DIR /S /B /A:D /O:N E:\Tesla\TeslaCam\Merged`) do (
# E:\Tesla\tesla_dashcam\tesla_dashcam.exe --merge --compression ultrafast --encoding x265 --quality HIGH --delete_source --output E:\Tesla\TeslaCam\Merged\%%~nxD\%%~nxD.mp4 E:\Tesla\TeslaCam\Merged\%%~nxD
)
)
I know you mentioned you'll take care of delete_source. I think that should be good enough for me. Even now, I can get rid of all files with _front _repeater and it will do it, but I'd rather have it be included in your script to guarantee that the files have been processed
Semi-related to this issue - I am also using this script with a NAS, which my Raspberry Pi copies to automatically upon connecting to WiFi (https://github.com/marcone/teslausb).
Would it be possible to have a different flavor of --monitor that periodically (I'm thinking in the middle of the night to avoid starting to process files while my Raspberry Pi is still copying files to the share) checks for new folders, instead of looking for TeslaCam to be mounted?
Semi-related to this issue - I am also using this script with a NAS, which my Raspberry Pi copies to automatically upon connecting to WiFi (https://github.com/marcone/teslausb).
Would it be possible to have a different flavor of --monitor that periodically (I'm thinking in the middle of the night to avoid starting to process files while my Raspberry Pi is still copying files to the share) checks for new folders, instead of looking for TeslaCam to be mounted?
This is something I've already been thinking off to be able to do. My thoughts on this was for example when someone is using it with a cloud service, they could copy the files up there and then it would start doing the merging etc.. But in the end solution would work for your case as well then. :-)
Would you mind just opening a new issue for this with the request? Won't be in next release (going on vacation so want to have the next release out before then as it also includes some fixes).
Done, thanks.
FYI,
Just released test version that includes option for --delete_source when not using --monitor and fix for reported issue:
https://github.com/ehendrix23/tesla_dashcam/releases/tag/v0.1.10b0
Release v.0.1.10 has been made available with the fix:
https://github.com/ehendrix23/tesla_dashcam/releases/tag/v0.1.10