Help?: Folder Monitoring doesn't unpack files.
YouveGotMeowxy opened this issue · 18 comments
I've mounted a folder for Unpackerr to monitor in my docker compose like this:
- '/mnt/e/Media/Unpack:/Media/unpack:rw'
and in my .conf have it like this:
[[folder]]
path = "/Media/unpack"
delete_files = false
delete_original = false
disable_log = false
move_back = false
extract_isos = false
disable_recursion = false
and according to the logs, Unpackerr is seeing it fine. The screenshot below shows the log, the view of the files within the container's internal mounted path, and the files in the external path:
so it appears, the mount itself is ok, but Unpackerr just never does anything with the files? I've even left it to sit all night while I slept and still never unpacked them.
Unpackerr v0.13.0
Latest image from: ghcr.io/hotio/unpackerr:release
Did I screw something up?
It only looks at things added after it starts, not pre-existing
Install the latest version and rename one of your files. Or make a new folder in your watch folder, and copy your archives there. They'll be extracted.
Thanks guys.
I think I've figured out the issue.
If I do any file manipulation from the Windows side (rename, drop a new file into the folder, move/copy files into subfolders, etc..) Unpackerr ignores (or simply doesn't see it), but if I do the manipulation from within the container itself (Linux obviously) Unpackerr then handles it IF the file is outside of a subfolder.
example:
My mounted folder is /Media/unpack:
- (within container) I create a subfolder named 'test', and then copy files into it; nothing happens.
- (within container) Renaming files within 'test'; nothing happens.
- (within container) Copying or moving files from 'test' up to the root folder; extracts fine.
- From Windows; no extraction happens no matter what or where the files/folders are in that mount.
I'm not sure about the subfolder handling, but regarding the lack of action when handling files from the Windows side, that's a shame, because aside from the usual *arr stuff within the docker containers, I also was really hoping by having a folder mounted to Windows I could use it to just drop things into whenever I wanted (from Windows) and also have it do it's thing there as well.
the lack of action when handling files from the Windows side, that's a shame
It's unfortunately the poor interaction between Docker and CIFS. Docker cannot see the changes you're making in Windows. Tuning your network mount may help, but there is nothing unpackerr can do to fix this. Try running it not in docker; might fix it.
Tuning your network mount may help
Can you please expand upon this? I'm not sure what you mean by "tune". :)
It's not a problem I've had to solve, so I don't have much to expand. My recommendation is to put unpackerr closer to your files so it can see them change. All the abstractions from CIFS and docker, and whatever the host storage may be doing are preventing that. Maybe it's a cache? I don't know... https://stackoverflow.com/questions/14920591/how-to-disable-caching-in-cifs-samba-on-client-side-in-linux
I have the same problem, running unpackerr in a docker compose and specifying a UN_FOLDER_0_PATH to a folder containing multiple downloads that are unpackaged/zipped and UN_FOLDER_0_EXTRACT_PATH to an output folder. For some reason it never unpacks it, even if I rename, move, etc. the folders within UN_FOLDER_0_PATH .
For example I have
- UN_FOLDER_0_PATH=/downloads/_unsorted
- UN_FOLDER_0_EXTRACT_PATH=/downloads/_sorted
Within _unsorted I have a recently downloaded folder unpackaged. Yet within the logs all I see it
2024-01-22 08:09:53 [INFO] 2024/01/22 08:09:53 [Folder] Watching (fsnotify): /downloads/_unsorted
2024-01-22 08:09:53 [INFO] 2024/01/22 08:09:53 [Folder] Polling @ 1s: /downloads/_unsorted
2024-01-22 08:09:53 [INFO] 2024/01/22 08:09:53 [Radarr] Updated (http://localhost:7878): 0 Items Queued, 0 Retrieved
2024-01-22 08:09:53 [INFO] 2024/01/22 08:09:53 [Sonarr] Updated (http://localhost:8989): 0 Items Queued, 0 Retrieved
2024-01-22 08:10:53 [INFO] 2024/01/22 08:10:53 [Unpackerr] Queue: [0 waiting] [0 queued] [0 extracting] [0 extracted] [0 imported] [0 failed] [0 deleted]
2024-01-22 08:10:53 [INFO] 2024/01/22 08:10:53 [Unpackerr] Totals: [0 retries] [0 finished] [0|0 webhooks] [0|0 cmdhooks] [stacks; event:0, hook:0, del:0]
I'm no coding wiz, but could it be a bug w/fsnotify? Which I believe is a 3rd party file system watcher?
@SachaTe It only sees new items you add to /downloads/_unsorted and will not see existing items.
@YouveGotMeowxy In addition to fsnotify, when docker is detected the app turns on a poller. It does the equivalent of ls on your watch folder every second looking for new items. In your case, CIFS is not updating Docker and Docker doesn't see the changes to the file system. If @SachaTe also has remote storage for his downloads then it's the same problem.
When you mount a network file system into a docker container: the performance and abilities vary widely. It's not recommended.
For both of you: Run the app not-in-docker and see how well things work. It runs on bsd, mac, windows and linux natively.
When you mount a network file system into a docker container: the performance and abilities vary widely. It's not recommended.
Ahh ok, interesting. Again, I'm somewhat intermediate in my skills in this stuff, so I'm still learning as I go; is a simple volume bind mount considered/treated as a "network" file system? I just mount it in the same usual way as all volume bind mounts:
- '/mnt/e/Media/Unpack:/Media/unpack:rw'
the weird thing (to me) is that if Unpacker is kinda doing a simple ls, and watches the /Media/unpack folder within the container, doing that ls does still show the files if I do the command manually inside the container; but for some reason Unpacker doesn't see them when it does it with the same command?
@YouveGotMeowxy It'd be a lot easier to troubleshoot your problem in Discord chat than here.
@davidnewhall ok, maybe one of these days I'll go on Discord. I'm at work rn though and no time to chat.
I've uncovered the bug with the folder poller. Will try to get it fixed.
See if the unstable docker tag works any better please.
See if the
unstabledocker tag works any better please.
I'm using the hotio container which has no unstable version; is the only difference between yours and hotio's the UID thing? if so I'll switch over from hotio.
The only "configuration difference" is the "uid thing" that's correct. You can set PUID and PGID on the golift container, but they don't do anything. Use user: as well. (but don't use user: on hotio).
ok, just wondering if switching containers would inadvertently screw anything else up. I'll try now.
SUCCESS!! woot!
Anything I drop into the folder on the Windows side extracts:
thank you for fixing this :)