Addon with docker results in Joex raising java.nio.file.NotDirectoryException
tiborrr opened this issue · 5 comments
tiborrr commented
Using an addon that depends on a docker image results in a error:
2024.05.22 10:00:01:0016 [io-comp...] [INFO ] docspell.scheduler.impl.LogSink.logInternal:41 - >>> 2024-05-22T08:00:01.663265642Z Info 61chMgjio.../1/process-item/High: Running 1 addon tasks for trigger Set(FinalProcessItem) (jobId: "61chMgjio9M-Y4vwYtzsoWT-2vYRKG27DLy-Y1TqfoRoFii", task: "process-item", group: "1", jobInfo: "61chMgjio.../1/process-item/High")
2024.05.22 10:00:01:0001 [io-comp...] [TRACE] docspell.store.impl.DoobieLogging.run:24 - SQL: INSERT INTO joblog (id, jid, level, created, message) VALUES (?,?,?,?,? ) (List(Ident(GQec5SbafBj-FrqbU1CaHFe-H36mXxrCBWq-j7tbTvzPbWA), Ident(61chMgjio9M-Y4vwYtzsoWT-2vYRKG27DLy-Y1TqfoRoFii), Info, Timestamp(2024-05-22T08:00:01.663265642Z), Running 1 addon tasks for trigger Set(FinalProcessItem))) executed in 366751 nanoseconds
2024.05.22 10:00:01:0017 [io-comp...] [WARN ] docspell.backend.joex.AddonPrepare.logResult:35 - Addon task '2PFFh4F5eiw-mqKsBKkEqUP-LmUho82NfuF-VPg2p2vH1Zt' has failed
java.nio.file.NotDirectoryException: /tmp/docspell-addons/addon-17931550648262311479/addons/rita-addon-0.33.0/docspell-addon.yml
at sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:419)
at java.nio.file.Files.newDirectoryStream(Files.java:482)
at java.nio.file.Files.list(Files.java:3792)
at fs2.io.file.FilesCompanionPlatform$AsyncFiles.$anonfun$list$1(FilesPlatform.scala:304)
at make @ docspell.common.ThreadFactories$.executorResource(ThreadFactories.scala:48)
at make @ docspell.common.ThreadFactories$.executorResource(ThreadFactories.scala:48)
at uncancelable @ fs2.Pull$.acquireCancelable(Pull.scala:423)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at handleErrorWith @ fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at uncancelable @ fs2.Compiler$Target.uncancelable(Compiler.scala:165)
at unique @ fs2.Compiler$Target$ConcurrentTarget.unique(Compiler.scala:194)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at ref @ fs2.concurrent.SignallingRef$.of(Signal.scala:240)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
(addon-task-id: "2PFFh4F5eiw-mqKsBKkEqUP-LmUho82NfuF-VPg2p2vH1Zt")
2024.05.22 10:00:01:0002 [io-comp...] [DEBUG] docspell.scheduler.impl.LogSink.logInternal:43 - >>> 2024-05-22T08:00:01.666582010Z Debug 61chMgjio.../1/process-item/High: Preparing item data 'test_text.pdf' for addon (jobId: "61chMgjio9M-Y4vwYtzsoWT-2vYRKG27DLy-Y1TqfoRoFii", task: "process-item", group: "1", jobInfo: "61chMgjio.../1/process-item/High")
2024.05.22 10:00:01:0024 [io-comp...] [TRACE] docspell.store.impl.DoobieLogging.run:24 - SQL: INSERT INTO joblog (id, jid, level, created, message) VALUES (?,?,?,?,? ) (List(Ident(8cJ7TAGsS4m-eVDqWL3hEs8-kusPQkHmvVr-qQoE55a8WYM), Ident(61chMgjio9M-Y4vwYtzsoWT-2vYRKG27DLy-Y1TqfoRoFii), Debug, Timestamp(2024-05-22T08:00:01.666582010Z), Preparing item data 'test_text.pdf' for addon)) executed in 320437 nanoseconds
2024.05.22 10:00:01:0018 [io-comp...] [TRACE] docspell.store.impl.DoobieLogging.run:24 - SQL: SELECT job.retries FROM job WHERE job.jid = ? (List(Ident(61chMgjio9M-Y4vwYtzsoWT-2vYRKG27DLy-Y1TqfoRoFii))) executed in 431290 nanoseconds
2024.05.22 10:00:01:0019 [io-comp...] [ERROR] docspell.scheduler.impl.SchedulerImpl.wrapTask:304 - Job 61chMgjio.../1/process-item/High execution failed. Retrying later.
You can use the following link to check the repo and docker image that I have used:
https://github.com/tiborrr/docspell-addon-example/releases/tag/0.6.0
meta:
name: "docspell-addon-example"
version: "0.7.0"
description: |
This is a minimal example of a docspell addon. It does nothing but print Hello and the argument that was passed to it.
triggers:
- final-process-item
runner:
docker:
enable: true
image: tiborr/docspell-addon-example:latest
options:
networking: false
collectOutput: trueThis code should be run in this container.
import sys
def main():
if len(sys.argv) > 1:
print(f"Hello {sys.argv[1]}")
else:
print("Hello")
if __name__ == "__main__":
main()Full implementation can be found here:
eikek commented
Thank you for the report with details to reproduce! I hope I can find some time soon to check it out
eikek commented
Hi @tiborrr I just pushed a fix for extracting addons. You can test it out using the nightly version. When I now test it, extraction works fine, but then it fails when trying to attempt to download the image
Debug 9QZZ5cfQD.../1/process-item/High: >> [docspell-addon-example-0.7.0 (err)] Unable to find image 'tiborr/docspell-addon-example:latest' locally
Debug 9QZZ5cfQD.../1/process-item/High: >> [docspell-addon-example-0.7.0 (err)] docker: Error response from daemon: pull access denied for tiborr/docspell-addon-example, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
This now seems to be an access issue.
eikek commented
Mergify closed it :-) I reopen to let you have a look
tiborrr commented
I do not think it works correctly yet.
I am now getting this error:
Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /tmp/docspell-addons/addon-14483220073169593010.
Thu, May 30th, 2024, 14:16: Running external command: docker run --rm --name docspell-addon-example-0.7.0-4B1nxM --mount type=bind,source=/tmp/docspell-addons/addon-14483220073169593010,target=/mnt/work --mount type=bind,source=/tmp/docspell-addons/addon-output-10138378789386319765,target=/mnt/output --mount type=bind,source=/tmp/docspell-addon-cache/EXpNaSFCvTP-eXs6FcYKdaS-41zyL95tTUR-jJERLW2CNkj,target=/mnt/cache --network none --env ADDON_DIR=addons/docspell-addon-example-0.7.0 --env ITEM_DATA_JSON=item/item-data.json --env DSC_DOCSPELL_URL=http://localhost:7880 --env ITEM_DIR=item --env TMP_DIR=temp --env TMPDIR=temp --env CACHE_DIR=/mnt/cache --env ITEM_ARGS_JSON=item/given-data.json --env ITEM_ORIGINAL_JSON=item/source-files.json --env DSC_SESSION=1717078614008-MS9ldHMvNldTa1dQaTl5dDYtTXROMmYzVXlwRGctY2tRQTZka1lOZG4tTlRIczJjTnhkTkgvYWRtaW4=-false-900-$2a$10$/xwLHsshpLKSDGii55kbUu-a8eFZrPefIXqO6qxnktUXLwPC/A= --env ITEM_PDF_JSON=item/pdf-files.json --env OUTPUT_DIR=/mnt/output --env ITEM_ORIGINAL_DIR=item/originals --env ITEM_PDF_DIR=item/pdfs -w /mnt/work tiborr/docspell-addon-example:latest arguments/user-input
Thu, May 30th, 2024, 14:16: Waiting for command to terminate…
Thu, May 30th, 2024, 14:16: >> [docspell-addon-example-0.7.0 (err)] docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /tmp/docspell-addons/addon-14483220073169593010.
Thu, May 30th, 2024, 14:16: >> [docspell-addon-example-0.7.0 (err)] See 'docker run --help'.
Thu, May 30th, 2024, 14:16: >> [docspell-addon-example-0.7.0 (err)]
I also think it's weird you are getting this pull access denied for tiborr/docspell-addon-example because this is a public repo.
Can you check what happens if you do docker pull tiborr/docspell-addon-example in your joex instance