MDKPredator/degoo_drive

Fail to copy files to and from Degoo volume

miikka75 opened this issue · 10 comments

miikka75 commented

I created docker container according to instructions using docker-compose.yml (at the root of cloned repository degoo_drive folder):

version: '3'

services:
  degoo-drive:
    build: .
    command: --debug --debug-fuse
    container_name: degoo-drive
    privileged: true
    volumes:
      - ${DEGOO_MOUNT}:/source_dir
      - ./degoo_config:/root/.config/degoo
    restart: always
    network_mode: host

Login didn't work using credentials.json but I modified Token and RefreshToken in keys.json with values from logging in with browser (Edge).

Init:

2022-11-14 12:14:53.132 MainThread: [__main__] Mode:                lazy
2022-11-14 12:14:53.132 MainThread: [__main__] Reading Degoo content from directory /
2022-11-14 12:14:53.136 MainThread: [urllib3.connectionpool] Starting new HTTPS connection (1): production-appsync.degoo.com:443
2022-11-14 12:14:53.673 MainThread: [urllib3.connectionpool] https://production-appsync.degoo.com:443 "POST /graphql HTTP/1.1" 200 None
2022-11-14 12:14:53.676 MainThread: [__main__] _add_path for 0, /
2022-11-14 12:14:53.676 MainThread: [__main__] _add_path for 15884293431, /Web
2022-11-14 12:14:53.676 MainThread: [__main__] Mounting...
2022-11-14 12:14:53.688 MainThread: [pyfuse3] Initializing pyfuse3
2022-11-14 12:14:53.688 MainThread: [pyfuse3] Calling fuse_session_new
FUSE library version: 3.10.3
2022-11-14 12:14:53.688 MainThread: [pyfuse3] Calling fuse_session_mount
2022-11-14 12:14:53.689 MainThread: [__main__] Entering main loop..
2022-11-14 12:14:53.691 MainThread: [pyfuse3] pyfuse-02: No tasks waiting, starting another worker (now 2 total).
unique: 2, opcode: INIT (26), nodeid: 0, insize: 56, pid: 0
INIT: 7.34
flags=0x33fffffb
max_readahead=0x00020000
   INIT: 7.31
   flags=0x0041b039
   max_readahead=0x00020000
   max_write=0x00100000
   max_background=0
   congestion_threshold=0
   time_gran=1
   unique: 2, success, outsize: 80
unique: 4, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 8
   unique: 4, success, outsize: 120
unique: 6, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 8
   unique: 6, success, outsize: 32
unique: 8, opcode: READDIRPLUS (44), nodeid: 1, insize: 80, pid: 8
2022-11-14 12:15:14.449 MainThread: [__main__] reading /
2022-11-14 12:15:14.449 MainThread: [__main__] _add_path for 15884293431, /Web
   unique: 8, success, outsize: 360
unique: 10, opcode: READDIRPLUS (44), nodeid: 1, insize: 80, pid: 8
2022-11-14 12:15:14.450 MainThread: [__main__] reading /
   unique: 10, success, outsize: 16
unique: 12, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 12, error: -38 (Function not implemented), outsize: 16
unique: 14, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 8
   unique: 14, success, outsize: 120
unique: 16, opcode: OPENDIR (27), nodeid: 15884293431, insize: 48, pid: 8
   unique: 16, success, outsize: 32
unique: 18, opcode: READDIRPLUS (44), nodeid: 15884293431, insize: 80, pid: 8

Trying to copy file from Degoo drive to local filesystem:

cp /home/degoo/Web/kansi.pdf /source_dir/

Empty file (size 0) kansi.pdf file is created at target path.

2022-11-14 12:15:15.388 MainThread: [__main__] reading /Web
2022-11-14 12:15:15.391 MainThread: [urllib3.connectionpool] Starting new HTTPS connection (1): production-appsync.degoo.com:443
2022-11-14 12:15:15.971 MainThread: [urllib3.connectionpool] https://production-appsync.degoo.com:443 "POST /graphql HTTP/1.1" 200 None
2022-11-14 12:15:15.974 MainThread: [__main__] _add_path for 18778209180, /Web/kansi.pdf
2022-11-14 12:15:15.975 MainThread: [__main__] _add_path for 18778209180, /Web/kansi.pdf
   unique: 18, success, outsize: 184
unique: 20, opcode: READDIRPLUS (44), nodeid: 15884293431, insize: 80, pid: 8
2022-11-14 12:15:15.975 MainThread: [__main__] reading /Web
   unique: 20, success, outsize: 16
unique: 22, opcode: RELEASEDIR (29), nodeid: 15884293431, insize: 64, pid: 0
   unique: 22, error: -38 (Function not implemented), outsize: 16
unique: 24, opcode: GETATTR (3), nodeid: 15884293431, insize: 56, pid: 8
   unique: 24, success, outsize: 120
unique: 26, opcode: OPEN (14), nodeid: 18778209180, insize: 48, pid: 16
   unique: 26, success, outsize: 32
unique: 28, opcode: READ (15), nodeid: 18778209180, insize: 80, pid: 16
   unique: 28, error: -2 (No such file or directory), outsize: 16
unique: 30, opcode: READ (15), nodeid: 18778209180, insize: 80, pid: 16
   unique: 30, error: -2 (No such file or directory), outsize: 16
unique: 32, opcode: FLUSH (25), nodeid: 18778209180, insize: 64, pid: 16
   unique: 32, error: -38 (Function not implemented), outsize: 16
unique: 34, opcode: RELEASE (18), nodeid: 18778209180, insize: 64, pid: 0
2022-11-14 12:15:19.661 MainThread: [__main__] Releasing file kansi.pdf
   unique: 34, success, outsize: 16
unique: 36, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 8
   unique: 36, success, outsize: 32
unique: 38, opcode: READDIRPLUS (44), nodeid: 1, insize: 80, pid: 8

Trying to copy file from local filesystem to Degoo drive:

cp /source_dir/degoo_token_refresh_token.png /home/degoo/Web/

No file is created at Degoo web drive.

2022-11-14 12:15:37.290 MainThread: [__main__] reading /
2022-11-14 12:15:37.290 MainThread: [__main__] _add_path for 15884293431, /Web
   unique: 38, success, outsize: 360
unique: 40, opcode: READDIRPLUS (44), nodeid: 1, insize: 80, pid: 8
2022-11-14 12:15:37.290 MainThread: [__main__] reading /
   unique: 40, success, outsize: 16
unique: 42, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 42, error: -38 (Function not implemented), outsize: 16
unique: 44, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 8
   unique: 44, success, outsize: 120
unique: 46, opcode: LOOKUP (1), nodeid: 15884293431, insize: 70, pid: 17
2022-11-14 12:15:38.699 MainThread: [__main__] lookup for degoo_token_refresh_token.png in 15884293431
   unique: 46, error: -2 (No such file or directory), outsize: 16
unique: 48, opcode: LOOKUP (1), nodeid: 15884293431, insize: 70, pid: 17
2022-11-14 12:15:38.700 MainThread: [__main__] lookup for degoo_token_refresh_token.png in 15884293431
   unique: 48, error: -2 (No such file or directory), outsize: 16
unique: 50, opcode: CREATE (35), nodeid: 15884293431, insize: 86, pid: 17
2022-11-14 12:15:38.700 MainThread: [__main__] _add_path for 3295235, /tmp/degoo/degoo_token_refresh_token.png
   unique: 50, success, outsize: 160
unique: 52, opcode: GETATTR (3), nodeid: 3295235, insize: 56, pid: 17
   unique: 52, success, outsize: 120
unique: 54, opcode: GETATTR (3), nodeid: 3295235, insize: 56, pid: 17
   unique: 54, success, outsize: 120
unique: 56, opcode: WRITE (16), nodeid: 3295235, insize: 9073, pid: 17
   unique: 56, success, outsize: 24
unique: 58, opcode: RELEASE (18), nodeid: 3295235, insize: 64, pid: 0
   unique: 58, success, outsize: 16

I can create new folders and delete folders and uploaded (via web-UI) files but file download/upload doesn't work.
Any help would be appreciated!

MDKPredator commented

Virtual drives are not visible from host to docker container. You can:

  • Mount degoo drive directly over your host
  • Copy the file to container (docker cp) and then, copy it from container to degoo
miikka75 commented

same error happens when I try to copy files from within docker container:

root@miibuntu:/# cp /degoo_drive/fuse_degoo.py /home/degoo/Web/
root@miibuntu:/# ls -la /home/degoo/Web/
total 94
-rw-rw-r-- 1 root root 96245 Apr 19  2022 kansi.pdf
root@miibuntu:/#  cp /home/degoo/Web/kansi.pdf /root
cp: error reading '/home/degoo/Web/kansi.pdf': No such file or directory
root@miibuntu:/# ls -l /root/
total 0
-rw-r--r-- 1 root root    0 Nov 15 18:03 kansi.pdf

Logs don't have anything new compared to the ones I previously posted...

MDKPredator commented

I did a quick test and it seems that the upload/download does not work. They probably changed something in the endpoint, I'll try to check it this weekend.

  • 👍1
MDKPredator commented

I have uploaded a fix. Keep in mind I have created a new config file, overlay4.txt

miikka75 commented

Thanks, I tested the changes and now I can download files from degoo (remote) drive but upload still fails...

MDKPredator commented

This is a known problem

miikka75 commented

I really appreciate the time and effort you have put into resolving this problem.
Maybe my wording was inaccurate. Upload doesn't work and there is no warning in the logs about failed upload. It just "silently" fails/doesn't upload anything.
This line should be also added to Dockerfile:

COPY degoo_config/overlay4.txt /root/.config/degoo/
miikka75 commented

I tried again from scratch using new token and refresh token but I still can't copy (upload) files to remote degoo drive - downloading works:

root@7f9d535e1da1:/# ls -la /home/degoo/Web/kansi.pdf 
-rw-rw-r-- 1 root root 96245 Apr 19  2022 /home/degoo/Web/kansi.pdf
root@7f9d535e1da1:/# cp /home/degoo/Web/kansi.pdf .
root@7f9d535e1da1:/# ls -la kansi.pdf 
-rw-r--r-- 1 root root 96245 Nov 24 19:32 kansi.pdf
root@7f9d535e1da1:/# ls -la /home/degoo/Web/test
total 0
root@7f9d535e1da1:/# cp kansi.pdf /home/degoo/Web/test
root@7f9d535e1da1:/# ls -la /home/degoo/Web/test
total 0

Corresponding log:

2022-11-24T19:31:52.132498465Z unique: 146, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 22
2022-11-24T19:31:52.132610943Z    unique: 146, success, outsize: 32
2022-11-24T19:31:52.132622674Z unique: 148, opcode: READDIRPLUS (44), nodeid: 1, insize: 80, pid: 22
2022-11-24T19:31:52.132780726Z 2022-11-24 19:31:52.132 MainThread: [__main__] reading /
2022-11-24T19:31:52.132952899Z 2022-11-24 19:31:52.132 MainThread: [__main__] _add_path for 4988159340, /DESKTOP-IEON0O8 (2019-09-10)
2022-11-24T19:31:52.133128117Z 2022-11-24 19:31:52.132 MainThread: [__main__] _add_path for 15884293431, /Web
2022-11-24T19:31:52.133143953Z    unique: 148, success, outsize: 360
2022-11-24T19:31:52.133408536Z unique: 150, opcode: READDIRPLUS (44), nodeid: 1, insize: 80, pid: 22
2022-11-24T19:31:52.133556670Z 2022-11-24 19:31:52.133 MainThread: [__main__] reading /
2022-11-24T19:31:52.134535738Z    unique: 150, success, outsize: 16
2022-11-24T19:31:52.134601884Z unique: 152, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
2022-11-24T19:31:52.134614893Z    unique: 152, error: -38 (Function not implemented), outsize: 16
2022-11-24T19:31:53.864499008Z unique: 154, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 22
2022-11-24T19:31:53.864541439Z    unique: 154, success, outsize: 120
2022-11-24T19:31:53.864747139Z unique: 156, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 22
2022-11-24T19:31:53.864762699Z    unique: 156, success, outsize: 32
2022-11-24T19:31:53.865079759Z unique: 158, opcode: READDIRPLUS (44), nodeid: 1, insize: 80, pid: 22
2022-11-24T19:31:53.865305995Z 2022-11-24 19:31:53.865 MainThread: [__main__] reading /
2022-11-24T19:31:53.865570310Z 2022-11-24 19:31:53.865 MainThread: [__main__] _add_path for 4988159340, /DESKTOP-IEON0O8 (2019-09-10)
2022-11-24T19:31:53.865694760Z 2022-11-24 19:31:53.865 MainThread: [__main__] _add_path for 15884293431, /Web
2022-11-24T19:31:53.865710045Z    unique: 158, success, outsize: 360
2022-11-24T19:31:53.867103028Z unique: 160, opcode: READDIRPLUS (44), nodeid: 1, insize: 80, pid: 22
2022-11-24T19:31:53.867157898Z 2022-11-24 19:31:53.865 MainThread: [__main__] reading /
2022-11-24T19:31:53.867167780Z    unique: 160, success, outsize: 16
2022-11-24T19:31:53.867175811Z unique: 162, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
2022-11-24T19:31:53.867183844Z    unique: 162, error: -38 (Function not implemented), outsize: 16
2022-11-24T19:31:53.867208242Z unique: 164, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 22
2022-11-24T19:31:53.867216806Z    unique: 164, success, outsize: 120
2022-11-24T19:31:55.768530093Z unique: 166, opcode: OPENDIR (27), nodeid: 15884293431, insize: 48, pid: 22
2022-11-24T19:31:55.768613424Z    unique: 166, success, outsize: 32
2022-11-24T19:31:55.768726581Z unique: 168, opcode: READDIRPLUS (44), nodeid: 15884293431, insize: 80, pid: 22
2022-11-24T19:31:55.768901042Z 2022-11-24 19:31:55.768 MainThread: [__main__] reading /Web
2022-11-24T19:31:55.769141323Z 2022-11-24 19:31:55.768 MainThread: [__main__] _add_path for 18786917526, /Web/kansi.pdf
2022-11-24T19:31:55.769300352Z 2022-11-24 19:31:55.769 MainThread: [__main__] _add_path for 18811163292, /Web/test
2022-11-24T19:31:55.769317077Z    unique: 168, success, outsize: 344
2022-11-24T19:31:55.769564677Z unique: 170, opcode: READDIRPLUS (44), nodeid: 15884293431, insize: 80, pid: 22
2022-11-24T19:31:55.769695298Z 2022-11-24 19:31:55.769 MainThread: [__main__] reading /Web
2022-11-24T19:31:55.769877462Z    unique: 170, success, outsize: 16
2022-11-24T19:31:55.770076759Z unique: 172, opcode: RELEASEDIR (29), nodeid: 15884293431, insize: 64, pid: 0
2022-11-24T19:31:55.770092818Z    unique: 172, error: -38 (Function not implemented), outsize: 16
2022-11-24T19:31:55.770484743Z unique: 174, opcode: GETATTR (3), nodeid: 15884293431, insize: 56, pid: 22
2022-11-24T19:31:55.770514826Z    unique: 174, success, outsize: 120
2022-11-24T19:32:10.229047140Z unique: 176, opcode: OPEN (14), nodeid: 18786917526, insize: 48, pid: 30
2022-11-24T19:32:10.229079052Z    unique: 176, success, outsize: 32
2022-11-24T19:32:10.230083655Z unique: 178, opcode: RELEASE (18), nodeid: 18786917526, insize: 64, pid: 0
2022-11-24T19:32:10.230116700Z 2022-11-24 19:32:10.229 MainThread: [__main__] Releasing file kansi.pdf
2022-11-24T19:32:10.230124588Z    unique: 178, success, outsize: 16
2022-11-24T19:32:33.753340270Z unique: 180, opcode: OPENDIR (27), nodeid: 18811163292, insize: 48, pid: 32
2022-11-24T19:32:33.753391837Z    unique: 180, success, outsize: 32
2022-11-24T19:32:33.753401178Z unique: 182, opcode: READDIRPLUS (44), nodeid: 18811163292, insize: 80, pid: 32
2022-11-24T19:32:33.753408625Z 2022-11-24 19:32:33.751 MainThread: [__main__] reading /Web/test
2022-11-24T19:32:33.753415407Z    unique: 182, success, outsize: 16
2022-11-24T19:32:33.753421925Z unique: 184, opcode: RELEASEDIR (29), nodeid: 18811163292, insize: 64, pid: 0
2022-11-24T19:32:33.753428647Z    unique: 184, error: -38 (Function not implemented), outsize: 16
2022-11-24T19:32:55.221842568Z unique: 186, opcode: GETATTR (3), nodeid: 18811163292, insize: 56, pid: 33
2022-11-24T19:32:55.222100992Z    unique: 186, success, outsize: 120
2022-11-24T19:32:55.222395431Z unique: 188, opcode: LOOKUP (1), nodeid: 18811163292, insize: 50, pid: 33
2022-11-24T19:32:55.222655738Z 2022-11-24 19:32:55.222 MainThread: [__main__] lookup for kansi.pdf in 18811163292
2022-11-24T19:32:55.222690094Z    unique: 188, error: -2 (No such file or directory), outsize: 16
2022-11-24T19:32:55.222987257Z unique: 190, opcode: LOOKUP (1), nodeid: 18811163292, insize: 50, pid: 33
2022-11-24T19:32:55.223265793Z 2022-11-24 19:32:55.222 MainThread: [__main__] lookup for kansi.pdf in 18811163292
2022-11-24T19:32:55.223283694Z    unique: 190, error: -2 (No such file or directory), outsize: 16
2022-11-24T19:32:55.223557457Z unique: 192, opcode: LOOKUP (1), nodeid: 18811163292, insize: 50, pid: 33
2022-11-24T19:32:55.223818937Z 2022-11-24 19:32:55.223 MainThread: [__main__] lookup for kansi.pdf in 18811163292
2022-11-24T19:32:55.223900406Z    unique: 192, error: -2 (No such file or directory), outsize: 16
2022-11-24T19:32:55.224301779Z unique: 194, opcode: CREATE (35), nodeid: 18811163292, insize: 66, pid: 33
2022-11-24T19:32:55.224694475Z 2022-11-24 19:32:55.224 MainThread: [__main__] _add_path for 3409067, /tmp/degoo/kansi.pdf
2022-11-24T19:32:55.224894002Z    unique: 194, success, outsize: 160
2022-11-24T19:32:55.225235255Z unique: 196, opcode: GETATTR (3), nodeid: 3409067, insize: 56, pid: 33
2022-11-24T19:32:55.225332442Z    unique: 196, success, outsize: 120
2022-11-24T19:32:55.225802541Z unique: 198, opcode: GETATTR (3), nodeid: 3409067, insize: 56, pid: 33
2022-11-24T19:32:55.225820891Z    unique: 198, success, outsize: 120
2022-11-24T19:32:55.225988672Z unique: 200, opcode: WRITE (16), nodeid: 3409067, insize: 96325, pid: 33
2022-11-24T19:32:55.226647329Z    unique: 200, success, outsize: 24
2022-11-24T19:32:55.226701448Z unique: 202, opcode: RELEASE (18), nodeid: 3409067, insize: 64, pid: 0
2022-11-24T19:32:55.227116633Z    unique: 202, success, outsize: 16
2022-11-24T19:32:55.227898147Z unique: 204, opcode: FORGET (2), nodeid: 3409067, insize: 48, pid: 0
2022-11-24T19:32:58.310572979Z unique: 206, opcode: GETATTR (3), nodeid: 18811163292, insize: 56, pid: 34
2022-11-24T19:32:58.310631027Z    unique: 206, success, outsize: 120
2022-11-24T19:32:58.312176137Z unique: 208, opcode: OPENDIR (27), nodeid: 18811163292, insize: 48, pid: 34
2022-11-24T19:32:58.312223447Z    unique: 208, success, outsize: 32
2022-11-24T19:32:58.312233436Z unique: 210, opcode: READDIRPLUS (44), nodeid: 18811163292, insize: 80, pid: 34
2022-11-24T19:32:58.312241690Z 2022-11-24 19:32:58.311 MainThread: [__main__] reading /Web/test
2022-11-24T19:32:58.312249197Z    unique: 210, success, outsize: 16
2022-11-24T19:32:58.312256326Z unique: 212, opcode: RELEASEDIR (29), nodeid: 18811163292, insize: 64, pid: 0
2022-11-24T19:32:58.312263869Z    unique: 212, error: -38 (Function not implemented), outsize: 16

Are you able to upload files? Did you use new token and refresh token when you verified that uploading works? Is there any other debugging options I could enable to investigate why uploading doesnät work for me?

MDKPredator commented

Have you deleted the "Token" key value in keys.json? The file should look like this:

{ "Token": "", "RefreshToken":"<login bypass>", "x-api-key": "da2-vs6twz5vnjdavpqndtbzg3prra"}

Even if the upload failed, the trace Upload of file <file> finished. Id <id> Url <url> should be output. However, the trace does not appear in your logs.