11-to-12 migration fails with s3ds plugin
ShadowJonathan opened this issue · 1 comments
ShadowJonathan commented
Currently i'm in the middle of an angry and frustrated debug session for this, but so far I have observed:
- It does not properly query beyond 1000 keys (the default max for one list call)
- It does not swap anything because it "cannot find the key":
root@infinistore:~/fs-repo-migrations# ~/unpack/fs-repo-migrations/fs-repo-migrations -to 12
Found fs-repo version 11 at /data/ipfs
Do you want to upgrade this to version 12? [y/n] y
2023/01/22 17:27:03 Looking for suitable migration binaries.
2023/01/22 17:27:03 Running migration fs-repo-11-to-12 ...
2023/01/22 17:27:03 => Running: /root/bin/fs-repo-11-to-12 -path=/data/ipfs -verbose=true
applying 11-to-12 repo migration
locking repo at "/data/ipfs"
- verifying version is '11'
- loading repo configurations
- starting CIDv1 to raw multihash block migration
- backup file will be written to /data/ipfs/11-to-12-cids.txt
- Adding keys in prefix /blocks to backup file
1000 CIDv1 keys added to backup file for /blocks
- Adding keys in prefix /filestore to backup file
0 CIDv1 keys added to backup file for /filestore
- backup file will be read from /data/ipfs/11-to-12-cids.txt
ERROR: could not swap /blocks/ipfs/AFKREIA234FMYJUK6JEMCMKMNOOVP4DH2JEIKLMXSFHXICRT23GTWOENXA->/blocks/ipfs/CIQBVXYKZQTIV4SIYEYUY245K7YGPUSIQUWZPEKPOQFDHVWNHM4I3OA. Could not find /blocks/ipfs/AFKREIA234FMYJUK6JEMCMKMNOOVP4DH2JEIKLMXSFHXICRT23GTWOENXA even though it was in the backup file. Skipping.
ERROR: could not swap /blocks/ipfs/AFKREIA2464TM23YDSLXW55WZQFZFCJ4BYQ3LFUX3XX3O3UHJWGWOA27AM->/blocks/ipfs/CIQBVZ5ZGZVXQHEXPN33NTALSKETYDRBWWLJPXPPW5XIOTMNM4BV6AY. Could not find /blocks/ipfs/AFKREIA2464TM23YDSLXW55WZQFZFCJ4BYQ3LFUX3XX3O3UHJWGWOA27AM even though it was in the backup file. Skipping.
ERROR: could not swap /blocks/ipfs/AFKREIA246V6J6SNG7BXZKS4R7WDYY6YESKD3JB7PNG3LVGN7SF6OW26OY->/blocks/ipfs/CIQBVZ5L4T5E2N6DPSVFZD7MHRR5QJEUHWSD662NWXKM37EL45NV45Q. Could not find /blocks/ipfs/AFKREIA246V6J6SNG7BXZKS4R7WDYY6YESKD3JB7PNG3LVGN7SF6OW26OY even though it was in the backup file. Skipping.
...
I built this with: ./build-plugin.sh 11-to-12 github.com/ipfs/go-ds-s3@v0.7.0
ShadowJonathan commented
I have finally gotten this working, using my own fork:
./build-plugin.sh 11-to-12 github.com/shadowjonathan/go-ds-s3@working-0.5-migration