12月15号还可以上传，今天去查看不行了
Closed this issue · 5 comments
大文件一直报错，如下
Traceback (most recent call last):
File "/home/buyi/anaconda3/bin/bypy", line 8, in
sys.exit(main())
File "/home/buyi/anaconda3/lib/python3.8/site-packages/bypy/bypy.py", line 3754, in main
result = getattr(by, args.command[0])(*uargs)
File "/home/buyi/anaconda3/lib/python3.8/site-packages/bypy/bypy.py", line 2953, in syncup
subresult = self.__syncup_single(localdir, rpath, compare_result)
File "/home/buyi/anaconda3/lib/python3.8/site-packages/bypy/bypy.py", line 2916, in __syncup_single
subresult = self.__syncup_local_one(rpath, localdir, l)
File "/home/buyi/anaconda3/lib/python3.8/site-packages/bypy/bypy.py", line 2880, in __syncup_local_one
subresult = self.__upload_file(lcpath, rcpath)
File "/home/buyi/anaconda3/lib/python3.8/site-packages/bypy/bypy.py", line 1761, in __upload_file
result = self.__upload_file_slices(localpath, remotepath, ondup)
File "/home/buyi/anaconda3/lib/python3.8/site-packages/bypy/bypy.py", line 1523, in __upload_file_slices
progress = jsonload(const.ProgressPath)
File "/home/buyi/anaconda3/lib/python3.8/site-packages/bypy/util.py", line 241, in jsonload
return json.load(f)
File "/home/buyi/anaconda3/lib/python3.8/json/init.py", line 293, in load
return loads(fp.read(),
File "/home/buyi/anaconda3/lib/python3.8/json/init.py", line 357, in loads
return _default_decoder.decode(s)
File "/home/buyi/anaconda3/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/home/buyi/anaconda3/lib/python3.8/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Unterminated string starting at: line 490 column 7 (char 20448)
小文件上传成功，稍微大一点文件就不行了，之前上传20G的文件都没问题。
谢谢反馈。
progress = jsonload(const.ProgressPath)
看上去是上传/下载进展的文件损坏？你去~/.bypy/下面找到一个大概叫bypy.progress.json的文件，把它删除试试?
确实是进度文件的问题，在~/.bypy/文件夹下面有bypy.hashcache.json，bypy.parts.json应该是记录上传进度的文件，目前删除后可以上传。
我猜你是不是之前用了--processes这个参数导致进度文件损坏？刚刚改进了一下，json读取加了个try，出现异常程序不会退出，并且用multiprocess的情况下各个进程写入json文件会序列化。你升级到最新版可能就会避免出现这样的错误了。
出这个问题之前我没有试过--progress, 只用过bypy upload downfile之类的命令，后面出这个问题试了挺多各种命令的。 我一开始上传一个大文件叫a.tar.gz 3G上传成功以后，我把这个文件修改了一下变成了15G，上传，然后好像传到了3G他出的问题，是不是有可能是这个造成的呢？