Cannot create dir when uploading

Lines 190 to 194 in 1f14699

    
           err = os.MkdirAll(filepath.Dir(destPath), 0755) 
        
           if err != nil { 
        
           	api.SendTransferError(oid, 14, fmt.Sprintf("Cannot create dir %q: %v", filepath.Dir(destPath), err), writer, errWriter) 
        
           	return 
        
           }

When I try to push I get an error saying Cannot create dir "D:\LFSPath\b8\69\": mkdir D:\LFSPath\b8: Cannot create a file when that file already exists. I believe the error comes from the code above. As seen from the part of the git trace log below, 2 files with hashes starting with b8 begin to be processed simultaneously before any of them is completed. Therefore the first one is completed successfully while the second one is not. But when 2 files with hashes starting with the same two letters are tried to be uploaded after one of them is completed, there is no error. That is why I think it is because they begin to be processed simultaneously. But I try to push again the error disappears.

14:25:44.592584 trace git-lfs: xfer[lfs-folderstore]: Received upload request for b870643b7cd0586c672b60551ab009757df14cc90b155e2f777c66617c45a0a7
14:25:44.599585 trace git-lfs: xfer[lfs-folderstore]: Received upload request for b8699ae114979db009d1bae81fcfda8e9ebf8c5577a449cd17143c05d5c28a4d
14:25:44.604589 trace git-lfs: xfer[lfs-folderstore]: Sent message {"event":"complete","oid":"b8699ae114979db009d1bae81fcfda8e9ebf8c5577a449cd17143c05d5c28a4d","error":{"code":14,"message":"Cannot create dir \"D:\\\\LFSPath\\\\b8\\\\69\": mkdir D:\\LFSPath\\b8: Cannot create a file when that file already exists."}}
14:25:44.604589 trace git-lfs: xfer[lfs-folderstore]: Received upload request for ed2aa2b7ddc90258333d767499c2b02cbcdfcbaed3f1a3de3ef62a1c895f2db9
14:25:44.609586 trace git-lfs: xfer[lfs-folderstore]: Sent message {"event":"complete","oid":"5ba2abfcb8cceb9067b1f72eedf4b1f4f55e7dd58cd7190356e8499259a2bf46"}
14:25:44.610585 trace git-lfs: xfer[lfs-folderstore]: Received upload request for 031d367026ddf265c70764075ea403979e804e641a0746174a4bcbe65439a2b4
14:25:44.847643 trace git-lfs: xfer[lfs-folderstore]: Sent message {"event":"progress","oid":"e144b75053622a9a3f6ffb305e7bcb3fc9b9f2b4fdf74f24d74f246d1c61f769","bytesSoFar":618828,"bytesSinceLast":618828}
14:25:44.847643 trace git-lfs: xfer[lfs-folderstore]: Sent message {"event":"progress","oid":"8e67c9186c7a16d1e092eb5d5bb1dfe888acfcddb06c8b977a634cf8702fef41","bytesSoFar":589340,"bytesSinceLast":589340}
14:25:44.848645 trace git-lfs: xfer[lfs-folderstore]: Sent message {"event":"progress","oid":"05ff58f0ec43dacee595480219512ee73c3da664143cdc3239271c2cba8d3392","bytesSoFar":589000,"bytesSinceLast":589000}
14:25:44.848645 trace git-lfs: xfer[lfs-folderstore]: Sent message {"event":"progress","oid":"c8660108abd9f5b531021045f6db2cb2c51f76ecf2191493e6e56cfa970210f9","bytesSoFar":617555,"bytesSinceLast":617555}
14:25:44.849644 trace git-lfs: xfer[lfs-folderstore]: Sent message {"event":"progress","oid":"7161c96ca48a606ba9ec3aa0be272945e1f6cd3b3801a37fbcc8bb513b043260","bytesSoFar":587105,"bytesSinceLast":587105}
14:25:44.849644 trace git-lfs: xfer[lfs-folderstore]: Sent message {"event":"progress","oid":"b870643b7cd0586c672b60551ab009757df14cc90b155e2f777c66617c45a0a7","bytesSoFar":569586,"bytesSinceLast":569586}
14:25:44.853645 trace git-lfs: xfer[lfs-folderstore]: Sent message {"event":"progress","oid":"ed2aa2b7ddc90258333d767499c2b02cbcdfcbaed3f1a3de3ef62a1c895f2db9","bytesSoFar":535317,"bytesSinceLast":535317}
14:25:44.854645 trace git-lfs: xfer[lfs-folderstore]: Sent message {"event":"progress","oid":"031d367026ddf265c70764075ea403979e804e641a0746174a4bcbe65439a2b4","bytesSoFar":534677,"bytesSinceLast":534677}
14:25:44.872374 trace git-lfs: xfer[lfs-folderstore]: Sent message {"event":"complete","oid":"e144b75053622a9a3f6ffb305e7bcb3fc9b9f2b4fdf74f24d74f246d1c61f769"}
14:25:44.872374 trace git-lfs: xfer[lfs-folderstore]: Received upload request for ad82ea138ddd7d2f0c8f1aef711f22fdc31e778521c5a7ce73a0ceb19656482c
14:25:44.873892 trace git-lfs: xfer[lfs-folderstore]: Sent message {"event":"complete","oid":"8e67c9186c7a16d1e092eb5d5bb1dfe888acfcddb06c8b977a634cf8702fef41"}
14:25:44.873892 trace git-lfs: xfer[lfs-folderstore]: Received upload request for 4be5fc87ec70133d03f65d474eb9166bce340531448bbcf8fb2c7527bdaff2d3
14:25:44.875892 trace git-lfs: xfer[lfs-folderstore]: Sent message {"event":"complete","oid":"05ff58f0ec43dacee595480219512ee73c3da664143cdc3239271c2cba8d3392"}
14:25:44.875892 trace git-lfs: xfer[lfs-folderstore]: Received upload request for f223709caf2529c54e66fcea539a02ef8286cdf74f23babe80b562507066bc7d
14:25:44.877893 trace git-lfs: xfer[lfs-folderstore]: Sent message {"event":"complete","oid":"c8660108abd9f5b531021045f6db2cb2c51f76ecf2191493e6e56cfa970210f9"}
14:25:44.877893 trace git-lfs: xfer[lfs-folderstore]: Received upload request for b561b788c0b528e5985448db08c6cf1e7d13da66df2f6e909e3fc3b1e62f5860
14:25:44.879891 trace git-lfs: xfer[lfs-folderstore]: Sent message {"event":"complete","oid":"b870643b7cd0586c672b60551ab009757df14cc90b155e2f777c66617c45a0a7"}

Hmm, I guess there's a potential race condition here. Given that there's no particular need to parallelise folderstore operations in most cases, the easy workaround is to set this config in your repo

git config lfs.concurrenttransfers 1

	err = os.MkdirAll(filepath.Dir(destPath), 0755)
	if err != nil {
	api.SendTransferError(oid, 14, fmt.Sprintf("Cannot create dir %q: %v", filepath.Dir(destPath), err), writer, errWriter)
	return
	}