rakyll/drive

Push hangs, error on pull

Opened this issue · 8 comments

ar4 commented

Drive hangs when I try to push a file, using about 100% of one CPU core, and when I run drive pull I get the following output:

Resolving...
runtime/cgo: pthread_create failed: Resource temporarily unavailable
SIGABRT: abort
PC=0xb773ac7c

goroutine 1 [semacquire]:
sync.runtime_Semacquire(0x18982e48)
/usr/lib/go/src/pkg/runtime/sema.goc:199 +0x34
sync.(_WaitGroup).Wait(0x18972f20)
/usr/lib/go/src/pkg/sync/waitgroup.go:127 +0x12a
github.com/rakyll/drive.(_Commands).resolveChangeListRecv(0x1880a470, 0x0, 0x188005e0, 0x1, 0x189adc00, ...)
/home/alanr/gocode/src/github.com/rakyll/drive/changes.go:86 +0x3d6
github.com/rakyll/drive.(_Commands).Pull(0x1880a470, 0x0, 0x0)
/home/alanr/gocode/src/github.com/rakyll/drive/pull.go:45 +0x193
main.(_pullCmd).Run(0x188004e8, 0x1880a010, 0x0, 0x0)
/home/alanr/gocode/src/github.com/rakyll/drive/cmd/drive/main.go:75 +0xb3
github.com/rakyll/command.Run()
/home/alanr/gocode/src/github.com/rakyll/command/command.go:155 +0x70
github.com/rakyll/command.ParseAndRun()
/home/alanr/gocode/src/github.com/rakyll/command/command.go:162 +0x23
main.main()
/home/alanr/gocode/src/github.com/rakyll/drive/cmd/drive/main.go:45 +0x68d

...

I am using a clean install of Lubuntu 14.10.

I am having issues with drive hanging when I push and hogging a core. The file I am pushing is almost 2GB. How big is the file you are pushing?

ar4 commented

The issue occurs for me even when I push a file that is only a few bytes.

see http://stackoverflow.com/a/12844718/558774 if it clarify your situation. "you are probably hitting address space limits", means system can't create more threads because of shortage of RAM. you should post some detailed report of resources available on your machine at the time you run drive. also, you can try to sync only one file at time from that machine.

ar4 commented

It's a 32-bit system with 1GB RAM, of which about 800MB was free at the
time of launching drive. It has no problem running Firefox, so it should
presumably be possible for it to run drive.

I've hitting the same issue (I've had it from diff, pull and push). I have 8G of RAM and the machine is doing very little else. From some other searching, it may be that there are too many threads, caused by too many goroutines. The stack trace I got has 606 goroutines in it. Also, sometimes the the commands work just fine. So it seems like some kind of random element is involved. Perhaps if the network is fast the goroutines complete quickly and don't accumulate but if it's slow, the backlog builds up and eventually exceeds a thread-limit.

(I'm on Ubuntu 14.04)

pmig commented

@fergald I recommend you using this branch: https://github.com/odeke-em/drive. Emmanuel did serval significant improvements.

I already am :(.

Should I reopen this bug on the other project?

Ah, already there.

odeke-em#22