alonho/pystuck

Fiddly to use in a multiprocessing context

Closed this issue · 3 comments

Basically, if you have a multithreaded application (using real threads, e.g. the multiprocessing module), they all try to open the same port, and you get crashes.

Basically, it seems pystuck assumes it's installed into only one process. This is /probably/ not really something that can be fixed easily, given the current system architecture, but it's a good thing to note clearly in the documentation, if anything.

A not-too-elegant-but-functional solution is to fiddle with the port:


def install_pystuck():
    import pystuck
    stuck_port = 6666
    while 1:
        try:
            pystuck.run_server(port=stuck_port)
            print("PyStuck installed to process, running on port %s" % stuck_port)
            break
        except OSError:
            stuck_port += 1

You can't dump the stack from each process, but it at least doesn't crash my app anymore.

That's a general issue in multithreaded programs: using global resources like sockets or files should be handles carefully. That's why the general guideline is create them in the main thread before it spawns any other thread. (usually I put run_server at the first line of the main() of my programs!)

Also, you mentioned multiprocessing which supports both threads and processes. Forking into other processes would make it impossible to debug all children with a single server. It's untested and the results are undefined.. In that case perhaps you should explicitly pass a port to each child process.

s/multithreaded/multiprocessing/g

I've been doing too much c++. Bleh.