can not parallel when there is "while True" in the routine

[kernel8liang]

I have a data_process function, it will continue running to process a data stream.
And there are a bundle of streams, so I want use goless.go to process them in parallel .
But, I found if there is "While True" in the routine, only the first routine is started.

Is the a way to start all of them ?

[rgalanakis]

Can you post more of your code example? Remember goless depends on cooperate scheduling on a single thread, so a while True loop will not work, if the body of the loop never yeids to other tasks (it'll loop forever). You need to do something like backend.yield() to do this (which calls stackless.schedule(), or gevent.sleep(). Note that since time.sleep is blocking, it won't work (you can think of it like a while True loop). You need to use yield, or some sort of green sleep, like gevent.sleep if you are using it.

[kernel8liang]

def  process_data (...):
       while True:
                #processing
                #time.sleep(),  here may be change to some signal
      
for symbol in symbols:
     goless.go(p.process_data, pre_path, symbol)

I will try gevent.sleep().

[kernel8liang]

I tried gevent.sleep(). it still can not run all process_data in parallel, it seems run process_data one by one. only 1 cpu core is working.

[rgalanakis]

Ok, I misunderstood the problem. I thought you meant that one goroutine is never allowing the other ones to run- this doesn't seem to be that, you're just saying that one thread/goroutine is only ever running at a time, rather than in parallel/at the same time.

Python is single threaded, so it will only ever use 1 core at a time. Goless does not change this fact. If you need multiple cores running python, try something like the multiprocessing module. You could also spawn new threads yourself, but that doesn't remove the limitations of the GIL (only one thread will ever actually be doing anything at a time- this will end up being slower than a single thread in most cases). Please tell me if, after researching this some more yourself, that answer is clear, or if I am still misunderstanding your problem.

[kernel8liang]

got it.