How to use Multiprocessing

[latot]

Hi!, IIRC this was working before, but now seems to now works:

import alive_progress
from multiprocessing import Pool

with alive_progress.alive_bar(10) as bar:
  def f(block):
    time.sleep(block)
    print(block)
    bar()
  with Pool(13) as p:
     ret = p.map(f, list(range(10)))
on 0: 0
on 0: 1
on 0: 2
on 0: 3
on 0: 4
on 0: 5
on 0: 6
on 0: 7
on 0: 8
on 0: 9
|⚠︎                                       | (!) 0/10 [0%] in 9.0s (0.00/s) 

Seems to not works...

Thx!

[takanuva15]

Hey I was actually dealing with a similar issue just today lol! When I tried calling bar() from a multithreaded context, the bar slowed down significantly and the bar UI got very laggy. (However, it did update the batch number correctly?)
Here's some sample code:

from concurrent.futures import ThreadPoolExecutor
from alive_progress import alive_bar

with alive_bar(title="Processing batches...", unit="batches") as bar:
	def callback(future):
		bar()

	with ThreadPoolExecutor(max_workers=2, thread_name_prefix="worker") as executor:
		for _ in range(10):
			f = executor.submit(time.sleep, 2)
			f.add_done_callback(callback)

[rsalmei]

Hi @latot ,

You are not using it correctly, it is impossible to call bar() from another process. Other processes do not receive the alive_bar instance from the main one, so Python on other processes calls some bogus object. It should throw an error in my opinion, but unfortunately, it silently ignores it. The correct way is to call bar() always from the main process.

I'm not an expert in multiprocessing, but it should be something like this:

from multiprocessing import Pool

def f(block):
    time.sleep(block)

with alive_bar(10) as bar, Pool(13) as p:
    for _ in p.imap_unordered(f, range(10)):
        bar()

[rsalmei]

Regarding you @takanuva15, you use multithreading, not multiprocessing, so it isn't related at all.
It slowed down by design, you are making 0.88batches/s, which is VERY VERY slow in absolute terms, the bar will render with the slowest allowed refresh rate, which is 4 frames per second, or it would be much more sluggish.
You need to calibrate it with your expected maximum throughput.