concurrent.futures.process.BrokenProcessPool('A process in the process pool was terminated abruptly while the future was running or pending.')

[EnzoBustos]

• Parallel TQDM version: latest
• Python version: latest
• Operating System: Windows

Description

Describe what you were trying to get done.
Tell us what happened, what went wrong, and what you expected to happen.

What I Did

from textblob import TextBlob
from pqdm.processes import pqdm
import multiprocessing


def translate_text(text):
    try:
        return str(TextBlob(text).translate(from_lang="pt", to="en"))
    except:
        return np.nan

result = pqdm(df["texts"].tolist(), translate_text, n_jobs=multiprocessing.cpu_count())
df["texts_en"] = result
df.head()

I´m trying to run the following code, it runs properly in google collab, but when I try to run on my local machine all I get from output is this error "concurrent.futures.process.BrokenProcessPool('A process in the process pool was terminated abruptly while the future was running or pending.')", do you have anny idea how to solve it?

[hitokun-s]

@EnzoBustos
Hi, maybe too late for you, but I left a note for others.
I assume you're Windows user and executing the code as python script locally.
If so, please try like this (with if __name__ == “__main__” statement )

from multiprocessing import freeze_support

if __name__ == '__main__':
    freeze_support() # unnecessary?
    # prepare "df" DataFrame variable in advance
    result = pqdm(df["texts"].tolist(), translate_text, n_jobs=multiprocessing.cpu_count())
    df["texts_en"] = result

ref: https://stackoverflow.com/questions/24374288/where-to-put-freeze-support-in-a-python-script