parallel is still running even after each model fits. 60GB ram, cpu-per-task ==60
Opened this issue · 0 comments
kalkite commented
Hello, it's wrong issue. i don't know how to contact you . but i created issue to contact for my problem.
def dt_hyper_parameter_model(df,feature_number,data_samples):
class_name = df.iloc[:, -1].name
#print(class_name)
le = LabelEncoder()
df[class_name] = le.fit_transform(df.iloc[:, -1])
data = df.groupby(class_name).apply(lambda x: x.sample(n=data_samples)).reset_index(drop=True)
X_train, X_test, y_train, y_test = train_test_split(data.iloc[:, 0:feature_number], data.iloc[:, -1], test_size=0.2,random_state=42)
params = {
"criterion": ("gini", "entropy"),
"splitter": ("best", "random"),
"max_depth": (list(range(1, 50))),
"min_samples_split": [2,3,4,5,6,7,8,9,10],
'max_features': ['auto', 'sqrt', 'log2'],
"min_samples_leaf": list(range(1, 50)),
}
tree_clf = DecisionTreeClassifier(random_state=42)
tree_cv = GridSearchCV(tree_clf, params, scoring="accuracy",verbose=1, cv=2,n_jobs=-1)
tree_cv.fit(X_train, y_train)
best_params = tree_cv.best_params_
dt_model = DecisionTreeClassifier(**best_params)
return dt_model.fit(X_train, y_train)
result = Parallel(n_jobs=9 ,backend="threading")(delayed(dt_hyper_parameter_model)(class3_df[i], 2,5 ) for i in range(len(class3_df)))
I trying to fit the model for model for the each dataframe of class3df . but result variable still continouly running even after fitting the model
could you please tell me the what the problem is it. I am sorry for posting the issue here.