alteryx/predict-loan-repayment

Abnormally long run time?

xiaodaigh opened this issue · 2 comments

Bug/Feature Request Title

Abnormally long expected run time. The estimate keep climbing! And progress remains 0.

I ran on Windows 10 and featuretools 0.10.1

I was just following along this tutorial https://github.com/Featuretools/predict-loan-repayment/blob/master/Automated%20Loan%20Repayment.ipynb

except I couldn't get this line to run

#default_trans_primitives =  ["day", "year", "month", "weekday", "haversine", "numwords", "characters"]

so I dropped two primitives

#default_trans_primitives =  ["day", "year", "month", "weekday", "haversine"]

Bug/Feature Request Description

I was just following along this tutorial https://github.com/Featuretools/predict-loan-repayment/blob/master/Automated%20Loan%20Repayment.ipynb

Expected Output

Within the hour

Output of featuretools.show_info()

Elapsed: 19:46 | Progress: 0%| | Remaining: 899:20:03

FWIW, this is what I see just before the Elapsed: ...

uilt 2188 features
Total size of entityset: 11.28767 gb.
Total number of cpus detected: 12.
Total size of system memory: 68.67001 gb.
Built 2188 features
Elapsed: 02:33 | Progress:   0%|          

This is a large dataset where you are calculating many features. The best approach for speeding it up would be to follow the instructions in this article about using Dask + Featuretools. We have an example notebook for using dask with this exact dataset here.