endless DataFrame warning at preprocessing.py
Closed this issue · 2 comments
I got this endless warning when running https://github.com/big-data-lab-umbc/multivariate-weather-data-clustering/blob/main/example/st_agglomerative.ipynb, which is quite annoying. We'd better find a way to fix it.
/Users/jianwu/Code/weather-data-clustering-3/multivariate-weather-data-clustering/mwdc/preprocessing/preprocessing.py:188: PerformanceWarning: DataFrame is highly fragmented. This is usually the result of calling frame.insert many times, which has poor performance. Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use newframe = frame.copy()
trans_data.loc[pd_df.time[i], c] = pd_df[j][i] # Based on the column name (var+ (lat,lon)), the correct value of each variable will sit in the right place.
Ok Will take a look at that over the weekend. Let's hope replacing the present code will solve the issue.
Resolved by adopting Omar's code.