big-data-lab-umbc/multivariate-weather-data-clustering

Time complexity resolving in Evaluation functions

Closed this issue · 3 comments

As we all (@OmarFaruqueUMBC @MostafaCham @Frankie0609) had discussed a issue today in meeting , the execution of the functions (st_rmse and st_correlation) is taking a very long time to execute. Coming to the requirements we need to able to input our Raw (.nc data )data as an input in these functions. In the current function we are directly processing the raw data for transformation which is taking very long time when the user already has a transformed data.
image

What if we add a hyper parameter which will ask the user if he needs to do a transformation or not ( basically putting it through if else case). I have created a similar thing the compute_silhoutte_score function .
something like this
image
If possible Can professor @jianwuwang and Dr. @xuezhengllnl please state your suggestions on this one. Also it will be good to understand what input would go smoother as a user?

add a new argument preprocessed = false. If it is true, we skip transformation().

Ok will do that and the get the updated function running by Monday.

Resolved by both adding the option for the user to use transformed data as well as proceeding with Omar's preprocessing code.