AttributeError: 'KNN' object has no attribute 'complete'
hamsterLee opened this issue · 11 comments
THIS IS MY ERROR
dataset.dtypes
dataset.isnull().sum()
hour=364324
test_cl=dataset[0:hour]
train_cl=dataset[hour:]
train.isnull().sum()
test.isnull().sum()
Xcol =dataset.columns
Xcol=Xcol.drop('aqhi')
Ycol='aqhi'
X = train_cl.loc[:, Xcol]
Y = train_cl.loc[:, Ycol]
def standardize(s):
return s.sub(s.min()).div((s.max() - s.min()))
Xnorm = X.apply(standardize, axis=0)
kvals = np.linspace(1, 100, 20, dtype='int64')
knn_errs = []
for k in kvals:
knn_err = []
Xknn = KNN(k=k, verbose=False).complete(Xnorm)
knn_err = cross_val_score(rf, Xknn, Y, cv=24, n_jobs=-1).mean()
knn_errs.append(knn_err)
print("[KNN] Estimated RF Test Error (n = {}, k = {}, 10-fold CV): {}".format(len(Xknn), k, np.mean(knn_err)))
sns.set_style("darkgrid")
plt.plot(kvals, knn_errs)
plt.xlabel('K')
plt.ylabel('10-fold CV Error Rate')
knn_err = max(knn_errs)
k_opt = kvals[knn_errs.index(knn_err)]
Xknn = KNN(k=k_opt, verbose=False).complete(Xnorm)
Yknn = Y
print("[BEST KNN] Estimated RF Test Error (n = {}, k = {}, 10-fold CV): {}".format(len(Xknn), k_opt, np.mean(knn_err)))
Traceback (most recent call last):
File "", line 39, in
Xknn = KNN(k=k, verbose=False).complete(Xnorm)
AttributeError: 'KNN' object has no attribute 'complete'
...:
X_train = train_cl.loc[:, Xcol]
Y_train = train_cl.loc[:, Ycol]
X_test = test_cl.loc[:, Xcol]
Y_test = test_cl.loc[:, Ycol]
X_filled_ii = IterativeImputer().fit_transform(X_train)
X_filled_knn = KNN(k=16).fit_transform(X_train)
X_filled_nnm = NuclearNormMinimization().fit_transform(X_train)
X_incomplete_normalized = BiScaler().fit_transform(X_train)
X_filled_softimpute = SoftImpute().fit_transform(X_incomplete_normalized)
ii_mse = ((X_filled_ii[missing_mask] - X[missing_mask]) ** 2).mean()
print("Iterative Imputer norm minimization MSE: %f" % ii_mse)
nnm_mse = ((X_filled_nnm[missing_mask] - X[missing_mask]) ** 2).mean()
print("Nuclear norm minimization MSE: %f" % nnm_mse)
softImpute_mse = ((X_filled_softimpute[missing_mask] - X[missing_mask]) ** 2).mean()
print("SoftImpute MSE: %f" % softImpute_mse)
knn_mse = ((X_filled_knn[missing_mask] - X[missing_mask]) ** 2).mean()
print("knnImpute MSE: %f" % knn_mse)
fancyimpute\solver.py:58: UserWarning: Input matrix is not missing any values
warnings.warn("Input matrix is not missing any values")
Traceback (most recent call last):
File "", line 34, in
X_filled_knn = KNN(k=16).fit_transform(X_train)
File "fancyimpute\solver.py", line 189, in fit_transform
X_result = self.solve(X_filled, missing_mask)
File "knn.py", line 103, in solve
print_interval=self.print_interval)
File "few_observed_entries.py", line 51, in knn_impute_few_observed
knn_initialize(X, missing_mask, verbose=verbose)
File "knnimpute\common.py", line 39, in knn_initialize
D = all_pairs_normalized_distances(X_row_major)
File "normalized_distance.py", line 38, in all_pairs_normalized_distances
D = np.ones((n_rows, n_rows), dtype="float32", order="C") * np.inf
File "core\numeric.py", line 203, in ones
a = empty(shape, dtype, order)
MemoryError
How big is your data? Looks like it is too big for knn imputation.
If you have a train/test type of setup, I would just use IterativeImputer. It's the only one that can work transductively.
I don't know what your error means. Which line is it happening in?
I have other problem : Why i from fancyimpute import MICE, it said
ImportError: cannot import name 'MICE'.
Could i solve this problem?
In one of my python imputation problem i faced the same error AttributeError: 'KNN' object has no attribute 'complete'
replacing ".complete" with ".fit_transform" resolved issue
Yes, we have stopped using .complete and now use .fit_transform everywhere to match sklearn api.