iskandr/fancyimpute

AttributeError: 'KNN' object has no attribute 'complete'

hamsterLee opened this issue · 11 comments

THIS IS MY ERROR
dataset.dtypes
dataset.isnull().sum()
hour=364324
test_cl=dataset[0:hour]
train_cl=dataset[hour:]
train.isnull().sum()
test.isnull().sum()

Xcol =dataset.columns
Xcol=Xcol.drop('aqhi')
Ycol='aqhi'
X = train_cl.loc[:, Xcol]
Y = train_cl.loc[:, Ycol]
def standardize(s):
return s.sub(s.min()).div((s.max() - s.min()))

Xnorm = X.apply(standardize, axis=0)
kvals = np.linspace(1, 100, 20, dtype='int64')

knn_errs = []
for k in kvals:
knn_err = []
Xknn = KNN(k=k, verbose=False).complete(Xnorm)
knn_err = cross_val_score(rf, Xknn, Y, cv=24, n_jobs=-1).mean()

knn_errs.append(knn_err)
print("[KNN] Estimated RF Test Error (n = {}, k = {}, 10-fold CV): {}".format(len(Xknn), k, np.mean(knn_err)))

sns.set_style("darkgrid")
plt.plot(kvals, knn_errs)
plt.xlabel('K')
plt.ylabel('10-fold CV Error Rate')

knn_err = max(knn_errs)
k_opt = kvals[knn_errs.index(knn_err)]

Xknn = KNN(k=k_opt, verbose=False).complete(Xnorm)
Yknn = Y

print("[BEST KNN] Estimated RF Test Error (n = {}, k = {}, 10-fold CV): {}".format(len(Xknn), k_opt, np.mean(knn_err)))
Traceback (most recent call last):

File "", line 39, in
Xknn = KNN(k=k, verbose=False).complete(Xnorm)

AttributeError: 'KNN' object has no attribute 'complete'

...:
X_train = train_cl.loc[:, Xcol]
Y_train = train_cl.loc[:, Ycol]
X_test = test_cl.loc[:, Xcol]
Y_test = test_cl.loc[:, Ycol]

X_filled_ii = IterativeImputer().fit_transform(X_train)

X_filled_knn = KNN(k=16).fit_transform(X_train)

X_filled_nnm = NuclearNormMinimization().fit_transform(X_train)

X_incomplete_normalized = BiScaler().fit_transform(X_train)
X_filled_softimpute = SoftImpute().fit_transform(X_incomplete_normalized)

ii_mse = ((X_filled_ii[missing_mask] - X[missing_mask]) ** 2).mean()
print("Iterative Imputer norm minimization MSE: %f" % ii_mse)

nnm_mse = ((X_filled_nnm[missing_mask] - X[missing_mask]) ** 2).mean()
print("Nuclear norm minimization MSE: %f" % nnm_mse)

softImpute_mse = ((X_filled_softimpute[missing_mask] - X[missing_mask]) ** 2).mean()
print("SoftImpute MSE: %f" % softImpute_mse)

knn_mse = ((X_filled_knn[missing_mask] - X[missing_mask]) ** 2).mean()
print("knnImpute MSE: %f" % knn_mse)
fancyimpute\solver.py:58: UserWarning: Input matrix is not missing any values
warnings.warn("Input matrix is not missing any values")
Traceback (most recent call last):

File "", line 34, in
X_filled_knn = KNN(k=16).fit_transform(X_train)

File "fancyimpute\solver.py", line 189, in fit_transform
X_result = self.solve(X_filled, missing_mask)

File "knn.py", line 103, in solve
print_interval=self.print_interval)

File "few_observed_entries.py", line 51, in knn_impute_few_observed
knn_initialize(X, missing_mask, verbose=verbose)

File "knnimpute\common.py", line 39, in knn_initialize
D = all_pairs_normalized_distances(X_row_major)

File "normalized_distance.py", line 38, in all_pairs_normalized_distances
D = np.ones((n_rows, n_rows), dtype="float32", order="C") * np.inf

File "core\numeric.py", line 203, in ones
a = empty(shape, dtype, order)

MemoryError

How big is your data? Looks like it is too big for knn imputation.

If you have a train/test type of setup, I would just use IterativeImputer. It's the only one that can work transductively.

I don't know what your error means. Which line is it happening in?

I have other problem : Why i from fancyimpute import MICE, it said
ImportError: cannot import name 'MICE'.
Could i solve this problem?

In one of my python imputation problem i faced the same error AttributeError: 'KNN' object has no attribute 'complete'
replacing ".complete" with ".fit_transform" resolved issue

Yes, we have stopped using .complete and now use .fit_transform everywhere to match sklearn api.