ValueError: No objects to concatenate error using pdp.pdp_isolate
chriseal opened this issue · 4 comments
chriseal commented
I've got pdp_isolate to work on other features, but it throws an exception "ValueError: No objects to concatenate" when plotted for column "f66". Does this mean there isn't enough information to make a plot of this feature?
Code that draws the error is:
pdp_feature = pdp.pdp_isolate(
model=model,
dataset=generic_features,
model_features=generic_features.columns.tolist(),
feature="f66",
num_grid_points=10, grid_type='percentile', percentile_range=None, grid_range=None, cust_grid_points=None, memory_limit=0.5, n_jobs=1, predict_kwds={}, data_transformer=None
)
Error printout is:
ValueError Traceback (most recent call last)
<ipython-input-48-65f3cae83da8> in <module>
7 model_features=self._xgb_col_names,
8 feature=''.join(['f', str(self._featureName_to_featureIdx_map[feature])]),
----> 9 num_grid_points=10, grid_type='percentile', percentile_range=None, grid_range=None, cust_grid_points=None, memory_limit=0.5, n_jobs=1, predict_kwds={}, data_transformer=None
10 )
~\AppData\Local\Continuum\anaconda3\lib\site-packages\pdpbox\pdp.py in pdp_isolate(model, dataset, model_features, feature, num_grid_points, grid_type, percentile_range, grid_range, cust_grid_points, memory_limit, n_jobs, predict_kwds, data_transformer)
165 ice_lines.append(ice_line_n_class)
166 else:
--> 167 ice_lines = pd.concat(grid_results, axis=1)
168
169 # calculate the counts
~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\reshape\concat.py in concat(objs, axis, join, join_axes, ignore_index, keys, levels, names, verify_integrity, sort, copy)
226 keys=keys, levels=levels, names=names,
227 verify_integrity=verify_integrity,
--> 228 copy=copy, sort=sort)
229 return op.get_result()
230
~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\reshape\concat.py in __init__(self, objs, axis, join, join_axes, keys, levels, names, ignore_index, verify_integrity, copy, sort)
260
261 if len(objs) == 0:
--> 262 raise ValueError('No objects to concatenate')
263
264 if keys is None:
ValueError: No objects to concatenate
The array of data is:
array([ 1.42646187e-04, 2.20339505e-03, -3.71780779e-02, -3.07990126e-02,
-1.16102087e-03, -1.56650202e-02, -2.06472276e-03, -2.08325083e-02,
1.91286310e-02, 8.36141875e-03, -8.11077609e-04, -1.92386611e-02,
2.00920603e-02, -3.85844310e-02, -3.05273896e-03, 1.50174930e-03,
4.46882570e-03, -2.53156515e-02, 7.88625472e-03, -4.30667359e-03,
-9.39565665e-03, 8.74410281e-04, 5.34033060e-02, -8.75319328e-03,
-9.87543920e-03, -5.46208786e-03, -6.50628504e-03, -7.57054927e-03,
-3.93052166e-03, 2.16708143e-03, 1.42646187e-04, -7.25448253e-03,
-1.06866675e-02, 3.59743997e-02, -3.22111433e-03, -7.26964438e-03,
2.44697544e-02, 1.66259945e-02, -3.95567627e-03, 1.93527167e-02,
1.64243560e-03, 2.66140257e-02, 3.36265867e-02, -1.45173875e-03,
-6.50628504e-03, 2.01777145e-02, 5.00185982e-03, -2.68591401e-02,
5.35982088e-03, 7.15823115e-02, -4.28643707e-03, 1.62057894e-02,
-6.83444130e-03, 5.02746656e-02, 2.21616214e-03, -9.68313760e-03,
-3.70581025e-03, 8.12628710e-03, -3.70581025e-03, -8.75319328e-03,
9.87255512e-03, 6.73500014e-03, 5.45383777e-02, -3.86630195e-03,
2.85044068e-03, 3.06522130e-02, 3.29460041e-03, 9.06824300e-03,
-3.16553335e-03, 6.66199922e-03, 5.24429083e-04, -3.71585490e-03,
6.27464537e-03, -8.96973965e-03, 8.17747652e-03, -5.51070443e-04,
5.00562964e-03, -2.66748822e-02, 1.38245766e-02, 6.50494563e-04,
-9.44020712e-03, 1.96937834e-02, -2.77638032e-03, -1.82182974e-04,
7.87300556e-03, 4.61156800e-03, -4.69126662e-02, 1.10852780e-02,
9.89909316e-04, 3.60295491e-04, 1.57297735e-02, 4.62766075e-02,
1.33140739e-02, 6.19039552e-03, 1.39541582e-02, -1.69532139e-03,
-1.16102087e-03, 3.80017680e-03, -6.50628504e-03, 6.10508974e-03,
1.15785507e-03, 1.68395233e-02, 3.28879539e-03, 1.30998048e-02,
-7.75223512e-03, 1.76696777e-02, -2.06107510e-03, 2.55053545e-02,
-4.71198937e-02, 6.10950021e-02, 4.04423462e-05, -3.27229081e-02,
-1.28115640e-03, -8.02660978e-02, 4.14655803e-02, -2.31697945e-02,
3.25156125e-02, 3.42166041e-02, -8.96781154e-02, 1.60982957e-03,
4.81951811e-02, 1.33224275e-03, 1.14265095e-02, -2.24592418e-02,
-2.36253054e-03, 2.93535839e-02, 1.33670513e-02, 6.95616360e-03,
-4.05554484e-02, -1.17983934e-02, -4.75658884e-03, -1.99465422e-02,
1.67505552e-02, -3.21242621e-03, 4.16127660e-03, -3.64498680e-02,
-2.67821052e-03, -1.50303162e-02, 1.17175118e-02, 1.08760602e-02,
-5.16998437e-03, -2.69782135e-04, -2.70069166e-03, -4.10286181e-02,
1.24384512e-03, 1.07679725e-03, -2.89405080e-02, -1.72169721e-03,
6.27027032e-02, -5.21718738e-04, 3.55731258e-02, -2.26476632e-02,
1.39934603e-02, 1.19824431e-02, -1.38995866e-02, 4.92600824e-03,
-7.93047226e-03, 4.36035646e-02, -1.69812569e-04, -6.50628504e-03,
-7.92308831e-02, 1.37241203e-02, 1.26096808e-02, 1.72127291e-03,
-2.10604483e-02, -3.86630195e-03, 1.76504359e-02, -6.50628504e-03,
-4.36824084e-03, 3.47388067e-03, 2.25694330e-03, -1.09188288e-02,
-2.17498918e-02, -2.32474048e-02, 2.99691884e-02, -2.52163580e-02,
1.43217263e-02, -8.43296067e-04, 2.70521511e-02, -4.18762082e-02,
-1.55742569e-02, 9.67830047e-02, 3.12716409e-02, 1.42646187e-04,
3.46233434e-02, -4.70378050e-03, -1.00597171e-02, -3.10874944e-03,
8.58906318e-04, -5.80679408e-04, -1.04566714e-02, 7.34680904e-03,
1.05924652e-02, -2.06613277e-02, -3.16553335e-03, 1.77879256e-03,
-4.37393282e-02, -5.83798230e-03, -6.18573615e-03, -1.14453858e-02,
-7.59866074e-03, -2.21871359e-02, -3.16553335e-03, -1.00102993e-04,
1.18235288e-03, 8.06930693e-03, -1.96816126e-04, 1.84993239e-02,
2.13461349e-02, 7.07266741e-03, 1.42646187e-04, -5.47744218e-03,
-1.03254946e-02, -1.73400374e-02, -2.58117361e-04, 1.24010993e-02,
3.92948733e-03, -1.82873090e-02, 1.75378801e-04, -2.85078366e-03,
5.82887766e-03, -9.36706224e-03, 2.93535839e-02, 1.69861258e-02,
1.29833871e-02, -2.34977297e-03, 4.12969746e-04, -4.58429383e-03,
-1.97912848e-02, -1.71597336e-02, 1.00754861e-02, 1.46873022e-02,
1.29070842e-02, -8.75319328e-03, -2.44563897e-02, -2.40753008e-02,
-2.29969049e-03, 5.32787323e-03, -9.60620144e-04, 5.82650104e-03,
1.23422518e-03, 1.23086831e-02, -4.98560347e-03, -2.14837586e-03,
-2.19718025e-03, 3.04158477e-03, 1.42646187e-04, -6.03845600e-03,
2.68922183e-03, 4.90456739e-03, -2.64048818e-02, -4.25583690e-03,
1.13904307e-03, 1.82139863e-02, 5.86379367e-03, 3.09502502e-02,
1.04408664e-02, 3.30992549e-02, 3.26630269e-03, -8.84241306e-03,
2.53524525e-03, -1.93375418e-02, -1.65741420e-03, 3.01436365e-02,
-3.25712075e-04, -3.89183492e-02, -2.70689577e-03, 1.42646187e-04,
-4.11961023e-02, -1.19694286e-02, 3.78084147e-03, 5.03621425e-03,
-1.31770086e-03, 1.73333962e-03, 1.83711198e-02, 4.24728106e-03,
4.81215520e-03, -2.05966443e-03, 1.55374284e-02, 2.09652199e-04,
-5.55169662e-02, 1.13634914e-02, 2.09302581e-02, 1.42646187e-04,
1.55923464e-02, 5.68731417e-03, -3.66920155e-03, 3.35823081e-02,
2.59721886e-02, 1.55376680e-02, 1.98637057e-03, -5.16036812e-02,
2.23336166e-02, 8.63353952e-03, 7.94881900e-03, -6.61724505e-04,
1.48939556e-02, 1.56944799e-02, -1.52858182e-02, 1.28012873e-03,
-1.90668483e-02, -6.50628504e-03, 7.59052706e-03, -3.16190751e-02,
-2.24662311e-02, 1.34058745e-03, -2.34977297e-03, 6.91485104e-03,
2.07674911e-02, -3.43720015e-03, 5.12697636e-03, 5.68418815e-03,
-1.98302377e-02, 3.17619165e-03, 2.87206334e-02, -4.16068265e-04,
-1.70174842e-02, -4.25886490e-03, -1.90537711e-02, 1.40145629e-02,
1.91054495e-03, -1.76012040e-02, 5.62004485e-02, -7.56296167e-03,
1.93897412e-02, -1.08269795e-03, -4.12494685e-02, -8.52674004e-02,
2.93535839e-02, -2.34977297e-03, -4.35269843e-02, 8.49059462e-04,
-3.10987210e-02, -2.11906108e-03, 2.75347206e-02, -1.16102087e-03,
-2.14935108e-04, -9.37163282e-04, 2.08917108e-02, -1.49624887e-04,
-1.20360853e-03, -1.84003701e-02, 5.69942226e-02, -6.51881530e-03,
1.19581645e-02, 9.34846000e-03, 1.95967201e-02, 1.83098260e-02,
1.50381701e-02, -1.01716106e-02, 1.19326200e-02, -8.45938115e-05,
-5.39846427e-02, 4.12628460e-02, 5.94171647e-02, -3.16112209e-02,
-1.04566714e-02, 2.13981361e-03, 2.17117295e-03, 5.24219656e-03,
-7.49931840e-04, -7.69360893e-02, 1.04539920e-03, -1.25689487e-02,
-1.11945172e-02, -1.06232041e-02, 7.95284125e-03, -1.19005699e-03,
-7.01669455e-03, 1.35516141e-03, -1.18424677e-02, 7.34416499e-03,
1.82665318e-02, -6.50628504e-03, -1.66917762e-04, 7.93482177e-03,
7.15381390e-02, 7.11127527e-05, -1.48615269e-03, 2.64163194e-02,
-4.44932982e-02, -2.34977297e-03, 4.55293195e-02, -8.35069547e-03,
3.70642948e-03, 1.53445914e-02, 5.59201845e-03, -4.88137601e-02,
1.52571208e-03, 2.52508119e-03, -1.16102087e-03, 1.21301764e-02,
7.62865196e-03, -9.38930734e-03, -1.87311721e-03, 1.73434631e-02,
-9.26327361e-03, -7.73302270e-03, -8.66310438e-03, 1.59627685e-02,
2.15100444e-02, -2.01720393e-02, -6.26028935e-03, -1.04566714e-02,
1.94997264e-02, 7.40022891e-03, 6.63133282e-02, -2.86199529e-02,
1.88261116e-03, 1.98344906e-03, 2.16258155e-03, 1.30730564e-02,
-1.16102087e-03, -2.62273463e-02, -6.53665300e-03, -2.45184235e-02,
-4.12609788e-02, -2.54090496e-03, -2.54805998e-02, 6.54251064e-03,
-6.50628504e-03, 1.90305898e-02, 3.77064587e-02, 7.18832359e-03,
-3.14250026e-04, -3.75230486e-02, -3.92870243e-03, 1.00036256e-02,
1.99581498e-02, -1.45951448e-02, 3.94904799e-04, -1.47333697e-02,
-1.12384254e-02, -2.02326112e-02, 1.89755598e-02, -5.89852839e-03,
7.44942716e-03, -1.07168755e-02, -1.68000304e-02, 1.22239083e-03])
imranahmed96 commented
Just had this bug, but was able to resolve by removing nan values.
I can't see any nans in the array you posted above, but maybe try a quick variable clean?
chriseal commented
I've checked the data, and as far as I can tell, as long as the feature doesn't include any null values in the array, it is not drawing this error. Since I used XGBoost, I had to mask the feature set for non-null values in the column of choice.
So it seems, I was unable to recreate the error. Though, I'm not entirely sure which dataset I was using when I posted this, so can't entirely attribute it to a variable clean. I'll repost if I find it again.
mattharrison commented
Ran into this issue as well. Any plans for supporting NaN? (since xgboost and other boosting libraries do)
wuyuyu1024 commented
My data has no NaN but still runs to this issue...btw I'm using XGBoost