PanelOLS : Dropping absorbed variables doesn't work with sample weights

[ignacio-rh]

Hi,

Really love the package, thanks for hosting it. Filling a very important hole in python's data analysis stack.

I encountered a strange bug, which you can see below. In a PanelOLS model, dropping absorbed variables doesn't work if the weights option is added. With no weights, dropping absorbed variables works as intended.

Best
Ignacio

from linearmodels.datasets import wage_panel

data = wage_panel.load()
year = pd.Categorical(data.year)
data = data.set_index(["nr", "year"])
data["year"] = year

#and random number between 0 and 1 
data['rand'] = np.random.rand(data.shape[0])
data['absorbe'] = data.groupby('nr')['union'].transform('mean')


##Works
PanelOLS.from_formula("lwage ~1+absorbe + married + EntityEffects", 
                data = data,
                drop_absorbed=True
               ).fit()


##Doesnt work
PanelOLS.from_formula("lwage ~1+absorbe + married + EntityEffects", 
                data = data,
                # check_rank=False, 
                weights=data['rand'] , 
                drop_absorbed=True
               ).fit()

[bashtage]

Thanks for the report. FIxed in main, and will be out in the next release. I think you can use the options use_lsdv to work around this for now.

[ignacio-rh]

Hey, thanks a lot for the fix!