py-econometrics/pyfixest

Feols: speed up the creation of interacted fixed effects via `fe1^fe2` syntax

s3alfisc opened this issue · 6 comments

s3alfisc commented

Context

Currently, the creation of interacted fixed effects is extremely slow for large data sets.

    if "^" in fval:
        for val in fval.split("+"):
            if "^" in val:
                vars = val.split("^")
                data[val.replace("^", "_")] = data[vars].apply(
                    lambda x: (
                        "^".join(x.dropna().astype(str)) if x.notna().all() else np.nan
                    ),
                    axis=1,
                )

Likely, there is a way to speed this up =)

To Do

Re-implement _fixef_interactions and speed it up.

s3alfisc commented

Sorry, I had not activated the app for the py-econometrics project. One more attempt:

allcontributors commented

@s3alfisc

I've put up a pull request to add @app! 🎉

I've put up a pull request to add @leostimpfle! 🎉

allcontributors commented

@s3alfisc

@leostimpfle already contributed before to code