Add "lean" argument to feols() and fepois()

[mcsqr]

(unless I somehow missed this) fixest has a lean parameter to strip the result of large components eating a lot of memory. It's really useful for large computations, where using pyfixest results in oom errors.

[s3alfisc]

Good that you mention this, it is bothering me as well 😀 note that you can already economize a bit by setting 'store_date=False' and 'copy_data=False', though this still saves a range of large-memory attributes.

I suppose an argument 'lean=True' should also drop the Y and X attributes and garbage collection memory.

Should be an easy enough addition =)

[s3alfisc]

Context

Quite a lot of large objects are stored in Feols / Feiv / Fepois objects. To avoid out-of-memory errors when working with big data sets, we want to add a function argument lean to all three classes mentioned above and the feols() and fepois() APIs.

Task

Add the end of the run_all_models method of the FixestMulti class, if lean = True, set the following attributes to None:

• self._X
• self._Y
• self._Z
• self._cluster_df
• self._data

In code, do something as

import gc
if lean: 
   del self._X
   del self._Y
   del self._Z
   del self._cluster_df
   del self._data
   gc.collect()

[mcsqr]

👏

[s3alfisc]

Done 👍

You can now specify a lean function argument:

%load_ext autoreload
%autoreload 2

import pyfixest as pf 
data = pf.get_data()
fit = pf.feols("Y ~ X1", data = data, lean = True)
hasattr(fit, "_X")
# False