Feols, Fepois, Feiv: Implement Option to use different Solver for Normal Equation

[s3alfisc]

Context

Currently, all fit methods solve the OLS normal equation.

E.g. for OLS, we have

pyfixest/pyfixest/estimation/feols_.py

Line 310 in d922bfc

self._beta_hat = np.linalg.solve(self._tZX, self._tZy).flatten()

, for IV we have

pyfixest/pyfixest/estimation/feiv_.py

Line 145 in d922bfc

self._beta_hat = np.linalg.solve(A, B).flatten()

, and for Fepois we have

pyfixest/pyfixest/estimation/fepois_.py

Line 218 in d922bfc

delta_new = np.linalg.solve(XWX, XWZ) # eq (10), delta_new -> reg_z

.

It would be cool to allow users to specify different solvers, e.g. np.linalg.lstsq.

To do

• Add a function argument solver to Feols, Feiv, Fepois, feols() and fepois.
• Allow users to choose between normal (the current default) and np.linalg.lstsq
• Additionally, consider to allow users to provide additional arguments to solvers via function kwargs.
• Add unit tests to confirm that different solvers lead to identical results.

@apoorvalal , any other solvers you'd like to see supported?

[apoorvalal]

Good idea; I think using a solver that 'adapts' to the structure of the data would be useful.

Candidates for linear

• scipy.sparse.linalg.lsqr [default in scikit's LinearRegression]
• lineax, gains are particularly large in minimum norm interpolant (p>n) settings like here

For poisson

• glum's GLM solvers have some very tailored solutions [I don't know much about efficient poisson estimation fwiw]

[s3alfisc]

Hi @saidamir, could you comment in the issue? Else I cannot "officially" assign it to you. And thanks for taking a crack at it! =)

[saidamir]

Commenting.

[saidamir]

Hi @s3alfisc still struggling to set up my local environment (I am following your onboarding instruction from Contributing section), as there are quite a few errors when I install the just and r packages. Also is there a requirements.txt file to import all required libraries? Thanks!

[s3alfisc]

Hi @saidamir - we use poetry for package management. I should update the contributing.md with details on how to set it up. Will do so later today!

[saidamir]

I tried installing the packages locally using poetry, however there was an error in the processor compatibility.

[s3alfisc]

Hm, that's strange, can you post the error message?

[s3alfisc]

There's no requirements.txt. Is there an advantage of having one over poetry? I.e. that one can install dependencies without poetry? We could add poetry export -f requirements.txt --output requirements.txt to the CI to automatically create a setup.py file based on the poetry.toml.

Could you list all the errors you get when trying to install just and the R dependencies? Note that they are not strictly needed for development - they're used only for unit tests, which you can also run on github actions when pushing to a fork / opening a PR.

[saidamir]

I imported libraries from using the poetry file as the requirements, however when I tried to run a jupyter notebook on my vscode, I had the errors that I didn't have numpy etc. I started imported them, however got the following error. So finally, what I did, is I started from the beginning, created a new folders, new venv and imported the libraries but didn't use the poetry file. So far it is working ok. Here is the error:
ImportError: dlopen(/Users/kamasam/Documents/pyfixest/pythonProject1/venv/lib/python3.11/site-packages/pandas/_libs/pandas_parser.cpython-311-darwin.so, 0x0002): tried: '/Users/kamasam/Documents/pyfixest/pythonProject1/venv/lib/python3.11/site-packages/pandas/_libs/pandas_parser.cpython-311-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')),
'/System/Volumes/Preboot/Cryptexes/OS/Users/kamasam/Documents/pyfixest/pythonProject1/venv/lib/python3.11/site-packages/pandas/_libs/pandas_parser.cpython-311-darwin.so' (no such file), '/Users/kamasam/Documents/pyfixest/pythonProject1/venv/lib/python3.11/site-packages/pandas/_libs/pandas_parser.cpython-311-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))

[s3alfisc]

I imported libraries from using the poetry file as the requirements, however when I tried to run a jupyter notebook on my vscode, I had the errors that I didn't have numpy etc. I started imported them, however got the following error. So finally, what I did, is I started from the beginning, created a new folders, new venv and imported the libraries but didn't use the poetry file. So far it is working ok. Here is the error:

Sounds like a pragmatic solution! Not really sure how to help with your error - my trusted LLM adviser says this:

If you are using a Mac with an M1 or M2 chip, you need to ensure that your Python environment is built for arm64. Verify that your Python version is build for the right architecture by running python -c "import platform; print(platform.architecture())".

You can install all r packages from the justfile by running just install-r - I just merged a PR that fixes a typo in the command. And note that if you're using the justfile from a mac, you need to delete the first line which tells just to use the powershell:

set shell := ["powershell.exe", "-c"]

[saidamir]

Thanks. My architecture is arm64. So far, I am able to work on this package, as I created a new folder and avoided using poetry file.

[saidamir]

Hi I just pushed a pr for one class feols. I would like to see if it is accepted and if yes, continue working on other classes and methods on this ticket.