lsqr() for fixed effects solver

[greenguy33]

Closes #469

[greenguy33]

Looks like method _matmul_dispatch() in scipy.sparse is throwing an error based on this change:

        if issparse(other):
            if self.shape[-1] != other.shape[0]:
>               raise ValueError('dimension mismatch')
E               ValueError: dimension mismatch

../../../.cache/pypoetry/virtualenvs/pyfixest-va_uSH9_-py3.9/lib/python3.9/site-packages/scipy/sparse/_base.py:603: ValueError

I'll take a look at what's going asap.

[s3alfisc]

I think lsqr might return a tuple? https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.lsqr.html so we'd have to pick the first return element.

[s3alfisc]

Another option to fix the failing tests is to tighten the convergence criteria for the iterative lsqr solver. I.e. by setting atol=1e-10 and btol=1e-10, all tests pass:

        alpha = lsqr(D2, uhat, atol = 1e-10, btol = 1e-10)[0]

Note that scikit-learns LinearRegession class also builds on the lsqr solver, and they are satisfied with the default tolerances for atol and btol of 1e-06.

I think I would suggest the following: we keep the 1e-06 defaults as our defaults, but add function arguments atol and btol to the .predict() and .fixef() methods so that users can specify their desired numerical accuracy.

What do you think Hayden (@greenguy33)? Any feedback from a user's perspective on this, @b-knight?

[greenguy33]

That sounds good to me. I am mostly offline this weekend but can definitely make this change in the coming week. I guess it will require some changes in the tests as well, to supply the passing values of 1e-10?

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Files	Coverage Δ
pyfixest/estimation/feols_.py	90.59% <100.00%> (ø)
tests/test_did.py	100.00% <100.00%> (ø)
tests/test_predict_resid_fixef.py	92.80% <100.00%> (+0.49%)	⬆️

... and 26 files with indirect coverage changes

[greenguy33]

@s3alfisc Looks like tests are passing now. However in order to get that to happen I changed the tolerance values for np.testing.assert_allclose in two test files. I'm not sure if these tests were passing before because it doesn't appear that they are running the lsqr solver.
Supplying tolerance values of atol = 1e-05, rtol = 1e-05 seemed to work, and smaller values tend to fail

[s3alfisc]

Could you add docstrings that explain atol and btol and that refer to the scipy lsqr docs?

https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.lsqr.html

[s3alfisc]

Ah, I've seen you did it already for predict. Thanks!

[s3alfisc]

Just merged, thank you @greenguy33!