Fepois: More rigorous tests for separation

[s3alfisc]

Context

In alignment with fixest, pyfixest currently only implements a very basic check for separation.

PyFixest currently implements the most basic separation check, which is equivalent to ppmlhdfe's separation(fe).

Ppmlhdfe implements three more separation checks; it would be nice if fepois() would support some of them.

For a primer on separation, see the primer on separation by the ppmlhdfe team.

To Do:

• implement additional separation checks for the check_for_separation function.
• Allow users to specify separation checks via the fepois and Fepois interfaces.

[leostimpfle]

I'd be interested in working on this

[s3alfisc]

Awesome, then I'll assign this issue to it. Maybe we could approach it one separation check at a time, with multiple PRs? This is just a suggestion, I am happy to follow your lead on how to structure the PR(s) =)

[leostimpfle]

Yes, seems sensible to do it method by method. I was thinking of initially focussing on ir (iterative rectifier) because it is recommended in combination with fe for regressions with many high-dimensional fixed effects.

[leostimpfle]

With #660 the user can specify separtion_checks for fepois. Currently, there are two checks implemented: the iterative rectifier ir and the original fe check.

ppmlhdfe implements two further methods:

• simplex: does not handle separation arising from fixed effects (which is arguably pyfixest's main use case)
• mu: a simple heuristic (if y=0 and exp(xb) is also very close to zero, then the observations are likely separated)

My understanding is that simplex is less relevant for pyfixest as it doesn't cover separation arising from fixed effects and that mu is a nice to have but much less powerful than the iterative rectifier.

Would it therefore make sense to close this issue and open two separate issues for the implementation of simplex and mu @s3alfisc ?

[s3alfisc]

Yes, I think we can close this - I think with the two separation tests we have, we're actually covered really well =)