Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests fail with pandas==1.1(metadata lost on groupby) #26

Closed
Flix6x opened this issue Sep 15, 2020 · 3 comments · Fixed by #23
Closed

Tests fail with pandas==1.1(metadata lost on groupby) #26

Flix6x opened this issue Sep 15, 2020 · 3 comments · Fixed by #23
Assignees
Labels
Continuous integration Subclassing Subclassing pandas data structures

Comments

@Flix6x
Copy link
Collaborator

Flix6x commented Sep 15, 2020

Until this issue is fixed, timely-beliefs only supports pandas<1.1.

@nhoening
Copy link
Contributor

nhoening commented Sep 15, 2020 via email

@Flix6x Flix6x added the Subclassing Subclassing pandas data structures label Nov 14, 2020
@Flix6x Flix6x linked a pull request Dec 11, 2020 that will close this issue
@Flix6x
Copy link
Collaborator Author

Flix6x commented Dec 11, 2020

The fix was released as part of pandas 1.1.5. There was a backport, too, but our relevant tests still fail. @nhoening can you comment if my procedure below is correct? I am probably misunderstanding how backports actually work. Basically, I thought a backport of a fix would lead to that fix being present in the code of relevant older versions as available on pypi, perhaps as a developer release. However, the release of 1.1.4 on pypi is on October 30th, while the backport was merged on November 4th.

Here I try to install a previous version of pandas with the backport supposedly in place:

pip install pandas==1.1.4 --no-cache --force-reinstall

Then I run the test that I would expect to pass (if this focused test fails, other integration tests fail, too, of course):

pytest -k test_groupby_retains_metadata

This test, and all others, too, are passing against pandas==1.1.5 on #23 .

@nhoening
Copy link
Contributor

It was indeed not obvious to find, but they had it set on 1.2 in the beginning of the PR, but then backported it to 1.1.5 when they were done.
You can see this here (click "show resolved"), where they removed the release note entry from the 1.2 part (comment: "Force pushed to drop this commit").

Flix6x added a commit that referenced this issue Dec 21, 2020
This PR adds no additional logic, only a test to check whether Issue #35 is successfully resolved (plus some refactoring of test util functions). Note that this is a merge into resample-while-keeping-metadata, and that branch (with PR #23) actually contains the logic that resolves this issue (as a side effect, because I was actually addressing deeper issues there: #22 and #26).

* Refactor metadata propagation checks to util function.

* Add test for metadata propagation upon multiplication (GH 35).

Co-authored-by: F.N. Claessen <[email protected]>
Flix6x added a commit that referenced this issue Dec 21, 2020
This PR drops support for pandas<1.1.5 and adds various tests to ensure metadata is propagated on various operations.
Specific code changes:

- Every constructor now calls finalize.
- for_each_belief_ is no longer passed self as the df to work on.

* Call finalize in all constructors to inherit metadata.
Prepare tests for upcoming pandas==1.1.1 functionality: inherit metadata when resampling and grouping.

* Update reference in test.
Add pandas test.

* Finish rebase.

* Complete the ordering of BeliefSources by name.

* Fix computation order.

* Workaround for aggregation function when resampling.

* Fix tests:
- Separate tests for mean resampling and aggregate resampling.
- Separate tests for temporary attributes and subclass attributes.
- Test metadata propagation for groupby of subclassed DataFrames.

* Update pandas dependency.
Drop support for pandas<1.1.5.
Bump timely-beliefs version with major release.

* Prepare dtype test for empty frames.

* Add comment about which pandas version fixed test_groupby_retains_metadata.

* Rename downsampling function.

* Simplify calls to for_each_belief.

* Fix plotting integer values (int64 is not JSON serializable).

* Add reference in test docs to relevant pandas issue.

* Issue 35 metadata lost on multiplication (#43)

This PR adds no additional logic, only a test to check whether Issue #35 is successfully resolved (plus some refactoring of test util functions). Note that this is a merge into resample-while-keeping-metadata, and that branch (with PR #23) actually contains the logic that resolves this issue (as a side effect, because I was actually addressing deeper issues there: #22 and #26).

* Refactor metadata propagation checks to util function.

* Add test for metadata propagation upon multiplication (GH 35).

Co-authored-by: F.N. Claessen <[email protected]>

Co-authored-by: Felix Claessen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Continuous integration Subclassing Subclassing pandas data structures
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants