Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce redirection in ops #19649

Merged
merged 3 commits into from
Feb 19, 2018
Merged

Reduce redirection in ops #19649

merged 3 commits into from
Feb 19, 2018

Conversation

jbrockmendel
Copy link
Member

Among other things, this will make it easier to handle the sparse vs non-sparse Frame ops in one place.

@codecov
Copy link

codecov bot commented Feb 12, 2018

Codecov Report

Merging #19649 into master will decrease coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #19649      +/-   ##
==========================================
- Coverage    91.6%   91.59%   -0.01%     
==========================================
  Files         150      150              
  Lines       48864    48830      -34     
==========================================
- Hits        44762    44728      -34     
  Misses       4102     4102
Flag Coverage Δ
#multiple 89.97% <100%> (-0.01%) ⬇️
#single 41.76% <100%> (ø) ⬆️
Impacted Files Coverage Δ
pandas/core/sparse/array.py 91.4% <ø> (+0.02%) ⬆️
pandas/core/sparse/series.py 95.26% <ø> (ø) ⬆️
pandas/core/panel.py 97.3% <100%> (ø) ⬆️
pandas/core/ops.py 95.57% <100%> (-0.88%) ⬇️
pandas/plotting/_converter.py 65.22% <0%> (-1.74%) ⬇️
pandas/core/indexes/datetimes.py 95.25% <0%> (-0.07%) ⬇️
pandas/core/frame.py 97.17% <0%> (-0.06%) ⬇️
pandas/core/indexes/period.py 92.99% <0%> (-0.06%) ⬇️
pandas/core/indexes/category.py 97.26% <0%> (-0.06%) ⬇️
... and 19 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 718d067...9355ce2. Read the comment docs.

@gfyoung gfyoung added Refactor Internal refactoring of code Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff labels Feb 12, 2018
@@ -1298,18 +1371,19 @@ def f(self, other, axis=0):
comp_method=_comp_method_PANEL,
bool_method=_arith_method_PANEL)

panel_flex_funcs = dict(flex_arith_method=_flex_method_PANEL,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are you adding support for this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changes zero logic, just puts a dict here instead of passing these directly in panel.

@@ -1379,3 +1454,12 @@ def wrapper(self, other):
name = name[2:-2]
wrapper.__name__ = name
return wrapper


sparse_array_special_funcs = dict(arith_method=_arith_method_SPARSE_ARRAY,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are these new?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changes zero logic, just puts a dict here instead of passing these directly in sparse.array.

@@ -1528,8 +1528,7 @@ def _extract_axis(self, data, axis=0, intersect=False):
'minor_axis': 'columns'})

ops.add_special_arithmetic_methods(Panel, **ops.panel_special_funcs)
ops.add_flex_arithmetic_methods(Panel, ops._flex_method_PANEL,
flex_comp_method=ops._comp_method_PANEL)
ops.add_flex_arithmetic_methods(Panel, **ops.panel_flex_funcs)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why don't we name these the same for all classes

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in fact, why do we need this logic at all, cannot

ops.add_flex_arithmetic_method(cls) just do this?

or even better
ops.add_arithmetic_methods(cls)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this idea. SparseSeries does some double-calls that I'll need to sort out to make this work, so let's roll this into the next pass which is Sparse-specific.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok this looks pretty good. I think should take this one step further (can be in another PR). where you then only need to call something like

ops.add_arithmetic_methods(cls)

in each class and push the logic of which flex/special are added to core ops

@jbrockmendel
Copy link
Member Author

#19613 and #19611 should both be ready; want to make sure they don't fall off the radar screen.

@jbrockmendel
Copy link
Member Author

I think this is the only outstanding PR that os core.ops-centric. Let's try to close this out before moving forward with the remaining series/frame-implementation-de-duplication.

@jreback jreback added this to the 0.23.0 milestone Feb 19, 2018
@jreback
Copy link
Contributor

jreback commented Feb 19, 2018

if you can rebase and then ping on green.

@jbrockmendel
Copy link
Member Author

semi-ping. clicking through to appveyor it shows green and said it finished an hour ago; the icon hasn't updated here though.

@jreback jreback merged commit 8aa55a9 into pandas-dev:master Feb 19, 2018
@jreback
Copy link
Contributor

jreback commented Feb 19, 2018

thanks. ideally would like to see a PR to simplify this further as my above comment.

@jbrockmendel jbrockmendel deleted the opsolete branch February 20, 2018 03:07
@jbrockmendel
Copy link
Member Author

The appveyor build from 9355ce2 isn't actually still running is it? If so could that be clogging the appveyor pipeline?

harisbal pushed a commit to harisbal/pandas that referenced this pull request Feb 28, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff Refactor Internal refactoring of code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants