Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add pipes supporting for numpy objects in the pipeline #229

Merged
merged 11 commits into from
Jan 31, 2023

Conversation

hollandjg
Copy link
Member

@hollandjg hollandjg commented Jan 12, 2023

Description

Provide some new pipe-functions (Pipeline steps) which convert pool sequences into arrays.
Sequences it accept look like range(5), zip(range(0, 10), range(0, 10)), and the functions pass a numpy array or a numpy recarray to the next pipeline step.

The general idea is that you drop the pipe sequence_to_array into your pipeline after a step which uses one of the more primitive pooling functions, and it'll make you a numpy array of the right type so you don't have to worry about it in the next pipeline step. You can switch the results back to the sequence form at the end, by using array_to_sequence.
Something like

Pipeline([
    ("pool", grid_pool),
    ("convert_for_next_steps", sequence_to_array),
    ("summed_dissimilarity_sampler", summed_dissimilarity_sampler),
    ("revert_to_original_form_for_output", array_to_sequence),
])

... and we could then take out the complex coercing of the input from the summed_dissimilarity_sampler, say.

Type of change:

  • New feature (non-breaking change which adds functionality)

@hollandjg hollandjg self-assigned this Jan 31, 2023
@hollandjg hollandjg marked this pull request as ready for review January 31, 2023 14:29
Copy link
Collaborator

@benwandrew benwandrew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nifty! works like a charm. wonderful documentation, and i got to learn about recarray!

@hollandjg hollandjg merged commit c005ecd into main Jan 31, 2023
@hollandjg hollandjg deleted the feat/numpy-pipeline branch January 31, 2023 21:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants