Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get execution time for each block #127

Closed
sarahmish opened this issue Jul 28, 2020 · 1 comment · Fixed by #128
Closed

Get execution time for each block #127

sarahmish opened this issue Jul 28, 2020 · 1 comment · Fixed by #128
Assignees
Milestone

Comments

@sarahmish
Copy link
Contributor

Record the time it takes to fit each primitive. This feature will become handy in debugging pipelines and understanding where the overhead is.

# mlpipeline
def get_time(self):
"""Get the execution time of each block.

If called before fitting the pipeline, it will return an empty dictionary.

Returns:
    dict:
        A dictionary containing the block names as keys and
        the execution time in seconds as values.
"""
@csala
Copy link
Contributor

csala commented Jul 28, 2020

Hi @sarahmish

Capturing debug information sounds good, but I would set this as an optional feature rather than the default behavior.

More precisely, when it comes to debugging I think that it would make sense to enable a debug mode (fit(..., debug=True) and predict(..., debug=True) for the pipeline which, if enabled, makes the pipeline return a dict with information about what happened in each step, including the elapsed time but also input and output arguments. This would also allow us to later on add other profiling information, such as CPU time vs IO time information or memory consumption.

@csala csala added this to the 0.4.0 milestone Dec 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants