Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Persist timing info for failed nodes #7353

Merged
merged 5 commits into from
Apr 24, 2023
Merged

Conversation

stu-k
Copy link
Contributor

@stu-k stu-k commented Apr 13, 2023

resolves #5476

Description

Since python's context managers __exit__ method still runs if the code they wrap throws an exception, added a callback to the collect_timing_info context manager to handle persisting the timing info for failed nodes to ctx.timing.

Checklist

@cla-bot cla-bot bot added the cla:yes label Apr 13, 2023
@stu-k stu-k marked this pull request as ready for review April 17, 2023 16:21
@stu-k stu-k requested a review from a team April 17, 2023 16:21
@stu-k stu-k requested review from a team as code owners April 17, 2023 16:21
@stu-k stu-k requested review from ChenyuLInx, peterallenwebb, aranke, a team and gshank and removed request for a team and peterallenwebb April 17, 2023 16:21
Copy link
Member

@aranke aranke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you write a test here?

@stu-k stu-k requested a review from aranke April 20, 2023 18:23

def test_timing_exists(self, project):
results = run_dbt(["run"])
assert results.results[0].timing
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you want to do something like this:

for result in results.results:
    assert isinstance(result, TimingInfo)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

results.results is a list of RunResults, not TimingInfo. RunResult has a timing attribute on it which is a list of TimingInfos. What I'm doing here is asserting that this list of TimingInfos is not empty.

There should however only ever be one result in results.results since only one model is run for each of these tests, so I will add an assertion for that.

@stu-k stu-k requested a review from aranke April 24, 2023 15:22
Copy link
Member

@aranke aranke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@stu-k stu-k merged commit de75777 into main Apr 24, 2023
@stu-k stu-k deleted the CT-846/persist-timing-with-failures branch April 24, 2023 16:13
@m-graf
Copy link

m-graf commented Apr 25, 2023

Go @stu-k :)

@aranke aranke mentioned this pull request May 9, 2023
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CT-846] Detailed timing info not collected for models with error status
3 participants