exception cleanup #6347

emmyoop · 2022-11-30T22:35:27Z

resolves #6339

Description

Decorator to be added in #6413

This PR

converts all exceptions thrown by a function to directly raise an exception.
adds a few more type properties.
moves the exception functions used in jinja to a separate file in core/context
Organizes exception classes some
Updates most exceptions to take in parameters and build the exceptions in the class instead of just being passed a message

This PR does not

Convert every site of parent exceptions (ParsingException, CompilationException, etc) into a unique exception
Add any kind of logging
Add a decorator to the functions no longer used within core

Checklist

I have read the contributing guide and understand what's expected of me
I have signed the CLA
I have run this code in development and it appears to resolve the stated issue
This PR includes tests, or tests are not required/relevant for this PR
I have opened an issue to add/update docs, or docs changes are not required/relevant for this PR
I have run changie new to create a changelog entry

github-actions · 2022-11-30T22:35:45Z

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

github-actions · 2022-11-30T22:35:46Z

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

core/dbt/adapters/cache.py

peterallenwebb · 2022-12-14T17:12:00Z

core/dbt/adapters/base/impl.py

@@ -22,13 +22,20 @@
 import pytz

 from dbt.exceptions import (


Maybe we've hit the tipping point here where we should just import the dbt.exceptions module?

Is there a reason to do that other than not importing them individually?

I think some Python codebases have policies around that but we don't. I guess if the import list gets really long it might be easier to just import the package. If we did import dbt..exceptions as exc we could specify every one as exc.SomeException. I think some people prefer the specific imports because it makes it clearer which ones are used in this file. I'm kind of agnostic on the whole thing.

some people prefer the specific imports because it makes it clearer which ones are used in this file

I'm in this camp for sure.

core/dbt/exceptions.py

emmyoop · 2022-12-15T22:09:14Z

core/dbt/task/run.py

@@ -400,7 +400,7 @@ def safe_run_hooks(
                    thread_id="main",
                    timing=[],
                    message=f"{hook_type.value} failed, error:\n {exc.msg}",
-                    adapter_response=exc.msg,


The BaseResult was added here recently as part of #6059. My understanding is that the adapter response isn't important. Because i added types, this was failing since exc.msg is a string but adapter_response is a Dict[str, Any].

I think there was some conversation recently about the adapter_response, but I'm pretty vague on what was discussed. I think it often doesn't actually have any content, but maybe should? But in any case assigning a string to it is clearly wrong.

gshank

This is a huge huge improvement over what we had before. It's so much easier to see all of the different exceptions that got hidden behind the generic functions.

I do confess to preferring the use of keyword arguments for constructing them, but wouldn't ask for that at this point.

Also ... Do you think it would be better to use dataclasses? The exceptions would then looks a lot more like the logging events, and we could do a standard "build the message" thing in a base class, kind of like we do for logs.

gshank · 2022-12-16T20:02:23Z

core/dbt/adapters/base/impl.py

                method_name="get_missing_columns",
                arg_name="from_relation",
                got_value=from_relation,
                expected_type=self.Relation,
+                version="0.13.0",


Can we get rid of the "version" piece entirely, including in the message? We haven't supported version 0.13.0 in forever, so just a "this isn't supported" should be fine.

gshank · 2022-12-16T20:16:13Z

core/dbt/task/run.py

@@ -400,7 +400,7 @@ def safe_run_hooks(
                    thread_id="main",
                    timing=[],
                    message=f"{hook_type.value} failed, error:\n {exc.msg}",
-                    adapter_response=exc.msg,


I think there was some conversation recently about the adapter_response, but I'm pretty vague on what was discussed. I think it often doesn't actually have any content, but maybe should? But in any case assigning a string to it is clearly wrong.

aranke

Looks great, but should we rename every exception to end in Error similar to Python's built-in exceptions?

At the very least, we should standardize on using either Error or Exception uniformly everywhere?

cla-bot bot added the cla:yes label Nov 30, 2022

emmyoop commented Nov 30, 2022

View reviewed changes

core/dbt/adapters/cache.py Outdated Show resolved Hide resolved

emmyoop force-pushed the er/ct-1567-exception-cleanup branch from e5b8c76 to a25bfb9 Compare December 5, 2022 20:13

emmyoop mentioned this pull request Dec 6, 2022

[CT-1612] Add deprecation decorator for functions #6393

Closed

emmyoop force-pushed the er/ct-1567-exception-cleanup branch from 9d19015 to 358746a Compare December 8, 2022 02:49

peterallenwebb reviewed Dec 14, 2022

View reviewed changes

emmyoop added 22 commits December 15, 2022 08:29

starting to move jinja exceptions

3d1d938

convert some exceptions

8797c5d

add back old functions for backward compatibility

023baef

organize

0971e32

more conversions

c02fc01

more conversions

493c997

add changelog

9f1d65c

split out CacheInconsistency

0c5d720

more conversions

685d58a

convert even more

9478bb3

convert parsingexceptions

1037aeb

fix tests

4b56659

more conversions

3a739b8

more conversions

72a0072

finish converting exception functions

d98f4f3

convert more tests

62f8b42

standardize to msg

d397c38

remove some TODOs

1925c11

fix test param and check the rest

c5f9d30

add comment, move exceptions

c0406c8

add types

b3bf0be

fix type errors

e85fb67

emmyoop force-pushed the er/ct-1567-exception-cleanup branch from bd34660 to e85fb67 Compare December 15, 2022 14:34

fix type for adapter_response

5b86c8b

emmyoop changed the title ~~[WIP] exception cleanup~~ exception cleanup Dec 15, 2022

emmyoop marked this pull request as ready for review December 15, 2022 21:41

emmyoop requested a review from a team as a code owner December 15, 2022 21:41

emmyoop requested a review from a team December 15, 2022 21:41

emmyoop requested review from a team as code owners December 15, 2022 21:41

emmyoop requested review from Fleid, aranke and gshank December 15, 2022 21:41

emmyoop commented Dec 15, 2022

View reviewed changes

gshank approved these changes Dec 16, 2022

View reviewed changes

remove 0.13 version from message

a6dbe51

peterallenwebb approved these changes Dec 19, 2022

View reviewed changes

aranke reviewed Dec 20, 2022

View reviewed changes

emmyoop mentioned this pull request Dec 20, 2022

[CT-1687] Rename every exception to end in Error #6460

Closed

emmyoop merged commit 304797b into main Dec 20, 2022

emmyoop deleted the er/ct-1567-exception-cleanup branch December 20, 2022 01:36

emmyoop mentioned this pull request Dec 20, 2022

[CT-1689] Check exceptions dbt-labs/dbt-rpc#111

Closed

iknox-fa mentioned this pull request Jan 30, 2023

testing merge strat - do not merge ns/add to committer list #6783

Closed

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

exception cleanup #6347

exception cleanup #6347

emmyoop commented Nov 30, 2022 •

edited

Loading

github-actions bot commented Nov 30, 2022

github-actions bot commented Nov 30, 2022

peterallenwebb Dec 14, 2022

emmyoop Dec 14, 2022

gshank Dec 16, 2022

emmyoop Dec 16, 2022

emmyoop Dec 15, 2022

gshank Dec 16, 2022

gshank left a comment

gshank Dec 16, 2022

gshank Dec 16, 2022

aranke left a comment

exception cleanup #6347

exception cleanup #6347

Conversation

emmyoop commented Nov 30, 2022 • edited Loading

Description

Checklist

github-actions bot commented Nov 30, 2022

github-actions bot commented Nov 30, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gshank left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aranke left a comment

Choose a reason for hiding this comment

emmyoop commented Nov 30, 2022 •

edited

Loading