Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-1440] [Bug] Connection reused event is not shown #6168

Closed
2 tasks done
JCZuurmond opened this issue Oct 28, 2022 · 2 comments · Fixed by #6605
Closed
2 tasks done

[CT-1440] [Bug] Connection reused event is not shown #6168

JCZuurmond opened this issue Oct 28, 2022 · 2 comments · Fixed by #6605
Assignees
Labels
bug Something isn't working logging support_rotation A good task to pick up during support rotation

Comments

@JCZuurmond
Copy link
Contributor

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

The connection reused event does not show in the logs: Re-using an available connection from the pool (formerly {self.conn_name})

I think this is because the connection reused event is never fired.

I am not completely sure if I am reading the code below correctly, but I think the ConnectionReused is misplaced and maybe never reached:

        conn = self.get_if_exists()
        if conn is None:
            conn = Connection(
                type=Identifier(self.TYPE),
                name=None,
                state=ConnectionState.INIT,
                transaction_open=False,
                handle=None,
                credentials=self.profile.credentials,
            )
            self.set_thread_connection(conn)

        if conn.name == conn_name and conn.state == "open":
            return conn

        fire_event(NewConnection(conn_name=conn_name, conn_type=self.TYPE))

        if conn.state == "open":
            fire_event(ConnectionReused(conn_name=conn_name))
        else:
            conn.handle = LazyHandle(self.open)

        conn.name = conn_name
        return conn

I read the following

  1. The code fetches an existing connection
  2. If no connection exists, it creates a new one and adds the connection to be used later
  3. If the connection has the same name and is open, the connection is returned immediately
  4. If not, a new connection event is sent
  5. Then if the connection is open, a connection reused event is sent
  6. Else lazy handler is set
  7. Connection is returned

I do not understand why the connection reused event might be sent right after the new connection event is sent.

Also, I do not understand when this conn.name == conn_name evaluates to False. When is a different connection name used within the same thread?

Expected Behavior

        conn = self.get_if_exists()
        if conn is None:
            conn = Connection(
                type=Identifier(self.TYPE),
                name=None,
                state=ConnectionState.INIT,
                transaction_open=False,
                handle=None,
                credentials=self.profile.credentials,
            )
            self.set_thread_connection(conn)

        if conn.state == ConnectionState.OPEN.
            fire_event(ConnectionReused(conn_name=conn_name))
        else:
            fire_event(NewConnection(conn_name=conn_name, conn_type=self.TYPE))
            conn.handle = LazyHandle(self.open)
            conn.name = conn_name
        return conn

Steps To Reproduce

NA

Relevant log output

NA

Environment

- OS: NA
- Python: NA
- dbt: v.1.3.0

Which database adapter are you using with dbt?

No response

Additional Context

No response

@JCZuurmond JCZuurmond added bug Something isn't working triage labels Oct 28, 2022
@github-actions github-actions bot changed the title [Bug] Connection reused event is not shown [CT-1440] [Bug] Connection reused event is not shown Oct 28, 2022
@lostmygithubaccount
Copy link
Contributor

thanks @JCZuurmond! your read on this makes sense to me

@emmyoop emmyoop added the support_rotation A good task to pick up during support rotation label Nov 17, 2022
@gshank gshank self-assigned this Jan 12, 2023
@gshank
Copy link
Contributor

gshank commented Jan 13, 2023

There is a connection per thread. If a connection already exists, it is reused and named with the new connection name (usually the unique_id of the node). I think it will be more clear in the modified ConnectionReused message.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working logging support_rotation A good task to pick up during support rotation
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants