Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adding db auto discovery support to postgres #124

Merged
merged 10 commits into from
Jul 3, 2024

Conversation

noapinsler
Copy link
Contributor

When using PostgreSQL, if the connection specifies exclude or include (only one can be specified), the exporter will create connections to the jobs based on the required databases. This parameter accepts a list of databases that can also be specified using regex, like so:
'postgres://postgres@localhost/exclude:postgres,test_db*?sslmode=disable'

@noapinsler noapinsler changed the title adding db auditdiscovry support to postgres dbs adding db autodiscovry support to postgres Apr 17, 2024
@noapinsler noapinsler changed the title adding db autodiscovry support to postgres adding db auto discovery support to postgres Apr 17, 2024
@rezonick
Copy link

Exactly what we need 👍🏼

@dewey
Copy link
Member

dewey commented Jun 19, 2024

@rezonick Do you have capacity to try this PR? I don't have availability to test it, but if it works for two people we can merge it.

@orilani
Copy link

orilani commented Jun 20, 2024

@dewey i can confirm that this PR is working...

@orilani
Copy link

orilani commented Jun 25, 2024

@dewey to elaborate more i took @noapinsler branch, build the dockerfile as is, and deploy it instead of the original sql exporter docker image with same config and values.
i then used this configmap for connecting to my postgres:
postgres://{{POSTGRES_USER}}:{{POSTGRES_PASSWORD}}@{{POSTGRES_URL}}:5432/include:org_*?sslmode=disable

postgres://{{POSTGRES_USER}}:{{POSTGRES_PASSWORD}}@{{POSTGRES_URL}}:5432/exclude:postgres?sslmode=disable

few logs example of top-tables logs that run with the second connection excluding the posgres db:
{"caller":"job.go:389","job":"top-tables","level":"debug","msg":"Running Query","query":"top_tables","ts":"2024-06-24T20:08:32.161510244Z"} {"caller":"job.go:389","job":"top-tables","level":"debug","msg":"Running Query","query":"top_tables","ts":"2024-06-24T20:08:32.161640149Z"} {"caller":"job.go:389","job":"top-tables","level":"debug","msg":"Running Query","query":"top_tables","ts":"2024-06-24T20:08:32.16169112Z"} {"caller":"job.go:389","job":"top-tables","level":"debug","msg":"Running Query","query":"top_tables","ts":"2024-06-24T20:08:32.163286563Z"}

its being working for few days now and works good!

@dewey
Copy link
Member

dewey commented Jun 25, 2024

@noapinsler Would you have time to replace the print statements with logging statements like in the rest of the file? Once that's done I can merge and create a release.

@orilani Thanks for taking the time to test it, sounds good!

@noapinsler
Copy link
Contributor Author

@dewey , I changed the printf to logger as in the rest of the file.

@dewey dewey merged commit 25f9333 into justwatchcom:master Jul 3, 2024
@dewey
Copy link
Member

dewey commented Jul 3, 2024

Thanks!

@dewey
Copy link
Member

dewey commented Jul 5, 2024

A new release is out, the Docker image is still being pushed but will be available shortly. Thanks for the contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants