Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Unified Fides] Health Checks Opening Up Too Many Connections #1144

Closed
pattisdr opened this issue Sep 30, 2022 · 0 comments · Fixed by #1182
Closed

[Unified Fides] Health Checks Opening Up Too Many Connections #1144

pattisdr opened this issue Sep 30, 2022 · 0 comments · Fixed by #1182
Assignees
Labels
bug Something isn't working Unified Fides

Comments

@pattisdr
Copy link
Contributor

pattisdr commented Sep 30, 2022

Bug Description

The health check endpoints are opening up too many connections against the application database.

This was previously addressed in fidesops ethyca/fidesops#1107 by:

  • Stop creating a new engine as part of the process of running a health check.
  • Create an engine to be used across the application, and pass this into get_db_session when we go to get a single connection, instead of creating a new engine. (Note we are currently using the default pool_size and max_overflow parameters. This is something that may need to be configurable in the future). This is used for all our API endpoints.
    Note that our celery processes intentionally create their own separate engines for managing connections for running privacy requests.

Steps to Reproduce

  1. Bring up services nox -s dev -- shell ui mongo postgres
  2. (fides_test database is incorrectly being used right now, but monitor connections there psql -h localhost -p 5432 -U postgres -d fides_test -c "SELECT sum(numbackends) FROM pg_stat_database"
  3. Observe as periodic health checks are made in the console: 2022-09-30 01:50:36.458 [INFO] (main:log_request:262): Request received | {'method': 'GET', 'status_code': 200, 'handler_time': '363.21500000000003ms', 'path': '/health'}
  4. Watch the number of connections increase linearly over time
 sum
-----
  22

Expected behavior

A description of what you expected to happen.

Screenshots

If applicable, add screenshots to help explain your problem.

Environment

  • Version:
  • OS:
  • Python Version:
  • Docker Version:

Additional context

Add any other context about the problem here.

@pattisdr pattisdr added bug Something isn't working Unified Fides labels Sep 30, 2022
@pattisdr pattisdr self-assigned this Sep 30, 2022
@pattisdr pattisdr linked a pull request Oct 3, 2022 that will close this issue
14 tasks
@pattisdr pattisdr closed this as completed Oct 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Unified Fides
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant