Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

netbox: fix psycopg dependency #263168

Closed
wants to merge 1 commit into from
Closed

netbox: fix psycopg dependency #263168

wants to merge 1 commit into from

Conversation

xanderio
Copy link
Contributor

@xanderio xanderio commented Oct 24, 2023 •
edited
Loading

Description of changes

netbox requires psycopg version 3, this version is packaged as psycopg.

nixos tests are currently failing do to #254268 (comment). I've locally modified the tests to use postgresql 14 which result in them passing. Should I include this change in this PR?

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.11 Release Notes (or backporting 23.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@xanderio
netbox: fix psycopg dependency
e293b32 
netbox requires psycopg version 3, this version is packaged as psycopg.
@xanderio xanderio marked this pull request as ready for review October 24, 2023 15:45
mweinelt
mweinelt reviewed Oct 24, 2023
View reviewed changes
pkgs/servers/web-apps/netbox/generic.nix
psycopg2
psycopg
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't this break netbox_3_5, which people could still want to migrate from?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unsure, the netbox_3_5 tests don't evaluate as they are eol. I need to figure out how I can run them.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xanderio you can run them with NIXPKGS_ALLOW_INSECURE=1 nix build --impure '.#nixosTests.netbox_3_5' if you have flakes enabled.

Without flakes enabled, you can do NIXPKGS_ALLOW_INSECURE=1 nix-build -A 'nixosTests.netbox_3_5'

And the same for the migration test.

@xanderio xanderio marked this pull request as draft October 24, 2023 16:11
@minijackson minijackson mentioned this pull request Oct 24, 2023
Merged
12 tasks
@xanderio
Copy link
Contributor Author

Fascinating and worrying both 3.5 and 3.6 tests pass, but when I apply these changes to a prod system netbox doesn't start.

Oct 24 18:10:44 netbox systemd[1]: Started NetBox WSGI Service.
Oct 24 18:10:45 netbox gunicorn[166826]: [2023-10-24 18:10:45 +0200] [166826] [INFO] Starting gunicorn 20.1.0
Oct 24 18:10:45 netbox gunicorn[166826]: [2023-10-24 18:10:45 +0200] [166826] [INFO] Listening at: http://[::1]:8001 (166826)
Oct 24 18:10:45 netbox gunicorn[166826]: [2023-10-24 18:10:45 +0200] [166826] [INFO] Using worker: sync
Oct 24 18:10:45 netbox gunicorn[166827]: [2023-10-24 18:10:45 +0200] [166827] [INFO] Booting worker with pid: 166827
Oct 24 18:10:45 netbox gunicorn[166827]: [2023-10-24 18:10:45 +0200] [166827] [ERROR] Exception in worker process
Oct 24 18:10:45 netbox gunicorn[166827]: Traceback (most recent call last):
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/5dvkjr7q7qzl6bxbiw3v0c9vbchbdhwr-python3.10-gunicorn-20.1.0/lib/python3.10/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
Oct 24 18:10:45 netbox gunicorn[166827]:     worker.init_process()
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/5dvkjr7q7qzl6bxbiw3v0c9vbchbdhwr-python3.10-gunicorn-20.1.0/lib/python3.10/site-packages/gunicorn/workers/base.py", line 134, in init_process
Oct 24 18:10:45 netbox gunicorn[166827]:     self.load_wsgi()
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/5dvkjr7q7qzl6bxbiw3v0c9vbchbdhwr-python3.10-gunicorn-20.1.0/lib/python3.10/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
Oct 24 18:10:45 netbox gunicorn[166827]:     self.wsgi = self.app.wsgi()
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/5dvkjr7q7qzl6bxbiw3v0c9vbchbdhwr-python3.10-gunicorn-20.1.0/lib/python3.10/site-packages/gunicorn/app/base.py", line 67, in wsgi
Oct 24 18:10:45 netbox gunicorn[166827]:     self.callable = self.load()
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/5dvkjr7q7qzl6bxbiw3v0c9vbchbdhwr-python3.10-gunicorn-20.1.0/lib/python3.10/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
Oct 24 18:10:45 netbox gunicorn[166827]:     return self.load_wsgiapp()
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/5dvkjr7q7qzl6bxbiw3v0c9vbchbdhwr-python3.10-gunicorn-20.1.0/lib/python3.10/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
Oct 24 18:10:45 netbox gunicorn[166827]:     return util.import_app(self.app_uri)
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/5dvkjr7q7qzl6bxbiw3v0c9vbchbdhwr-python3.10-gunicorn-20.1.0/lib/python3.10/site-packages/gunicorn/util.py", line 359, in import_app
Oct 24 18:10:45 netbox gunicorn[166827]:     mod = importlib.import_module(module)
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/5ywaphn9qls4161gv3j1081hnmshdqka-python3-3.10.13/lib/python3.10/importlib/__init__.py", line 126, in import_module
Oct 24 18:10:45 netbox gunicorn[166827]:     return _bootstrap._gcd_import(name[level:], package, level)
Oct 24 18:10:45 netbox gunicorn[166827]:   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
Oct 24 18:10:45 netbox gunicorn[166827]:   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
Oct 24 18:10:45 netbox gunicorn[166827]:   File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
Oct 24 18:10:45 netbox gunicorn[166827]:   File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
Oct 24 18:10:45 netbox gunicorn[166827]:   File "<frozen importlib._bootstrap_external>", line 883, in exec_module
Oct 24 18:10:45 netbox gunicorn[166827]:   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/1bir42s5v5yh3il1dzcwzq7rvqxjz1k2-netbox-3.6.3/opt/netbox/netbox/netbox/wsgi.py", line 7, in <module>
Oct 24 18:10:45 netbox gunicorn[166827]:     application = get_wsgi_application()
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/3plg5pnzsvr38pq8s364g0zlfkkrd3zw-python3.11-Django-4.2.6/lib/python3.11/site-packages/django/core/wsgi.py", line 12, in get_wsgi_application
Oct 24 18:10:45 netbox gunicorn[166827]:     django.setup(set_prefix=False)
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/3plg5pnzsvr38pq8s364g0zlfkkrd3zw-python3.11-Django-4.2.6/lib/python3.11/site-packages/django/__init__.py", line 19, in setup
Oct 24 18:10:45 netbox gunicorn[166827]:     configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/3plg5pnzsvr38pq8s364g0zlfkkrd3zw-python3.11-Django-4.2.6/lib/python3.11/site-packages/django/conf/__init__.py", line 102, in __getattr__
Oct 24 18:10:45 netbox gunicorn[166827]:     self._setup(name)
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/3plg5pnzsvr38pq8s364g0zlfkkrd3zw-python3.11-Django-4.2.6/lib/python3.11/site-packages/django/conf/__init__.py", line 89, in _setup
Oct 24 18:10:45 netbox gunicorn[166827]:     self._wrapped = Settings(settings_module)
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/3plg5pnzsvr38pq8s364g0zlfkkrd3zw-python3.11-Django-4.2.6/lib/python3.11/site-packages/django/conf/__init__.py", line 217, in __init__
Oct 24 18:10:45 netbox gunicorn[166827]:     mod = importlib.import_module(self.SETTINGS_MODULE)
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/5ywaphn9qls4161gv3j1081hnmshdqka-python3-3.10.13/lib/python3.10/importlib/__init__.py", line 126, in import_module
Oct 24 18:10:45 netbox gunicorn[166827]:     return _bootstrap._gcd_import(name[level:], package, level)
Oct 24 18:10:45 netbox gunicorn[166827]:   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
Oct 24 18:10:45 netbox gunicorn[166827]:   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
Oct 24 18:10:45 netbox gunicorn[166827]:   File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
Oct 24 18:10:45 netbox gunicorn[166827]:   File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
Oct 24 18:10:45 netbox gunicorn[166827]:   File "<frozen importlib._bootstrap_external>", line 883, in exec_module
Oct 24 18:10:45 netbox gunicorn[166827]:   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/1bir42s5v5yh3il1dzcwzq7rvqxjz1k2-netbox-3.6.3/opt/netbox/netbox/netbox/settings.py", line 20, in <module>
Oct 24 18:10:45 netbox gunicorn[166827]:     from netbox.config import PARAMS
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/1bir42s5v5yh3il1dzcwzq7rvqxjz1k2-netbox-3.6.3/opt/netbox/netbox/netbox/config/__init__.py", line 8, in <module>
Oct 24 18:10:45 netbox gunicorn[166827]:     from .parameters import PARAMS
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/1bir42s5v5yh3il1dzcwzq7rvqxjz1k2-netbox-3.6.3/opt/netbox/netbox/netbox/config/parameters.py", line 2, in <module>
Oct 24 18:10:45 netbox gunicorn[166827]:     from django.contrib.postgres.forms import SimpleArrayField
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/3plg5pnzsvr38pq8s364g0zlfkkrd3zw-python3.11-Django-4.2.6/lib/python3.11/site-packages/django/contrib/postgres/forms/__init__.py", line 3, in <module>
Oct 24 18:10:45 netbox gunicorn[166827]:     from .ranges import *  # NOQA
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/3plg5pnzsvr38pq8s364g0zlfkkrd3zw-python3.11-Django-4.2.6/lib/python3.11/site-packages/django/contrib/postgres/forms/ranges.py", line 3, in <module>
Oct 24 18:10:45 netbox gunicorn[166827]:     from django.db.backends.postgresql.psycopg_any import (
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/3plg5pnzsvr38pq8s364g0zlfkkrd3zw-python3.11-Django-4.2.6/lib/python3.11/site-packages/django/db/backends/postgresql/psycopg_any.py", line 5, in <module>
Oct 24 18:10:45 netbox gunicorn[166827]:     from psycopg import ClientCursor, IsolationLevel, adapt, adapters, errors, sql
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/mf2zrql2hv20flfn2bg1ni003jb8ygzc-python3.11-psycopg-3.1.12/lib/python3.11/site-packages/psycopg/__init__.py", line 9, in <module>
Oct 24 18:10:45 netbox gunicorn[166827]:     from . import pq  # noqa: F401 import early to stabilize side effects
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/mf2zrql2hv20flfn2bg1ni003jb8ygzc-python3.11-psycopg-3.1.12/lib/python3.11/site-packages/psycopg/pq/__init__.py", line 114, in <module>
Oct 24 18:10:45 netbox gunicorn[166827]:     import_from_libpq()
Oct 24 18:10:45 netbox gunicorn[166827]:   File "/nix/store/mf2zrql2hv20flfn2bg1ni003jb8ygzc-python3.11-psycopg-3.1.12/lib/python3.11/site-packages/psycopg/pq/__init__.py", line 94, in import_from_libpq
Oct 24 18:10:45 netbox gunicorn[166827]:     __impl__ = module.__impl__
Oct 24 18:10:45 netbox gunicorn[166827]: AttributeError: module 'psycopg_c.pq' has no attribute '__impl__'
Oct 24 18:10:45 netbox gunicorn[166827]: [2023-10-24 18:10:45 +0200] [166827] [INFO] Worker exiting (pid: 166827)
Oct 24 18:10:46 netbox gunicorn[166826]: [2023-10-24 18:10:46 +0200] [166826] [INFO] Shutting down: Master
Oct 24 18:10:46 netbox gunicorn[166826]: [2023-10-24 18:10:46 +0200] [166826] [INFO] Reason: Worker failed to boot.
Oct 24 18:10:46 netbox systemd[1]: netbox.service: Main process exited, code=exited, status=3/NOTIMPLEMENTED
Oct 24 18:10:46 netbox systemd[1]: netbox.service: Failed with result 'exit-code'.
Oct 24 18:10:46 netbox systemd[1]: netbox.service: Consumed 1.335s CPU time, no IP traffic.

@RaitoBezarius
Copy link
Member

Fresh horrors, great. :D

@xanderio
Copy link
Contributor Author

xanderio commented Dec 4, 2023

This seems to be fixed in nixos 23.11

@xanderio xanderio closed this Dec 4, 2023
@xanderio xanderio deleted the fix-netbox branch November 29, 2024 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mweinelt mweinelt mweinelt left review comments

@n0emis n0emis Awaiting requested review from n0emis

@minijackson minijackson Awaiting requested review from minijackson

@RaitoBezarius RaitoBezarius Awaiting requested review from RaitoBezarius

Assignees
No one assigned
Labels
10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@xanderio @RaitoBezarius @mweinelt @minijackson