Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace black, flake8, pydocstyle, bandit with ruff #314

Closed
wants to merge 27 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
d45f23c
build: ➕ Replace black and flake8 with ruff
jdrew82 Jan 4, 2024
88c1ff8
build: Remove black and flake8 tasks
jdrew82 Jan 4, 2024
bc9c618
build: ➖ Remove pydocstyle as covered by ruff
jdrew82 Jan 4, 2024
db55636
build: 🏗️ Update settings for pylint and add ruff settings to pyproje…
jdrew82 Jan 4, 2024
c8886a9
build: 🏗️ Add ruff task and add to tests command.
jdrew82 Jan 4, 2024
53056f8
build: ⬆️ Update project dependencies lockfile
jdrew82 Jan 4, 2024
763158c
build: 🏗️ Add ignore for SNOW third party package folder
jdrew82 Jan 4, 2024
ec74089
fix: 🐛 Correct folders to check with ruff in task
jdrew82 Jan 4, 2024
b4b8e96
style: 🚨 Fix formatting for ruff
jdrew82 Jan 4, 2024
aef3e34
fix:
jdrew82 Jan 4, 2024
1bee050
fix: 🐛 Correct syntax to use not in
jdrew82 Jan 4, 2024
303b0a3
test: 🚨 Addressing ruff linter complaints
jdrew82 Jan 4, 2024
7b9a31e
build: ➖ Remove bandit dependency as covered by flake8-bandit in ruff
jdrew82 Jan 4, 2024
d381242
test: 🚨 Ignore RUF013 check in ruff
jdrew82 Jan 4, 2024
10cdf04
build: ⏪️ Restore some settings from black for ruff
jdrew82 Jan 4, 2024
cf33584
ci: 👷 Replace black, pydocstyle, flake8, and bandit with ruff in CI
jdrew82 Jan 4, 2024
d598f73
test: 🚨 Remove unused linter toggles
jdrew82 Jan 4, 2024
d1e3176
build: ⬆️ Update project dependency lockfile
jdrew82 Jan 5, 2024
b9dc826
build: ⏪️ Revert pylint message control to original messages.
jdrew82 Jan 5, 2024
d77ee7f
build: ➕ Enable isort linter and fix all import sorting issues with r…
jdrew82 Jan 5, 2024
ac1a767
build: 🚨 Enable pydocstyle rules in ruff, copy settings from yml sett…
jdrew82 Jan 5, 2024
9be9c94
build: 🔥 Remove bandit setting file as covered by ruff settings in py…
jdrew82 Jan 5, 2024
b790d30
refactor: ♻️ Update ruff task to match core
jdrew82 Jan 5, 2024
564a127
build: 🙈 Add ruff cache to gitignore
jdrew82 Jan 5, 2024
a49582c
refactor: ♻️ Change random color to pull from ColorChoices
jdrew82 Jan 5, 2024
e0d0dac
refactor: ♻️ Tweak use of mark_safe
jdrew82 Jan 5, 2024
1167a9c
test: 🚨 Fix sorts
jdrew82 Jan 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions .bandit.yml

This file was deleted.

11 changes: 0 additions & 11 deletions .flake8

This file was deleted.

49 changes: 5 additions & 44 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ env:
PLUGIN_NAME: "nautobot-app-ssot"

jobs:
black:
ruff:
runs-on: "ubuntu-22.04"
env:
INVOKE_NAUTOBOT_SSOT_LOCAL: "True"
Expand All @@ -25,41 +25,8 @@ jobs:
uses: "actions/checkout@v4"
- name: "Setup environment"
uses: "networktocode/gh-action-setup-poetry-environment@v4"
- name: "Linting: black"
run: "poetry run invoke black"
bandit:
runs-on: "ubuntu-22.04"
env:
INVOKE_NAUTOBOT_SSOT_LOCAL: "True"
steps:
- name: "Check out repository code"
uses: "actions/checkout@v4"
- name: "Setup environment"
uses: "networktocode/gh-action-setup-poetry-environment@v4"
- name: "Linting: bandit"
run: "poetry run invoke bandit"
pydocstyle:
runs-on: "ubuntu-22.04"
env:
INVOKE_NAUTOBOT_SSOT_LOCAL: "True"
steps:
- name: "Check out repository code"
uses: "actions/checkout@v4"
- name: "Setup environment"
uses: "networktocode/gh-action-setup-poetry-environment@v4"
- name: "Linting: pydocstyle"
run: "poetry run invoke pydocstyle"
flake8:
runs-on: "ubuntu-22.04"
env:
INVOKE_NAUTOBOT_SSOT_LOCAL: "True"
steps:
- name: "Check out repository code"
uses: "actions/checkout@v4"
- name: "Setup environment"
uses: "networktocode/gh-action-setup-poetry-environment@v4"
- name: "Linting: flake8"
run: "poetry run invoke flake8"
- name: "Linting: ruff"
run: "poetry run invoke ruff"
poetry:
runs-on: "ubuntu-22.04"
env:
Expand All @@ -84,12 +51,9 @@ jobs:
run: "poetry run invoke yamllint"
pylint:
needs:
- "bandit"
- "pydocstyle"
- "flake8"
- "poetry"
- "yamllint"
- "black"
- "ruff"
runs-on: "ubuntu-22.04"
strategy:
fail-fast: true
Expand Down Expand Up @@ -127,12 +91,9 @@ jobs:
run: "poetry run invoke pylint"
check-migrations:
needs:
- "bandit"
- "pydocstyle"
- "flake8"
- "poetry"
- "yamllint"
- "black"
- "ruff"
runs-on: "ubuntu-22.04"
strategy:
fail-fast: true
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# Ansible Retry Files
*.retry

# Ruff Stuff
.ruff_cache

# Swap files
*.swp

Expand Down
5 changes: 3 additions & 2 deletions nautobot_ssot/__init__.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
"""App declaration for nautobot_ssot."""
import os
from importlib import metadata
import os

from django.conf import settings
from nautobot.extras.plugins import NautobotAppConfig

from nautobot.core.settings_funcs import is_truthy
from nautobot.extras.plugins import NautobotAppConfig

from nautobot_ssot.integrations.utils import each_enabled_integration_module
from nautobot_ssot.utils import logger
Expand Down
6 changes: 3 additions & 3 deletions nautobot_ssot/contrib.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@
from collections import defaultdict
from dataclasses import dataclass

import pydantic
from diffsync import DiffSyncModel, DiffSync
from diffsync import DiffSync, DiffSyncModel
from django.contrib.contenttypes.models import ContentType
from django.core.exceptions import ValidationError, MultipleObjectsReturned
from django.core.exceptions import MultipleObjectsReturned, ValidationError
from django.db.models import Model
import pydantic
from typing_extensions import get_type_hints


Expand Down
2 changes: 1 addition & 1 deletion nautobot_ssot/filters.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
"""Filtering logic for Sync and SyncLogEntry records."""

import django_filters
from django.db.models import Q
import django_filters

from nautobot.apps.filters import BaseFilterSet

Expand Down
2 changes: 1 addition & 1 deletion nautobot_ssot/forms.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
from django import forms

from nautobot.apps.forms import add_blank_choice
from nautobot.core.forms import BootstrapMixin, BOOLEAN_WITH_BLANK_CHOICES
from nautobot.core.forms import BOOLEAN_WITH_BLANK_CHOICES, BootstrapMixin

from .choices import SyncLogEntryActionChoices, SyncLogEntryStatusChoices
from .models import Sync, SyncLogEntry
Expand Down
1 change: 1 addition & 0 deletions nautobot_ssot/integrations/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""Single Source of Truth integrations."""
1 change: 1 addition & 0 deletions nautobot_ssot/integrations/aci/diffsync/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""ACI integration DiffSync adapters and models."""
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""ACI integration DiffSync adapters."""
29 changes: 17 additions & 12 deletions nautobot_ssot/integrations/aci/diffsync/adapters/aci.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,27 +2,30 @@
# pylint: disable=duplicate-code


from ipaddress import ip_network
import logging
import os
import re
from typing import Optional
from ipaddress import ip_network

from diffsync import DiffSync
from diffsync.exceptions import ObjectNotFound

from nautobot_ssot.integrations.aci.constant import PLUGIN_CFG
from nautobot_ssot.integrations.aci.diffsync.models import NautobotTenant
from nautobot_ssot.integrations.aci.diffsync.models import NautobotVrf
from nautobot_ssot.integrations.aci.diffsync.models import NautobotDeviceType
from nautobot_ssot.integrations.aci.diffsync.models import NautobotDeviceRole
from nautobot_ssot.integrations.aci.diffsync.models import NautobotDevice
from nautobot_ssot.integrations.aci.diffsync.models import NautobotInterfaceTemplate
from nautobot_ssot.integrations.aci.diffsync.models import NautobotInterface
from nautobot_ssot.integrations.aci.diffsync.models import NautobotIPAddress
from nautobot_ssot.integrations.aci.diffsync.models import NautobotPrefix
from nautobot_ssot.integrations.aci.diffsync.client import AciApi
from nautobot_ssot.integrations.aci.diffsync.models import (
NautobotDevice,
NautobotDeviceRole,
NautobotDeviceType,
NautobotInterface,
NautobotInterfaceTemplate,
NautobotIPAddress,
NautobotPrefix,
NautobotTenant,
NautobotVrf,
)
from nautobot_ssot.integrations.aci.diffsync.utils import load_yamlfile


logger = logging.getLogger(__name__)


Expand Down Expand Up @@ -58,6 +61,8 @@ def __init__(self, *args, job=None, sync=None, client, **kwargs):
job (object, optional): Aci job. Defaults to None.
sync (object, optional): Aci DiffSync. Defaults to None.
client (object): Aci credentials.
*args: Variable length argument list.
**kwargs: Arbitrary keyword arguments.
"""
super().__init__(*args, **kwargs)
self.job = job
Expand All @@ -80,7 +85,7 @@ def load_tenants(self):
"""Load tenants from ACI."""
tenant_list = self.conn.get_tenants()
for _tenant in tenant_list:
if not _tenant["name"] in PLUGIN_CFG.get("ignore_tenants"):
if _tenant["name"] not in PLUGIN_CFG.get("ignore_tenants"):
tenant_name = f"{self.tenant_prefix}:{_tenant['name']}"
new_tenant = self.tenant(
name=tenant_name,
Expand Down
36 changes: 22 additions & 14 deletions nautobot_ssot/integrations/aci/diffsync/adapters/nautobot.py
Original file line number Diff line number Diff line change
@@ -1,27 +1,31 @@
"""DiffSync Adapter for Nautobot."""
# pylint: disable=duplicate-code

import logging
from collections import defaultdict
import logging

from diffsync import DiffSync
from diffsync.enum import DiffSyncModelFlags
from django.contrib.contenttypes.models import ContentType
from django.db.models import ProtectedError
from nautobot.tenancy.models import Tenant
from nautobot.dcim.models import DeviceType, Device, InterfaceTemplate, Interface
from nautobot.extras.models import Role

from nautobot.dcim.models import Device, DeviceType, Interface, InterfaceTemplate
from nautobot.extras.models import Role, Tag
from nautobot.ipam.models import IPAddress, Prefix, VRF
from nautobot.extras.models import Tag
from nautobot_ssot.integrations.aci.diffsync.models import NautobotTenant
from nautobot_ssot.integrations.aci.diffsync.models import NautobotVrf
from nautobot_ssot.integrations.aci.diffsync.models import NautobotDeviceType
from nautobot_ssot.integrations.aci.diffsync.models import NautobotDeviceRole
from nautobot_ssot.integrations.aci.diffsync.models import NautobotDevice
from nautobot_ssot.integrations.aci.diffsync.models import NautobotInterfaceTemplate
from nautobot_ssot.integrations.aci.diffsync.models import NautobotInterface
from nautobot_ssot.integrations.aci.diffsync.models import NautobotIPAddress
from nautobot_ssot.integrations.aci.diffsync.models import NautobotPrefix
from nautobot.tenancy.models import Tenant

from nautobot_ssot.integrations.aci.constant import PLUGIN_CFG
from nautobot_ssot.integrations.aci.diffsync.models import (
NautobotDevice,
NautobotDeviceRole,
NautobotDeviceType,
NautobotInterface,
NautobotInterfaceTemplate,
NautobotIPAddress,
NautobotPrefix,
NautobotTenant,
NautobotVrf,
)

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -60,6 +64,8 @@ def __init__(self, *args, job=None, sync=None, client, **kwargs):
job (object, optional): Nautobot job. Defaults to None.
sync (object, optional): Nautobot DiffSync. Defaults to None.
client (object): ACI credentials.
*args: Variable length argument list.
**kwargs: Arbitrary keyword arguments.
"""
super().__init__(*args, **kwargs)
self.job = job
Expand All @@ -76,6 +82,8 @@ def sync_complete(self, source: DiffSync, *args, **kwargs):

Args:
source (DiffSync): DiffSync
*args: Variable length argument list.
**kwargs: Arbitrary keyword arguments.
"""
for grouping in (
"ipaddress",
Expand Down
32 changes: 17 additions & 15 deletions nautobot_ssot/integrations/aci/diffsync/client.py
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
"""All interactions with ACI.""" # pylint: disable=too-many-lines, too-many-instance-attributes, too-many-arguments
# pylint: disable=invalid-name

import sys
from datetime import datetime, timedelta
from ipaddress import ip_network
import logging
from datetime import datetime
from datetime import timedelta
import re
from ipaddress import ip_network
import sys
from typing import Optional

import requests
import urllib3

from .utils import tenant_from_dn, ap_from_dn, node_from_dn, pod_from_dn, fex_id_from_dn, interface_from_dn

from .utils import ap_from_dn, fex_id_from_dn, interface_from_dn, node_from_dn, pod_from_dn, tenant_from_dn

urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

Expand Down Expand Up @@ -59,7 +59,9 @@ def _login(self):
self.refresh_timeout = int(resp.json()["imdata"][0]["aaaLogin"]["attributes"]["refreshTimeoutSeconds"])
return resp

def _handle_request(self, url: str, params: dict = None, request_type: str = "get", data: dict = None) -> object:
def _handle_request(
self, url: str, params: Optional[dict] = None, request_type: str = "get", data: Optional[dict] = None
) -> object:
"""Send a REST API call to the APIC."""
try:
resp = requests.request(
Expand Down Expand Up @@ -92,7 +94,7 @@ def _handle_error(self, response: object):
f"Error: {response.status_code}, Reason: {response.reason}"
)

def _get(self, uri: str, params: dict = None) -> object:
def _get(self, uri: str, params: Optional[dict] = None) -> object:
"""Method to retrieve data from the ACI fabric."""
url = self.base_uri + uri
if self._refresh_token():
Expand All @@ -108,7 +110,7 @@ def _get(self, uri: str, params: dict = None) -> object:
return resp
return self._handle_error(resp)

def _post(self, uri: str, params: dict = None, data=None) -> object:
def _post(self, uri: str, params: Optional[dict] = None, data=None) -> object:
"""Method to post data to the ACI fabric."""
url = self.base_uri + uri
if self._refresh_token():
Expand Down Expand Up @@ -384,16 +386,16 @@ def get_nodes(self) -> dict:
resp = self._get('/api/class/topSystem.json?query-target-filter=ne(topSystem.role,"controller")')

for node in resp.json()["imdata"]:
if node["topSystem"]["attributes"]["oobMgmtAddr"] != "0.0.0.0": # nosec: B104
if node["topSystem"]["attributes"]["oobMgmtAddr"] != "0.0.0.0": # noqa: S104
mgmt_addr = f"{node['topSystem']['attributes']['oobMgmtAddr']}/{node['topSystem']['attributes']['oobMgmtAddrMask']}"
elif (
node["topSystem"]["attributes"]["address"] != "0.0.0.0" # nosec: B104
node["topSystem"]["attributes"]["address"] != "0.0.0.0" # noqa: S104
and node["topSystem"]["attributes"]["tepPool"]
):
mgmt_addr = f"{node['topSystem']['attributes']['address']}/{ip_network(node['topSystem']['attributes']['tepPool'], strict=False).prefixlen}"
else:
mgmt_addr = ""
if node["topSystem"]["attributes"]["tepPool"] != "0.0.0.0": # nosec: B104
if node["topSystem"]["attributes"]["tepPool"] != "0.0.0.0": # noqa: S104
subnet = node["topSystem"]["attributes"]["tepPool"]
elif mgmt_addr:
subnet = ip_network(mgmt_addr, strict=False).with_prefixlen
Expand Down Expand Up @@ -437,16 +439,16 @@ def get_controllers(self) -> dict:
node_dict[node_id]["site"] = self.site
resp = self._get('/api/class/topSystem.json?query-target-filter=eq(topSystem.role,"controller")')
for node in resp.json()["imdata"]:
if node["topSystem"]["attributes"]["oobMgmtAddr"] != "0.0.0.0": # nosec: B104
if node["topSystem"]["attributes"]["oobMgmtAddr"] != "0.0.0.0": # noqa: S104
mgmt_addr = f"{node['topSystem']['attributes']['oobMgmtAddr']}/{node['topSystem']['attributes']['oobMgmtAddrMask']}"
elif (
node["topSystem"]["attributes"]["address"] != "0.0.0.0" # nosec: B104
node["topSystem"]["attributes"]["address"] != "0.0.0.0" # noqa: S104
and node["topSystem"]["attributes"]["tepPool"]
):
mgmt_addr = f"{node['topSystem']['attributes']['address']}/{ip_network(node['topSystem']['attributes']['tepPool'], strict=False).prefixlen}"
else:
mgmt_addr = ""
if node["topSystem"]["attributes"]["tepPool"] != "0.0.0.0": # nosec: B104
if node["topSystem"]["attributes"]["tepPool"] != "0.0.0.0": # noqa: S104
subnet = node["topSystem"]["attributes"]["tepPool"]
elif mgmt_addr:
subnet = ip_network(mgmt_addr, strict=False).with_prefixlen
Expand Down
8 changes: 4 additions & 4 deletions nautobot_ssot/integrations/aci/diffsync/models/__init__.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
"""Initialize models for Nautobot and ACI."""
from .nautobot import (
NautobotTenant,
NautobotVrf,
NautobotDevice,
NautobotDeviceRole,
NautobotDeviceType,
NautobotInterfaceTemplate,
NautobotInterface,
NautobotPrefix,
NautobotInterfaceTemplate,
NautobotIPAddress,
NautobotPrefix,
NautobotTenant,
NautobotVrf,
)

__all__ = [
Expand Down
Loading