Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: ✨ Decommissioning Job #85

Merged
merged 43 commits into from
Jan 4, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
62559fe
Updates pyproject.toml for 3.11
jvanderaa Aug 1, 2023
625e680
fix: Updates for OSRB documentation review
abates Oct 23, 2023
b264d0a
docs: The custom `indent` filter was removed in favor of using the bu…
abates Oct 23, 2023
6faec79
docs: Documentation updates
abates Oct 24, 2023
1c96d02
Merge branch 'develop' into support311
abates Dec 7, 2023
85e4ed0
docs: Removed unneeded doc template
abates Dec 11, 2023
b3950ac
Merge branch 'develop' into osrb-docs-review
abates Dec 11, 2023
d88f578
feat: :sparkles: Decommissioning Job
chadell Dec 12, 2023
1bd206d
Apply suggestions from code review
chadell Dec 13, 2023
e96367c
Merge pull request #81 from networktocode-llc/osrb-docs-review
abates Dec 13, 2023
b11e238
Merge pull request #62 from networktocode-llc/support311
abates Dec 13, 2023
1999e21
Decouple pre decommission job
chadell Dec 14, 2023
8a3f890
use callable
chadell Dec 14, 2023
59ea087
generalize hook
chadell Dec 15, 2023
d106fa7
wip
chadell Dec 15, 2023
ca01bd0
wip
chadell Dec 18, 2023
0962021
fix: Fixed test failures.
abates Dec 18, 2023
08aba58
refactor: JournalEntry can now be reverted.
abates Dec 19, 2023
ea007da
fix tests
chadell Dec 19, 2023
0121d1b
fix dict logic
chadell Dec 19, 2023
11eea60
docs: Documented why refreshing `design_object` is necessary
abates Dec 19, 2023
a17cf92
refactor: Refactored revert code into `Journal` model
abates Dec 19, 2023
c8b7faf
docs: Updated branding from `plugin` to `app`
abates Jan 2, 2024
8333422
Merge branch 'osrb' into feature_delices_decom_job
abates Jan 2, 2024
92ee1ed
refactor: Refactored decom code to model and hooks to signals
abates Jan 2, 2024
f1fe686
style: Autoformatting
abates Jan 2, 2024
2132ac9
Add l3vpn design example
chadell Jan 2, 2024
63ea09f
fix old dict value null
chadell Jan 2, 2024
5efb783
Merge branch 'feature_delices_decom_job' of github.com:networktocode-…
chadell Jan 2, 2024
f911eec
refactor: Minor refactoring of `JournalEntry` revert and the model it…
abates Jan 2, 2024
42e4e19
fix tests
chadell Jan 3, 2024
d8ad207
update the l3vppn example and add a hook to validate input data
chadell Jan 3, 2024
fbad8f5
bump version
chadell Jan 3, 2024
9bf5ff4
avoid overwrite of method
chadell Jan 3, 2024
1d665ea
clean up some leftovers
chadell Jan 3, 2024
ccdd49f
Missing part of previous commit
chadell Jan 3, 2024
4513c5e
Rename variables for consistency
chadell Jan 3, 2024
192874f
fix: Fixed extra `{% endmacro %}` that got added at some point.
abates Jan 3, 2024
b39a54f
fix: Now logging the design instance and journal objects.
abates Jan 3, 2024
1cc2184
Adjust test with warning, improve logging plus journalentry retrieve …
chadell Jan 4, 2024
358390c
adjust logging
chadell Jan 4, 2024
4aabdce
mre info
chadell Jan 4, 2024
31de2a4
fix exception chain
chadell Jan 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
---
regions:
- "!update:name": "{{ region.name }}"
description: "new description for testing"

sites:
- name: "{{ site_name }}"
region__slug: "{{ region.slug }}"
Expand Down Expand Up @@ -51,6 +55,15 @@ racks:
status__name: "Reserved"
{% endmacro %}

{% macro device_edit(device_name) -%}
- "!update:name": "{{ device_name }}"
local_context_data: {
"snmp-location": "some-cool-place",
}

{% endmacro %}

devices:
{{ device("core01", "rack1", core_1_loopback) }}
{{ device("core02", "rack2", core_2_loopback) }}
{{ device_edit("core02.iad5") }}
chadell marked this conversation as resolved.
Show resolved Hide resolved
6 changes: 6 additions & 0 deletions nautobot_design_builder/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,5 +33,11 @@ def context_repository(cls):
"""Retrieve the Git Repository slug that has been configured for the Design Builder."""
return settings.PLUGINS_CONFIG[cls.name]["context_repository"]

# pylint: disable=no-self-argument
@classproperty
def pre_decommission_hook(cls) -> str:
"""Retrieve the pre decommission hook callable for the Design Builder, if configured."""
return settings.PLUGINS_CONFIG[cls.name].get("pre_decommission_hook", "")


config = DesignBuilderConfig # pylint:disable=invalid-name
166 changes: 166 additions & 0 deletions nautobot_design_builder/jobs.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
"""Generic Design Builder Jobs."""
from django.contrib.contenttypes.models import ContentType
from django.utils.module_loading import import_string

from nautobot.extras.models import Status
from nautobot.extras.jobs import Job, MultiObjectVar

from nautobot_design_builder import DesignBuilderConfig
from nautobot_design_builder.models import DesignInstance, JournalEntry
from nautobot_design_builder.choices import DesignInstanceStatusChoices


class DesignInstanceDecommissioning(Job):
"""Job to decommission Design Instances."""

design_instances = MultiObjectVar(
model=DesignInstance,
query_params={"status": "active"},
description="Design Instances to decommission.",
)

class Meta: # pylint: disable=too-few-public-methods
"""Meta class."""

name = "Decommission Design Instances."
description = """Job to decommission one or many Design Instances from Nautobot."""

def _proceed_after_pre_decommission_hook(self, design_instance):
"""If configured, run a pre decomission hook.

It should return True if it's good to go, or False and the reason of the failure.
"""
pre_decommission_hook = DesignBuilderConfig.pre_decommission_hook
if not pre_decommission_hook:
return True

self.log_info(
f"Checking if the design instance {design_instance} can be decommissioned by external dependencies."
)

try:
func = import_string(pre_decommission_hook)
except ImportError as error:
msg = (
"There was an issue attempting to import the pre decommission hook "
f"{pre_decommission_hook}, this is expected with a local configuration issue and not related to"
" the Design Builder App, please contact your system admin for further details.\n"
)
raise ValueError(msg + str(error)) from error
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of this approach - does it make sense/is it possible to instead just pass the callable directly? I.e. have the settings import it and then just use it here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, it would make it easier. I will try it.


result, reason = func(design_instance)

if not result:
self.log_failure(f"The pre hook validation failed due to: {reason}")
return False

self.log_success(f"No dependency issues found for {design_instance}.")
return True

def _process_journal_entry_with_full_control(self, journal_entry):
"""It takes care of decommission changes for objects with full control.

Returns True if the decommissioning was successful.
"""
other_journal_entries = (
JournalEntry.objects.filter(_design_object_id=journal_entry.design_object.id)
.exclude(id=journal_entry.id)
.exclude(journal__design_instance__status__name=DesignInstanceStatusChoices.DECOMMISSIONED)
)

if other_journal_entries:
self.log_failure(
journal_entry.design_object,
message=(
"This object is referenced by other active Journals: ",
f"{list(other_journal_entries.values_list('id', flat=True))}",
),
)
return False

journal_entry.design_object.delete()

self.log_success(obj=journal_entry.design_object, message=f"Object {journal_entry.design_object} removed.")

return True

def _process_journal_entry_without_full_control(self, journal_entry):
"""It takes care of decommission changes for objects without full control.

Returns True if the decommissioning was successful.
"""
for attribute in journal_entry.changes["differences"].get("added", {}):
value_changed = journal_entry.changes["differences"]["added"][attribute]
old_value = journal_entry.changes["differences"]["removed"][attribute]
if isinstance(value_changed, dict):
# If the value is a dictionary (e.g., config context), we only update the
# keys changed, honouring the current value of the attribute
current_value = getattr(journal_entry.design_object, attribute)
keys_to_remove = []
for key in current_value:
if key in value_changed:
if old_value:
current_value = old_value[key]
else:
keys_to_remove.append(key)
for key in keys_to_remove:
del current_value[key]
setattr(journal_entry.design_object, attribute, current_value)
else:
setattr(journal_entry.design_object, attribute, old_value)

journal_entry.design_object.save()

self.log_success(
obj=journal_entry.design_object,
message="Because full control is not given, we have restored the object to its previous state.",
)

return True

def run(self, data, commit):
"""Execute job."""
design_instances = data["design_instances"]
self.log_info(
message=f"Starting decommissioning of design instances: {', '.join([instance.name for instance in design_instances])}",
)

found_cross_references = False

for design_instance in design_instances:
if not self._proceed_after_pre_decommission_hook(design_instance):
self.log_warning(design_instance, message="Dependency issues found to this Design Instance.")
continue

self.log_info(obj=design_instance, message="Working on resetting objects for this Design Instance...")

# TODO: When update mode is available, this should cover the journals stacked
latest_journal = design_instance.journal_set.order_by("created").last()
self.log_info(latest_journal, "Journal to be decommissioned.")

for journal_entry in latest_journal.entries.exclude(_design_object_id=None).order_by("-last_updated"):
self.log_debug(f"Decommissioning changes for {journal_entry.design_object}.")

if journal_entry.full_control:
# With full control, we can delete the design_object is there are no active references by other Journals
if not self._process_journal_entry_with_full_control(journal_entry):
found_cross_references = True
else:
# If we don't have full control, we recover the value of the items changed to the previous value
self._process_journal_entry_without_full_control(journal_entry)

content_type = ContentType.objects.get_for_model(DesignInstance)
design_instance.status = Status.objects.get(
content_types=content_type, name=DesignInstanceStatusChoices.DECOMMISSIONED
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
content_type = ContentType.objects.get_for_model(DesignInstance)
design_instance.status = Status.objects.get(
content_types=content_type, name=DesignInstanceStatusChoices.DECOMMISSIONED
)
design_instance.status = Status.objects.get(
name=DesignInstanceStatusChoices.DECOMMISSIONED
)

This should be the same, shouldn't it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

likely. I guess a Status can't have the same name and belong to a different content_type. will check

design_instance.save()

if found_cross_references:
raise ValueError(
"Because of cross-references between design instances, decommissioning has been cancelled."
)

self.log_success(f"{design_instance} has been successfully decommissioned from Nautobot.")


jobs = (DesignInstanceDecommissioning,)
10 changes: 9 additions & 1 deletion nautobot_design_builder/tables.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ class DesignTable(BaseTable):

job = Column(linkify=True)
name = Column(linkify=True)
instance_count = Column(accessor=Accessor("instance_count"), verbose_name="Instances")
instance_count = Column(linkify=True, accessor=Accessor("instance_count"), verbose_name="Instances")
actions = ButtonsColumn(Design, buttons=("changelog",), prepend_template=DESIGNTABLE)

class Meta(BaseTable.Meta): # pylint: disable=too-few-public-methods
Expand All @@ -29,6 +29,13 @@ class Meta(BaseTable.Meta): # pylint: disable=too-few-public-methods
fields = ("name", "job", "instance_count")


DESIGNINSTANCETABLE = """
<a href="{% url "extras:job" class_path="plugins/nautobot_design_builder.jobs/DesignInstanceDecommissioning" %}?design_instances={{record.pk}}" class="btn btn-xs btn-primary" title="Decommission">
<i class="mdi mdi-delete-sweep"></i>
</a>
"""


class DesignInstanceTable(StatusTableMixin, BaseTable):
"""Table for list view."""

Expand All @@ -41,6 +48,7 @@ class DesignInstanceTable(StatusTableMixin, BaseTable):
"delete",
"changelog",
),
prepend_template=DESIGNINSTANCETABLE,
)

class Meta(BaseTable.Meta): # pylint: disable=too-few-public-methods
Expand Down
Loading