diff --git a/README.md b/README.md index 86cd5e80..0883879d 100644 --- a/README.md +++ b/README.md @@ -15,6 +15,8 @@ Design Builder is a Nautobot application for easily populating data within Nautobot using standardized design files. These design files are just Jinja templates that describe the Nautobot objects to be created or updated. +It also introduces the concept of a design-oriented Source of Truth with a complete lifecycle management of the design deployments (i.e., an instantiation of a design with concrete input data). With this approach, the users of the application can not only create (or populate) data within Nautobot but also update or decommission it while enforcing data protection and dependency. + ## Documentation Full documentation for this App can be found over on the [Nautobot Docs](https://docs.nautobot.com) website: diff --git a/development/nautobot_config.py b/development/nautobot_config.py index b04a2f43..91126021 100644 --- a/development/nautobot_config.py +++ b/development/nautobot_config.py @@ -25,7 +25,8 @@ if "debug_toolbar.middleware.DebugToolbarMiddleware" not in MIDDLEWARE: # noqa: F405 MIDDLEWARE.insert(0, "debug_toolbar.middleware.DebugToolbarMiddleware") # noqa: F405 -MIDDLEWARE.insert(0, "nautobot_design_builder.middleware.GlobalRequestMiddleware") # noqa: F405 +if "nautobot_design_builder.middleware.GlobalRequestMiddleware" not in MIDDLEWARE: # noqa: F405 + MIDDLEWARE.insert(0, "nautobot_design_builder.middleware.GlobalRequestMiddleware") # noqa: F405 # # Misc. settings diff --git a/docs/user/app_getting_started.md b/docs/user/app_getting_started.md index 4c6dccc3..8e0aa8b3 100644 --- a/docs/user/app_getting_started.md +++ b/docs/user/app_getting_started.md @@ -12,6 +12,8 @@ The easiest way to experience Design Builder is to either add the [demo-designs] ## What are the next steps? + + The Design Builder demo designs ship with some sample designs to demonstrate capabilities. Once the application stack is ready, you should have several jobs listed under the "Jobs" -> "Jobs" menu item. ![Jobs list](../images/screenshots/sample-design-jobs-list.png) diff --git a/docs/user/app_overview.md b/docs/user/app_overview.md index d7f79768..c1b31c2c 100644 --- a/docs/user/app_overview.md +++ b/docs/user/app_overview.md @@ -7,12 +7,15 @@ This document provides an overview of the App including critical information and ## Description -Design Builder provides a system where standardized network designs can be developed to produce collections of objects within Nautobot. These designs are text based templates that can create and update hierarchical data structures within Nautobot. +Design Builder provides a system where standardized network designs can be developed to produce or update collections of objects within Nautobot. These designs are text based templates that can create and update hierarchical data structures within Nautobot. + +The deployment of a design comes with a complete lifecycle management of all the changes connected as a single entity. Thus, the design deployment can be updated or decommissioned after its creation, and the all the changes introduced can be honored when accessing the data outside of the design builder app. ## Audience (User Personas) - Who should use this App? - Network engineers who want to have reproducible sets of Nautobot objects based on some standard design. - Automation engineers who want to be able to automate the creation of Nautobot objects based on a set of standard designs. +- Users who want to leverage abstracted network services defined by network engineers in a simplfied way. ## Authors and Maintainers diff --git a/docs/user/design_development.md b/docs/user/design_development.md index 19bb8c9e..6eac1c71 100644 --- a/docs/user/design_development.md +++ b/docs/user/design_development.md @@ -114,6 +114,18 @@ The value of the `context_class` metadata attribute should be any Python class t This attribute is optional. A report is a Jinja template that is rendered once the design has been implemented. Like `design_file` the design builder will look for this template relative to the filename that defines the design job. This is helpful to generate a custom view of the data that was built during the design build. +### `version` + +It's an optional string attribute that is used to define the versioning reference of a design job. This will enable in the future the versioning lifecycle of design deployments. For example, one a design evolves from one version to another, the design deployment will be able to accommodate the new changes. + +### `description` + +This optional attribute that is a string that provides a high-level overview of the intend of the design job. This description is displayed int the design detail view. + +### `docs` + +This attribute is also displayed on the design detail view. The `docs` attribute can utilize markdown format and should provide more detailed information than the description. This should help the users of the `Design` to understand the goal of the design and the impact of the input data. + ## Design Context Primary Purpose: @@ -339,41 +351,3 @@ class DesignJobWithExtensions(DesignJob): extensions = [ext.BGPPeeringExtension] ``` -## Design LifeCycle - -Design implementations can have a full life cycle: creation, update, and decommission. - - - -Once a design is "deployed" in Nautobot, a Design Instance is created with the report of the changes implemented, and with actions to decommission or update it. - -### Design Decommission - -This feature allows to rollback all the changes implemented by a design instance to the previous state. This rollback depends on the scope of the change: - -- If the object was created by the design implementation, this object will be removed. -- If only some attributes were changes, the affected attributes will be rolled back to the previous state. - -The decommissioning feature takes into account potential dependencies between design implementations. For example, if a new l3vpn design depends on devices that were created by another design, this previous design won't be decommissioned until the l3vpn dependencies are also decommissioned to warrant consistency. - -Once a design instance is decommissioned, it's still visible in the API/UI to check the history of changes but without any active relationship with Nautobot objects. After decommissioning, the design instance can be deleted completely from Nautobot. - -### Design Updates - -This feature allows to re run a design instance with different input data to update the implemented design with the new changes: additions and removals. - -It leverages a complete tracking of previous design implementation and a reduce function for the new design to understand the changes to be implemented and the objects to be decommissioned (leveraging the previous decommissioning feature for only a specific object). - -The update feature comes with a few assumptions: - -- All the design objects that have an identifier have to use identifier keys to identify the object to make them comparable across designs. -- Object identifiers should keep consistent in multiple design runs. For example, you can't target a device with the device name and update the name on the same design. -- When design provides a list of objects, the objects are assumed to be in the same order. For example, if the first design creates `[deviceA1, deviceB1]`, if expanded, it should be `[deviceA1, deviceB1, deviceA2, deviceB2]`, not `[deviceA1, deviceA2, deviceB1, deviceB2]`. - - diff --git a/docs/user/design_lifecycle.md b/docs/user/design_lifecycle.md new file mode 100644 index 00000000..2584c0fe --- /dev/null +++ b/docs/user/design_lifecycle.md @@ -0,0 +1,67 @@ +# Design LifeCycle + + + +According to a design-oriented approach, the Design Builder App provides not only the capacity to create and update data in Nautobot but also a complete lifecycle management of each deployment: update, versioning (in the future), and decommissioning. + + + +All the Design Builder UI navigation menus are under the Design Builder tab. + +## `Design` + +A `Design` is a one to one mapping with a Nautobot `Job`, enriched with some data from the Design Builder `DesignJob` definition. In concrete, it stores: + +- A `Job` reference. +- A `version` string from the `DesignJob`. +- A `description` string from the `DesignJob`. +- A `docs` string from the `DesignJob`. + + + +From the `Design`, the user can manage the associated `Job`, and trigger its execution, which creates a `DesignInstance` or Design Deployment + +## Design Deployment or `DesignInstance` + +Once a design is "deployed" in Nautobot, a Design Deployment (or `DesignInstance`) is created with the report of the changes implemented (i.e. `Journals`), and with actions to update or decommission it (see next subsections). + +The `DesignInstance` stores: + +- The `name` of the deployment, within the context of the `Design`. +- The `Design` reference. +- The `version` from the `Design` when it was deployed or updated. +- When it was initially deployed or last updated. +- The `status` of the design, and the `live_state` or operational status to signal its state in the actual network. + + + +### Design Deployment Update + +This feature provides a means to re-run a design instance with different input data. Re-running the job will update the implemented design with the new changes: additions and removals. + +It leverages a complete tracking of previous design implementations and a function to combine the new design and previous design, to understand the changes to be implemented and the objects to be decommissioned (leveraging the previous decommissioning feature for only a specific object). + +The update feature comes with a few assumptions: + +- All the design objects that have an identifier have to use identifier keys to identify the object to make them comparable across designs. +- Object identifiers should keep consistent in multiple design runs. For example, you can't target a device with the device name and update the name on the same design. +- When design provides a list of objects, the objects are assumed to be in the same order. For example, if the first design creates `[deviceA1, deviceB1]`, if expanded, it should be `[deviceA1, deviceB1, deviceA2, deviceB2]`, not `[deviceA1, deviceA2, deviceB1, deviceB2]`. + + + +### Design Deployment Decommission + +This feature allows to rollback all the changes implemented by a design instance to the previous state. This rollback depends on the scope of the change: + +- If the object was created by the design implementation, this object will be removed. +- If only some attributes were changes, the affected attributes will be rolled back to the previous state. + +The decommissioning feature takes into account potential dependencies between design implementations. For example, if a new l3vpn design depends on devices that were created by another design, this previous design won't be decommissioned until the l3vpn dependencies are also decommissioned to warrant consistency. + +Once a design instance is decommissioned, it's still visible in the API/UI to check the history of changes but without any active relationship with Nautobot objects. After decommissioning, the design instance can be deleted completely from Nautobot. diff --git a/docs/user/design_quickstart.md b/docs/user/design_quickstart.md index 545556ef..fed93c42 100644 --- a/docs/user/design_quickstart.md +++ b/docs/user/design_quickstart.md @@ -10,6 +10,8 @@ To add a new design you will need (at a minimum) a class extending `nautobot_des For more information on creating designs see [Getting Started with Designs](design_development.md). +Once the designs are loaded, you can start managing them from the "Design Builder" navigation tab. + ## Sample Data Much of the time, designs will need some data to exist in Nautobot before they can be built. In a development and testing environment it is necessary to generate this data for testing purposes. The Design Builder application comes with a `load_design` management command that will read a design YAML file (not a template) and will build the design in Nautobot. This can be used to produce sample data for a development environment. Simply create a YAML file that includes all of the object definitions needed for testing and load the file with `invoke build-design `. This should read the file and build all of the objects within Nautobot. diff --git a/examples/custom_design/designs/initial_data/jobs.py b/examples/custom_design/designs/initial_data/jobs.py index 39f01ee1..941719d4 100644 --- a/examples/custom_design/designs/initial_data/jobs.py +++ b/examples/custom_design/designs/initial_data/jobs.py @@ -19,3 +19,13 @@ class Meta: commit_default = False design_file = "designs/0001_design.yaml.j2" context_class = InitialDesignContext + version = "1.0.0" + description = "Establish the devices and site information for four sites: IAD5, LGA1, LAX11, SEA11." + docs = """This design creates the following objects in the source of truth to establish the initia network environment in four sites: IAD5, LGA1, LAX11, SEA11. + +These sites belong to the America region (and different subregions), and use Juniper PTX10016 devices. + +The user input data is: + - Number of devices per site (integer) + - The description for one of the regions (string) +""" diff --git a/mkdocs.yml b/mkdocs.yml index 9c07e15c..8c0173d4 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -107,6 +107,7 @@ nav: - Getting Started: "user/app_getting_started.md" - Design Quick Start: "user/design_quickstart.md" - Design Development: "user/design_development.md" + - Design LifeCycle: "user/design_lifecycle.md" - Frequently Asked Questions: "user/faq.md" - Git-based Config Context: "user/git_config_context.md" - Administrator Guide: diff --git a/nautobot_design_builder/api/serializers.py b/nautobot_design_builder/api/serializers.py index a34daf8f..3545ca47 100644 --- a/nautobot_design_builder/api/serializers.py +++ b/nautobot_design_builder/api/serializers.py @@ -40,6 +40,8 @@ class DesignInstanceSerializer(NautobotModelSerializer, TaggedModelSerializerMix url = HyperlinkedIdentityField(view_name="plugins-api:nautobot_design_builder-api:design-detail") design = NestedDesignSerializer() live_state = NestedStatusSerializer() + created_by = SerializerMethodField() + last_updated_by = SerializerMethodField() class Meta: """Serializer options for the design model.""" @@ -50,13 +52,22 @@ class Meta: "url", "design", "name", - "owner", + "created_by", "first_implemented", + "last_updated_by", "last_implemented", "status", "live_state", ] + def get_created_by(self, instance): + """Get the username of the user who created the object.""" + return instance.created_by + + def get_last_updated_by(self, instance): + """Get the username of the user who update the object last time.""" + return instance.last_updated_by + class JournalSerializer(NautobotModelSerializer, TaggedModelSerializerMixin): """Serializer for the journal model.""" diff --git a/nautobot_design_builder/design.py b/nautobot_design_builder/design.py index b9c67bdf..6dd5724e 100644 --- a/nautobot_design_builder/design.py +++ b/nautobot_design_builder/design.py @@ -617,7 +617,6 @@ def get_extension(self, ext_type: str, tag: str) -> ext.Extension: extn["object"] = extn["class"](self) return extn["object"] - # TODO: this is a breaking change that needs to be revisited because it's used by Django commands directly @transaction.atomic def implement_design_changes(self, design: Dict, deprecated_design: Dict, design_file: str, commit: bool = False): """Iterates through items in the design and creates them. diff --git a/nautobot_design_builder/design_job.py b/nautobot_design_builder/design_job.py index 7478b656..4b47e5a9 100644 --- a/nautobot_design_builder/design_job.py +++ b/nautobot_design_builder/design_job.py @@ -24,7 +24,7 @@ from nautobot_design_builder.context import Context from nautobot_design_builder import models from nautobot_design_builder import choices -from nautobot_design_builder.recursive import reduce_design +from nautobot_design_builder.recursive import combine_designs from .util import nautobot_version @@ -38,7 +38,6 @@ class DesignJob(Job, ABC, LoggingMixin): # pylint: disable=too-many-instance-at """ instance_name = StringVar(label="Instance Name", max_length=models.DESIGN_NAME_MAX_LENGTH) - owner = StringVar(label="Implementation Owner", required=False, max_length=models.DESIGN_OWNER_MAX_LENGTH) if nautobot_version >= "2.0.0": from nautobot.extras.jobs import DryRunVar # pylint: disable=no-name-in-module,import-outside-toplevel @@ -183,14 +182,14 @@ def implement_design(self, context, design_file, commit): for key, new_value in design.items(): old_value = previous_design[key] future_value = self.builder.builder_output[design_file][key] - reduce_design(new_value, old_value, future_value, deprecated_design, key) + combine_designs(new_value, old_value, future_value, deprecated_design, key) self.log_debug(f"Design to implement after reduction: {design}") self.log_debug(f"Design to deprecate after reduction: {deprecated_design}") self.builder.implement_design_changes(design, deprecated_design, design_file, commit) - def _setup_journal(self, instance_name: str, design_owner: str): + def _setup_journal(self, instance_name: str): try: instance = models.DesignInstance.objects.get(name=instance_name, design=self.design_model()) self.log_info(message=f'Existing design instance of "{instance_name}" was found, re-running design job.') @@ -200,13 +199,13 @@ def _setup_journal(self, instance_name: str, design_owner: str): content_type = ContentType.objects.get_for_model(models.DesignInstance) instance = models.DesignInstance( name=instance_name, - owner=design_owner, design=self.design_model(), last_implemented=datetime.now(), status=Status.objects.get(content_types=content_type, name=choices.DesignInstanceStatusChoices.ACTIVE), live_state=Status.objects.get( content_types=content_type, name=choices.DesignInstanceLiveStateChoices.PENDING ), + version=self.design_model().version, ) instance.validated_save() @@ -238,7 +237,7 @@ def run(self, **kwargs): # pylint: disable=arguments-differ,too-many-branches,t else: self.job_result.job_kwargs = self.serialize_data(data) - journal = self._setup_journal(data.pop("instance_name"), data.pop("owner")) + journal = self._setup_journal(data.pop("instance_name")) self.log_info(message=f"Building {getattr(self.Meta, 'name')}") extensions = getattr(self.Meta, "extensions", []) self.builder = Builder( diff --git a/nautobot_design_builder/filters.py b/nautobot_design_builder/filters.py index 66d2cec3..9488711a 100644 --- a/nautobot_design_builder/filters.py +++ b/nautobot_design_builder/filters.py @@ -40,7 +40,16 @@ class Meta: """Meta attributes for filter.""" model = DesignInstance - fields = ["id", "design", "name", "owner", "first_implemented", "last_implemented", "status", "live_state"] + fields = [ + "id", + "design", + "name", + "first_implemented", + "last_implemented", + "status", + "live_state", + "version", + ] class JournalFilterSet(NautobotFilterSet): @@ -50,7 +59,7 @@ class JournalFilterSet(NautobotFilterSet): design_instance = NaturalKeyOrPKMultipleChoiceFilter( queryset=DesignInstance.objects.all(), - label="Design Instance (ID)", + label="Design Deployment (ID)", ) job_result = NaturalKeyOrPKMultipleChoiceFilter( diff --git a/nautobot_design_builder/forms.py b/nautobot_design_builder/forms.py index fe45868e..e36dbeed 100644 --- a/nautobot_design_builder/forms.py +++ b/nautobot_design_builder/forms.py @@ -1,6 +1,6 @@ """Forms for the design builder app.""" -from django.forms import NullBooleanField +from django.forms import NullBooleanField, CharField from nautobot.extras.forms import NautobotFilterForm from nautobot.extras.models import Job, JobResult from nautobot.utilities.forms import TagFilterField, DynamicModelChoiceField, StaticSelect2, BOOLEAN_WITH_BLANK_CHOICES @@ -13,8 +13,9 @@ class DesignFilterForm(NautobotFilterForm): model = Design - job = DynamicModelChoiceField(queryset=Job.objects.all()) + job = DynamicModelChoiceField(queryset=Job.objects.all(), required=False) tag = TagFilterField(model) + version = CharField(max_length=20, required=False) class DesignInstanceFilterForm(NautobotFilterForm): @@ -24,6 +25,7 @@ class DesignInstanceFilterForm(NautobotFilterForm): design = DynamicModelChoiceField(queryset=Design.objects.all()) tag = TagFilterField(model) + version = CharField(max_length=20, required=False) class JournalFilterForm(NautobotFilterForm): diff --git a/nautobot_design_builder/jobs.py b/nautobot_design_builder/jobs.py index 24f6d385..8e9952be 100644 --- a/nautobot_design_builder/jobs.py +++ b/nautobot_design_builder/jobs.py @@ -6,20 +6,23 @@ from .models import DesignInstance +name = "Design Builder" # pylint: disable=invalid-name + + class DesignInstanceDecommissioning(Job): """Job to decommission Design Instances.""" design_instances = MultiObjectVar( model=DesignInstance, query_params={"status": "active"}, - description="Design Instances to decommission.", + description="Design Deployments to decommission.", ) class Meta: # pylint: disable=too-few-public-methods """Meta class.""" - name = "Decommission Design Instances." - description = """Job to decommission one or many Design Instances from Nautobot.""" + name = "Decommission Design Deployments" + description = """Job to decommission one or many Design Deployments from Nautobot.""" def run(self, data, commit): """Execute Decommissioning job.""" diff --git a/nautobot_design_builder/migrations/0005_auto_20240415_0455.py b/nautobot_design_builder/migrations/0005_auto_20240415_0455.py new file mode 100644 index 00000000..587582ab --- /dev/null +++ b/nautobot_design_builder/migrations/0005_auto_20240415_0455.py @@ -0,0 +1,30 @@ +# Generated by Django 3.2.20 on 2024-04-15 04:55 + +from django.db import migrations, models + + +class Migration(migrations.Migration): + dependencies = [ + ("nautobot_design_builder", "0004_support_update_design"), + ] + + operations = [ + migrations.AlterModelOptions( + name="designinstance", + options={"verbose_name": "Design Deployment", "verbose_name_plural": "Design Deployments"}, + ), + migrations.RemoveField( + model_name="designinstance", + name="owner", + ), + migrations.AddField( + model_name="designinstance", + name="version", + field=models.CharField(blank=True, default="", max_length=20), + ), + migrations.AlterField( + model_name="designinstance", + name="name", + field=models.CharField(max_length=255), + ), + ] diff --git a/nautobot_design_builder/models.py b/nautobot_design_builder/models.py index 9426ffd6..8ce2e6ea 100644 --- a/nautobot_design_builder/models.py +++ b/nautobot_design_builder/models.py @@ -16,7 +16,7 @@ from nautobot.utilities.querysets import RestrictedQuerySet from nautobot.utilities.choices import ColorChoices -from .util import nautobot_version +from .util import nautobot_version, get_created_and_last_updated_usernames_for_model from . import choices from .errors import DesignValidationError @@ -103,11 +103,9 @@ class Design(PrimaryModel): to a saved graphql query at some point in the future. """ - # TODO: Add version field (future feature) # TODO: Add saved graphql query (future feature) # TODO: Add a template mapping to get custom payload (future feature) job = models.ForeignKey(to=JobModel, on_delete=models.PROTECT, editable=False) - objects = DesignQuerySet.as_manager() class Meta: @@ -139,6 +137,27 @@ def __str__(self): """Stringify instance.""" return self.name + @property + def description(self): + """Get the description from the Job.""" + if self.job.job_class and hasattr(self.job.job_class.Meta, "description"): + return self.job.job_class.Meta.description + return "" + + @property + def version(self): + """Get the version from the Job.""" + if self.job.job_class and hasattr(self.job.job_class.Meta, "version"): + return self.job.job_class.Meta.version + return "" + + @property + def docs(self): + """Get the docs from the Job.""" + if self.job.job_class and hasattr(self.job.job_class.Meta, "docs"): + return self.job.job_class.Meta.docs + return "" + class DesignInstanceQuerySet(RestrictedQuerySet): """Queryset for `DesignInstance` objects.""" @@ -148,9 +167,7 @@ def get_by_natural_key(self, design_name, instance_name): return self.get(design__job__name=design_name, name=instance_name) -DESIGN_NAME_MAX_LENGTH = 100 - -DESIGN_OWNER_MAX_LENGTH = 100 +DESIGN_NAME_MAX_LENGTH = 255 @extras_features("statuses") @@ -167,14 +184,12 @@ class DesignInstance(PrimaryModel, StatusModel): post_decommission = Signal() - # TODO: add version field to indicate which version of a design - # this instance is on. (future feature) design = models.ForeignKey(to=Design, on_delete=models.PROTECT, editable=False, related_name="instances") name = models.CharField(max_length=DESIGN_NAME_MAX_LENGTH) - owner = models.CharField(max_length=DESIGN_OWNER_MAX_LENGTH, blank=True, default="") first_implemented = models.DateTimeField(blank=True, null=True, auto_now_add=True) last_implemented = models.DateTimeField(blank=True, null=True) live_state = StatusField(blank=False, null=False, on_delete=models.PROTECT) + version = models.CharField(max_length=20, blank=True, default="") objects = DesignInstanceQuerySet.as_manager() @@ -190,6 +205,8 @@ class Meta: unique_together = [ ("design", "name"), ] + verbose_name = "Design Deployment" + verbose_name_plural = "Design Deployments" def clean(self): """Guarantee that the design field cannot be changed.""" @@ -236,6 +253,18 @@ def delete(self, *args, **kwargs): raise ValidationError("A Design Instance can only be delete if it's Decommissioned and not Deployed.") return super().delete(*args, **kwargs) + @property + def created_by(self): + """Get the username of the user who created the object.""" + created_by, _ = get_created_and_last_updated_usernames_for_model(self) + return created_by + + @property + def last_updated_by(self): + """Get the username of the user who update the object last time.""" + _, last_updated_by = get_created_and_last_updated_usernames_for_model(self) + return last_updated_by + class Journal(PrimaryModel): """The Journal represents a single execution of a design instance. diff --git a/nautobot_design_builder/navigation.py b/nautobot_design_builder/navigation.py index aa286886..d8061a53 100644 --- a/nautobot_design_builder/navigation.py +++ b/nautobot_design_builder/navigation.py @@ -9,11 +9,11 @@ menu_items = ( NavMenuTab( - name="Jobs", - weight=150, + name="Designs", + weight=1000, groups=( NavMenuGroup( - name="Designs", + name="Design Builder", weight=100, items=( NavMenuItem( @@ -24,16 +24,10 @@ ), NavMenuItem( link="plugins:nautobot_design_builder:designinstance_list", - name="Design Instances", + name="Design Deployments", permissions=["nautobot_design_builder.view_designinstance"], buttons=(), ), - NavMenuItem( - link="plugins:nautobot_design_builder:journal_list", - name="Journals", - permissions=["nautobot_design_builder.view_journal"], - buttons=(), - ), ), ), ), diff --git a/nautobot_design_builder/recursive.py b/nautobot_design_builder/recursive.py index 784223ee..b4858b3b 100644 --- a/nautobot_design_builder/recursive.py +++ b/nautobot_design_builder/recursive.py @@ -58,7 +58,7 @@ def inject_nautobot_uuids(initial_data, final_data, only_ext=False): # pylint: # TODO: could we make it simpler? -def reduce_design( +def combine_designs( new_value, old_value, future_value, decommissioned_objects, type_key ): # pylint: disable=too-many-locals,too-many-return-statements,too-many-branches,too-many-statements """Recursive function to simplify the new design by comparing with a previous design. @@ -104,11 +104,11 @@ def reduce_design( # be taken into account to be decommissioned before. inject_nautobot_uuids(old_element, new_element, only_ext=True) - reduce_design({}, old_element, {}, decommissioned_objects, type_key) + combine_designs({}, old_element, {}, decommissioned_objects, type_key) # When the elements have the same identifier, we progress on the recursive reduction analysis - elif reduce_design(new_element, old_element, future_element, decommissioned_objects, type_key): - # As we are iterating over the new_value list, we keep the elements that the `reduce_design` + elif combine_designs(new_element, old_element, future_element, decommissioned_objects, type_key): + # As we are iterating over the new_value list, we keep the elements that the `combine_designs` # concludes that must be deleted as not longer relevant for the new design. new_value.remove(new_element) @@ -188,11 +188,11 @@ def reduce_design( decommissioned_objects[inner_key] = [] decommissioned_objects[inner_key].append((obj[NAUTOBOT_ID], get_object_identifier(obj))) - reduce_design({}, obj, {}, decommissioned_objects, inner_key) + combine_designs({}, obj, {}, decommissioned_objects, inner_key) elif isinstance(inner_value, (dict, list)) and inner_key in old_value: # If an attribute is a dict or list, explore it recursively to reduce it - if reduce_design( + if combine_designs( inner_value, old_value[inner_key], future_value[inner_key], diff --git a/nautobot_design_builder/signals.py b/nautobot_design_builder/signals.py index 1c3ce072..e98ae0f3 100644 --- a/nautobot_design_builder/signals.py +++ b/nautobot_design_builder/signals.py @@ -5,14 +5,14 @@ from django.apps import apps from django.contrib.contenttypes.models import ContentType -from django.db.models.signals import post_save +from django.db.models.signals import post_save, post_delete from django.dispatch import receiver from django.conf import settings from django.db.models.signals import pre_delete from django.db.models import ProtectedError from nautobot.core.signals import nautobot_database_ready -from nautobot.extras.models import Job, Status +from nautobot.extras.models import Job, Status, Tag from nautobot.utilities.choices import ColorChoices from nautobot.extras.registry import registry from nautobot_design_builder.models import JournalEntry @@ -67,6 +67,7 @@ def create_design_model(sender, instance: Job, **kwargs): # pylint:disable=unus instance (Job): Job instance that has been created or updated. """ if instance.job_class and issubclass(instance.job_class, DesignJob): + _, created = Design.objects.get_or_create(job=instance) if created: _LOGGER.debug("Created design from %s", instance) @@ -105,3 +106,9 @@ def load_pre_delete_signals(): load_pre_delete_signals() + + +@receiver(signal=post_delete, sender=DesignInstance) +def handle_post_delete_design_instance(sender, instance, **kwargs): # pylint: disable=unused-argument + """Cleaning up the Tag created for a design instance.""" + Tag.objects.get(name=f"Managed by {instance}").delete() diff --git a/nautobot_design_builder/tables.py b/nautobot_design_builder/tables.py index b20cc345..55cf6686 100644 --- a/nautobot_design_builder/tables.py +++ b/nautobot_design_builder/tables.py @@ -7,10 +7,16 @@ from nautobot_design_builder.models import Design, DesignInstance, Journal, JournalEntry - DESIGNTABLE = """ + + + + - + + + + """ @@ -18,23 +24,24 @@ class DesignTable(BaseTable): """Table for list view.""" - job = Column(linkify=True) name = Column(linkify=True) - instance_count = Column(linkify=True, accessor=Accessor("instance_count"), verbose_name="Instances") - actions = ButtonsColumn(Design, buttons=("changelog",), prepend_template=DESIGNTABLE) + instance_count = Column(linkify=True, accessor=Accessor("instance_count"), verbose_name="Deployments") + actions = ButtonsColumn(Design, buttons=("changelog", "delete"), prepend_template=DESIGNTABLE) + job_last_synced = Column(accessor="job.last_updated", verbose_name="Last Synced Time") class Meta(BaseTable.Meta): # pylint: disable=too-few-public-methods """Meta attributes.""" model = Design - fields = ("name", "job", "instance_count") + fields = ("name", "version", "job_last_synced", "description", "instance_count") DESIGNINSTANCETABLE = """ +{% load utils %} - @@ -46,7 +53,11 @@ class DesignInstanceTable(StatusTableMixin, BaseTable): name = Column(linkify=True) design = Column(linkify=True) - live_state = ColoredLabelColumn() + first_implemented = Column(verbose_name="Deployment Time") + last_implemented = Column(verbose_name="Last Update Time") + created_by = Column(verbose_name="Deployed by") + last_updated_by = Column(verbose_name="Last Updated by") + live_state = ColoredLabelColumn(verbose_name="Operational State") actions = ButtonsColumn( DesignInstance, buttons=( @@ -60,15 +71,25 @@ class Meta(BaseTable.Meta): # pylint: disable=too-few-public-methods """Meta attributes.""" model = DesignInstance - fields = ("name", "design", "owner", "first_implemented", "last_implemented", "status", "live_state") + fields = ( + "name", + "design", + "version", + "created_by", + "first_implemented", + "last_updated_by", + "last_implemented", + "status", + "live_state", + ) class JournalTable(BaseTable): """Table for list view.""" pk = Column(linkify=True, verbose_name="ID") - design_instance = Column(linkify=True) - job_result = Column(linkify=True) + design_instance = Column(linkify=True, verbose_name="Deployment") + job_result = Column(accessor=Accessor("job_result.created"), linkify=True, verbose_name="Design Job Result") journal_entry_count = Column(accessor=Accessor("journal_entry_count"), verbose_name="Journal Entries") active = BooleanColumn(verbose_name="Active Journal") diff --git a/nautobot_design_builder/templates/nautobot_design_builder/design_list.html b/nautobot_design_builder/templates/nautobot_design_builder/design_list.html new file mode 100644 index 00000000..21fdc25d --- /dev/null +++ b/nautobot_design_builder/templates/nautobot_design_builder/design_list.html @@ -0,0 +1,69 @@ +{% extends 'generic/object_list.html' %} +{% load buttons %} +{% load static %} +{% load helpers %} + +{% block extra_styles %} +{{ block.super }} + +{% endblock %} +{% block content %} + {{ block.super }} + + +{% endblock %} + +{% block javascript %} + + +{% endblock %} diff --git a/nautobot_design_builder/templates/nautobot_design_builder/design_retrieve.html b/nautobot_design_builder/templates/nautobot_design_builder/design_retrieve.html index 3ec300c1..cb5324ee 100644 --- a/nautobot_design_builder/templates/nautobot_design_builder/design_retrieve.html +++ b/nautobot_design_builder/templates/nautobot_design_builder/design_retrieve.html @@ -2,26 +2,48 @@ {% load helpers %} {% block content_left_page %} -
-
- Design -
- - - - - - - - - -
Status - {{ object.get_status_display }} -
Job{{ object.job|hyperlinked_object }}
-
+
+
+ Design +
+ + + + + + + + + + + + + + + + + +
Job{{ object.job|hyperlinked_object }}
Job Last Synced{{ object.job.last_updated }}
Version{{ object.version }}
Description{{ object.description }}
+
{% endblock content_left_page %} +{% block content_right_page %} +
+
+ Documentation +
+ + + + +
+ {{ object.docs | render_markdown }} +
+
+ +{% endblock content_right_page %} + {% block content_full_width_page %} -{% include 'utilities/obj_table.html' with table=instances_table table_template='panel_table.html' heading='Instances' %} +{% include 'utilities/obj_table.html' with table=instances_table table_template='panel_table.html' heading='Design Deployments' %}
{% endblock content_full_width_page %} diff --git a/nautobot_design_builder/templates/nautobot_design_builder/designinstance_retrieve.html b/nautobot_design_builder/templates/nautobot_design_builder/designinstance_retrieve.html index 1dc68240..45133453 100644 --- a/nautobot_design_builder/templates/nautobot_design_builder/designinstance_retrieve.html +++ b/nautobot_design_builder/templates/nautobot_design_builder/designinstance_retrieve.html @@ -4,7 +4,7 @@ {% block content_left_page %}
- Design Instance + Design Deployment
@@ -12,15 +12,23 @@ - - + + - + + + + + - + + + + + @@ -34,7 +42,7 @@ - + @@ -43,7 +51,7 @@ {% endblock content_left_page %} -{% block content_full_width_page %} +{% block content_right_page %} {% include 'utilities/obj_table.html' with table=journals_table table_template='panel_table.html' heading='Journals' %}
-{% endblock content_full_width_page %} +{% endblock content_right_page %} diff --git a/nautobot_design_builder/templates/nautobot_design_builder/designprotection_tab.html b/nautobot_design_builder/templates/nautobot_design_builder/designprotection_tab.html index 99e18d0c..b84d8b94 100644 --- a/nautobot_design_builder/templates/nautobot_design_builder/designprotection_tab.html +++ b/nautobot_design_builder/templates/nautobot_design_builder/designprotection_tab.html @@ -15,7 +15,7 @@ - + diff --git a/nautobot_design_builder/templates/nautobot_design_builder/journal_retrieve.html b/nautobot_design_builder/templates/nautobot_design_builder/journal_retrieve.html index 0417ec50..645b50b8 100644 --- a/nautobot_design_builder/templates/nautobot_design_builder/journal_retrieve.html +++ b/nautobot_design_builder/templates/nautobot_design_builder/journal_retrieve.html @@ -12,7 +12,7 @@ - + diff --git a/nautobot_design_builder/templates/nautobot_design_builder/markdown_render.html b/nautobot_design_builder/templates/nautobot_design_builder/markdown_render.html new file mode 100644 index 00000000..dbab3b28 --- /dev/null +++ b/nautobot_design_builder/templates/nautobot_design_builder/markdown_render.html @@ -0,0 +1,23 @@ +{% load helpers %} +{% load static %} + + + + + + +

{{ design_name }} design

+ + +
+
+ 
{{ text_content | render_markdown }}
+
+
diff --git a/nautobot_design_builder/templatetags/__init__.py b/nautobot_design_builder/templatetags/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/nautobot_design_builder/templatetags/utils.py b/nautobot_design_builder/templatetags/utils.py new file mode 100644 index 00000000..a41f3252 --- /dev/null +++ b/nautobot_design_builder/templatetags/utils.py @@ -0,0 +1,14 @@ +"""Jinja filters for design_builder.""" + +from django import template +from django_jinja import library + + +register = template.Library() + + +@library.filter() +@register.filter() +def get_last_journal(design_instance): + """Get last run journal in a design instance.""" + return design_instance.journals.order_by("last_updated").last() diff --git a/nautobot_design_builder/tests/__init__.py b/nautobot_design_builder/tests/__init__.py index baada528..1368ffaa 100644 --- a/nautobot_design_builder/tests/__init__.py +++ b/nautobot_design_builder/tests/__init__.py @@ -21,7 +21,6 @@ def setUp(self): super().setUp() self.data = { "instance_name": "Test Design", - "owner": "", } self.logged_messages = [] self.git_patcher = patch("nautobot_design_builder.ext.GitRepo") diff --git a/nautobot_design_builder/tests/test_decommissioning_job.py b/nautobot_design_builder/tests/test_decommissioning_job.py index da84f7be..77453392 100644 --- a/nautobot_design_builder/tests/test_decommissioning_job.py +++ b/nautobot_design_builder/tests/test_decommissioning_job.py @@ -64,7 +64,7 @@ def setUp(self): ) self.job1.validated_save() - self.design1, _ = models.Design.objects.get_or_create(job=self.job1) + self.design1, _ = models.Design.objects.get_or_create(job=self.job1, defaults={"version": "0.0.1"}) self.content_type = ContentType.objects.get_for_model(models.DesignInstance) self.design_instance = models.DesignInstance( design=self.design1, @@ -73,6 +73,7 @@ def setUp(self): live_state=Status.objects.get( content_types=self.content_type, name=choices.DesignInstanceLiveStateChoices.PENDING ), + version=self.design1.version, ) self.design_instance.validated_save() @@ -83,6 +84,7 @@ def setUp(self): live_state=Status.objects.get( content_types=self.content_type, name=choices.DesignInstanceLiveStateChoices.PENDING ), + version=self.design1.version, ) self.design_instance_2.validated_save() diff --git a/nautobot_design_builder/tests/test_model_design_instance.py b/nautobot_design_builder/tests/test_model_design_instance.py index 17588e37..7cd332d1 100644 --- a/nautobot_design_builder/tests/test_model_design_instance.py +++ b/nautobot_design_builder/tests/test_model_design_instance.py @@ -28,6 +28,7 @@ def create_design_instance(design_name, design): live_state=Status.objects.get( content_types=content_type, name=choices.DesignInstanceLiveStateChoices.PENDING ), + version=design.version, ) design_instance.validated_save() return design_instance diff --git a/nautobot_design_builder/tests/test_reduce.py b/nautobot_design_builder/tests/test_reduce.py index 324b4a38..a719936f 100644 --- a/nautobot_design_builder/tests/test_reduce.py +++ b/nautobot_design_builder/tests/test_reduce.py @@ -6,7 +6,7 @@ import json from parameterized import parameterized -from nautobot_design_builder.recursive import reduce_design +from nautobot_design_builder.recursive import combine_designs # pylint: disable=missing-class-docstring @@ -35,7 +35,7 @@ def setUp(self): ], ] ) - def test_reduce_design(self, folder_name): # pylint: disable=too-many-locals + def test_combine_designs(self, folder_name): # pylint: disable=too-many-locals folder_path = os.path.join(os.path.dirname(__file__), "testdata_reduce") design_filename = os.path.join(folder_path, folder_name, "design.json") previous_design_filename = os.path.join(folder_path, folder_name, "previous_design.json") @@ -60,7 +60,7 @@ def test_reduce_design(self, folder_name): # pylint: disable=too-many-locals for key, new_value in design.items(): old_value = previous_design[key] future_value = future_design[key] - to_delete = reduce_design(new_value, old_value, future_value, elements_to_be_decommissioned, key) + to_delete = combine_designs(new_value, old_value, future_value, elements_to_be_decommissioned, key) if to_delete: ext_keys_to_be_simplified.append(key) diff --git a/nautobot_design_builder/tests/util.py b/nautobot_design_builder/tests/util.py index c790be93..4a769127 100644 --- a/nautobot_design_builder/tests/util.py +++ b/nautobot_design_builder/tests/util.py @@ -7,25 +7,8 @@ from nautobot_design_builder.models import Design, DesignInstance, Journal, JournalEntry -def populate_sample_data(): - """Populate the database with some sample data.""" - job = Job.objects.get(name="Initial Data") - job_result, _ = JobResult.objects.get_or_create( - name="Test", obj_type=ContentType.objects.get_for_model(Job), job_id=job.pk - ) - - design, _ = Design.objects.get_or_create(job=job) - design_instance, _ = DesignInstance.objects.get_or_create(design=design, name="Initial Data", owner="Test User") - Journal.objects.get_or_create(design_instance=design_instance, job_result=job_result) - - def create_test_view_data(): """Creates test data for view and API view test cases.""" - owners = [ - "Peter Müller", - "Maria Meyer", - "Otto Fischer", - ] for i in range(1, 4): # Core models job = Job.objects.create(name=f"Fake Design Job {i}") @@ -36,7 +19,7 @@ def create_test_view_data(): # Design Builder models design = Design.objects.create(job=job) - instance = DesignInstance.objects.create(design=design, name=f"Test Instance {i}", owner=owners[i - 1]) + instance = DesignInstance.objects.create(design=design, name=f"Test Instance {i}") journal = Journal.objects.create(design_instance=instance, job_result=job_result) full_control = i == 1 # Have one record where full control is given, more than one where its not. JournalEntry.objects.create(journal=journal, design_object=object_created_by_job, full_control=full_control) diff --git a/nautobot_design_builder/util.py b/nautobot_design_builder/util.py index e19d4c1d..2d614148 100644 --- a/nautobot_design_builder/util.py +++ b/nautobot_design_builder/util.py @@ -1,5 +1,6 @@ """Main design builder app module, contains DesignJob and base methods and functions.""" +# pylint: disable=import-outside-toplevel import functools import importlib import inspect @@ -14,6 +15,8 @@ from packaging.specifiers import Specifier import yaml +from django.contrib.contenttypes.models import ContentType +from django.db.models import Model from django.conf import settings import nautobot from nautobot.extras.models import GitRepository @@ -350,6 +353,54 @@ def custom_delete_order(key: str) -> int: return 0 +# TODO: this is only available in Nautobot 2.x, recreating it here to reuse for Nautobot 1.x +def get_changes_for_model(model): + """Return a queryset of ObjectChanges for a model or instance. + + The queryset will be filtered by the model class. If an instance is provided, + the queryset will also be filtered by the instance id. + """ + from nautobot.extras.models import ObjectChange # prevent circular import + + if isinstance(model, Model): + return ObjectChange.objects.filter( + changed_object_type=ContentType.objects.get_for_model(model._meta.model), + changed_object_id=model.pk, + ) + if issubclass(model, Model): + return ObjectChange.objects.filter(changed_object_type=ContentType.objects.get_for_model(model._meta.model)) + raise TypeError(f"{model!r} is not a Django Model class or instance") + + +def get_created_and_last_updated_usernames_for_model(instance): + """Get the user who created and last updated an instance. + + Args: + instance (Model): A model class instance + + Returns: + created_by (str): Username of the user that created the instance + last_updated_by (str): Username of the user that last modified the instance + """ + from nautobot.extras.choices import ObjectChangeActionChoices + from nautobot.extras.models import ObjectChange + + object_change_records = get_changes_for_model(instance) + created_by = None + last_updated_by = None + try: + created_by_record = object_change_records.get(action=ObjectChangeActionChoices.ACTION_CREATE) + created_by = created_by_record.user_name + except ObjectChange.DoesNotExist: + pass + + last_updated_by_record = object_change_records.order_by("time").last() + if last_updated_by_record: + last_updated_by = last_updated_by_record.user_name + + return created_by, last_updated_by + + @functools.total_ordering class _NautobotVersion: """Utility for comparing Nautobot versions.""" diff --git a/nautobot_design_builder/views.py b/nautobot_design_builder/views.py index 2650cf66..1f6a096e 100644 --- a/nautobot_design_builder/views.py +++ b/nautobot_design_builder/views.py @@ -2,6 +2,9 @@ from django_tables2 import RequestConfig from django.apps import apps as global_apps +from django.shortcuts import render + +from rest_framework.decorators import action from nautobot.core.views.mixins import ( ObjectDetailViewMixin, @@ -13,7 +16,7 @@ from nautobot.utilities.paginator import EnhancedPaginator, get_paginate_count from nautobot.utilities.utils import count_related from nautobot.core.views.generic import ObjectView - +from nautobot.core.views.mixins import PERMISSIONS_ACTION_MAP from nautobot_design_builder.api.serializers import ( DesignSerializer, @@ -37,11 +40,19 @@ from nautobot_design_builder.tables import DesignTable, DesignInstanceTable, JournalTable, JournalEntryTable +PERMISSIONS_ACTION_MAP.update( + { + "docs": "view", + } +) + + class DesignUIViewSet( # pylint:disable=abstract-method ObjectDetailViewMixin, ObjectListViewMixin, ObjectChangeLogViewMixin, ObjectNotesViewMixin, + ObjectDestroyViewMixin, ): """UI views for the design model.""" @@ -70,6 +81,17 @@ def get_extra_context(self, request, instance=None): context["instances_table"] = instances_table return context + @action(detail=True, methods=["get"]) + def docs(self, request, pk, *args, **kwargs): + """Additional action to handle docs.""" + design = Design.objects.get(pk=pk) + context = { + "design_name": design.name, + "is_modal": request.GET.get("modal"), + "text_content": design.docs, + } + return render(request, "nautobot_design_builder/markdown_render.html", context) + class DesignInstanceUIViewSet( # pylint:disable=abstract-method ObjectDetailViewMixin, @@ -87,12 +109,19 @@ class DesignInstanceUIViewSet( # pylint:disable=abstract-method table_class = DesignInstanceTable action_buttons = () lookup_field = "pk" + verbose_name = "Design Deployment" + verbose_name_plural = "Design Deployments" def get_extra_context(self, request, instance=None): """Extend UI.""" context = super().get_extra_context(request, instance) if self.action == "retrieve": - journals = Journal.objects.restrict(request.user, "view").filter(design_instance=instance) + journals = ( + Journal.objects.restrict(request.user, "view") + .filter(design_instance=instance) + .order_by("last_updated") + .annotate(journal_entry_count=count_related(JournalEntry, "journal")) + ) journals_table = JournalTable(journals) journals_table.columns.hide("design_instance")
{{ object.name }}
Owner{{ object.owner|placeholder }}Version{{ object.version }}
First implementedDeployed by{{ object.created_by|placeholder }}
Deployment Time {{ object.first_implemented|placeholder }}
Last implementedLast Updated by{{ object.last_updated_by|placeholder }}
Last Update Time {{ object.last_implemented|placeholder }}
Live StateOperational State {{ object.live_state }}
AttributeReferencing Design InstanceReferencing Design Deployments
{{ object.job_result|hyperlinked_object }}
Design InstanceDesign Deployment {{ object.design_instance|hyperlinked_object }}