Dev/service parcels (#2)

* Start work on Service models

* Service VPN passes for the internal vManage

* Refactor Service VPN converter. Instantiate the converter in the create_parcel_from_template function. Convert the create_parcel methods to instance methods, allowing them to access class attributes and methods. Split large create_parcel functions

* Add Interface GRE model. Add unittests. Add integration tests. Change feature profile integration test structure. Add more Castable literals to the normalizer. Change name factory method to parcel_factory. Change VPN model type to lan/vpn.

* Add SVI Interface model. Add unit tests and integration test for the moedl creation. Remove static UUID in tests, instead create dynamicly. Add IPv6Interface and IPv4Interface cast in normalizer. Change models to use casted Global[Interface] values. Fix Svi model

* Add Ethernet Interface. Add Unit tests. Add integratio tests. Fix VPN for sdwan demo data.

* Rename Feature Profile builder file. Use from typing_extensions import Annotated

* Add IPSEC interface model. Add unittests. Add integration tests.

* Add Ethernet interface. Add Builders for other and system profile. Improve logging during pushing ux2 config. Minor fixes for service models. Refactor config pusher. Add pushing service FP and parcels. Working for internal vmanage

* Fixes

* Add default=None for Optional fields.

* Add missing imports

* Prepare the rest service parcels for converter

* Fix add default=None for optional field

* Add OSPF converter

* Add OSPF intergration test

* Add OSPFv3IP4 converter.

* Add ospfv3ipv6 converter. add unittest add integration tests

* Add ospf model to transform. Add service feature profile in transform for creation. Add default values for ospf model (helps whe default template is empty and endpoint needs values). Comment logic for interface assigement since there can be many vpns and interfaces in one feature profile correct implementation is needed

* update deprecated github actions (#544)

* update deprecated github actions

* fix type error

* Remove annotation

* Log feature templates that cant be assigned to feature profile

---------

Co-authored-by: Jakub Krajewski <[email protected]>

.github/workflows/documentation.yml

@@ -6,9 +6,9 @@ jobs:
   build:
     runs-on: ubuntu-latest
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
     - name: Set up Python
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v5
       with:
         python-version: 3.8
     - name: Set Up Poetry
@@ -20,7 +20,7 @@ jobs:
     - name: Build HTML
       run: poetry run sphinx-build -M html docs/source docs/build
     - name: Upload artifacts
-      uses: actions/upload-artifact@v3
+      uses: actions/upload-artifact@v4
       with:
         name: html-docs
         path: docs/build/html/

.github/workflows/linting.yml

@@ -19,9 +19,9 @@ jobs:
         run: |
             curl -fsSL https://deb.nodesource.com/setup_12.x | sudo -E bash -
             sudo apt-get install -y nodejs
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
       - name: Set up Python
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
       - name: Set Up Poetry

.github/workflows/release.yml

@@ -17,9 +17,9 @@ jobs:
         run: |
           curl -fsSL https://deb.nodesource.com/setup_12.x | sudo -E bash -
           sudo apt-get install -y nodejs
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
       - name: Set up Python
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: 3.8
       - name: Set Up Poetry

.github/workflows/unittests.yml

@@ -19,9 +19,9 @@ jobs:
         run: |
             curl -fsSL https://deb.nodesource.com/setup_12.x | sudo -E bash -
             sudo apt-get install -y nodejs
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
       - name: Set up Python
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
       - name: Set Up Poetry

.github/workflows/version.yml

@@ -12,9 +12,9 @@ jobs:
         shell: bash
 
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
       - name: Set up Python
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: 3.8
       - name: Set Up Poetry

catalystwan/api/api_container.py

@@ -15,6 +15,7 @@
 )
 from catalystwan.api.alarms_api import AlarmsAPI
 from catalystwan.api.basic_api import DevicesAPI, DeviceStateAPI
+from catalystwan.api.builders import BuilderAPI
 from catalystwan.api.config_device_inventory_api import ConfigurationDeviceInventoryAPI
 from catalystwan.api.config_group_api import ConfigGroupAPI
 from catalystwan.api.dashboard_api import DashboardAPI
@@ -67,3 +68,4 @@ def __init__(self, session: ManagerSession):
         self.policy = PolicyAPI(session)
         self.sd_routing_feature_profiles = SDRoutingFeatureProfilesAPI(session)
         self.sdwan_feature_profiles = SDWANFeatureProfilesAPI(session)
+        self.builders = BuilderAPI(session)

catalystwan/api/builders/__init__.py

@@ -0,0 +1,13 @@
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+from catalystwan.api.builders.feature_profiles.builder_factory import FeatureProfileBuilderFactory
+
+if TYPE_CHECKING:
+    from catalystwan.session import ManagerSession
+
+
+class BuilderAPI:
+    def __init__(self, session: ManagerSession):
+        self.feature_profiles = FeatureProfileBuilderFactory(session=session)

catalystwan/api/builders/feature_profiles/builder_factory.py

@@ -0,0 +1,42 @@
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Callable, Mapping, Union
+
+from catalystwan.api.builders.feature_profiles.other import OtherFeatureProfileBuilder
+from catalystwan.api.builders.feature_profiles.service import ServiceFeatureProfileBuilder
+from catalystwan.api.builders.feature_profiles.system import SystemFeatureProfileBuilder
+from catalystwan.exceptions import CatalystwanException
+from catalystwan.models.configuration.feature_profile.common import ProfileType
+
+if TYPE_CHECKING:
+    from catalystwan.session import ManagerSession
+
+FeatureProfileBuilder = Union[ServiceFeatureProfileBuilder, SystemFeatureProfileBuilder, OtherFeatureProfileBuilder]
+
+BUILDER_MAPPING: Mapping[ProfileType, Callable] = {
+    "service": ServiceFeatureProfileBuilder,
+    "system": SystemFeatureProfileBuilder,
+    "other": OtherFeatureProfileBuilder,
+}
+
+
+class FeatureProfileBuilderFactory:
+    def __init__(self, session: ManagerSession):
+        self.session = session
+
+    def create_builder(self, profile_type: ProfileType) -> FeatureProfileBuilder:
+        """
+        Creates a builder for the specified feature profile.
+
+        Args:
+            feature_profile_name (str): The name of the feature profile.
+
+        Returns:
+            FeatureProfileBuilder: The builder for the specified feature profile.
+
+        Raises:
+            CatalystwanException: If the feature profile is not found or has an unsupported type.
+        """
+        if profile_type not in BUILDER_MAPPING:
+            raise CatalystwanException(f"Unsupported builder for type {profile_type}")
+        return BUILDER_MAPPING[profile_type](self.session)

catalystwan/api/builders/feature_profiles/other.py

@@ -0,0 +1,69 @@
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, List
+from uuid import UUID
+
+from catalystwan.api.feature_profile_api import OtherFeatureProfileAPI
+from catalystwan.endpoints.configuration.feature_profile.sdwan.other import OtherFeatureProfile
+from catalystwan.models.configuration.feature_profile.common import FeatureProfileCreationPayload
+from catalystwan.models.configuration.feature_profile.sdwan.other import AnyOtherParcel
+
+if TYPE_CHECKING:
+    from catalystwan.session import ManagerSession
+
+
+class OtherFeatureProfileBuilder:
+    """
+    A class for building Other feature profiles.
+    """
+
+    def __init__(self, session: ManagerSession) -> None:
+        """
+        Initialize a new instance of the Service class.
+
+        Args:
+            session (ManagerSession): The ManagerSession object used for API communication.
+            profile_uuid (UUID): The UUID of the profile.
+        """
+        self._profile: FeatureProfileCreationPayload
+        self._api = OtherFeatureProfileAPI(session)
+        self._endpoints = OtherFeatureProfile(session)
+        self._independent_items: List[AnyOtherParcel] = []
+
+    def add_profile_name_and_description(self, feature_profile: FeatureProfileCreationPayload) -> None:
+        """
+        Adds a name and description to the feature profile.
+
+        Args:
+            name (str): The name of the feature profile.
+            description (str): The description of the feature profile.
+
+        Returns:
+            None
+        """
+        self._profile = feature_profile
+
+    def add_parcel(self, parcel: AnyOtherParcel) -> None:
+        """
+        Adds a parcel to the feature profile.
+
+        Args:
+            parcel (AnySystemParcel): The parcel to add.
+
+        Returns:
+            None
+        """
+        self._independent_items.append(parcel)
+
+    def build(self) -> UUID:
+        """
+        Builds the feature profile.
+
+        Returns:
+            UUID: The UUID of the created feature profile.
+        """
+
+        profile_uuid = self._endpoints.create_sdwan_other_feature_profile(self._profile).id
+        for parcel in self._independent_items:
+            self._api.create_parcel(profile_uuid, parcel)
+        return profile_uuid

catalystwan/api/builders/feature_profiles/service.py

@@ -0,0 +1,130 @@
+from __future__ import annotations
+
+import logging
+from collections import defaultdict
+from typing import TYPE_CHECKING, Dict, List, Union
+from uuid import UUID, uuid4
+
+from pydantic import Field
+from typing_extensions import Annotated
+
+from catalystwan.api.feature_profile_api import ServiceFeatureProfileAPI
+from catalystwan.endpoints.configuration.feature_profile.sdwan.service import ServiceFeatureProfile
+from catalystwan.models.configuration.feature_profile.common import FeatureProfileCreationPayload
+from catalystwan.models.configuration.feature_profile.sdwan.service import (
+    AppqoeParcel,
+    InterfaceEthernetParcel,
+    InterfaceGreParcel,
+    InterfaceIpsecParcel,
+    InterfaceSviParcel,
+    LanVpnDhcpServerParcel,
+    LanVpnParcel,
+)
+
+if TYPE_CHECKING:
+    from catalystwan.session import ManagerSession
+
+logger = logging.getLogger(__name__)
+
+IndependedParcels = Annotated[Union[AppqoeParcel, LanVpnDhcpServerParcel], Field(discriminator="type_")]
+DependedInterfaceParcels = Annotated[
+    Union[InterfaceGreParcel, InterfaceSviParcel, InterfaceEthernetParcel, InterfaceIpsecParcel],
+    Field(discriminator="type_"),
+]
+
+
+class ServiceFeatureProfileBuilder:
+    """
+    A class for building service feature profiles.
+    """
+
+    def __init__(self, session: ManagerSession):
+        """
+        Initialize a new instance of the Service class.
+
+        Args:
+            session (ManagerSession): The ManagerSession object used for API communication.
+            profile_uuid (UUID): The UUID of the profile.
+        """
+        self._profile: FeatureProfileCreationPayload
+        self._api = ServiceFeatureProfileAPI(session)
+        self._endpoints = ServiceFeatureProfile(session)
+        self._independent_items: List[IndependedParcels] = []
+        self._independent_items_vpns: Dict[UUID, LanVpnParcel] = {}
+        self._depended_items_on_vpns: Dict[UUID, List[DependedInterfaceParcels]] = defaultdict(list)
+
+    def add_profile_name_and_description(self, feature_profile: FeatureProfileCreationPayload) -> None:
+        """
+        Adds a name and description to the feature profile.
+
+        Args:
+            name (str): The name of the feature profile.
+            description (str): The description of the feature profile.
+
+        Returns:
+            None
+        """
+        self._profile = feature_profile
+
+    def add_parcel(self, parcel: IndependedParcels) -> None:
+        """
+        Adds an independent parcel to the builder.
+
+        Args:
+            parcel (IndependedParcels): The independent parcel to add.
+
+        Returns:
+            None
+        """
+        self._independent_items.append(parcel)
+
+    def add_parcel_vpn(self, parcel: LanVpnParcel) -> UUID:
+        """
+        Adds a VPN parcel to the builder.
+
+        Args:
+            parcel (LanVpnParcel): The VPN parcel to add.
+
+        Returns:
+            UUID: The UUID tag of the added VPN parcel.
+        """
+        vpn_tag = uuid4()
+        logger.debug(f"Adding VPN parcel {parcel.parcel_name} with tag {vpn_tag}")
+        self._independent_items_vpns[vpn_tag] = parcel
+        return vpn_tag
+
+    def add_parcel_vpn_interface(self, vpn_tag: UUID, parcel: DependedInterfaceParcels) -> None:
+        """
+        Adds an interface parcel dependent on a VPN to the builder.
+
+        Args:
+            vpn_tag (UUID): The UUID of the VPN.
+            parcel (DependedInterfaceParcels): The interface parcel to add.
+
+        Returns:
+            None
+        """
+        logger.debug(f"Adding interface parcel {parcel.parcel_name} to VPN {vpn_tag}")
+        self._depended_items_on_vpns[vpn_tag].append(parcel)
+
+    def build(self) -> UUID:
+        """
+        Builds the feature profile by creating parcels for independent items,
+        VPNs, and interface parcels dependent on VPNs.
+
+        Returns:
+            Service feature profile UUID
+        """
+        profile_uuid = self._endpoints.create_sdwan_service_feature_profile(self._profile).id
+
+        for parcel in self._independent_items:
+            self._api.create_parcel(profile_uuid, parcel)
+
+        for vpn_tag, vpn_parcel in self._independent_items_vpns.items():
+            vpn_uuid = self._api.create_parcel(profile_uuid, vpn_parcel).id
+
+            for interface_parcel in self._depended_items_on_vpns[vpn_tag]:
+                logger.debug(f"Creating interface parcel {interface_parcel.parcel_name} to VPN {vpn_tag}")
+                self._api.create_parcel(profile_uuid, interface_parcel, vpn_uuid)
+
+        return profile_uuid