Converting VS with lsn pool is not compliant with schema reference

[azahajkiewicz]

Environment

• Application Services Version:
  {
  "version": "3.25.0",
  "release": "3",
  "schemaCurrent": "3.25.0",
  "schemaMinimum": "3.0.0"
  }
• BIG-IP Version:
  Product: BIG-IP
  Version: 14.1.3
  Build: 0.0.7
  Sequence: 14.1.3-0.0.7.0
  BaseBuild: 0.0.7
  Edition: Final

VELOS platform

Summary

Attaching LSN pool to “snat” field is not valid, according to the schema reference:
F5 schema-reference
Values of the "snat" field should be: "auto", "none" or "self" only.

This impacts migrations to VELOS platform, as Charon creates AS3 declaration, which cannot be loadable on it: CGNAT/lsn pool is not supported on VELOS. Loading lsn pool objects cause failure.

Steps To Reproduce

Steps to reproduce the behavior:

1. Convert following config:

ltm virtual /Common/cgnat_vs {
    creation-time 2020-11-26:03:52:47
    destination /Common/10.144.18.44:0
    ip-protocol tcp
    last-modified-time 2020-11-26:03:52:47
    mask 255.255.255.255
    profiles {
        /Common/tcp { }
    }
    source 0.0.0.0/24
    source-address-translation {
        pool /Common/lsn_pool_test
        type lsn
    }
    translate-address disabled
    translate-port disabled
}

ltm lsn-pool /Common/lsn_pool_test {
    egress-interfaces {
        /Common/vlan2
    }
    egress-interfaces-disabled
    members {
        10.10.10.0/23
    }
}

2. Observe the following error message:
   Not tested on the classic BIG-IP.
   On VELOS there is an incompatibility, as the new platform does not support CGNAT (-> lsn pool) yet - loading such configuration would cause an error.

Problem lies in the declaration itself, because, according to the schema reference, AS3 should not support lsn pool (so referencing such to the "snat" field is incorrect).

Expected Behavior

Charon should never reference lsn pools in AS3 declarations.

Actual Behavior

Charon references lsn pool in "snat" field.

[mdditt2000]

ACC converted the configuration

{
    "class": "ADC",
    "schemaVersion": "3.25.0",
    "id": "urn:uuid:d317d335-b57b-4b42-a1f1-378b750d8eab",
    "label": "Converted Declaration",
    "remark": "Auto-generated by Project Charon",
    "Common": {
        "class": "Tenant",
        "Shared": {
            "class": "Application",
            "template": "shared",
            "cgnat_vs": {
                "layer4": "tcp",
                "translateServerAddress": false,
                "translateServerPort": false,
                "class": "Service_TCP",
                "profileTCP": {
                    "bigip": "/Common/tcp"
                },
                "virtualAddresses": [
                    [
                        "10.144.18.44",
                        "0.0.0.0/24"
                    ]
                ],
                "virtualPort": 0,
                "persistenceMethods": [],
                "snat": {
                    "use": "/Common/Shared/lsn_pool_test"
                }
            }
        }
    }
}

This is a valid configuration and conversion for ACC as cgnat_vs and lsn_pool_test are valid names. ACC doesn't know anything about gnat and nor should it. Closing issue.

https://github.com/mdditt2000/f5-appsvcs-acc/tree/master/Github/9

[azahajkiewicz]

Did you try to load it on any platform?
Yes, lsn_pool_test or cgnat_vs are valid names, but that is not the point of this bug.

First of all, the application "cgnat_vs" cannot be deployed on BIG-IP because of errors related to the declaration itself.
Secondly, the snat field for Service_TCP application type is:

snat (string | Service_TCP_snat)	“auto”	“none”, “self”, “auto”, -	Name of built-in SNAT method or AS3 pointer to SNAT pool. If ‘self’, the system uses the virtual-server address as SNAT address

There is no information in schema reference (link in bug description), that a reference to a BIG-IP pool can be used. The only values allowed are: "auto", "none", "self".

1. UCS with cgnat_vs loaded successfully:

Extracting manifest: /var/local/ucs/integrationOld.ucs
Product : BIG-IP
Platform: UCS   : Z100
          System: C116
Version : 14.1.2
Edition : Final
Hostname: journeys-test-system.f5.com
Installing --full-- configuration on host journeys-test-system.f5.com
Installing configuration...
Post-processing...
usermod: no changes
Reloading License and configuration - this may take a few minutes...
Warnings were generated:
    Load balancing feature not licensed.

Platform migrate loaded successfully. Saving configuration.
/var/local/ucs/integrationOld.ucs is loaded.

bigip.conf file contains cgnat_vs with lsn pool reference:

ltm virtual /Common/cgnat_vs {
    creation-time 2020-11-26:03:52:47
    destination /Common/10.144.18.44:0
    ip-protocol tcp
    last-modified-time 2020-11-26:03:52:47
    mask 255.255.255.255
    profiles {
        /Common/tcp { }
    }
    source 0.0.0.0/24
    source-address-translation {
        pool /Common/lsn_pool_test
        type lsn
    }
    translate-address disabled
    translate-port disabled
}

2. Convertion of this UCS with latest charon image (1.10.0)

WRW-ML-00029908:bbb zahajkiewicz$ docker run --rm -v $(pwd):/app/data f5-appsvcs-acc:1.10.0 -o data/deploy.json -u data/integrationOld.ucs
stdout: 152 BIG-IP objects detected total
31 BIG-IP objects recognized by AS3
22 BIG-IP objects supported by Charon
13 AS3 stanzas generated

deploy.json:

{
    "class": "ADC",
    "schemaVersion": "3.25.0",
    "id": "urn:uuid:7346ff06-30ac-4185-a070-46d6cd4f48b7",
    "label": "Converted Declaration",
    "remark": "Auto-generated by Project Charon",
    "Common": {
        "class": "Tenant",
        "Shared": {
            "class": "Application",
            "template": "shared",
            "pool_test": {
                "members": [
                    {
                        "addressDiscovery": "static",
                        "servicePort": 22,
                        "serverAddresses": [
                            "10.146.65.121"
                        ],
                        "shareNodes": true
                    }
                ],
                "monitors": [
                    {
                        "bigip": "/Common/tcp_half_open"
                    }
                ],
                "class": "Pool"
            },
            "VS_test": {
                "layer4": "tcp",
                "pool": "pool_test",
                "translateServerAddress": true,
                "translateServerPort": false,
                "class": "Service_L4",
                "profileL4": {
                    "bigip": "/Common/fastL4"
                },
                "virtualAddresses": [
                    [
                        "10.144.18.39",
                        "0.0.0.0/24"
                    ]
                ],
                "virtualPort": 0,
                "persistenceMethods": [],
                "snat": "none",
                "allowVlans": [
                    {
                        "bigip": "/Common/vlan1"
                    },
                    {
                        "bigip": "/Common/vlan2"
                    },
                    {
                        "bigip": "/Common/vlanGroup_test"
                    }
                ]
            },
            "cgnat_vs": {
                "layer4": "tcp",
                "translateServerAddress": false,
                "translateServerPort": false,
                "class": "Service_TCP",
                "profileTCP": {
                    "bigip": "/Common/tcp"
                },
                "virtualAddresses": [
                    [
                        "10.144.18.44",
                        "0.0.0.0/24"
                    ]
                ],
                "virtualPort": 0,
                "persistenceMethods": [],
                **"snat": {
                    "use": "/Common/Shared/lsn_pool_test"
                }**
            },
            "pem_listeners_ANY_IP": {
                "remark": "Created by Web Configuration Utility for PEM Listener pem_listeners",
                "translateServerAddress": true,
                "translateServerPort": false,
                "class": "Service_Generic",
                "profileClassification": {
                    "bigip": "/Common/classification_pem"
                },
                "profileIPOther": {
                    "bigip": "/Common/ipother"
                },
                "profileEnforcement": {
                    "use": "/Common/Shared/pem_listeners_pem_profile"
                },
                "virtualAddresses": [
                    [
                        "10.144.18.36",
                        "0.0.0.0/24"
                    ]
                ],
                "virtualPort": 0,
                "persistenceMethods": [],
                "snat": "none"
            },
            "pem_listeners_FAST_L4": {
                "remark": "Created by Web Configuration Utility for PEM Listener pem_listeners",
                "layer4": "tcp",
                "translateServerAddress": true,
                "translateServerPort": true,
                "class": "Service_HTTP",
                "profileClassification": {
                    "bigip": "/Common/classification_pem"
                },
                "profileL4": {
                    "bigip": "/Common/fastL4"
                },
                "profileHTTP": {
                    "bigip": "/Common/http"
                },
                "profileEnforcement": {
                    "use": "/Common/Shared/pem_listeners_pem_profile"
                },
                "virtualAddresses": [
                    [
                        "10.144.18.36",
                        "0.0.0.0/32"
                    ]
                ],
                "virtualPort": 80,
                "persistenceMethods": [],
                "snat": "none"
            },
            "pem_listeners_HTTP": {
                "remark": "Created by Web Configuration Utility for PEM Listener pem_listeners",
                "layer4": "tcp",
                "translateServerAddress": true,
                "translateServerPort": true,
                "class": "Service_Generic",
                "profileClassification": {
                    "bigip": "/Common/classification_pem"
                },
                "profileEnforcement": {
                    "use": "/Common/Shared/pem_listeners_pem_profile"
                },
                "virtualAddresses": [
                    [
                        "10.144.18.36",
                        "0.0.0.0/24"
                    ]
                ],
                "virtualPort": 80,
                "persistenceMethods": [],
                "snat": "none"
            },
            "pem_listeners_L4_1": {
                "remark": "Created by Web Configuration Utility for PEM Listener pem_listeners",
                "layer4": "tcp",
                "translateServerAddress": true,
                "translateServerPort": false,
                "class": "Service_Generic",
                "profileClassification": {
                    "bigip": "/Common/classification_pem"
                },
                "profileEnforcement": {
                    "use": "/Common/Shared/pem_listeners_pem_profile"
                },
                "virtualAddresses": [
                    [
                        "10.144.18.36",
                        "0.0.0.0/24"
                    ]
                ],
                "virtualPort": 0,
                "persistenceMethods": [],
                "snat": "none"
            },
            "pem_listeners_L4_2": {
                "remark": "Created by Web Configuration Utility for PEM Listener pem_listeners",
                "layer4": "udp",
                "translateServerAddress": true,
                "translateServerPort": false,
                "class": "Service_Generic",
                "profileClassification": {
                    "bigip": "/Common/classification_pem"
                },
                "profileEnforcement": {
                    "use": "/Common/Shared/pem_listeners_pem_profile"
                },
                "virtualAddresses": [
                    [
                        "10.144.18.36",
                        "0.0.0.0/24"
                    ]
                ],
                "virtualPort": 0,
                "persistenceMethods": [],
                "snat": "none"
            },
            "pem_listeners": {
                "enforcementProfile": {
                    "use": "/Common/Shared/pem_listeners_pem_profile"
                },
                "services": [
                    {
                        "use": "/Common/Shared/pem_listeners_ANY_IP"
                    },
                    {
                        "use": "/Common/Shared/pem_listeners_HTTP"
                    },
                    {
                        "use": "/Common/Shared/pem_listeners_L4_1"
                    },
                    {
                        "use": "/Common/Shared/pem_listeners_L4_2"
                    }
                ],
                "class": "Enforcement_Listener"
            },
            "policy_subs": {
                "rules": [
                    {
                        "name": "rule_subs",
                        "precedence": 10,
                        "modifyHttpHeader": {
                            "headerName": "Encoding",
                            "operation": "insert",
                            "valueContent": "BLABLA"
                        },
                        "DTOSTethering": {
                            "detectDtos": true,
                            "detectTethering": true,
                            "reportDestinationHsl": {
                                "highSpeedLogPublisher": {
                                    "bigip": "/Common/local-db-publisher"
                                }
                            }
                        }
                    }
                ],
                "class": "Enforcement_Policy"
            },
            "policy_unknown": {
                "enable": false,
                "class": "Enforcement_Policy"
            },
            "pem_listeners_pem_profile": {
                "remark": "\"Created by Web Configuration Utility for PEM Listener pem_listeners\"",
                "connectionOptimizationEnabled": true,
                "connectionOptimizationService": {
                    "use": "/Common/Shared/pem_listeners_FAST_L4"
                },
                "class": "Enforcement_Profile"
            },
            "test_addr_list": {
                "addresses": [
                    "10.144.65.121",
                    "10.145.65.122"
                ],
                "remark": "test_addr_list",
                "class": "Firewall_Address_List"
            }
        }
    }
}

3. Deployment of the declaration generated by Charon:

- First error is that Charon created a reference to lsn pool wrongly:

{
    "code": 422,
    "errors": [
        "/Common/Shared/cgnat_vs/snat/use: contains path to non-existent object lsn_pool_test"
    ],
    "declarationFullId": "",
    "message": "declaration is invalid"
}

lsn_pool_test is defined in bigip.conf as /Common/lsn_pool_test, not /Common/Shared/lsn_pool_test:

ltm lsn-pool /Common/lsn_pool_test {
    egress-interfaces {
        /Common/vlan2
    }
    egress-interfaces-disabled
    members {
        10.10.10.0/23
    }
}

- If the reference is fixed manually, then I get an error:

{
    "code": 422,
    "errors": [
        "/Common/Shared/cgnat_vs/snat/use: contains invalid path (Tenant without application)"
    ],
    "declarationFullId": "",
    "message": "declaration is invalid"
}

- If I change snat value to the valid one, eg. "none":

            "cgnat_vs": {
                "layer4": "tcp",
                "translateServerAddress": false,
                "translateServerPort": false,
                "class": "Service_TCP",
                "profileTCP": {
                    "bigip": "/Common/tcp"
                },
                "virtualAddresses": [
                    [
                        "10.144.18.44",
                        "0.0.0.0/24"
                    ]
                ],
                "virtualPort": 0,
                "persistenceMethods": [],
                "snat": "none"
            },

I will still get an error, but it is filed in the other bug (#8):

{
    "code": 422,
    "errors": [
        "/Common/Shared/pem_listeners_ANY_IP/remark: should match format \"f5remark\""
    ],
    "declarationFullId": "",
    "message": "declaration is invalid"
}

Then, if I change the remark to the correct format, I finally get a response:

{
    "results": [
        {
            "code": 422,
            "message": "declaration failed",
            "response": "0107176c:3: Invalid Node, the IP address 10.146.65.121 already exists.",
            "host": "localhost",
            "tenant": "Common",
            "runTime": 2532
        },
        {
            "code": 422,
            "message": "declaration failed",
            "response": "01020036:3: The requested Node (/Common/10.146.65.121) was not found.",
            "host": "localhost",
            "tenant": "Common",
            "runTime": 2966
        }
    ],
    "declaration": {
        "class": "ADC",
        "schemaVersion": "3.25.0",
        "id": "urn:uuid:7346ff06-30ac-4185-a070-46d6cd4f48b7",
        "label": "Converted Declaration",
        "remark": "Auto-generated by Project Charon",
        "updateMode": "selective",
        "controls": {
            "archiveTimestamp": "2021-02-24T11:03:23.307Z"
        }
    },
    "code": 422
}

which is O.K., because I already have pools and members defined (ucs load contained it), so the declaration is duplicating it (I can mention in this place, that the duplication error happened, because declaration is deployed in different tenant than original applications were loaded: /Common vs /Common/shared. If theese were consistent, we would not see this error)

- If I remove those addresses and other duplicated objects from BIG-IP, I can then successfully deploy the declaration:

{
    "results": [
        {
            "code": 200,
            "message": "success",
            "lineCount": 35,
            "host": "localhost",
            "tenant": "Common",
            "runTime": 1903
        },
        {
            "code": 200,
            "message": "no change",
            "host": "localhost",
            "tenant": "Common",
            "runTime": 1378
        }
    ],
    "declaration": {
        "class": "ADC",
        "schemaVersion": "3.25.0",
        "id": "urn:uuid:7346ff06-30ac-4185-a070-46d6cd4f48b7",
        "label": "Converted Declaration",
        "remark": "Auto-generated by Project Charon",
        "Common": {
            "class": "Tenant",
            "Shared": {
                "class": "Application",
                "template": "shared",
                "pool_test": {
                    "members": [
                        {
                            "addressDiscovery": "static",
                            "servicePort": 22,
                            "serverAddresses": [
                                "10.146.65.121"
                            ],
                            "shareNodes": true
                        }
                    ],
                    "monitors": [
                        {
                            "bigip": "/Common/tcp_half_open"
                        }
                    ],
                    "class": "Pool"
                },
                "VS_test": {
                    "layer4": "tcp",
                    "pool": "pool_test",
                    "translateServerAddress": true,
                    "translateServerPort": false,
                    "class": "Service_L4",
                    "profileL4": {
                        "bigip": "/Common/fastL4"
                    },
                    "virtualAddresses": [
                        [
                            "10.144.18.39",
                            "0.0.0.0/24"
                        ]
                    ],
                    "virtualPort": 0,
                    "persistenceMethods": [],
                    "snat": "none",
                    "allowVlans": [
                        {
                            "bigip": "/Common/vlan1"
                        },
                        {
                            "bigip": "/Common/vlan2"
                        },
                        {
                            "bigip": "/Common/vlanGroup_test"
                        }
                    ]
                },
                "cgnat_vs": {
                    "layer4": "tcp",
                    "translateServerAddress": false,
                    "translateServerPort": false,
                    "class": "Service_TCP",
                    "profileTCP": {
                        "bigip": "/Common/tcp"
                    },
                    "virtualAddresses": [
                        [
                            "10.144.18.44",
                            "0.0.0.0/24"
                        ]
                    ],
                    "virtualPort": 0,
                    "persistenceMethods": [],
                    "snat": "none"
                },
                "pem_listeners_ANY_IP": {
                    "remark": "em_listeners",
                    "translateServerAddress": true,
                    "translateServerPort": false,
                    "class": "Service_Generic",
                    "profileClassification": {
                        "bigip": "/Common/classification_pem"
                    },
                    "profileIPOther": {
                        "bigip": "/Common/ipother"
                    },
                    "profileEnforcement": {
                        "use": "/Common/Shared/pem_listeners_pem_profile"
                    },
                    "virtualAddresses": [
                        [
                            "10.144.18.36",
                            "0.0.0.0/24"
                        ]
                    ],
                    "virtualPort": 0,
                    "persistenceMethods": [],
                    "snat": "none"
                },
                "pem_listeners_FAST_L4": {
                    "remark": "pem_listeners",
                    "layer4": "tcp",
                    "translateServerAddress": true,
                    "translateServerPort": true,
                    "class": "Service_HTTP",
                    "profileClassification": {
                        "bigip": "/Common/classification_pem"
                    },
                    "profileL4": {
                        "bigip": "/Common/fastL4"
                    },
                    "profileHTTP": {
                        "bigip": "/Common/http"
                    },
                    "profileEnforcement": {
                        "use": "/Common/Shared/pem_listeners_pem_profile"
                    },
                    "virtualAddresses": [
                        [
                            "10.144.18.36",
                            "0.0.0.0/32"
                        ]
                    ],
                    "virtualPort": 80,
                    "persistenceMethods": [],
                    "snat": "none"
                },
                "pem_listeners_HTTP": {
                    "remark": "pem_listeners",
                    "layer4": "tcp",
                    "translateServerAddress": true,
                    "translateServerPort": true,
                    "class": "Service_Generic",
                    "profileClassification": {
                        "bigip": "/Common/classification_pem"
                    },
                    "profileEnforcement": {
                        "use": "/Common/Shared/pem_listeners_pem_profile"
                    },
                    "virtualAddresses": [
                        [
                            "10.144.18.36",
                            "0.0.0.0/24"
                        ]
                    ],
                    "virtualPort": 80,
                    "persistenceMethods": [],
                    "snat": "none"
                },
                "pem_listeners_L4_1": {
                    "remark": "em_listeners",
                    "layer4": "tcp",
                    "translateServerAddress": true,
                    "translateServerPort": false,
                    "class": "Service_Generic",
                    "profileClassification": {
                        "bigip": "/Common/classification_pem"
                    },
                    "profileEnforcement": {
                        "use": "/Common/Shared/pem_listeners_pem_profile"
                    },
                    "virtualAddresses": [
                        [
                            "10.144.18.36",
                            "0.0.0.0/24"
                        ]
                    ],
                    "virtualPort": 0,
                    "persistenceMethods": [],
                    "snat": "none"
                },
                "pem_listeners_L4_2": {
                    "remark": "pem_listeners",
                    "layer4": "udp",
                    "translateServerAddress": true,
                    "translateServerPort": false,
                    "class": "Service_Generic",
                    "profileClassification": {
                        "bigip": "/Common/classification_pem"
                    },
                    "profileEnforcement": {
                        "use": "/Common/Shared/pem_listeners_pem_profile"
                    },
                    "virtualAddresses": [
                        [
                            "10.144.18.36",
                            "0.0.0.0/24"
                        ]
                    ],
                    "virtualPort": 0,
                    "persistenceMethods": [],
                    "snat": "none"
                },
                "pem_listeners": {
                    "enforcementProfile": {
                        "use": "/Common/Shared/pem_listeners_pem_profile"
                    },
                    "services": [
                        {
                            "use": "/Common/Shared/pem_listeners_ANY_IP"
                        },
                        {
                            "use": "/Common/Shared/pem_listeners_HTTP"
                        },
                        {
                            "use": "/Common/Shared/pem_listeners_L4_1"
                        },
                        {
                            "use": "/Common/Shared/pem_listeners_L4_2"
                        }
                    ],
                    "class": "Enforcement_Listener"
                },
                "policy_subs": {
                    "rules": [
                        {
                            "name": "rule_subs",
                            "precedence": 10,
                            "modifyHttpHeader": {
                                "headerName": "Encoding",
                                "operation": "insert",
                                "valueContent": "BLABLA"
                            },
                            "DTOSTethering": {
                                "detectDtos": true,
                                "detectTethering": true,
                                "reportDestinationHsl": {
                                    "highSpeedLogPublisher": {
                                        "bigip": "/Common/local-db-publisher"
                                    }
                                }
                            }
                        }
                    ],
                    "class": "Enforcement_Policy"
                },
                "policy_unknown": {
                    "enable": false,
                    "class": "Enforcement_Policy"
                },
                "pem_listeners_pem_profile": {
                    "remark": "em_listeners",
                    "connectionOptimizationEnabled": true,
                    "connectionOptimizationService": {
                        "use": "/Common/Shared/pem_listeners_FAST_L4"
                    },
                    "class": "Enforcement_Profile"
                },
                "test_addr_list": {
                    "addresses": [
                        "10.144.65.121",
                        "10.145.65.122"
                    ],
                    "remark": "test_addr_list",
                    "class": "Firewall_Address_List"
                }
            }
        },
        "updateMode": "selective",
        "controls": {
            "archiveTimestamp": "2021-02-24T11:07:33.303Z"
        }
    }
}

[azahajkiewicz]

This scenario indicates three problems:

• referencing to the object which is not supported by Charon - lsn-pool (there is no such class in schema reference, and pool class is not the same as lsn-pool, at least not from the BIG-IP point of view),

• wrong reference (probably Charon was supposed to convert lsn-pool and place it in /Common/Shared, but declaration does not contain it)

which lead to the main problem:

• putting a(ny) pool in snat field

[mdditt2000]

@azahajkiewicz based on users input above a snat pool was provided.

    source-address-translation {
        pool /Common/lsn_pool_test
        type lsn

Therefore ACC outputs the snat pool

       "snat": {
                    "use": "/Common/Shared/lsn_pool_test"
                }

ACC shouldn't change snat options "auto", "none", "self". if this wasn't the input

[rjouhann]

@mdditt2000 I think @azahajkiewicz point was the AS3 converted declaration is pointing to an object which does not exist in the converted as3 declaration: /Common/Shared/lsn_pool_test.

ltm lsn-pool /Common/lsn_pool_test {
    egress-interfaces {
        /Common/vlan2
    }
    egress-interfaces-disabled
    members {
        10.10.10.0/23
    }
}

/Common/lsn_pool_test is not converted/represented in the ACC output.

[mdditt2000]

@rjouhann ACC like the /Shared tomuch :). I can file a defect on this to make sure that ACC output the snat pool as

    "snat": {
                    "use": "/Common/Isn_pool_test"
                }

[rjouhann]

@mdditt2000 actually it should be:

    "snat": {
                    "bigip": "/Common/Isn_pool_test"
                }

Then journey can take care of removing it for velos migration scenario.

[mdditt2000]

Filed CHARON-180 in Jira for this issue.

[mdditt2000]

Merge branch 'snat_pool' into 'develop' - Incorrect use of /Common/Shared/snat-pool

[mdditt2000]

resolved snat pool issue. correct use of snat-pool now at /Common/snat-pool. Coming in release ACC 1.11. Closing issue