fix defaults for node_selector and tolerations

[dmarav]

When setting node_selector and tolerations values for all deployments/staefulsets, the playbook in the operator fails with the following error:

unsupported operand type(s) for +=: 'dict' and 'str'

The CRD defines the tolerations as an array, and the node_selector as an object.

The defaults for the roles are empty strings, so when trying to combine the <component> and _<component> dictionaries to render the final configuration, a string is added to an array/dict and I get the aforementioned error.

It can be reproduced with this manifest using operator version 1.02:

apiVersion: eda.ansible.com/v1alpha1
kind: EDA
metadata:
  name: eda-demo
  namespace: eda
spec:
  no_log: false
  automation_server_url: http://awx-service.awx
  image: quay.io/ansible/eda-server
  image_version: main
  image_web: quay.io/ansible/eda-ui
  image_web_version: main
  ipv6_disabled: true
  ingress_type: ingress
  ingress_path: "/"
  ingress_path_type: Prefix
  hostname: my-eda.example.com
  api:
    node_selector:
      system/dedicated: my-node-group
    tolerations:
      - key: "system/dedicated"
        operator: "Equal"
        value: "my-node-group"
        effect: "NoSchedule"
  ui:
    node_selector:
      system/dedicated: my-node-group
    tolerations:
      - key: "system/dedicated"
        operator: "Equal"
        value: "my-node-group"
        effect: "NoSchedule"
  scheduler:
    node_selector:
      system/dedicated: my-node-group
    tolerations:
      - key: "system/dedicated"
        operator: "Equal"
        value: "my-node-group"
        effect: "NoSchedule"
  default_worker:
    node_selector:
      system/dedicated: my-node-group
    tolerations:
      - key: "system/dedicated"
        operator: "Equal"
        value: "my-node-group"
        effect: "NoSchedule"
  activation_worker:
    node_selector:
      system/dedicated: my-node-group
    tolerations:
      - key: "system/dedicated"
        operator: "Equal"
        value: "my-node-group"
        effect: "NoSchedule"
  redis:
    node_selector:
      system/dedicated: my-node-group
    tolerations:
      - key: "system/dedicated"
        operator: "Equal"
        value: "my-node-group"
        effect: "NoSchedule"
  database:
    node_selector:
      system/dedicated: my-node-group
    tolerations:
      - key: "system/dedicated"
        operator: "Equal"
        value: "my-node-group"
        effect: "NoSchedule"

[rooftopcellist]

@dmarav early on, in the awx-operator, we naively used a string value for this and documented specifying the values like this:

• https://ansible.readthedocs.io/projects/awx-operator/en/latest/user-guide/advanced-configuration/assigning-awx-pods-to-specific-nodes.html?h=node_selector#assigning-awx-pods-to-specific-nodes

Which should also work for the eda-server-operator.

We plan to cut a new apiVersion for the awx-operator with improvements like these in the coming year. I think it we change it now without backwards compat, it may break users who have been defining it as a string on the CR spec...

I think we should probably leave this PR open until we do the new apiVersion for both AWX and EDA...

cc @TheRealHaoLiu @rcarrillocruz re: new apiVersion

[dmarav]

Hi @rooftopcellist! Thanks for the quick response!

@dmarav early on, in the awx-operator, we naively used a string value for this and documented specifying the values like this:

• https://ansible.readthedocs.io/projects/awx-operator/en/latest/user-guide/advanced-configuration/assigning-awx-pods-to-specific-nodes.html?h=node_selector#assigning-awx-pods-to-specific-nodes

This was my first approach, to set the tolerations and node_selector as string, since that's how it was defined in the awx-operator, but in the CRD node_selector is defined as a dict and tolerations is defined as a list of dicts.

Which should also work for the eda-server-operator.

It doesn't because the CRD expects one type, and the defaults set in the roles is string, so when you combine the defaults with the user provided values, ansible fails with the error in my first comment. It cannot combine a dictionary/list with a string.

With this manifest:

---
apiVersion: eda.ansible.com/v1alpha1
kind: EDA
metadata:
  name: eda-dbops
  namespace: eda
spec:
  no_log: false
  automation_server_url: http://awx-service.awx
  image: quay.io/ansible/eda-server
  image_version: main
  image_web: quay.io/ansible/eda-ui
  image_web_version: main
  ipv6_disabled: true
  ingress_type: ingress
  ingress_path: "/"
  ingress_path_type: Prefix
  hostname: my-eda.example.com
  ui:
    node_selector: |
      system/dedicated: my-node-group
    tolerations: |
      - key: "system/dedicated"
        operator: "Equal"
        value: "my-node-group"
        effect: "NoSchedule"

I get this response from kubernetes

# kubectl -n eda apply -f  eda-test-node-selector.yml
The EDA "eda-dbops" is invalid:
* spec.ui.tolerations: Invalid value: "string": spec.ui.tolerations in body must be of type array: "string"
* spec.ui.node_selector: Invalid value: "string": spec.ui.node_selector in body must be of type object: "string"

[rooftopcellist]

@dmarav you are absolutely right. The CRD schema expects object/dict for these settings, which does not line up with how that string is later templated into the deployment yaml.

I agree with your approach here and believe that we should standardize all of the ansible related operator around having node_selector etc. be a dict rather than a string (which necessitated the confusing | syntax).

I plan to test this out this week and get it merged. For awx-operator, we'll need to wait until we cut a new apiVersion, which is something we are planning to tackle in the coming months. Thanks for the contribution!

[rooftopcellist]

It would be great to have some docs for how to use these in this repo, particularly because it will be different from the awx-operator for a time. Help appreciated there, but I won't let that block this PR from merging.

[dmarav]

Hi @rooftopcellist! Should I create a separate PR for updating the docs and link to this one, or just update the docs in this one?

[rooftopcellist]

@dmarav I just merged, sorry for the delay. If you can open a separate PR for docs on how to use this, that would be wonderful!

[dmarav]

@dmarav I just merged, sorry for the delay. If you can open a separate PR for docs on how to use this, that would be wonderful!

@rooftopcellist I just created PR #214 with the documentation. I copied the same format/structure as in the AWX operator.