Merge Deployment Files Error

[tfullerbmx]

When running the merge deployment files playbook the perform deployment file merges task fails.
Here's the error:
TASK [Perform deployment file merges] *******************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: configparser.DuplicateOptionError: While reading from '' [line 51]: option '99999;' in section 'CoreServices' already exists fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n File \"/tmp/ansible_merge_viya_deployment_files_payload_8hrpjena/__main__.py\", line 172, in read_inventory\n File \"/usr/lib64/python3.6/configparser.py\", line 697, in read\n self._read(fp, filename)\n File \"/usr/lib64/python3.6/configparser.py\", line 1080, in _read\n raise MissingSectionHeaderError(fpname, lineno, line)\nconfigparser.MissingSectionHeaderError: File contains no section headers.\nfile: 'inventory.tmp', line: 1\n'deployTarget ansible_connection=local consul_bind_adapter=eth0\\n'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/tmp/.root.ansible/ansible-tmp-1608652696.3518848-28277903391481/AnsiballZ_merge_viya_deployment_files.py\", line 114, in \n _ansiballz_main()\n File \"/tmp/.root.ansible/ansible-tmp-1608652696.3518848-28277903391481/AnsiballZ_merge_viya_deployment_files.py\", line 106, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/tmp/.root.ansible/ansible-tmp-1608652696.3518848-28277903391481/AnsiballZ_merge_viya_deployment_files.py\", line 49, in invoke_module\n imp.load_module('__main__', mod, module, MOD_DESC)\n File \"/usr/lib64/python3.6/imp.py\", line 235, in load_module\n return load_source(name, filename, file)\n File \"/usr/lib64/python3.6/imp.py\", line 170, in load_source\n module = _exec(spec, sys.modules[name])\n File \"\", line 618, in _exec\n File \"\", line 678, in exec_module\n File \"\", line 219, in _call_with_frames_removed\n File \"/tmp/ansible_merge_viya_deployment_files_payload_8hrpjena/__main__.py\", line 661, in \n File \"/tmp/ansible_merge_viya_deployment_files_payload_8hrpjena/__main__.py\", line 606, in main\n File \"/tmp/ansible_merge_viya_deployment_files_payload_8hrpjena/__main__.py\", line 179, in read_inventory\n File \"/usr/lib64/python3.6/configparser.py\", line 763, in readfp\n self.read_file(fp, source=filename)\n File \"/usr/lib64/python3.6/configparser.py\", line 718, in read_file\n self._read(f, source)\n File \"/usr/lib64/python3.6/configparser.py\", line 1092, in _read\n fpname, lineno)\nconfigparser.DuplicateOptionError: While reading from '' [line 51]: option '99999;' in section 'CoreServices' already exists\n", "module_stdout": "2020-12-22 15:58:16,719 filemerge [INFO] \n2020-12-22 15:58:16,719 filemerge [INFO] Process that merges SAS Viya deployment files started.\n2020-12-22 15:58:16,719 filemerge [INFO] Current version of the merge_viya_deployment_files script: 19w34\n---\n# Recommended Ansible version\nMAXIMUM_RECOMMENDED_ANSIBLE_VERSION: 2.8\n\n# DEPLOYMENT_LABEL sets the datacenter for the Consul server. In addition, any\n# local Consul agents will use DEPLOYMENT_LABEL to become members of a\n# multi-machine deployment. A valid label can contain only lowercase characters,\n# numeric characters, and hyphens.\n\nDEPLOYMENT_LABEL: "{{ DEPLOYMENT_ID }}"\n\n# run deployment assessment during playbook execution\nVERIFY_DEPLOYMENT: true\n\n# Determine how Consul's ports are configured. Setting this variable to false\n# will only open the HTTP port (8500). Setting this variable to true will enable\n# the combination of the HTTP port (8500) and the HTTPS port (8501).\nSECURE_CONSUL: true\n\n# Toggles the status of the Consul port operating over HTTP.\n#\n# If this variable is set to true when SECURE_CONSUL is true,\n# then Consul will only be able to communicate over HTTPS (port 8501).\n#\n# If this variable is set to false when SECURE_CONSUL is true,\n# then Consul will be able to communicate over both HTTP (port 8500),\n# and HTTPS (port 8501).\n#\n# This setting is only honored when SECURE_CONSUL is true.\nDISABLE_CONSUL_HTTP_PORT: true\n\n# Path to the HTTPD certificate to distribute into SAS truststores\nHTTPD_CERT_PATH: "/etc/pki/tls/certs/localhost.crt"\n\n# The name of the license file on the Ansible machine.\nLICENSE_FILENAME: "SASViyaV0300_9CD1PD_Linux_x86-64.txt"\n\n# The name of the composite license file on the Ansible machine.\n# If both files are present, the playbook will use the\n# composite license file.\nLICENSE_COMPOSITE_FILENAME: "SASViyaV0300_9CD1PD_70201999_Linux_x86-64.jwt"\n\n# The default value of this variable is the base URL of the repository warehouse used when\n# the playbook was generated. After playbook generation, this variable can be used to manage\n# changes to the repository warehouse location. For example, if a mirrored repository is used\n# and that mirror location changes, that change can be managed with this variable.\n#\n# If the SAS packages are already available to the server (e.g. through a RHN satellite channel),\n# set this variable to "none" to indicate no additional repository is needed for deployment.\nREPOSITORY_WAREHOUSE: "https://ses.sas.download/ses/\"\n\n# Multiple invocation definitions\n# The machine reference (with deployTarget as the default) must be enclosed\n# in quotes if it is set to a value that starts with a number.\nINVOCATION_VARIABLES:\n deployTarget:\n pgpoolc:\n - PCP_PORT: '5430'\n PGPOOL_PORT: '5431'\n SANMOUNT: '{{ SAS_CONFIG_ROOT }}/data/sasdatasvrc'\n SERVICE_NAME: postgres\n sasdatasvrc:\n - NODE_NUMBER: '0'\n NODE_TYPE: P\n PG_PORT: '5432'\n SANMOUNT: '{{ SAS_CONFIG_ROOT }}/data/sasdatasvrc'\n SERVICE_NAME: postgres\n\n\n############################################################################\n\n# When sas_install_java is set to true, the deployment will install the SAS \n# default Java and use this Java during processing. If Java is\n# already available on the system, set sas_install_java to false and define \n# the path in sasenv_java_home below. When sas_install_java\n# is set to false and a custom path is not provided in sasenv_java_home, \n# the deployment will look for Java via the JAVA_HOME environment \n# variable first and if Java is not found, the deployment will look \n# at "which java". In the case that no Java is found and sas_install_java\n# is set to false, this will stop the deployment.\n#\n# If sas_install_java is set to true and Java is already on the system,\n# for Linux hosts the Java alternatives may change once the SAS deployment \n# is complete. Once the deployment completes, validate that the Java alternatives \n# are correct for your environment. If this action is not desired, set \n# sas_install_java to false and install Java prior to running the SAS \n# deployment and set the path to Java in sasenv_java_home.\nsas_install_java: true\n\n# User specific location of where Java resides on the host. \n# If not set, the deployment will use the Java defined on the host.\n# This is an optional setting.\n# Example:\n# sasenv_java_home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.101-3.b13.el6_8.x86_64/jre\nsasenv_java_home:\n\n# When set to true will try and create any user in the sas_users group below.\n# This will also create and setup ssh keys for that user across all hosts\n# default is true\nsetup_sas_users: true\n\n# Define the users so they can be created. The block includes user ID cas which should remain\n# so that SSH keys can be assigned to it. If you did not create a cas user named "cas", replace\n# "cas" below with the user you did create. Recreate the same block for any other users you\n# want to add. See the deployment guide for more information.\n# To assign a password, here is a way to create the password. \n# This will require the passlib library to be installed: sudo yum install python-passlib\n# At a command line prompt run:\n# python -c "from passlib.hash import sha512_crypt; import getpass; print sha512_crypt.encrypt(getpass.getpass(), rounds=10000)"\n# Take the value that is returned and paste it inbetween the single quotes below. Example:\n# password: 'secret'\n# If setup_home is set to true, will use the values of shell and home when creating the user\nsas_users:\n cas:\n group: sas\n password: ''\n setup_home: false\n shell:\n home:\n\n#############################################################################################\n#\n# DEPRECATION WARNING!!\n#\n# The two variables, setup_sas_packages and extra_packages, are deprecated and will be\n# removed in a future version of SAS Viya.\n#############################################################################################\n# When set to true, will install any additional packages defined in the extra_packages group\nsetup_sas_packages: false\n\n# Define the extra packages that need to be installed on all machines\nextra_packages:\n libselinux-python: support copying files\n#############################################################################################\n\n# When set to true, enables extra logging during the deployment\nsas_deploy_debug: false\n\n\n############################################################################\n## CAS Configuration\n############################################################################\n\n# The user that the CAS process will run under\n\ncasenv_user: cas\n\n# The group that the CAS user belongs to\n\ncasenv_group: sas\n\n# The following is the initial Admin user for use with CAS Server Monitor.\n# This is the user you will log into CAS Server Monitor with\n# in order to create global CAS libs and set access rights.\n# If not set, the casenv_user will be used by default.\n# If all defaults are taken, the "cas" user will not have a password\n# defined for it. To have one created by the deployment process,\n# review how to define a password as documented with the sas_users\n# collection above.\n\n#casenv_admin_user:\n\n\n#### CAS Specific ####\n# Anything in this list will end up in the cas.settings file\nCAS_SETTINGS:\n 1: ODBCSYSINI=/etc\n 2: ODBCINI=odbc.ini\n 3: ODBCINSTINI=odbcinst.ini\n 4: JAVA_HOME=/usr/lib/jvm/jre-1.8.0\n 5: LD_LIBRARY_PATH=/opt/microsoft/msodbcsql17/lib64/:$LD_LIBRARY_PATH\n\n# Anything in this list will end up in the casconfig.lua file\n# The env section will create a env.VARIABLE in the file\n# Example: env.CAS_DISK_CACHE = '/tmp'\n# The cfg section will create a cas.variable in the file\n# Example: cas.port = 5570\n#\n# If you have defined hosts for the sas_casserver_worker then the MODE will\n# automatically be set to 'mpp'. If the environment variables HADOOP_HOME and\n# HADOOP_NAMENODE are set, the COLOCATION option will automatically equal 'hdfs'.\n# If HADOOP_HOME and HADOOP_NAMENODE are not set, then the COLOCATION option\n# will automatically equal 'none'.\n\nCAS_CONFIGURATION:\n env:\n #CAS_DISK_CACHE: /tmp\n #CAS_VIRTUAL_HOST: 'loadbalancer.company.com'\n #CAS_VIRTUAL_PROTO: 'https'\n #CAS_VIRTUAL_PORT: 443\n cfg:\n #gcport: 0\n #httpport: 8777\n #port: 5570\n #colocation: 'none'\n #SERVICESBASEURL: 'https://loadbalancer.company.com'\n\n############################################################################\n## Foundation Configuration\n############################################################################\n\n# Optional: Will use the CAS controller host as defined in the inventory\n# file. If one is not defined it will default to localhost.\n# If you know the host of the controller you want to connect to,\n# provide that here\n#sasenv_cas_host:\n\n# Optional: If a value is not provided, the system will use the CAS port\n# as defined for the CAS controller.\n# If you know the port of the grid you want to connect to,\n# provide that here.\n#sasenv_cas_port:\n\n# Set the ports that SAS/CONNECT will listen on\n#sasenv_connect_port: 17551\n#sasenv_connect_mgmt_port: 17541\n\n# Updates the init_deployment.properties and appserver_deployment.sh\nSTUDIO_CONFIGURATION:\n init:\n #sasstudio.appserver.port_comment: '# Port that Studio is listening on'\n #sasstudio.appserver.port: 7080\n #sasstudio.appserver.https.port: 7443\n #webdms.workspaceServer.hostName: localhost\n #webdms.workspaceServer.port: 8591\n webdms.showSystemRoot: false\n appserver:\n #1: '# Comment about KEY'\n #2: KEY="value with spaces"\n\n# Updates spawner.cfg\nSPAWNER_CONFIGURATION:\n #sasPort: 8591\n\n# Updates the workspaceserver autoexec_deployment.sas\n#WORKSPACESERVER_CONFIGURATION:\n #1: '/* Comment about key /'\n #2: key=value;\n\n# Creates a workspaceserver sasenv_deployment file\nFOUNDATION_CONFIGURATION:\n 1: ODBCSYSINI=/etc\n 2: ODBCINI=odbc.ini\n 3: ODBCINSTINI=odbcinst.ini\n 4: LD_LIBRARY_PATH=/opt/microsoft/msodbcsql17/lib64/:$LD_LIBRARY_PATH\n\n# Creates a workspaceserver sasv9_deployment.cfg file\n#SASV9_CONFIGURATION:\n #1: '/ Comment about OPTION */'\n #2: 'OPTION value'\n2020-12-22 15:58:16,768 filemerge [INFO] The merge_default_host is: deployTarget\n", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

Environment

• Ansible version:

• ansible 2.8.0
  config file = /opt/sas/update35/sas_viya_playbook/ansible.cfg
  configured module search path = ['/opt/sas/update35/sas_viya_playbook/library']
  ansible python module location = /usr/local/lib/python3.6/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.6.8 (default, Aug 13 2020, 07:46:32) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]

• python --version
  Python 2.7.5

• OS version:
  Red Hat Enterprise Linux Server release 7.9 (Maipo)

• Failed playbook tasks log (or entire playbook log) [Attach]

• What version of Viya 3.x is being deployed?
  3.5

To Reproduce
Steps to reproduce the behavior:

1. Firstly, run the following: ansible-playbook viya-ark-master/playbooks/merge-playbook/merge-viya-deployment-files.yml -e "current_inventory_file=/opt/sas/update_recent/sas_viya_playbook/inventory.ini" -e "merge_default_host=deployTarget"

2. Then, playbook fails at the above mentioned step

Expected behavior
Playbook to merge deployment files as intended.

[kevinlinglesas]

Thanks @tfullerbmx for the feedback. I'm pretty certain we recently came across this issue while doing some regression testing. A couple of issues were uncovered and we have changes that need to get aligned with a future release. Note also, that while your OS env may be python 2.7, it looks like ansible is using python 3.6.8. We were testing with python 3 when we uncovered the issue.

[tfullerbmx]

Yup, you're right. I tried with both variants. Thanks for the quick follow and the work you guys do.

[kevinlinglesas]

Fixed by Viya35-ark-1.12.