[CI] Use compatible version of heat-tempest-plugin

This is a combination of four patch to fix the gate:

1. Use compatible version of heat-tempest-plugin

heat-tempest-plugin dropped py38 support, hence it cannot be installed
from recent master, so we need to override the checkout to the version
that still supports py38 and compatible to the actual branch.

This is needed both for the grenade job and the functional job, because
these are using the heat-tempest-plugin.

2. Try archive path to download Fedora image

This is a temporal workaround to allow downloading Fedora 37 image
which was moved to the archive path.

3. Remove reference to devstack-gate

devstack-gate was deprecated in xena and is being retired now[1].

4. Set functional jobs as non-voting

heat-functional jobs try to install python-zaqarclient (via zaqar
project) from master branch, but on master the client dropped py38
support, hence the jobs fail. This patch sets it non-voting as a
quick workaround until a final fix arrives.

[1] https://review.opendev.org/c/openstack/governance/+/919629
[2] pypa/setuptools#4483

Changes:
  .zuul.yaml
  tox.ini

NOTE(elod.illes): change in .zuul.yaml is to adapt the patch to the
current branch ('<series>-last' needs to be used).
Also needed to remove grenade job as pre-victoria branches are all EOL.
periodic-stable job template is unnecessary as well.
setuptools needed to be capped as well to make >py36 based jobs pass.

Change-Id: I9b1702749976a2cea42a24130e5fec2931b75ce1
(cherry picked from commit a806b40)
(cherry picked from commit ae62258)
(cherry picked from commit 4fd9953)
(cherry picked from commit 81a8b93)
(cherry picked from commit 665ccfc)
(cherry picked from commit c18b344)
(cherry picked from commit dcf7403)
(cherry picked from commit 21ed95f)
(cherry picked from commit e742c31)

.zuul.yaml

@@ -10,11 +10,11 @@
       - zuul: opendev.org/openstack/devstack
       - zuul: opendev.org/openstack/tempest
     required-projects:
-      - openstack/devstack-gate
       - openstack/barbican
       - openstack/heat
       - openstack/heat-templates
-      - openstack/heat-tempest-plugin
+      - name: opendev.org/openstack/heat-tempest-plugin
+        override-checkout: victoria-last
       - openstack/octavia
       - openstack/neutron
       - openstack/oslo.messaging
@@ -23,7 +23,8 @@
       - openstack/heat-agents
       - openstack/python-zaqarclient
       - openstack/zaqar
-      - openstack/tempest
+      - name: openstack/tempest
+        override-checkout: victoria-last
     vars:
       configure_swap_size: 8192
       gabbi_tempest_path: heat_tempest_plugin.tests.api.gabbits
@@ -138,7 +139,8 @@
     parent: grenade-multinode
     required-projects:
       - opendev.org/openstack/heat
-      - opendev.org/openstack/heat-tempest-plugin
+      - name: opendev.org/openstack/heat-tempest-plugin
+        override-checkout: victoria-last
       - opendev.org/openstack/python-heatclient
     vars:
       grenade_devstack_localrc:
@@ -188,17 +190,24 @@
     templates:
       - openstack-cover-jobs
       - openstack-python3-victoria-jobs
-      - periodic-stable-jobs
       - publish-openstack-docs-pti
       - check-requirements
       - release-notes-jobs-python3
     check:
       jobs:
-        - grenade-heat-multinode
-        - heat-functional
-        - heat-functional-legacy
-    gate:
-      jobs:
-        - grenade-heat-multinode
-        - heat-functional
-        - heat-functional-legacy
+        # NOTE(elod.illes): functional jobs are failing due to
+        # python-zaqarclient dropped py38 support and even on
+        # this old branch the client is installed from master
+        # as a quick workaround this is now set as non-voting
+        - heat-functional:
+            voting: false
+        - heat-functional-legacy:
+            voting: false
+    # gate:
+    #   jobs:
+        # NOTE(elod.illes): functional jobs are failing due to
+        # python-zaqarclient dropped py38 support and even on
+        # this old branch the client is installed from master
+        # as a quick workaround this is now removed from gate
+        # - heat-functional
+        # - heat-functional-legacy

devstack/lib/heat

@@ -482,14 +482,17 @@ function configure_tempest_for_heat {
         source /etc/ci/mirror_info.sh
     fi
     HEAT_TEST_FEDORA_IMAGE_UPSTREAM=https://download.fedoraproject.org/pub/fedora/linux
-    HEAT_TEST_FEDORA_IMAGE_PATH=releases/36/Cloud/x86_64/images/Fedora-Cloud-Base-36-1.5.x86_64.qcow2
-    if curl --output /dev/null --silent --head --fail "${NODEPOOL_FEDORA_MIRROR}/${HEAT_TEST_FEDORA_IMAGE_PATH}"; then
+    HEAT_TEST_FEDORA_IMAGE_UPSTREAM_ARCHIVE=https://download.fedoraproject.org/pub/archive/fedora/linux/
+    HEAT_TEST_FEDORA_IMAGE_PATH=releases/37/Cloud/x86_64/images/Fedora-Cloud-Base-37-1.7.x86_64.qcow2
+    if curl --location --output /dev/null --silent --head --fail "${NODEPOOL_FEDORA_MIRROR}/${HEAT_TEST_FEDORA_IMAGE_PATH}"; then
         export HEAT_TEST_FEDORA_IMAGE="${NODEPOOL_FEDORA_MIRROR}/${HEAT_TEST_FEDORA_IMAGE_PATH}"
-    else
+    elif curl --location --output /dev/null --silent --head --fail "${HEAT_TEST_FEDORA_IMAGE_UPSTREAM}/${HEAT_TEST_FEDORA_IMAGE_PATH}"; then
         export HEAT_TEST_FEDORA_IMAGE="${HEAT_TEST_FEDORA_IMAGE_UPSTREAM}/${HEAT_TEST_FEDORA_IMAGE_PATH}"
+    else
+        export HEAT_TEST_FEDORA_IMAGE="${HEAT_TEST_FEDORA_IMAGE_UPSTREAM_ARCHIVE}/${HEAT_TEST_FEDORA_IMAGE_PATH}"
     fi
     TOKEN=$(openstack token issue -c id -f value)
-    local image_exists=$( openstack image list | grep "Fedora-Cloud-Base-36-1.5.x86_64" )
+    local image_exists=$( openstack image list | grep "Fedora-Cloud-Base-37-1.7.x86_64" )
     if [[ -z $image_exists ]]; then
         if is_service_enabled g-api; then
             upload_image $HEAT_TEST_FEDORA_IMAGE $TOKEN

tox.ini

@@ -9,6 +9,7 @@ skipsdist = True
 requires =
   virtualenv<20.24.0
   tox<4
+  setuptools<71.0.0
 
 [testenv]
 basepython = python3