Integrated e2e framework in drenv

Dump cluster kubconfigs and e2e config.yaml when starting an
environment. The config.yaml can be used with the e2e framework to run
tests using the environment.

To test integration quickly, a new e2e environment was added.

Example run (using #764):

    $ (cd test && drenv start envs/e2e.yaml)
    2023-08-01 13:36:04,454 INFO    [e2e] Starting environment
    2023-08-01 13:36:04,506 INFO    [dr1] Starting minikube cluster
    2023-08-01 13:36:05,534 INFO    [dr2] Starting minikube cluster
    2023-08-01 13:36:06,525 INFO    [hub] Starting minikube cluster
    2023-08-01 13:36:40,727 INFO    [dr1] Cluster started in 36.22 seconds
    2023-08-01 13:36:57,248 INFO    [dr2] Cluster started in 51.71 seconds
    2023-08-01 13:37:15,228 INFO    [hub] Cluster started in 68.70 seconds
    2023-08-01 13:37:15,228 INFO    [e2e] Dumping ramen e2e config to '/home/nsoffer/.config/drenv/e2e'
    2023-08-01 13:37:15,346 INFO    [e2e] Environment started in 70.89 seconds

    # There must be a better way...
    $ cp ~/.config/drenv/e2e/config.yaml e2e/config/config.yaml

    $ go test -v ./e2e
    === RUN   TestMain
    === RUN   TestMain/TestConnectionToCluster
    --- PASS: TestMain (0.02s)
        --- PASS: TestMain/TestConnectionToCluster (0.01s)
    PASS
    ok  	github.com/red-hat-storage/ramen/e2e	0.025s

    $ (cd test && drenv delete envs/e2e.yaml)
    2023-08-01 13:40:18,575 INFO    [e2e] Deleting environment
    2023-08-01 13:40:18,576 INFO    [dr1] Deleting cluster
    2023-08-01 13:40:18,577 INFO    [dr2] Deleting cluster
    2023-08-01 13:40:18,577 INFO    [hub] Deleting cluster
    2023-08-01 13:40:19,833 INFO    [dr1] Cluster deleted in 1.26 seconds
    2023-08-01 13:40:19,838 INFO    [hub] Cluster deleted in 1.26 seconds
    2023-08-01 13:40:19,839 INFO    [dr2] Cluster deleted in 1.26 seconds
    2023-08-01 13:40:19,839 INFO    [e2e] Removing config /home/nsoffer/.config/drenv/e2e
    2023-08-01 13:40:19,840 INFO    [e2e] Environment deleted in 1.26 seconds

Signed-off-by: Nir Soffer <[email protected]>

test/README.md

@@ -588,6 +588,7 @@ simpler and faster to work with a minimal environment.
 - `test.yaml` - for testing `drenv`
 - `example.yaml` - example for experimenting with the `drenv` tool
 - `demo.yaml` - interactive demo for exploring the `drenv` tool
+- `e2e.yaml` - example for testing integration with the e2e framework
 - `external.yaml` - example for using external clusters
 - `minio.yaml` - for testing `minio` deployment
 - `ocm.yaml` - for testing `ocm` deployment

test/drenv/__main__.py

@@ -17,6 +17,7 @@
 from . import commands
 from . import envfile
 from . import minikube
+from . import ramen
 
 CMD_PREFIX = "cmd_"
 ADDONS_DIR = "addons"
@@ -62,6 +63,9 @@ def cmd_start(env, args):
     )
     execute(run_worker, env["workers"], hooks=hooks)
 
+    if "ramen" in env:
+        ramen.dump_e2e_config(env)
+
     logging.info(
         "[%s] Environment started in %.2f seconds",
         env["name"],
@@ -84,6 +88,12 @@ def cmd_delete(env, args):
     start = time.monotonic()
     logging.info("[%s] Deleting environment", env["name"])
     execute(delete_cluster, env["profiles"])
+
+    env_config = drenv.config_dir(env["name"])
+    if os.path.exists(env_config):
+        logging.info("[%s] Removing config %s", env["name"], env_config)
+        shutil.rmtree(env_config)
+
     logging.info(
         "[%s] Environment deleted in %.2f seconds",
         env["name"],

test/drenv/ramen.py

@@ -1,8 +1,14 @@
 # SPDX-FileCopyrightText: The RamenDR authors
 # SPDX-License-Identifier: Apache-2.0
 
+import os
+import logging
+
 import yaml
 
+import drenv
+from . import kubectl
+
 
 def env_info(filename, name_prefix=None):
     """
@@ -18,3 +24,37 @@ def env_info(filename, name_prefix=None):
         info["clusters"] = [name_prefix + cluster for cluster in info["clusters"]]
 
     return info
+
+
+def dump_e2e_config(env):
+    """
+    Dump configuration for ramen e2e framework.
+    """
+    base = drenv.config_dir(env["name"])
+    logging.info("[%s] Dumping ramen e2e config to '%s'", env["name"], base)
+
+    kubeconfigs_dir = os.path.join(base, "kubeconfigs")
+    os.makedirs(kubeconfigs_dir, exist_ok=True)
+
+    e2e_config = {"Clusters": {}}
+
+    # Map e2e cluster names to actual cluster names.
+    clusters = zip(
+        [env["ramen"]["hub"], *env["ramen"]["clusters"]],
+        ["hub", "c1", "c2"],
+    )
+
+    for cluster_name, e2e_name in clusters:
+        if cluster_name is None:
+            continue
+
+        data = kubectl.config("view", "--flatten", context=cluster_name)
+        path = os.path.join(kubeconfigs_dir, cluster_name)
+        with open(path, "w") as f:
+            f.write(data)
+
+        e2e_config["Clusters"][e2e_name] = {"kubeconfigpath": path}
+
+    path = os.path.join(base, "config.yaml")
+    with open(path, "w") as f:
+        yaml.dump(e2e_config, f)

test/envs/e2e.yaml

@@ -0,0 +1,25 @@
+# SPDX-FileCopyrightText: The RamenDR authors
+# SPDX-License-Identifier: Apache-2.0
+
+# Enviroment for testing integration with e2e framework.
+---
+name: "e2e"
+
+ramen:
+  hub: hub
+  clusters: [dr1, dr2]
+  topology: regional-dr
+
+templates:
+  - name: cluster
+    driver: "$vm"
+    container_runtime: containerd
+    memory: "2g"
+
+profiles:
+  - name: dr1
+    template: cluster
+  - name: dr2
+    template: cluster
+  - name: hub
+    template: cluster