Refine unit tests

The new test:bb and test:jvm bb tasks replace the test task. I think this makes bb and jvm platform tests easier to discover. Extracted out test matrix generation to its own test-matrix bb task (was formerly invoked via a test task cmd line arg). Now employing babashka cli for test tasks. Using for both test:jvm and test:bb. Can now choose to run individual tests vars and namespaces. Our test tasks now accept --nses --patterns and --vars options which we pass thru to the cognitect test runner. Updates to babashka allow us to now no longer rely on babashka: - namespace fork (was need for test runner) - babashka spec.alpha fork (ide feature relies on spec.alpha). Closes clj-commons#507 For consitency: - dev task renamed to dev:jvm - dev-bb task renamed dev:bb. Adjusted this one to match test:bb re classpath construction. When I sent in my own --classpath I think Clojure's spec.alpha was getting in the way. Now explicitly relying on slingshot. Closes clj-commons#511 Docker run task corrected to accept args. User and dev guides updated accordingly. Closes clj-commons#506
lread · Dec 2, 2022 · fc7ab74 · fc7ab74
1 parent 829a899
commit fc7ab74
Show file tree

Hide file tree

Showing 10 changed files with 261 additions and 197 deletions.
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
@@ -43,7 +43,7 @@ jobs:
     - id: set-tests
       name: Set test var for matrix
       # run test.clj directly instead of via bb task to avoid generic task output
-      run: echo "tests=$(bb script/test.clj matrix-for-ci --format=json)" >> $GITHUB_OUTPUT
+      run: echo "tests=$(bb script/test_matrix.clj --format json)" >> $GITHUB_OUTPUT
 
   build:
     needs: setup

diff --git a/CHANGELOG.adoc b/CHANGELOG.adoc
@@ -11,6 +11,7 @@ A release with an intentional breaking changes is marked with:
 * bumped Etaoin dependencies
 * docs:
 ** https://github.com/clj-commons/etaoin/issues/447[#447]: Describe testing without a display in the link:doc/01-user-guide.adoc#headless-testing[user guide], including a new requirement for a windows manager when using a virtual display on Linux
+** https://github.com/clj-commons/etaoin/issues/507[#507]: The current version of babashka, which is the only one we support, no longer requires the a dependency to the babashka spec.alpha fork to use the Etaoin ide feature. Tests and docs updated accordingly.
 
 == v1.0.38 [minor breaking]
 

diff --git a/bb.edn b/bb.edn
@@ -11,24 +11,48 @@
              [clojure.string :as string]
              [helper.shell :as shell]
              [lread.status-line :as status])
-  :enter (let [{:keys [name]} (current-task)] (status/line :head "TASK %s %s" name (string/join " " *command-line-args*)))
-  :leave (let [{:keys [name]} (current-task)] (status/line :detail "\nTASK %s done." name))
+  :enter (let [{:keys [name]} (current-task)]
+           (when-not (string/starts-with? name "-")
+             (status/line :head "TASK %s %s" name (string/join " " *command-line-args*))))
+  :leave (let [{:keys [name]} (current-task)]
+           (when-not (string/starts-with? name "-")
+             (status/line :detail "\nTASK %s done." name)))
 
   ;; commands
-
-  dev            {:doc "start a Clojure nrepl server/prompt"
+  dev:jvm        {:doc "start a Clojure nrepl server/prompt"
                   :task (shell/command "clj" "-M:test:repl/cider")}
-  bb-dev         {:doc "start a Babashka nrepl server"
-                  :task (let [cp (-> (shell/clojure "-Spath -M:test:bb-spec:bb-test")
-                                     with-out-str)
-                              bbcp (cp/get-classpath)]
-                          (shell/command "bb"
-                                         "--classpath" (str cp fs/path-separator bbcp)
-                                         "--nrepl-server"))}
-  test           {:doc "run all or a subset of tests, use --help for args"
-                  :task test/-main}
+  dev:bb         {:doc "start a Babashka nrepl server"
+                  ;; repeat :test paths from deps.edn
+                  :extra-paths ["test" "env/test/resources"]
+                  :extra-deps  {;; inherit base deps from deps.edn
+                                etaoin/etaoin {:local/root "."}
+                                ;; repeat necessary :test deps from deps.edn
+                                io.github.cognitect-labs/test-runner {:git/tag "v0.5.1" :git/sha "dfb30dd"}}
+                  :task (babashka.nrepl.server/start-server!)}
+  test:jvm       {:doc "Runs tests under JVM Clojure [--help]"
+                  :task test/test-jvm}
+
+  -test:bb       {:doc "bb test runner, invoked within script/test.clj"
+                  :requires ([taoensso.timbre :as timbre])
+                  ;; repeat :test paths from deps.edn
+                  :extra-paths ["test" "env/test/resources"]
+                  :extra-deps  {;; inherit base deps from deps.edn
+                                etaoin/etaoin ({:local/root "."})
+                                ;; repeat necessary :test deps from deps.edn
+                                io.github.cognitect-labs/test-runner {:git/tag "v0.5.1" :git/sha "dfb30dd"}}
+                  :task (do
+                          ;; timbre default logging level is debug, which generates a lot of http logging noise
+                          (timbre/set-level! :info)
+                          (exec 'cognitect.test-runner.api/test))
+                  :org.babashka/cli {:coerce {:nses [:symbol]
+                                              :patterns [:string]
+                                              :vars [:symbol]}}}
+  test:bb        {:doc "Runs tests under Babashka [--help]"
+                  :task test/test-bb}
   test-doc       {:doc "test code blocks in user guide"
                   :task test-doc/-main}
+  test-matrix    {:doc "Returns a test matrix for CI [--help]"
+                  :task test-matrix/-main}
   drivers        {:doc "[list|kill] any running WebDrivers"
                   :task drivers/-main}
   lint           {:doc "[--rebuild] lint source code"
@@ -49,10 +73,10 @@
   docker-run     {:doc "run etaoin docker image (specify no commmands for interactive)"
                   :task (apply shell/command {:continue true}
                                "docker run -it --rm -v"
-                               (str (fs/cwd) ":/etaoin")
-                               "-w" "/etaoin"
-                               "--entrypoint" "/bin/bash"
-                               "etaoin:latest"
-                               *command-line-args*)}
+                               (cond-> [(str (fs/cwd) ":/etaoin")
+                                        "-w" "/etaoin"]
+                                 (when-not (seq *command-line-args*)) (conj "--entrypoint" "/bin/bash")
+                                 :always (conj "etaoin:latest")
+                                 :always (concat *command-line-args*)))}
   ci-release     {:doc "release tasks, use --help for args"
                   :task ci-release/-main }}}
diff --git a/deps.edn b/deps.edn
@@ -4,6 +4,7 @@
         babashka/process {:mvn/version "0.3.11"}
         clj-http/clj-http {:mvn/version "3.12.3"} ;; for jvm use
         org.clj-commons/clj-http-lite {:mvn/version "1.0.13"} ;; for babashka use
+        slingshot/slingshot {:mvn/version "0.12.2"}
         cheshire/cheshire {:mvn/version "5.11.0"}
         org.clojure/tools.cli {:mvn/version "1.0.214"}
         org.clojure/tools.logging {:mvn/version "1.2.4"}}
@@ -12,17 +13,16 @@
   :debug {:extra-paths ["env/dev/resources"]}
   :test {:extra-paths ["test" "env/test/resources"]
          :extra-deps {io.github.cognitect-labs/test-runner {:git/tag "v0.5.1" :git/sha "dfb30dd"}
+                      org.babashka/cli {:mvn/version "0.5.40"}
                       ch.qos.logback/logback-classic {:mvn/version "1.4.5"}
                       ;; for http-client which uses apache http client 4.x which uses commons logging
                       org.slf4j/jcl-over-slf4j {:mvn/version "2.0.5"}}
-         :main-opts ["-m" "cognitect.test-runner"]}
+         :exec-fn cognitect.test-runner.api/test
+         :org.babashka/cli {:coerce {:nses [:symbol]
+                                     :patterns [:string]
+                                     :vars [:symbol]}}
+         :main-opts ["-m" "babashka.cli.exec"]}
   :script {:extra-paths ["script"]}
-  ;; for babashka testing, needed for etaoin.ide
-  :bb-spec {:extra-deps {org.babashka/spec.alpha {:git/url "https://github.com/babashka/spec.alpha"
-                                                  :sha "8df0712896f596680da7a32ae44bb000b7e45e68"}}}
-  ;; for babashka testing, allows us to use cognitect test-runner
-  :bb-test {:extra-deps {org.clojure/tools.namespace {:git/url "https://github.com/babashka/tools.namespace"
-                                                      :git/sha "16b8c53174a5c9d89d6e0ce4128aa30b071aabdf"}}}
 
   ;; test-doc-blocks - gen tests
   :test-doc-blocks {:replace-deps {org.clojure/clojure {:mvn/version "1.11.1"}

diff --git a/doc/01-user-guide.adoc b/doc/01-user-guide.adoc
@@ -108,18 +108,6 @@ Add the following under `:deps` to your `bb.edn` file:
    etaoin/etaoin {:mvn/version "{lib-version}"}
 ----
 
-The Etaoin feature to <<selenium-ide, run Selenium IDE files>> employs clojure spec.
-If you are using this feature, you'll need to also enable clojure spec support in Babashka by adding `babashka/spec.alpha` to your `bb.edn` `:deps`:
-
-//:test-doc-blocks/skip
-[source,clojure]
-----
-   org.babashka/spec.alpha {:git/url "https://github.com/babashka/spec.alpha"
-                            :sha "8df0712896f596680da7a32ae44bb000b7e45e68"}
-----
-
-See https://github.com/babashka/spec.alpha[babashka/spec.alpha] for current docs.
-
 :url-webdriver: https://www.w3.org/TR/webdriver/
 :url-tests: https://github.com/{project-src-coords}/blob/master/test/etaoin/api_test.clj
 :url-chromedriver: https://sites.google.com/chromium.org/driver/

diff --git a/doc/02-developer-guide.adoc b/doc/02-developer-guide.adoc
@@ -67,9 +67,6 @@ All documentation is written in AsciiDoc.
 
 We host our docs on cljdoc and have support for <<cljdoc-preview,previewing>>
 
-
-
-
 == Babashka Tasks
 
 We use Babashka tasks, to see all available tasks run:
@@ -84,13 +81,13 @@ bb tasks
 For a Clojure REPL
 [source,shell]
 ----
-bb dev
+bb dev:jvm
 ----
 
 For a babashka REPL
 [source,shell]
 ----
-bb bb-dev
+bb dev:bb
 ----
 
 === Checking Tools Versions
@@ -105,17 +102,15 @@ bb tools-versions
 [[running-tests]]
 === Runing tests
 
-The `test` task provides a coarse grained facility to invoke tests.
-It was written to satisfy the use case of running tests in parallel on GitHub Actions.
-
+Use the `test:jvm` and `test:bb` tasks to invoke tests.
 
 [source,shell]
 ----
-bb test --help
+bb test:jvm --help
+bb test:bb --help
 ----
 
-We'll likely add finer grained test selection to satisfy developer needs.
-For now, temporarily tweak `./script/test.clj` if you need to.
+You can choose to invoke a test suite or individual tests.
 
 ==== Testing User Guide Code Blocks
 

diff --git a/script/bb_test_runner.clj b/script/bb_test_runner.clj