Cannot set the @disabled flag dynamically

[alekdavisintel]

How do you disable a request only if a previously executed request's response matches certain conditions?

For example, say you need to test a CRUD sequence, so you first execute a POST request to create an object, but if the response status is not '201 Created', then the subsequent GET, UPDATE, and DELETE requests would be disabled. Or, say, you run a GET request and based on the returned JSON object (or collection), you may or may not disable another GET request (e.g. I call a GET method to obtain a list of roles and if among the returned role objects I find a role with the owner property that is not null, I would call a GET by owner request, passing the owner ID, but if none of the items have a non-null owner property, then the GET by owner request would be disabled).

I know how to get and pass values from one request to the other, it's just I cannot figure out how to use them to dynamically turn the @disabled flag on and off. I tried all kinds of things: setting a value in pre-request script, post-request script of the dependent request, using $global, export, @variable. No matter what variable I passed to the @disabled meta flag, the request was always disabled.

So, I backed all the way to basics and tried to use the example from https://httpyac.github.io/guide/metaData.html#disabled to make the request enabled:

@callRequest={{true}}

# @disabled
GET /anything
Content-Type: text/html
foo: {{foo}}

It didn't work:

Then I just hard coded false in the metadata, and it also did not work:

Only if I remove the @disabled meta flag, the request would get executed:

Am I missing something basic?

[AnWeber]

Unfortunately, I commited a broken example, probably during manual testing. The classic error, nobody can double negate. I just stumbled across it myself and only noticed it when debugging. Disabled=true means that the request should be executed. a working example is this code. The request does not get executed

@callRequest={{false}}
###

# @disabled !callRequest
GET https://httpbin.org/json

[alekdavisintel]

I must be missing something but it's still not working for me:

If I remove the @disabled meta tag line, the request gets executed, but once I add it, it does not matter if I put a true or false condition next to it, the requests is always disabled.

[AnWeber]

callRequest=true you want to call Request. Set callRequest to false. this does not execute the request for me

@callRequest={{false}}
###

# @disabled !callRequest
GET https://httpbin.org/json

[alekdavisintel]

I must not be expressing myself correctly. No matter what I set the @disabled value to, the request is always disabled.

How do I set the @disabled value to ENABLE the request?

[AnWeber]

It may just be a UX problem. The Send button will not disappear. For performance reasons. This is a problem in vscode.rest-client. It executes a lot of code in the background so that it can generate the preview for the variables. I have deliberately decided against this.

[alekdavisintel]

Ah, I see. The Send button is not really a problem. The problem is that once you execute all tests in a folder, the results for the ones marked with the @disabled tag will always appear as skipped, even if they did run:

So, there is no way to easily see which tests completed and which ones skipped by looking at the Test Results tab or Testing tree view.

[alekdavisintel]

And ignoring the UI issue for now, how do you make the @disabled argument dependent on the value returned from a previous request. Say, I run a query to GET slide show results from httpbin.org/json, so if I find an item with a specific tilte I would then call a GET request against httpbin.org/anthing, but if not, then the next request would be skipped.

Here is my script, but if I run all tests, the returned request info shows that both test 3 and 4 are skipped. Shouldn't test 3 or 4 (at least, one of them) run? Or am I not passing the exported variable to the @disabled tag correctly? The exported values work fine in the request, but I'm not sure how they can be used in the @disabled condition.

@validTitle1=Overview
@validTitle2=Wake up to WonderWidgets!
@invalidTitle=XYZ
###

### Test 0 (initial test which is a dependency for test 1, 2, 3, 4)
# @name results
GET https://httpbin.org/json

### Test 1 (ok)
# @ref results
{{
    var item = results.slideshow.slides.find(item => item.title == validTitle1)?.title;
    exports.item = item
}}
GET https://httpbin.org/anything/{{item}}

### Test 2 (error 'invalidItem is not defined' as expected)
# @ref results
{{
    var item = results.slideshow.slides.find(item => item.title == invalidTitle)?.title;
    exports.item = item
}}
GET https://httpbin.org/anything/{{item}}

### Test 3 (must be enabled since item is a valid string)
# @ref results
{{
    var item = results.slideshow.slides.find(item => item.title == validTitle2)?.title;
    exports.item = item
}}
# @disabled !item
GET https://httpbin.org/anything/{{item}}

### Test 4 (must be disabled since item is undefined or null)
# @ref results
{{
    var item = results.slideshow.slides.find(item => item.title == invalidTitle)?.title;
    exports.item = item
}}
# @disabled !item
GET https://httpbin.org/anything/{{item}}

Responses (with unnecessary blank lines removed):

---------------------
=== Test 0 (initial test which is a dependency for test 1, 2, 3, 4) ===

GET https://httpbin.org/json
accept-encoding: gzip, deflate, br
accept: */*
user-agent: httpyac

HTTP/1.1 200  - OK
access-control-allow-credentials: true
access-control-allow-origin: *
connection: close
content-length: 429
content-type: application/json
date: Fri, 24 May 2024 22:20:17 GMT
server: gunicorn/19.9.0

{
  "slideshow": {
    "author": "Yours Truly", 
    "date": "date of publication", 
    "slides": [
      {
        "title": "Wake up to WonderWidgets!", 
        "type": "all"
      }, 
      {
        "items": [
          "Why <em>WonderWidgets</em> are great", 
          "Who <em>buys</em> WonderWidgets"
        ], 
        "title": "Overview", 
        "type": "all"
      }
    ], 
    "title": "Sample Slide Show"
  }
}
---------------------
=== Test 1 (ok) ===

GET https://httpbin.org/anything/Overview
accept-encoding: gzip, deflate, br
accept: */*
user-agent: httpyac

HTTP/1.1 200  - OK
access-control-allow-credentials: true
access-control-allow-origin: *
connection: close
content-length: 392
content-type: application/json
date: Fri, 24 May 2024 22:20:17 GMT
server: gunicorn/19.9.0

{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip, deflate, br", 
    "Host": "httpbin.org", 
    "User-Agent": "httpyac", 
    "X-Amzn-Trace-Id": "Root=1-665112a1-4136e523715a2c931e9b17f7"
  }, 
  "json": null, 
  "method": "GET", 
  "origin": "192.55.54.50", 
  "url": "https://httpbin.org/anything/Overview"
}

---------------------
=== Test 2 (error 'invalidItem is not defined' as expected) ===

So, there are no logs for requests from either test 3 or 4. And once I picked the send selected option and selected just test 3 and 4, I got:

---------------------

=== Test 3 (must be enabled since item is a valid string) ===

I must not be doing it right. What I am trying to do here is to make sure that subsequent requests only execute based on some condition from a dependency (e.g. if a POST to create an item fails, then subsequent requests for GET, PATCH, and DELETE should be disabled), and I cannot figure out if I am passing the criteria retrieved from the dependency correctly to the @disabled condition.

[AnWeber]

This is due to how up-to-date the disabled detection is implemented. I currently check the condition before starting the region and after each individual step of the execution. But you expect it to be checked only at the point where you write the disabled. I think I adapt the behavior to your expected behavior, as this seems more intuitive.
There is another variant, which may be poorly documented. You can also cancel the execution using a script exporting variable $cancel. However, I prefer a custom plugin for my scripts, which uses the method (PUT, POST) to set certain parameters in order to interrupt CRUD test scripts.

[alekdavisintel]

Thanks @AnWeber. I did not quite get your point about the disabled detection implementation. The two links in your response (starting with before... and after...) result in 404 Page Not Found, so I'm not sure if there is any useful info there. And if they intended to point to code, I'm not sure they would be meaningful (it's hard to read someone else's code from a domain you are not familiar with).

I did not see any description of the $cancel variable (and maybe I missed it), but I think I figured it out. I rewrote the tests using $cancel:

@validTitle1=Overview
@validTitle2=Wake up to WonderWidgets!
@invalidTitle=XYZ
###

### Test A (initial test which is a dependency for other tests)
# @name test_a_results
GET https://httpbin.org/json

### Test B (must be enabled since item is a valid string)
# @ref test_a_results
{{
    var item = test_a_results.slideshow.slides.find(item => item.title == validTitle1)?.title;
    if (item == undefined || item == null) {
        exports.$cancel = true;
    } else {
        exports.item = item;
    }
}}
GET https://httpbin.org/anything/{{item}}

### Test C (must be disabled since item is undefined or null)
# @ref test_a_results
{{
    var item = test_a_results.slideshow.slides.find(item => item.title == invalidTitle)?.title;
    if (item == undefined || item == null) {
        exports.$cancel = true;
    } else {
        exports.item = item;
    }
}}
GET https://httpbin.org/anything/{{item}}

### Test D (must be enabled since item is a valid string)
# @ref test_a_results
{{
    var item = test_a_results.slideshow.slides.find(item => item.title == validTitle2)?.title;
    if (item == undefined || item == null) {
        exports.$cancel = true;
    } else {
        exports.item = item;
    }
}}
GET https://httpbin.org/anything/{{item}}

And request output looks right (test C gets skipped):

---------------------
=== Test A (initial test which is a dependency for other tests) ===

GET https://httpbin.org/json
accept-encoding: gzip, deflate, br
accept: */*
user-agent: httpyac

HTTP/1.1 200  - OK
access-control-allow-credentials: true
access-control-allow-origin: *
connection: close
content-length: 429
content-type: application/json
date: Mon, 27 May 2024 00:22:14 GMT
server: gunicorn/19.9.0

{
  "slideshow": {
    "author": "Yours Truly", 
    "date": "date of publication", 
    "slides": [
      {
        "title": "Wake up to WonderWidgets!", 
        "type": "all"
      }, 
      {
        "items": [
          "Why <em>WonderWidgets</em> are great", 
          "Who <em>buys</em> WonderWidgets"
        ], 
        "title": "Overview", 
        "type": "all"
      }
    ], 
    "title": "Sample Slide Show"
  }
}
---------------------
=== Test B (must be enabled since item is a valid string) ===

GET https://httpbin.org/anything/Overview
accept-encoding: gzip, deflate, br
accept: */*
user-agent: httpyac

HTTP/1.1 200  - OK
access-control-allow-credentials: true
access-control-allow-origin: *
connection: close
content-length: 392
content-type: application/json
date: Mon, 27 May 2024 00:22:15 GMT
server: gunicorn/19.9.0

{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip, deflate, br", 
    "Host": "httpbin.org", 
    "User-Agent": "httpyac", 
    "X-Amzn-Trace-Id": "Root=1-6653d237-692cfffe7cea02ec3d5c420e"
  }, 
  "json": null, 
  "method": "GET", 
  "origin": "192.55.55.51", 
  "url": "https://httpbin.org/anything/Overview"
}
---------------------
=== Test D (must be enabled since item is a valid string) ===

GET https://httpbin.org/anything/Wake%20up%20to%20WonderWidgets!
accept-encoding: gzip, deflate, br
accept: */*
user-agent: httpyac

HTTP/1.1 200  - OK
access-control-allow-credentials: true
access-control-allow-origin: *
connection: close
content-length: 409
content-type: application/json
date: Mon, 27 May 2024 00:22:16 GMT
server: gunicorn/19.9.0

{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip, deflate, br", 
    "Host": "httpbin.org", 
    "User-Agent": "httpyac", 
    "X-Amzn-Trace-Id": "Root=1-6653d238-13d17b33508d90250ba26cea"
  }, 
  "json": null, 
  "method": "GET", 
  "origin": "192.55.55.51", 
  "url": "https://httpbin.org/anything/Wake up to WonderWidgets!"
}

The only problem is that to see which tests ran and which ones get skipped, I need to read the request log because in both the test tree results and test results panel, all tests look like they completed (including the one that was skipped):

Also, while running all test in the folder via the testing tree is fine, if I try to run them via the Send All code lens option, I noticed that according to the request output only testss A and B got executed (and tests C and D got skipped). I tried using the Send Selected option and it did the same as Send All. If I try the Send option on the test that should be cancelled, there is no indication that it was not executed. Don't know if this is VSCode quirks, or if this can be improved.

To summarize, while it would be nice to get the @disabled tag to work (and based on your comment, it sounds like it kind of works, but not as I might have expected), using the $cancel variable is a legit workaround. It would be nice if the test result indicators reflected the fact that the tests were cancelled/skipped/disabled for more accurate reporting. Thanks for all your help.

[alekdavisintel]

I deleted my two previous comments. It was my fault (forgot == in the condition). So, no issue with the failed test icons. Sorry for the confusion.

[AnWeber]

I'm not sure they would be meaningful (it's hard to read someone else's code from a domain you are not familiar with).

The two links still worked before my commit, but then I deleted the file 🤦I am happy to add links. At the end of the day it's just code and yes the lack of context may be the problem. But that's where the fun begins in my eyes. Raise others to the same level of understanding and use the feedback to improve yourself

The indicators will also work for $cancel with next release