Environment specific config doesn't work on v1.0.1

[briannqc]

myproject.zip

Steps to reproduce:

1. Create a new skeleton project with

mvn archetype:generate \ 
-DarchetypeGroupId=com.intuit.karate \
-DarchetypeArtifactId=karate-archetype \
-DarchetypeVersion=1.0.1 \
-DgroupId=com.mycompany \
-DartifactId=myproject

2. Replace karate-config.js with env specific config karate-config-sandbox.js

function fn() {
    return {
        baseUrl: 'https://jsonplaceholder.typicode.com'
    }
}

3. Replace fixed url in users.feature to configurable url baseUrl

Feature: sample karate test script
  for help, see: https://github.com/intuit/karate/wiki/IDE-Support

  Background:
    * url baseUrl

4. Run test and log

>> mvn clean test -Dkarate.env=sandbox 

[INFO] Scanning for projects...
[INFO] 
[INFO] ----------------------< com.mycompany:myproject >-----------------------
[INFO] Building myproject 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ myproject ---
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ myproject ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/ngobrian/Workspace/sandbox/myproject/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ myproject ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ myproject ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ myproject ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 source files to /Users/ngobrian/Workspace/sandbox/myproject/target/test-classes
[INFO] 
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ myproject ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running examples.ExamplesTest
10:08:23.800 [main] INFO  com.intuit.karate.Runner - using system property 'karate.env': sandbox
10:08:24.679 [pool-1-thread-2] ERROR com.intuit.karate - classpath:examples/users/users.feature:5
* url baseUrl
>>>> js failed:
01: baseUrl
<<<<
org.graalvm.polyglot.PolyglotException: ReferenceError: "baseUrl" is not defined
- <js>.:program(Unnamed:1)

classpath:examples/users/users.feature:5
10:08:24.679 [pool-1-thread-1] ERROR com.intuit.karate - classpath:examples/users/users.feature:5
* url baseUrl
>>>> js failed:
01: baseUrl
<<<<
org.graalvm.polyglot.PolyglotException: ReferenceError: "baseUrl" is not defined
- <js>.:program(Unnamed:1)

classpath:examples/users/users.feature:5
---------------------------------------------------------
feature: classpath:examples/users/users.feature
scenarios:  2 | passed:  0 | failed:  2 | time: 0.0191
---------------------------------------------------------

10:08:25.138 [pool-2-thread-1] INFO  com.intuit.karate.Suite - <<fail>> feature 1 of 1 (0 remaining) classpath:examples/users/users.feature
Karate version: 1.0.1
======================================================
elapsed:   1.34 | threads:    5 | thread time: 0.02 
features:     1 | skipped:    0 | efficiency: 0.00
scenarios:    2 | passed:     0 | failed: 2
======================================================
>>> failed features:
>>>> js failed:
01: baseUrl
<<<<
org.graalvm.polyglot.PolyglotException: ReferenceError: "baseUrl" is not defined
- <js>.:program(Unnamed:1)

classpath:examples/users/users.feature:5
>>>> js failed:
01: baseUrl
<<<<
org.graalvm.polyglot.PolyglotException: ReferenceError: "baseUrl" is not defined
- <js>.:program(Unnamed:1)

classpath:examples/users/users.feature:5
<<<

How did I find this bug?

I'm upgrading karate v0.9.6 to v1.0.1, the env specific config was working fine but stopped working on the new version.

[ptrthomas]

@brianngo313 won't look at this unless you follow this process: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue

next time please provide this feedback for RC releases when we ask, e.g: https://twitter.com/KarateDSL/status/1359010765219393538

so now this may take time to fix. you are welcome to submit a PR of course

[joelpramos]

20:58:18.081 [main] INFO com.intuit.karate.Runner - using system property 'karate.env': staging

this log is suspicious... are overriding the variable within one of your tests?

[briannqc]

20:58:18.081 [main] INFO com.intuit.karate.Runner - using system property 'karate.env': staging

this log is suspicious... are overriding the variable within one of your tests?

Well, sorry my bad, actually env1=staging, I anonymized my codes. Let me create a skeleton project and upload it here. Sorry for not reading the howto carefully

[briannqc]

@brianngo313 won't look at this unless you follow this process: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue

next time please provide this feedback for RC releases when we ask, e.g: https://twitter.com/KarateDSL/status/1359010765219393538

so now this may take time to fix. you are welcome to submit a PR of course

Sorry for the inconvenience, I updated the issue description following the howto guide

[ptrthomas]

reopening - right now tagged as help wanted

[ptrthomas]

@brianngo313 I'm closing this as won't fix. karate now expects you to have a "root" karate-config.js if you start using a env-specific over-ride. so the solution for you is create one, and it can do nothing or return an empty JSON.

the reason is we have started to look for karate-config.js to auto-detect if we are in "classpath" mode or not (standalone JAR) which gives us a few other advantages. you are welcome to challenge this, but then we insist on a PR. I must say you might be the only one ever who attempted to use a env-specifc config without a default one, and yes this used to work.

here is the relevant line number: https://github.com/intuit/karate/blob/v1.0.1/karate-core/src/main/java/com/intuit/karate/Runner.java#L238

you are also welcome to submit a PR for the docs if you want that updated, but I don't see an RoI :)

[briannqc]

Thank @ptrthomas for your times and helps. I'm okay with won't fix, however, I wonder whether a bit of elaborating in README can help other users avoid the same confusion in the future?

What do you think about this #1599?

[ptrthomas]

@brianngo313 perfect, thanks for the PR and I merged it

[briannqc]

Thank you @ptrthomas