Skip to content

Commit

Permalink
fixed #61 for js array and match, also some doc improvements
Browse files Browse the repository at this point in the history
  • Loading branch information
ptrthomas committed May 7, 2017
1 parent 7018c8e commit 1d83876
Show file tree
Hide file tree
Showing 6 changed files with 63 additions and 10 deletions.
27 changes: 18 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,7 +198,7 @@ src/test/java
Assuming you use JUnit, there are some good reasons for the recommended (best practice) naming convention and choice of file-placement shown above:
* Not using the `*Test.java` convention for the JUnit classes (e.g. `CatsRunner.java`) in the `cats` and `dogs` folder ensures that these tests will **not** be picked up when invoking `mvn test` (for the whole project) from the [command line](#command-line). But you can still invoke these tests from the IDE, which is convenient when in development mode.
* `AnimalsTest.java` (the only file that follows the `*Test.java` naming convention) acts as the 'test suite' for the entire project. By default, Karate will load all `*.feature` files from sub-directories as well. But since `some-reusable.feature` is _above_ `AnimalsTest.java` in the folder heirarchy, it will **not** be picked-up. Which is exactly what we want, because `some-reusable.feature` is designed to be [called](#calling-other-feature-files) only from one of the other test scripts (perhaps with some parameters being passed). You can also use [tags](#cucumber-tags) to skip files.
* `some-classpath-function.js` and `some-classpath-payload.js` are on the Java 'classpath' which means they can be easily [read](#reading-files) (and re-used) from any test-script by using the `classpath:` prefix, for e.g: `read('classpath:some-classpath-function.js')`.
* `some-classpath-function.js` and `some-classpath-payload.js` are in the 'root' of the Java 'classpath' which means they can be easily [read](#reading-files) (and re-used) from any test-script by using the `classpath:` prefix, for e.g: `read('classpath:some-classpath-function.js')`. Relative paths will also work.

For details on what actually goes into a script or `*.feature` file, refer to the
[syntax guide](#syntax-guide).
Expand Down Expand Up @@ -308,12 +308,14 @@ You can 'lock down' the fact that you only want to execute the single JUnit clas
</plugin>
```

This is actually the recommended configuration for generating CI-friendly reports when using Cucumber and note how the `cucumber.options` can be specified using the `<systemProperties>` configuration. Options here would over-ride corresponding options specified if a `@CucumberOptions` annotation is present (on `AnimalsTest.java`). So for the above example, any `plugin` options present on the annotation would not take effect, but anything else (for example `tags`) would continue to work.
Note how the `cucumber.options` can be specified using the `<systemProperties>` configuration. Options here would over-ride corresponding options specified if a `@CucumberOptions` annotation is present (on `AnimalsTest.java`). So for the above example, any `plugin` options present on the annotation would not take effect, but anything else (for example `tags`) would continue to work.

With the above in place, you don't have to use `-Dtest=AnimalsTest` on the command-line any more. You may need to point your CI to the location of the JUnit XML report (e.g. `target/cucumber-junit.xml`) so that test-reports are generated correctly.

The [Karate Demo](karate-demo) has a working example of this set-up. Also refer to the section on [switching the environment](#switching-the-environment) for more ways of running tests via Maven using the command-line.

The big drawback of the 'Cucumber-native' approach is that you cannot run tests in parallel. But you have the option of choosing other report formats, for e.g. `html`. The recommended approach for Karate reporting in a Continuous Integration set-up is described in the next section which focuses on emitting the [JUnit XML](https://wiki.jenkins-ci.org/display/JENKINS/JUnit+Plugin) format that most CI tools can consume.

## Parallel Execution
Karate can run tests in parallel, and dramatically cut down execution time. This is a 'core' feature and does not depend on JUnit, TestNG or even Maven.

Expand Down Expand Up @@ -355,7 +357,7 @@ scenarios: 12 | failed: 0 | skipped: 0

The [Karate Demo](karate-demo) has a working example of this set-up.

> Going forward, this is likely to be the preferred way of running all Karate tests in a project, mainly because the other Cucumber reports (e.g. HTML) are not thread-safe. In other words, please rely on the `CucumberRunner.parallel()` JUnit XML for CI build reporting, and if you see any problems, please submit a defect report.
This is the preferred way of automating the execution of all Karate tests in a project, mainly because the other Cucumber reports (e.g. HTML) are not thread-safe. In other words, please rely on the `CucumberRunner.parallel()` JUnit XML output for CI and test result reporting, and if you see any problems, or if your CI tool does not support the JUnit XML format, please submit a defect report.

## Logging
> This is optional, and Karate will work without the logging config in place, but the default
Expand Down Expand Up @@ -1015,6 +1017,8 @@ Given cookie foo = 'bar'

You also have the option of setting multiple cookies in one-step using the [`cookies`](#cookies-json) keyword.

Note that any cookies returned in the HTTP response would be automatically set for any future requests. Refer to the documentation of the built-in variable [`cookies`](#cookies) for more details.

## `form field`
HTML form fields would be URL-encoded when the HTTP request is submitted (by the [`method`](#method) step). You would typically use these to simulate a user sign-in and then grab a security token from the [`response`](#response). For example:

Expand Down Expand Up @@ -1263,9 +1267,9 @@ validations ! This example uses the [`match contains`](#match-contains) syntax,
this comes in useful will be apparent when we discuss [`match each`](#match-each).
```cucumber
Given def temperature = { celsius: 100, fahrenheit: 212 }
Then match temperature contains { fahrenheit: '#? _ == $.celsius * 1.8 + 32' }
Then match temperature == { celsius: '#number', fahrenheit: '#? _ == $.celsius * 1.8 + 32' }
# when validation logic is an 'equality' check, an embedded expression works better
Then match temperature == { celsius: '#number', fahrenheit: '#($.celsius * 1.8 + 32)' }
Then match temperature contains { fahrenheit: '#($.celsius * 1.8 + 32)' }
```

### `match` for Text and Streams
Expand Down Expand Up @@ -1398,8 +1402,8 @@ The `match` keyword can be made to iterate over all elements in a JSON array usi
* match each data.foo contains { baz: '#? isAbc(_)' }
```

Here is a contrived example that uses `match each`, `contains` and the `#?` 'predicate' marker to validate that the
value of `totalPrice` is always equal to the `roomPrice` of the first item in the `roomInformation` array.
Here is a contrived example that uses `match each`, [`contains`](#match-contains) and the [`#?`](#self-validation-expressions) 'predicate' marker to validate that the value of `totalPrice` is always equal to the `roomPrice` of the first item in the `roomInformation` array.

```cucumber
Given def json =
"""
Expand All @@ -1412,7 +1416,7 @@ Given def json =
"""
Then match each json.hotels contains { totalPrice: '#? _ == $.roomInformation[0].roomPrice' }
# when validation logic is an 'equality' check, an embedded expression works better
Then match each json.hotels == { roomInformation: '#array', totalPrice: '#($.roomInformation[0].roomPrice)' }
Then match each json.hotels contains { totalPrice: '#($.roomInformation[0].roomPrice)' }
```

## `get`
Expand All @@ -1436,7 +1440,9 @@ By now, it should be clear that [JsonPath]((https://github.com/jayway/JsonPath#p
```

### XPath Functions
When handling XML, you sometimes need to call [XPath functions](https://docs.oracle.com/javase/tutorial/jaxp/xslt/xpath.html), for example to get the count of a node-set. XPath functions are not supported directly within [`match`](#match) statements. But by using the `get` keyword, you should be able to achieve any assertion involving XPath functions in two steps. The examples below also show how 'standard' XPath can be used to do `match`.
When handling XML, you sometimes need to call [XPath functions](https://docs.oracle.com/javase/tutorial/jaxp/xslt/xpath.html), for example to get the count of a node-set. XPath functions are not supported directly within [`match`](#match) statements. But by using the `get` keyword, you should be able to achieve any assertion involving XPath functions in two steps.

The last line below also show how 'normal' (uncomplicated) XPath can be used to do a `match` in a single step.

```cucumber
* def myXml =
Expand All @@ -1461,6 +1467,7 @@ When handling XML, you sometimes need to call [XPath functions](https://docs.ora
```

# Special Variables
These are 'built-in' variables, there are only a few and all of them give you access to the HTTP response.

## `response`
After every HTTP call this variable is set with the response and is available until the next HTTP request over-writes it. You can easily assign `response` (or parts of it using Json-Path or XPath) to a variable and use it in later steps.
Expand Down Expand Up @@ -1676,6 +1683,8 @@ Examples of [defining and using JavaScript functions](#javascript-functions) app

In real-life scripts, you would typically also use this capability of Karate to [`configure headers`](#configure-headers) where the specified JavaScript function uses the variables that result from a [sign in](#calling-other-feature-files) to manipulate headers for all subsequent HTTP requests.

And it is worth mentioning that the Karate [configuration 'bootstrap'](#configuration) routine is itself a JavaScript function.

### The `karate` object
A JavaScript function at runtime has access to a utility object in a variable named: `karate`. This provides the following methods:

Expand Down
6 changes: 5 additions & 1 deletion karate-core/src/main/java/com/intuit/karate/Script.java
Original file line number Diff line number Diff line change
Expand Up @@ -662,9 +662,13 @@ public static AssertionResult matchJsonPath(MatchType matchType, ScriptValue act
ScriptValue expected = eval(expression, context);
Object expObject;
switch (expected.getType()) {
case JSON:
case JSON: // convert to map or list
expObject = expected.getValue(DocumentContext.class).read("$");
break;
case JS_ARRAY: // array returned by js function, needs conversion to list
ScriptObjectMirror som = expected.getValue(ScriptObjectMirror.class);
expObject = new ArrayList(som.values());
break;
default:
expObject = expected.getValue();
}
Expand Down
20 changes: 20 additions & 0 deletions karate-core/src/test/java/com/intuit/karate/ScriptTest.java
Original file line number Diff line number Diff line change
Expand Up @@ -322,6 +322,26 @@ public void testMatchAllJsonPath() {
assertTrue(Script.matchJsonPath(MatchType.EACH_CONTAINS, myJson, "$.foo", "{bar:'#number'}", ctx).pass);
assertTrue(Script.matchJsonPath(MatchType.EACH_CONTAINS, myJson, "$.foo", "{baz:'#string'}", ctx).pass);
assertFalse(Script.matchJsonPath(MatchType.EACH_EQUALS, myJson, "$.foo", "{bar:'#? _ < 3', baz:'#string'}", ctx).pass);
}

@Test
public void testMatchJsonObjectReturnedFromJs() {
ScriptContext ctx = getContext();
Script.assign("fun", "function(){ return { foo: 'bar' } }", ctx);
Script.assign("json", "{ foo: 'bar' }", ctx);
Script.assign("expected", "fun()", ctx);
assertTrue(Script.matchNamed("json", null, "expected", ctx).pass);
assertTrue(Script.matchNamed("json", null, "fun()", ctx).pass);
}

@Test
public void testMatchJsonArrayReturnedFromJs() {
ScriptContext ctx = getContext();
Script.assign("fun", "function(){ return [ 'foo', 'bar', 'baz' ] }", ctx);
Script.assign("json", "[ 'foo', 'bar', 'baz' ]", ctx);
Script.assign("expected", "fun()", ctx);
assertTrue(Script.matchNamed("json", null, "expected", ctx).pass);
assertTrue(Script.matchNamed("json", null, "fun()", ctx).pass);
}

@Test
Expand Down
1 change: 1 addition & 0 deletions karate-demo/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ as well as demonstrate various Karate features and best-practices.
[`call-table.feature`](src/test/java/demo/calltable/call-table.feature) | This is a great example of how Karate combines with Cucumber and JsonPath to give you an extremely readable data-driven test. Karate's [`table`](https://github.com/intuit/karate#table) keyword is a super-elegant and readable way to create JSON arrays, perfect for setting up all manner of data-driven tests.
[`call-dynamic-json.feature`](src/test/java/demo/calldynamic/call-dynamic-json.feature) | Shows how to dynamically create a JSON array and then use it to call a `*.feature` file in a loop. In this example, the JSON is created using a JavaScript function, but it can very well be the response from an HTTP call, the result of a JsonPath expression or even a List of HashMap-s acquired by [calling Java](https://github.com/intuit/karate#calling-java). This test actually calls a second `*.feature` file in a loop to validate a 'get by id'. Using JsonPath and [`match each`](https://github.com/intuit/karate#match-each) to validate all items within a JSON array is also demonstrated.
[`call-once.feature`](src/test/java/demo/callonce/call-once.feature) | Cucumber has a [limitation](https://github.com/cucumber/cucumber-jvm/issues/515) where `Background` steps are re-run for every `Scenario` and even for every `Examples` row within a `Scenario Outline`. This is a problem when you have expensive and time-consuming HTTP calls in your 'set-up' routines. Fortunately you have an elegant work-around with Karate's [`callonce`](https://github.com/intuit/karate#callonce) keyword.
[`polling.feature`](src/test/java/demo/polling/polling.feature) | If you need to keep polling until the response changes to something you exoect, you can achieve this by combining JavaScript functions with [calling another `*feature` file](https://github.com/intuit/karate#calling-other-feature-files).

## Configuration and Best Practices
| File | Demonstrates
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
package com.intuit.karate.junit4.demos;

import com.intuit.karate.junit4.Karate;
import cucumber.api.CucumberOptions;
import org.junit.runner.RunWith;

@RunWith(Karate.class)
@CucumberOptions(features = "classpath:com/intuit/karate/junit4/demos/js-arrays.feature")
public class JsArraysRunner {

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
Feature: test that arrays returned from js can be used in match

Scenario:
* def fun = function(){ return ['foo', 'bar', 'baz'] }
* def json = ['foo', 'bar', 'baz']
* match json == fun()
* def expected = fun()
* match json == expected

0 comments on commit 1d83876

Please sign in to comment.