Integration Testing with Karate Framework

What is Karate?

Karate is a framework for integration testing. It’s relevant to understand that although Karate borrows some concepts from Cucumber and BDD (Behavior Driven Development), it is not a framework for BDD. See Peter Thomas’ answer about this for more details.

Architecture of a Karate program

A Karate program is a Java program that runs the Karate Java library, which contains a JavaScript interpreter that runs a JavaScript program which is what the user provides. This JavaScript program is structured using a gherkin-like domain specific language.

The Java part of the program can be extended by user-provided Java code. The stack can be described as follows:

Karate script (JavaScript code inside a .feature file)

JavaScript interpreter

Karate Java library (with optional user-provided Java code)

Java Virtual Machine

The Karate Java library can be invoked from a JUnit test class for easy integration with build systems like Maven o Gradle, or can be invoked directly from a standalone runner.

Running a Karate test

Running with Maven

With this approach, you create a JUnit test class that kickstarts Karate, and run it with mvn -Dtest=MyRunner test, where MyRunner is the name if your test class. This way of running tests is the most natural if your script depends on custom Java code, because it ensures that your utility code gets built and then included in the classpath for running every time you run the test. This is a very common use case since things like checking a database between API calls requires Java JDBC code.

Note also that calling the test class as …Runner (without ending in Test or Tests) is useful so that Maven will not attempt to run it as part of the unit tests (but it will still get built and included in the classpath).

Given a REST service in a repo structured according to the standard Maven project structure, and a URL called myresource in your API that you want to test, follow these steps to create a Karate test and run it.

Add the following dependencies to the pom.xml:

<dependency>
    <groupId>com.intuit.karate</groupId>
    <artifactId>karate-apache</artifactId>
    <version>0.9.6</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>com.intuit.karate</groupId>
    <artifactId>karate-junit5</artifactId>
    <version>0.9.6</version>
    <scope>test</scope>
</dependency>

We want to create our .feature files under sr/test/java. In order to do that, add the following to the build section of the pom.xml:

<build>
    <testResources>
        <testResource>
            <directory>src/test/java</directory>
            <excludes>
                <exclude>**/*.java</exclude>
            </excludes>
        </testResource>
    </testResources>        
    <plugins>
    ...
    </plugins>
</build>

Create the following folder structure in your project:

src/test/java
    \-- integration
        +-- karate-config.js
        \-- myresource
            |
            +-- MyResourceRunner.java
            +-- myresource.feature

Create the MyResourceRunner class, with the following contents:

package integration.myresource;

import com.intuit.karate.junit5.Karate;

class MyResourceRunner {
    @Karate.Test
    Karate testMyResource() {
        return Karate.run("myresource").relativeTo(getClass());
    }
}

Create the myresource.feature test script, with the following contents:

Feature: tests the myresource API.
  more lines of description if needed.

Background:
  # this section is optional !
  # steps here are executed before each Scenario in this file
  # variables defined here will be 'global' to all scenarios
  # and will be re-initialized before every scenario

Scenario: Send a GET and validate the HTTP status code
Given url 'https://localhost:8080/api/v1/myresource'
When method GET
Then status 200

Create the karate-config.js config file, with the following contents:

function fn() {
    // Comments in this file cannot be in the first line
    karate.configure('ssl', { trustAll: true }); // This is necessary of your service uses a self-signed certificate. Otherwise, you can omit this line.
    return {};
}

Careful with comments in this file! The first line MUST start with the keyword function, otherwise Karate will not parse it correctly.

Karate looks for the karate-config.js file in the root of the classpath (that is src/test/java) be default. We prefer to have all Karate related files under src/test/java/integration, so we have put our karate-config.js there.

If you want to configure the log level used by the Karate logger, add the following in your logback.xml:

<logger name="com.intuit.karate" level="debug"/>

Use the following command to build and run the test:

mvn -Dtest=MyResourceRunner -Dkarate.config.dir=target/test-classes/integration/ test

Karate looks for the karate-config.js file in the root of the classpath (that is src/test/java/) be default, and this is the setup found on most examples in the documentation. We prefer to have all Karate related files under src/test/java/integration, so we have put our karate-config.js there, and that’s why we pass the karate.config.dir argument when running. If you want, you can leave the karate-config.js in src/test/java/, and then you can omit the argument -Dkarate.config.dir when running.

Running with the standalone executable

All of Karate (core API testing, parallel-runner / HTML reports, the debugger-UI, mocks and web / UI automation) is available as a single, executable JAR file, which includes even the karate-apache dependency. Under this mode of execution, you would run a Karate script with the following command:

java -jar karate.jar my-test.feature

If your script depends on custom Java code, you will need to add that code to the classpath when running the Karate jar. You can use the standalone JAR and still depend on external Java code – but you have to set the classpath for this to work. The entry-point for the Karate command-line app is com.intuit.karate.Main. Here is an example of using the Karate Robot library as a JAR file assuming it is in the current working directory.

java -cp karate.jar:karate-robot.jar com.intuit.karate.Main test.feature

If on Windows, note that the path-separator is ; instead of : as seen above for Mac / Linux. Refer this post for more details. The karate-config.js will be looked for in the classpath itself.

Splitting a test in several .feature files

This is already implicit in the above, but it’s still worth noting that you can split your test across several feature files by simply defining several test methods in the runner class, like so:

package integration.myresource;

import com.intuit.karate.junit5.Karate;

class MyResourceRunner {
    @Karate.Test
    Karate testMyResourcePositiveCases() {
        // Runs myResourcePositiveCases.feature
        return Karate.run("myResourcePositiveCases").relativeTo(getClass());
    }
    
    Karate testMyResourceNegativeCases() {
        // Runs myResourceNegativeCases.feature
        return Karate.run("myResourceNegativeCases").relativeTo(getClass());
    }
}

Setting and Using Variables

Take a look at the following examples:

# assigning a string value:
Given def myVar = 'world'

# using a variable
Then print myVar

# assigning a number (you can use '*' instead of Given / When / Then)
* def myNum = 5
* print 'the value of myNum is:', myNum

Note that def will over-write any variable that was using the same name earlier. Keep in mind that the start-up configuration routine could have already initialized some variables before the script even started. For details of scope and visibility of variables, see Script Structure. Note that url and request are not allowed as variable names. This is just to reduce confusion for users new to Karate who tend to do * def request = {} and expect the request body or similarly, the url to be set.

JavaScript integration

Remember, a karate script is a thinly-veiled JavaScript program, so JavaScript integration is straightforward.

Defining a one-line function

* def greeter = function(title, name) { return 'hello ' + title + ' ' + name }
* assert greeter('Mr.', 'Bob') == 'hello Mr. Bob'

Standard JavaScript syntax rules apply, but the right-hand-side (or contents of the *.js file if applicable) should begin with the function keyword. This means that JavaScript comments are not supported if they appear before the function body. Also note that ES6 arrow functions are not supported. Finally, especially when using stand-alone *.js files, you can use fn as the function name, so that your IDE does not complain about JavaScript syntax errors, e.g. function fn(x){ return x + 1 }.

Defining more complex functions

* def myFunc =
  """
  function(i) {
    return i + 2;
  } 
  """
* assert myFunc(2) == 4

Calling using the call construct

Regardless of how a JavaScript function is defined, if it’s a one-argument function, you can call it with the call construct. This makes for a somewhat more readable code because it makes the parameter names explicit at the point of invocation:

* def greeter = function(name){ return 'Hello ' + name.first + ' ' + name.last + '!' }
* def greeting = call greeter { first: 'John', last: 'Smith' }

The caveat mentioned above about comments in the karate-config.js file applies to all JavaScript files: the first line MUST start with the function keyword.

Java integration

Calling Java static methods

Calling static methods can be done as if they were Javascrpt native functions:

Utility	Recipe
System Time (as a string)	function(){ return java.lang.System.currentTimeMillis() + ” }
UUID	function(){ return java.util.UUID.randomUUID() + ” }
Random Number (0 to max-1)	function(max){ return Math.floor(Math.random() * max) }
Case Insensitive Comparison	function(a, b){ return a.equalsIgnoreCase(b) }
Sleep or Wait for pause milliseconds	function(pause){ java.lang.Thread.sleep(pause) }

Calling Java non-static methods

To access primitive and reference Java types from JavaScript, call the Java.type() function, which returns a type object that corresponds to the full name of the class passed in as a string.

* def dateStringToLong =
  """
  function(s) {
    var SimpleDateFormat = Java.type('java.text.SimpleDateFormat');
    var sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSZ");
    return sdf.parse(s).time; // '.getTime()' would also have worked instead of '.time'
  } 
  """
* assert dateStringToLong("2016-12-24T03:39:21.081+0000") == 1482550761081

Given this custom, user-defined Java class:

package com.mycompany;

import java.util.HashMap;
import java.util.Map;

public class JavaDemo {    
    
    public Map<String, Object> doWork(String fromJs) {
        Map<String, Object> map = new HashMap<>();
        map.put("someKey", "hello " + fromJs);
        return map;
    }

    public static String doWorkStatic(String fromJs) {
        return "hello " + fromJs;
    }   

}

This is how it can be called from a test-script via JavaScript, and yes, even static methods can be invoked:

* def doWork =
  """
  function(arg) {
    var JavaDemo = Java.type('com.mycompany.JavaDemo');
    var jd = new JavaDemo();
    return jd.doWork(arg);  
  }
  """
# in this case the solitary 'call' argument is of type string
* def result = call doWork 'world'
* match result == { someKey: 'hello world' }

# using a static method - observe how java interop is truly seamless !
* def JavaDemo = Java.type('com.mycompany.JavaDemo')
* def result = JavaDemo.doWorkStatic('world')
* assert result == 'hello world'

Note that JSON gets auto-converted to Map (or List) when making the cross-over to Java. Refer to the cats-java.feature demo for an example.

Modularizing tests

A Karate program is written in a feature file, which is stored with extension .feature. Each feature file can call other feature files. For every feature file, Karate creates a context object that the JavaScript program can mutate. Each feature file has its own isolated context by default, but using shared context is also possible. When you use the def construct to assign a variable, what you are doing is adding a new field in the context object.

To call another feature file, use the call function. To pass paramaters to a called script, pass a json object with the data you want. To return data from the called script to the caller, have the caller create fields in its context with def. The caller can then access the callee’s context as the return value of the call function. Example:

Caller script:

Feature: which makes a 'call' to another re-usable feature

Background:
  * configure headers = read('classpath:my-headers.js')
  * def signIn = call read('classpath:my-signin.feature') { username: 'john', password: 'secret' }
  * def authToken = signIn.authToken

Scenario: some scenario
  # main test steps

Called script:

Feature: here are the contents of 'my-signin.feature'

Scenario:
  Given url loginUrlBase
  And request { userId: '#(username)', userPass: '#(password)' }
  When method post
  Then status 200
  And def authToken = response

  # second HTTP call, to get a list of 'projects'
  Given path 'users', authToken.userId, 'projects'
  When method get
  Then status 200
  # logic to 'choose' first project
  And set authToken.projectId = response.projects[0].projectId;

Key takeaways:

• The call read syntax an abbreviation for two separate calls:
  • def someFunction = read(‘classpath:my-signin.feature’) : reads the file and evaluates it, returning a JavaScript function that contains the executable code.
  • call someFunction : executes the function.
• The called script uses set authToken.projectId to set a property in an object.
• The called script uses def authToken = response to store an object as a field in its context. Note the context is not referenced explicitely, rather you use def to access it.
• The caller accesses the context of the called script by assigning the result of the call function: def signIn = call, thus it can then acccess its fields as signIn.authToken.
• When the caller script is run, only the scenarios from the caller script are reported in the output as “Tests run”. If the called script has several scenarios, these will be executed, but they won’t be really counted.
• Also note that if you invoke the called script from the background, it will be run before every scenario of the parent script.
• The two previous points show that the call construct is meant to be used to call small feature files that contain routine code, not to separate a complex test with several scenarios per file.

Test selection with tags

You can tag a set of scenarios to be able to run only those. You can also tag an entire feature to select that feature. To add tags, use this syntax:

@myTagForTheFeature
Feature: tags demo - first

@myTagForAScenario
Scenario: Bla
...

Then you can run only the scenarios tagged with myTagForAScenario using the command: mvn -Dtest=MyRunner -Dkarate.options=”–tags @myTagForAScenario” test

SQL integration

We can work with a relational database within Karate scripts, by leveraging the Java integration and JDBC. The JDBC part can be further simplified by using Spring’s JdbcTemplate utility class. The dogs.featue example and its accompanying JDBC utility code in Karate’s repo show how to do this.

Sources

• Official guide in github: intuit/karate
• General information: https://www.softwaretestinghelp.com/api-testing-with-karate-framework/
• Baeldung’s tutorial: https://www.baeldung.com/karate-rest-api-testing