How to test a REST API with Arquillian

Testing a REST API on a real application server with Arquillian is easy – if you know what you need to do 😉

I have this simple REST service, that authenticates a user:

@POST
@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_JSON)
public Response authenticateUser(final UserData userData)
{
    ...
    return Response
        .status(401)
        .entity(false)
        .build();
}

Let’s write a test for this REST API, that uses a real application server and calls the interface from the outside.

Arquillian dependencies

The REST extensions for Arquillian make it fairly easy to test a deployed web application. Let’s start with their dependency in gradle.build:

// main BOM for Arquillian
'org.jboss.arquillian:arquillian-bom:1.1.11.Final',
// JUnit container
'org.jboss.arquillian.junit:arquillian-junit-container:1.1.11.Final',
// Chameleon (or any other specific container, e.g. JBoss, Wildfly etc.)
'org.arquillian.container:arquillian-container-chameleon:1.0.0.Alpha6',
// REST extensions
'org.jboss.arquillian.extension:arquillian-rest-client-impl-jersey:1.0.0.Alpha4'

Arquillian test

I can now write a test against the REST interface on the server like this:

@Deployment
public static WebArchive createDeployment()
{
    return ShrinkWrap
        .create(WebArchive.class)
        .addPackages(true, Filters.exclude(".*Test.*"),
            SessionResource.class.getPackage(),
            ...)
        .addAsWebInfResource(EmptyAsset.INSTANCE, "beans.xml");
}

@Test
@RunAsClient
public void authenticateUser(
    @ArquillianResteasyResource final WebTarget webTarget)
{
    final Response response = webTarget
        .path("/sessions")
        .request(MediaType.APPLICATION_JSON)
        .post(Entity.json(new UserData(
            "myuser",
            "mypassword")));
    assertEquals(true, response.readEntity(Boolean.class));
}

Note the @RunAsClient. This tells Arquillian to run the test as a client against the remote server and not within the remote server. The test class isn’t even deployed to the application server in this case (see Filters.exclude(".*Test.*")). That’s exactly what I needed, because I wanted to test the REST API from the outside, to make sure it’s available to its clients. Alternatively, you could set the whole deployment to @Deployment(testable = false), instead of configuring each test individually.

The REST extensions for Arquillian now call the test method with a WebTarget as a parameter. It points directly to the URL of the deployed web application, e.g. http://1.2.3.4:8080/044f5571-3b1a-4976-9baa-a64b56f2eec1/rest. So you don’t have to manually create the target URL everytime.

JBoss server configuration

It took me a while to find out how the target server – JBoss EAP 7 in my case – needs to be configured, to make this test pass, because the initial error message after my first attempt to run the test wasn’t very helpful at all:

javax.ws.rs.ProcessingException: java.net.ConnectException: Connection refused: connect
    at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:287)
    at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:255)
    ...
Caused by: java.net.ConnectException: Connection refused: connect
    at java.net.DualStackPlainSocketImpl.connect0(Native Method)
    at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79)

I could see in the server’s logs, that the deployment of the WAR file into JBoss was successful, so it had to be the REST client, that couldn’t connect to the API. A simple System.out.println(webTarget.getUri()); showed the problem: http://0.0.0.0:8080/044f5571-3b1a-4976-9baa-a64b56f2eec1/rest. Arquillian uses the IP configuration of the target server and I had configured JBoss to listen on any of its interfaces (hence 0.0.0.0).

To fix this problem I made JBoss listen on the exact public IP address of the server by adding this to \standalone\configuration\standalone.xml. Be sure to use the real IP address and not 0.0.0.0 or <any-address />.

<interfaces>
    <interface name="management">
        <inet-address value="1.2.3.4"/>
    </interface>
    <interface name="public">
        <inet-address value="1.2.3.4"/>
    </interface>
</interfaces>

JsonParseException when calling a REST service with curl on Windows

When I called a REST service (provided by webMethods Integration Server) with curl on my Windows machine, I got the following error:

org.codehaus.jackson.JsonParseException:Unexpected character (''' (code 39)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')\n at [Source: com.wm.net.HttpInputStream@74356f93; line: 1, column: 2]

The solution was quite simple: use double quotes instead of single quotes when providing JSON payload.

The curl call that led to the above message looked like this:

curl^
      -X POST^
      -H "Accept: application/json"^
      -H "Content-Type: application/json"^
      -d '{"Username":"Stefan","Password":"secret"}'^
      http://server/service

After changing the fifth line to -d "{\"Username\":\"Stefan\",\"Password\":\"secret\"}"^ everyhing worked as expected.

How to call a REST service in webMethods Integration Server from Java

After publishing a REST Resource in webMethods Integration Server and giving it a nice logical URL, you may want to call the service from a Java program. Here’s how to do this using the Jersey framework.

Automatically testing a REST Resource

On top of my unit tests for plain old Flow services (as introduced in Unit-testing Flow Services in webMethods’ Integration Server with JUnit and released in Test everything! – Lessons Learned from SOA-fying a Monolith) I need to automatically test my REST Resources as well, of course. For this reason, I took a closer look at the Jersey Java framework, which provides an easy way of calling REST services.

With Jersey, a complete integration test for a REST Resource looks like this (file CreateSessionShould.java):

package net.aokv.is.examplepackage.resources.session;

import static org.hamcrest.CoreMatchers.is;
import static org.junit.Assert.assertThat;

import javax.ws.rs.client.Client;
import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.client.Entity;
import javax.ws.rs.client.Invocation.Builder;
import javax.ws.rs.client.WebTarget;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import javax.xml.bind.annotation.XmlRootElement;

import org.glassfish.jersey.client.authentication.HttpAuthenticationFeature;
import org.junit.Before;
import org.junit.Test;

public class CreateSessionShould
{
    private Client client;
    private WebTarget webTarget;

    @XmlRootElement
    private static class InputParameters
    {
        public String Username;
        public String Password;
    }

    @XmlRootElement
    private static class OutputParameters
    {
        public boolean Authenticated;
    }

    @Before
    public void setup()
    {
        HttpAuthenticationFeature feature = HttpAuthenticationFeature.basic(
                "Administrator",
                "manage");

        client = ClientBuilder.newClient();
        client.register(feature);

        webTarget = client.target("http://localhost:5555/User");
    }

    private Builder createRequest()
    {
        return webTarget
                .path("/Session")
                .request(MediaType.APPLICATION_JSON);
    }

    private void shouldBeAuthenticated(Response response) throws Exception
    {
        assertThat(response.getStatus(), is(200));

        OutputParameters output = response.readEntity(OutputParameters.class);
        assertThat(output.Authenticated, is(true));
    }

    private void shouldNotBeAuthenticated(Response response) throws Exception
    {
        assertThat(response.getStatus(), is(401));

        OutputParameters output = response.readEntity(OutputParameters.class);
        assertThat(output.Authenticated, is(false));
    }

    @Test
    public void authenticateExistingUserWithCorrectPassword() throws Exception
    {
        InputParameters input = new InputParameters();
        input.Username = "Stefan";
        input.Password = "secret";

        Response response = createRequest()
                .post(Entity.entity(input, MediaType.APPLICATION_JSON));
        shouldBeAuthenticated(response);
    }

    @Test
    public void notAuthenticateExistingUserWithWrongPassword() throws Exception
    {
        InputParameters input = new InputParameters();
        input.Username = "Stefan";
        input.Password = "asdf";

        Response response = createRequest()
                .post(Entity.entity(input, MediaType.APPLICATION_JSON));
        shouldNotBeAuthenticated(response);
    }

    @Test
    public void notAuthenticateNotExistingUser() throws Exception
    {
        InputParameters input = new InputParameters();
        input.Username = "unknown";
        input.Password = "asdf";

        Response response = createRequest()
                .post(Entity.entity(input, MediaType.APPLICATION_JSON));
        shouldNotBeAuthenticated(response);
    }
}

You can simply run the test with JUnit:

Testing a REST Resource with Jersey and JUnit

How it works

Jersey provides means to simply call a REST service and convert the input and output parameters to and from POJOs. All you need to do is annotate the corresponding classes (InputParameters and OutputParameters in the example above) with @XmlRootElement. Even though JSON is used for data exchange, the underlying library needs this JAXB annotation to work properly. Everyhing else is handled for you by Jersey and some reflection magic.

The example uses Basic Authentication for the REST service (or in other words: username and password). And HttpAuthenticationFeature provides the means to use Basic Authentication. It’s added to the Client that calls our service.

HttpAuthenticationFeature feature = HttpAuthenticationFeature.basic(
    "Administrator",
    "manage");

This line of code builds the REST service’s URL. You can see how the URL is put together and JSON is used for transferring the payload to the service.

client.target("http://localhost:5555/User")
    .path("/Session")
    .request(MediaType.APPLICATION_JSON);

The response we get from the service contains the HTTP status code and the JSON payload, that is automatically converted to a POJO:

assertThat(response.getStatus(), is(200));
OutputParameters output = response.readEntity(OutputParameters.class);
assertThat(output.Authenticated, is(true));

Calling the service then is as easy as creating the input POJO and POSTing the payload as JSON:

InputParameters input = new InputParameters();
input.Username = "Stefan";
input.Password = "secret";

Response response = createRequest()
    .post(Entity.entity(input, MediaType.APPLICATION_JSON));

Recommended reading

I took a very good introductory course on REST with Jersey on Pluralsight: RESTFul Services in Java using Jersey. It goes into all the details needed to provide as well as consume REST services with Java. I would definitely recommend this course, if you want to get a deeper understanding of the code above.


RESTFul Services in Java using Jersey – $29.00

This course walks through developing RESTFul web services in Java using the Jersey Framework. It walks through all the configuration and setup to begin developing using this framework.

How to configure a URL Alias for a REST Resource in webMethods Integration Server

After I created my first REST Resource in webMethods Integration Server, I immediately saw potential for improvement regarding the URL under which it was made accessible:

http://localhost:5555/rest/ExamplePackage/Resources/Session

The default URL always starts with rest followed by the fully qualified name of the REST Resource. What bugs me is the latter: the path exposes the internal package structure to the outside. And if – for whatever reason – we decide to change this structure later, all clients need to be changed accordingly. And due to REST’s loose coupling you may not be aware of all consumers of your service and only find out late in the process that you missed to update a caller.

So, what we want is a logical URL instead of a physical one that represents our package structure, e.g.:

http://localhost:5555/User/Session

This logical URL should never change due to an internal restructuring.

URL Aliases

Integration Server provides an easy way of creating such logical URLs: URL Aliases.

You can find the dialog for creating URL Aliases in Integration Server’s web interface under Settings -> URL Aliases:

Creating a URL Alias in webMethods Integration Server

  • Alias
    This is the logical URL you want to define for your REST resource, e.g. User/Session.
  • URL Path
    The Alias above gets expanded to this fully qualified path, e.g. rest/ExamplePackage/Resources/Session.
  • Package
    The Alias will be saved to this package. It resides in an XML file called config/urlalias.cnf in the package’s folder in the file system and can be versioned along with the rest of the package’s contents.

Now I can call the REST Resource via the URL Alias:

curl^
      -u Administrator:manage^
      -X POST^
      -H "Accept: application/json"^
      -H "Content-Type: application/json"^
      -d "{\"Username\":\"Stefan\",\"Password\":\"secr23et\"}"^
      http://localhost:5555/User/Session

Calling a REST Resource via a URL Alias in webMethods Integration Server

Partial Matching

As stated in the REST Developer’s Guide, you may need to activate Partial Matching for your URL Aliases to work. See Integration Server Administrator’s Guide for details.

All you need to do is add this Extended Setting and restart IS:

watt.server.url.alias.partialMatching=true

How to create a REST service in webMethods Integration Server

A while ago I created my first REST service in webMethods Integration Server. It’s quite easy to do and works correctly out of the box. Software AG managed to implement the standards quite well and I didn’t have to go through hours of configuring (like I had to with SOAP webservices).

How to create a REST service in webMethods Integration Server

First of all, you don’t actually create a REST service, but a REST resource. A resource represents a business entity for wich you can call the basic CRUD operations through (multiple) services, that respond to the common HTTP methods (GET, POST, PUT, DELETE). So, a REST resource uses multiple Flow services that implement the logic. These services get called automatically by IS, when the corresponding HTTP call is made by the client.

My example implements a common scenario: Authenticating a user based on its username and password.

The first – and most important – question was, how to name the REST resource. I couldn’t name it User, as one might think at first, because this name doesn’t fit the notion of REST: I don’t create or read a user. The user already has to be there and simply gets authenticated. In a SOAP context I would create a Flow service like AuthenticateUser with Username and Password as parameters. But in REST we can only use the basic CRUD operations. And there is no HTTP method called AUTHENTICATE. So I had to find another name for the resource.

What I ended up with was the concept of a Session. When a user gets authenticated, a Session is created for him. When he logs off, the Session gets destroyed. I can also read the Session to find out, whether the user is already logged in. So it all seemed to make sense now.

After the nomenclature problem is solved, here’s a quick step-by-step guide on how I implemented the REST resource in IS.

Step-by-step guide

  • Create a Flow service that implements the logic you need.
    In my case, I created AuthenticateUser as described above (see Best practices for structuring packages in webMethods Integration Server for details on the package’s folder structure).
    Flow service AuthenticateUser
    As the REST resource is merely an additional access layer on top of existing logic (like a Web Service Descriptor), it’s good practice to keep the two concerns – business logic and access logic – separated. Another advantage of this approach is the fact, that you can simply re-use your existing Flow services and only have to add the REST part on top of it.
  • Make sure the Flow service works as expected.
    This step might seem obvious, but if you introduce another layer of complexity in your system, you better make sure the underlying parts already work correctly. Otherwise, the search for potential errors might take very long and you can’t be sure, if it’s really the REST part that goes wrong.
    Successful result of a call to AuthenticateUser
  • Create a REST Resource.
    This is a easy as selecting New -> REST Resource from Designer’s context menu, giving the resource a name and selecting the needed HTTP methods.
    Creating a REST Resource in webMethods Integration Server - Step 1
    Creating a REST Resource in webMethods Integration Server - Step 2
    Here, I used POST, because I want to create a new user session. If you are not exactly sure, which HTTP methods to use, here’s a quick mapping: GET -> read, POST -> create, PUT -> update, DELETE -> delete. Also see Using HTTP Methods for RESTful Services and When should we use PUT and when should we use POST?.
    Creating a REST Resource in webMethods Integration Server - Step 3
    VoilĂĄ, we created our first “REST Service”! 🙂
    Creating a REST Resource in webMethods Integration Server - Step 4
  • Define the parameters for the REST services.
    The newly created service _post has no parameters yet (aside from two technical parameters $resourceID and $path, see REST Developer’s Guide for an explanation of those). So you need to add them.
    Adding paramters to the REST service
    In my case, I simply used the same parameters from AuthenticateUser. But they also can be different.
  • Implement the REST service.
    Implementing the _post service is quite simple, because the main logic is already implemented in AuthenticateUser. Therefore, _post can simply call AuthenticateUser and only needs to take care of REST specialties, like setting a meaningful response code. Otherwise, IS returns 200 (see List of HTTP status codes for additional information) even if the authentication was not successful. Fortunately, there is an HTTP code that describes almost exactly what I need: 401 Unauthorized. You can set the response code with the IS service pub.flow:setResponseCode.
    Implementation of the _post service
  • Test the REST service.
    You can now call the _post service directly from Designer and make sure it works.
    Call _post
    Successful response from _post
    Unsuccessful response from _post
  • Test the REST service from outside IS.
    After you made sure, that the _post service works as expected, you can try and call it from outside IS. My tool of choice for this task is curl (which comes with the installation of Git for Windows, by the way). All REST resources are available in IS under the following URL without any further configuration:

    http://server:port/rest/FullyQualifiedResource
    

    Or translated to my example:

    http://localhost:5555/rest/ExamplePackage/Resources/Session
    

    You can call the REST service with this curl command on Windows (on Linux you need to replace ^ with \, see Windows: How to specify multiline command on command prompt?):

    curl^
      -u Administrator:manage^
      -X POST^
      -H "Accept: application/json"^
      -H "Content-Type: application/json"^
      -d "{\"Username\":\"Stefan\",\"Password\":\"secret\"}"^
      http://localhost:5555/rest/ExamplePackage/Resources/Session
    

    IS automatically unwraps the JSON payload and converts it to IS documents. You only need to provide the same document structure and field names as defined in your _post parameters.
    Result of curl call to _post service
    By the way: I had to use double quotes for the JSON payload and needed to escape the quotes inside the JSON data to make it work on Windows. When I used single quotes (e.g. -d '{"Username":"Stefan","Password":"secret"}'), I got this nice error:

    org.codehaus.jackson.JsonParseException:Unexpected character (''' (code 39)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')\n at [Source: com.wm.net.HttpInputStream@74356f93; line: 1, column: 2]
    

And this leaves you with a fully functional REST service in webMethods Integration Server! 🙂

Official Documentation

These documents helped me along the way:

Recommended reading

For a very good introduction to REST I recommend the Pluralsight course REST Fundamentals:


REST Fundamentals – $29.00

REST is an overloaded, and thus misunderstood term in architectural circles these days. This course attempts to clear up some of the misunderstandings about REST as well as provide a more practical approach for designing RESTful solutions – both clients and services. Additionally, the course looks at REST from the perspective of the cloud and describes how REST is well-suited to meet the demands that the cloud brings to bear on a modern architecture.

SOAP is dead – Lessons Learned from SOA-fying a Monolith

I’ll continue my series of blog posts regarding the lessons we learned while SOA-fying our monolithic Adabas/Natural application with a more technical lesson:

SOAP is dead.

SOAP is dead

This may be a harsh statement, taking into account that we started out with Webservices based on SOAP and at the moment our whole infrastructure is based on it. However, while we were still searching for the right solution to the communication problem two major programming languages stopped supporting SOAP out of the box: Groovy and Ruby. And we used both of them.

If you take a closer look at the current notions in architecture and distributed systems, you’ll find plenty of conference talks and frameworks for providing and consuming REST services. However, SOAP seems to be legacy technology already.

To be honest, we don’t have a single REST service in production, yet. But after working with SOAP Webservices for quite some time now, I can definitely see the advantages of a more loosely coupled approach like REST. The interface design for SOAP services can take quite a lot of time, because you have to define individual operations and data types. And changing the interface – e.g. adding a parameter – leads to a required re-build of all systems using it. With REST you only need simple CRUD operations and can use JSON as a very flexible data format that can be added to later on.

I’m glad that Integration Server provides an easy means to publish REST services. You can even put them on top of existing services and simply provide a more flexible interface to the consumers. This is definitely a solution I want to try out in the near future.

So, if you start with an SOA today, take a closer look at REST and don’t blindly use the default implementation SOAP as it may not be supported any more in the near future.