Using Gradle wrapper behind a proxy server with self-signed SSL certificates

Today, I wanted to add a Gradle Wrapper to my Java project but had a few issues. I am behind a proxy and it changes the SSL certificates to be able to scan traffic for viruses.

My first attempt to start gradlew build resulted in:

    Exception in thread "main" java.net.UnknownHostException: services.gradle.org
            at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
            at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
            at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
            at java.net.Socket.connect(Socket.java:589)
            at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:668)
            at sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:173)
            ...

Gradle didn’t use the proxy server and tried to connect to the internet directly. This was solved by setting the proxy server in %GRADLE_USER_HOME%\gradle.properties (see Gradlew behind a proxy):

    systemProp.http.proxyHost=192.168.1.1
    systemProp.http.proxyPort=80
    systemProp.http.proxyUser=userid
    systemProp.http.proxyPassword=password
    systemProp.https.proxyHost=192.168.1.1
    systemProp.https.proxyPort=80
    systemProp.https.proxyUser=userid
    systemProp.https.proxyPassword=password

The next try lead to:

    Downloading https://services.gradle.org/distributions/gradle-2.11-bin.zip

    Exception in thread "main" javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
            at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
            at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1949)
            at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302)
            at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)
            at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1509)
            ....

The reason for the SSLHandshakeException were the proxy’s selft-signed certificates, that could not be validated. I had to add them to the Java keystore (see Java: Ignore/Trust an invalid SSL cert for https communication and Cacerts default password? -> the default password for the Java keystore is changeit):

    "%JAVA_HOME%\bin\keytool" -import -trustcacerts -alias MY_ALIAS -file MY_CERT.crt -keystore "%JAVA_HOME%\jre\lib\security\cacerts"

Now, Gradle was able to connect to gradle.org to download the distribution. However, the proxy server would not let the ZIP file through:

    Exception in thread "main" java.io.IOException: Server returned HTTP response code: 403 for URL: https://downloads.gradle.org/distributions/gradle-2.11-bin.zip
            at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840)
            at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
            at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
            ...

So I configured Gradle to “download” the ZIP file from the local hard drive in %GRADLE_USER_HOME%\gradle.properties (see How to use gradle zip in local system without downloading when using gradle-wrapper):

    distributionBase=GRADLE_USER_HOME
    distributionPath=wrapper/dists
    zipStoreBase=GRADLE_USER_HOME
    zipStorePath=wrapper/dists
    distributionUrl=gradle-2.11-bin.zip

I manually downloaded the distribution file and put it into %GRADLE_USER_HOME%\wrapper\dists\gradle-2.11-bin\[SOME_HASH]\.

And finally the build was successful! 😀

    D:\MY_PROJECT>gradlew build
    Unzipping D:\GradleUserHome\wrapper\dists\gradle-2.11-bin\452syho4l32rlk2s8ivdjogs8\gradle-2.11-bin.zip to D:\GradleUserHome\wrapper\dists\gradle-2.11-bin\452syho4l32rlk2s8ivdjogs8
    Starting a new Gradle Daemon for this build (subsequent builds will be faster).
    Parallel execution with configuration on demand is an incubating feature.
    :compileJava UP-TO-DATE
    ...

How to call a REST service in webMethods Integration Server from Java

After publishing a REST Resource in webMethods Integration Server and giving it a nice logical URL, you may want to call the service from a Java program. Here’s how to do this using the Jersey framework.

Automatically testing a REST Resource

On top of my unit tests for plain old Flow services (as introduced in Unit-testing Flow Services in webMethods’ Integration Server with JUnit and released in Test everything! – Lessons Learned from SOA-fying a Monolith) I need to automatically test my REST Resources as well, of course. For this reason, I took a closer look at the Jersey Java framework, which provides an easy way of calling REST services.

With Jersey, a complete integration test for a REST Resource looks like this (file CreateSessionShould.java):

package net.aokv.is.examplepackage.resources.session;

import static org.hamcrest.CoreMatchers.is;
import static org.junit.Assert.assertThat;

import javax.ws.rs.client.Client;
import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.client.Entity;
import javax.ws.rs.client.Invocation.Builder;
import javax.ws.rs.client.WebTarget;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import javax.xml.bind.annotation.XmlRootElement;

import org.glassfish.jersey.client.authentication.HttpAuthenticationFeature;
import org.junit.Before;
import org.junit.Test;

public class CreateSessionShould
{
    private Client client;
    private WebTarget webTarget;

    @XmlRootElement
    private static class InputParameters
    {
        public String Username;
        public String Password;
    }

    @XmlRootElement
    private static class OutputParameters
    {
        public boolean Authenticated;
    }

    @Before
    public void setup()
    {
        HttpAuthenticationFeature feature = HttpAuthenticationFeature.basic(
                "Administrator",
                "manage");

        client = ClientBuilder.newClient();
        client.register(feature);

        webTarget = client.target("http://localhost:5555/User");
    }

    private Builder createRequest()
    {
        return webTarget
                .path("/Session")
                .request(MediaType.APPLICATION_JSON);
    }

    private void shouldBeAuthenticated(Response response) throws Exception
    {
        assertThat(response.getStatus(), is(200));

        OutputParameters output = response.readEntity(OutputParameters.class);
        assertThat(output.Authenticated, is(true));
    }

    private void shouldNotBeAuthenticated(Response response) throws Exception
    {
        assertThat(response.getStatus(), is(401));

        OutputParameters output = response.readEntity(OutputParameters.class);
        assertThat(output.Authenticated, is(false));
    }

    @Test
    public void authenticateExistingUserWithCorrectPassword() throws Exception
    {
        InputParameters input = new InputParameters();
        input.Username = "Stefan";
        input.Password = "secret";

        Response response = createRequest()
                .post(Entity.entity(input, MediaType.APPLICATION_JSON));
        shouldBeAuthenticated(response);
    }

    @Test
    public void notAuthenticateExistingUserWithWrongPassword() throws Exception
    {
        InputParameters input = new InputParameters();
        input.Username = "Stefan";
        input.Password = "asdf";

        Response response = createRequest()
                .post(Entity.entity(input, MediaType.APPLICATION_JSON));
        shouldNotBeAuthenticated(response);
    }

    @Test
    public void notAuthenticateNotExistingUser() throws Exception
    {
        InputParameters input = new InputParameters();
        input.Username = "unknown";
        input.Password = "asdf";

        Response response = createRequest()
                .post(Entity.entity(input, MediaType.APPLICATION_JSON));
        shouldNotBeAuthenticated(response);
    }
}

You can simply run the test with JUnit:

Testing a REST Resource with Jersey and JUnit

How it works

Jersey provides means to simply call a REST service and convert the input and output parameters to and from POJOs. All you need to do is annotate the corresponding classes (InputParameters and OutputParameters in the example above) with @XmlRootElement. Even though JSON is used for data exchange, the underlying library needs this JAXB annotation to work properly. Everyhing else is handled for you by Jersey and some reflection magic.

The example uses Basic Authentication for the REST service (or in other words: username and password). And HttpAuthenticationFeature provides the means to use Basic Authentication. It’s added to the Client that calls our service.

HttpAuthenticationFeature feature = HttpAuthenticationFeature.basic(
    "Administrator",
    "manage");

This line of code builds the REST service’s URL. You can see how the URL is put together and JSON is used for transferring the payload to the service.

client.target("http://localhost:5555/User")
    .path("/Session")
    .request(MediaType.APPLICATION_JSON);

The response we get from the service contains the HTTP status code and the JSON payload, that is automatically converted to a POJO:

assertThat(response.getStatus(), is(200));
OutputParameters output = response.readEntity(OutputParameters.class);
assertThat(output.Authenticated, is(true));

Calling the service then is as easy as creating the input POJO and POSTing the payload as JSON:

InputParameters input = new InputParameters();
input.Username = "Stefan";
input.Password = "secret";

Response response = createRequest()
    .post(Entity.entity(input, MediaType.APPLICATION_JSON));

Recommended reading

I took a very good introductory course on REST with Jersey on Pluralsight: RESTFul Services in Java using Jersey. It goes into all the details needed to provide as well as consume REST services with Java. I would definitely recommend this course, if you want to get a deeper understanding of the code above.


RESTFul Services in Java using Jersey – $29.00

This course walks through developing RESTFul web services in Java using the Jersey Framework. It walks through all the configuration and setup to begin developing using this framework.

Externalize Java helper methods with access to the service pipeline of webMethods Integration Server

As I’ve mentioned earlier, I like to re-use my existing code whenever possible (and if not possible I’ll try to find a way ;-)). When writing Java Services in webMethods’ Integration Server, I wanted to use an existing method that provides easy access and type safety when working with the service pipeline (i.e. interface IData). The problem was, that Software AG Designer (i.e. Eclipse) didn’t “know” the needed interfaces etc. to compile the source code locally (!). As it turns out, all you have to add to your Java project to program against Integration Server’s API is a single JAR file. However, to find this particular JAR file you have to dig deep into the Designer’s installation folder. I found it here: [...]\SoftwareAG\eclipse\v36\plugins\com.webmethods.process.model.upgrade.impl_8.2.2.0000-0211\lib (you’ll have to adjust the path to your version of IS) and it’s called client.jar. After adding the JAR file to the Eclipse project’s CLASSPATH I was able to externalize my short helper function into its own project and re-use it in all my Java Services. Here’s my code of a type safe access method for the pipeline:

public static <T> T getInputParameter(IData pipeline, String paramName) throws ServiceException
{
  IDataCursor pipelineCursor = pipeline.getCursor();
  Object paramValue = IDataUtil.get(pipelineCursor, paramName);
  pipelineCursor.destroy();

  if (paramValue == null)
  {
    throw new ServiceException("Required parameter <" + paramName + "> not set.");
  }

  try
  {
    return (T)paramValue;
  }
  catch (ClassCastException e)
  {
    throw new ServiceException("Required parameter <" + paramName + "> could not be casted.");
  }
}

Re-use your own existing Java code in a Java Service in webMethods Integration Server

Modularization is an important concept of programming. So, when I wanted to create a Java Service in webMethods’ Integration Server today, I wanted to re-use my existing Java code that already provided the feature I wanted to publish as a Service. However, this is not as easy as it sounds 😉 Here are the steps you need to perform to get your already existing Java code working in a Java Service:

  1. Compile and bundle your external Java project into a JAR file, e.g. using Apache Ant.
  2. Copy this JAR file into the subfolder code\jars\ of your package on Integration Server and reload the package. Example path: [...]\SoftwareAG\IntegrationServer\packages\YOURPACKAGE\code\jars\. Now, IS is able to use the JAR and automatically puts it into the package’s CLASSPATH.
  3. Copy the JAR file into the subfolder lib\ of your local Java project (in your local Eclipse workspace) and add it to the Eclipse project’s CLASSPATH, e.g. by adding <classpathentry kind="lib" path="lib/MyExternalProject.jar"/> to .classpath. Now, Software AG Designer (i.e. Eclipse) is able to resolve the external classes and can compile the code correctly, provide code completion etc.
  4. import the needed packages from your external project and use the classes in your code. Here’s some example code from the resulting Java Service:

    package XML.Services;
    
    import com.wm...; // all webMethods imports
    import it.macke.tools.saxxmlvalidator.SaxXmlValidator; // import from your external project
    
    public final class ValidateXML_SVC
    {
      /** 
       * The primary method for the Java service
       *
       * @param pipeline
       *            The IData pipeline
       * @throws ServiceException
       */
      public static final void ValidateXML(IData pipeline) throws ServiceException
      {
        xml = ISPipelineHelper.getInputParameter(pipeline, "xmlStream");
        schema = ISPipelineHelper.getInputParameter(pipeline, "schemaStream");
        validateXML();
      }
    
      // --- <<IS-BEGIN-SHARED-SOURCE-AREA>> ---  
    
      private static InputStream xml;
      private static InputStream schema;
      private static boolean isValid;
      private static String validationError;
    
      private static void validateXML() throws ServiceException 
      {
        isValid = true;
        validationError = "";
    
        // now you can use your own classes
        SaxXmlValidator validator = new SaxXmlValidator(xml, schema);
        try 
        {
          validator.validate();
        }
        catch (XmlNotValidException e) 
        {
          isValid = false;
          validationError = e.getMessage();
        }
        catch (Exception e)
        {
          throw new ServiceException(e);
        }
      }
    
      // --- <<IS-END-SHARED-SOURCE-AREA>> ---
    }
    

How to change the Java version for webMethods Integration Server

Changing the Java version under which webMethods Integration Server runs and compiles Java services is quite easy (after you searched for hours for the right files to change ;-)). You can check which version your Integration Server uses on the About page:

webMethods Integration Server About page: Java Version

Here’s how to change the version for Integration Server (in my case version 9.7 running on Windows Server 2012):

  1. Install the new JVM on the server Integration Server runs on, e.g. into C:\Program Files.
  2. In the Extended Settings of your Integration Server, set watt.server.compile to the new JVM, e.g. watt.server.compile=C:\Program Files\Java\jdk1.8.0_25\bin\javac -classpath {0} -d {1} {2}
    webMethods Integration Server Extended Settings: watt.server.compile
  3. Shutdown Integration Server.
  4. Edit [SAG]\profiles\IS_default\configuration\custom_wrapper.conf and add (or change) the setting wrapper.java.command, e.g. wrapper.java.command=D:\Dev\Java\jdk1.8.0_25\bin\java
  5. Deregister the Windows service (as Administrator) with [SAG]\IntegrationServer\instances\default\support\win32\installSvc.bat unreg:
    Unregister webMethods Integration Server Windows service
  6. Register the Windows service (as Administrator) with [SAG]\IntegrationServer\instances\default\support\win32\installSvc.bat:
    Register webMethods Integration Server Windows service
  7. Start Integration Server and check the About page for whether the new Java version was set correctly:
    webMethods Integration Server About page: Java Version

Unit-testing Flow Services in webMethods’ Integration Server with JUnit

Today, I built a small test bed for unit-testing Flow Services in webMethods’ Integration Server. If you develop Java Services for IS and follow the best practice of “no business logic in the service layer”, you can unit-test your logic in isolation in a plain old Java environment and deploy the library to IS, which then only acts as the service bus and simply forwards service calls to the library. However, if you create Flow Services directly in IS, which can get complicated pretty fast, I found no built-in tool for testing their logic. Here’s an example of a small Flow Service:

webMethods Integration Server Flow Service

So I went ahead and created a local Java project, in which I could unit-test the remote services. I know network access in unit-tests is considered a “boundary” which should not be crossed, because tests run slower and may fail due to network problems. However, I would rather have a suite of (relatively) slow running tests for my important services than have no tests at all and see programs fail due to regression. And debugging services in IS is no fun at all! Calling the IS services directly from Java is surprisingly easy. Software AG Designer generates all the needed source code for you. Right-click on the service and choose Generate Code. Then follow the steps in the wizard:

Calling IS Service Wizard 1

Calling IS Service Wizard 2

Calling IS Service Wizard 3

The generated code is executable directly, as long as you add references to IS’ client.jar and enttoolkit.jar to your Java project.

Testing IS Services References

The service call itself is implemented in this simple method:

public static IData invoke(Context context, IData inputDocument) 
    throws IOException, ServiceException
{
     IData out = context.invoke("PACKAGE", "NAME", inputDocument);
     IData outputDocument = out;
     return outputDocument;
}

However, as you can see above, the service accepts and returns IData objects, whose construction is not that easy and should not be scattered throughout your test code. Even providing a simple value requires at least the following code. And the code for getting the value out of IData to be able to call an assertion on it looks similar.

IData out = IDataFactory.create();
IDataCursor idc = out.getCursor();
idc.insertAfter("errorFlag", "true");
idc.destroy();

So, to make life easier for me, I spent some time and developed a small environment, in which I can work with plain old Java objects for service input and output in my tests and assert against the public fields of these objects, like this example of a simple date conversion service demonstrates:

protected ConvertDateToN8 sut;
protected ConvertDateToN8.Input input;

@Test
public void shouldConvertValidDate() throws Exception
{
    input.Date = "2008-10-15";
    ConvertDateToN8.Output output = sut.call(input);
    assertThat(output.DateN8, is("20081015"));
}

The service class looks like this:

public class ConvertDateToN8 extends ServiceCall<ConvertDateToN8.Input, ConvertDateToN8.Output>
{
    public class Input extends ServiceInput
    {
        public String Date;
    }

    public class Output extends ServiceOutput
    {
        public String DateN8;
    }

    public ConvertDateToN8(String host)
    {
        super(host, "Common", "ConvertDateToN8");
    }

    public Input createInput()
    {
        return new Input();
    }

    @Override
    protected Output createOutput()
    {
        return new Output();
    }
}

The only thing that needs to be added to the project to test another service is a class like the one above. I can simply define the input and output of the service in nested classes and everything else is magically built with generics and reflection. Here is an example of converting the generic Input object to IData in class ServiceCall:

protected IData createPipelineFromInput(ServiceInput input)
{
    IData pipeline = IDataFactory.create();
    IDataCursor idc = pipeline.getCursor();

    Class<?> c = input.getClass();
    for (Field publicField : c.getFields())
    {
        try
        {
            String fieldName = FieldNameConverter.convertToPipelineName(publicField.getName());
            Object fieldValue = FieldConverter.convertToPipeline(publicField.getType().getName(), publicField.get(input));
            IDataUtil.put(idc, fieldName, fieldValue);
        }
        catch (Exception e)
        {
            e.printStackTrace();
        }
    }
    idc.destroy();

    return pipeline;
}

Even more complex data structures like objects or arrays can be constructed by extending the classes FieldConverter and PipelineConverter which encapsulate the access to IData:

public class Input extends ServiceInput
{
    public String errorFlag;
    public String errorCode;
    public String errorMessage;
    public NaturalErrorInfo ERROR_INFO;
}

You can take a look at the implementation for converting the above input to and from IData in the source code, which can be downloaded below. Feel free to take a look at it and contact me for questions or additions!

Download