Kohei Nozaki's blog 

I introduced that a way to test a JSR352 job with Arquillian, but I guess it's might too complex for testing of a JSR352 job, so I'm going to try to test them without Arquillian.

Environment

• WildFly 8.0.0.Final
• Oracle JDK7u51

Sample project

• Whole resources are available at GitHub.
• Example test class is HelloJobTest.

How does it work?

1. JUnit test class lookups javax.batch.operations.JobOperator instance from Remote WildFly through Remote EJB Interface.
2. Test class kicks the job.
3. Test class will wait till the job finished.
4. Assert BatchStatus.

How do I run on other application servers?

• I don't know surely, but edit below resources may helps:
  • jndi.properties
  • AbstractJobTest.java (JNDI_NAME)
• Also appropriate jar files for remote EJB invocation are required.

Log of WildFly

17:06:18,214 INFO  [stdout] (batch-batch - 3) hello

Local log

2 26, 2014 5:14:39 午後 org.xnio.Xnio <clinit>
INFO: XNIO version 3.2.0.Final
2 26, 2014 5:14:40 午後 org.xnio.nio.NioXnio <clinit>
INFO: XNIO NIO Implementation Version 3.2.0.Final
2 26, 2014 5:14:40 午後 org.jboss.remoting3.EndpointImpl <clinit>
INFO: JBoss Remoting version 4.0.0.Final
2 26, 2014 5:14:40 午後 org.jboss.ejb.client.remoting.VersionReceiver handleMessage
INFO: EJBCLIENT000017: Received server version 2 and marshalling strategies [river]
2 26, 2014 5:14:40 午後 org.jboss.ejb.client.remoting.RemotingConnectionEJBReceiver associate
INFO: EJBCLIENT000013: Successful version handshake completed for receiver context EJBReceiverContext{clientContext=org.jboss.ejb.client.EJBClientContext@2db9e6d7, receiver=Remoting connection EJB receiver [connection=org.jboss.ejb.client.remoting.ConnectionPool$PooledConnection@7e244b5,channel=jboss.ejb,nodename=kyle-no-macbook]} on channel Channel ID c584a9f3 (outbound) of Remoting connection 09168b43 to localhost/127.0.0.1:8080
2 26, 2014 5:14:40 午後 org.jboss.ejb.client.EJBClient <clinit>
INFO: JBoss EJB Client version 2.0.0.Final

Remarks

• It's much faster than Arquillian Remoting on my environment.

Environment

• Commons Configuration 1.10
• WildFly 8.0.0.Final
• Oracle JDK7u51

Why go this way?

Various portable ways are available for manage configurations of the Java EE application. such as:

• Configuration table in DB
  • I guess it's not bad, but I feel that is bit tedious work to coding.
  • There's a way to use this mechanism via Commons Configuration, but I haven't tried it yet.
• Custom JNDI resources
  • I thought that it's most efficient at first, but negative points here:
    • Badness of lookup performance.
    • Editing JNDI resources is very annoying.

So sometimes I'm thinking of any better way, and I found Commons Configurations[1]. notable capabilities of Commons Configuration are:

• Simple API
• Auto-reloading and saving mechanism provided
• Support of various formats and placement styles

So I'd try it.

Prepare sample project and related resources

Create a system property

• Create a system property which specifies the path of configuration file.
• I'm using WildFly so go with jboss-cli:

[standalone@localhost:9990 /] /system-property=app.config:add(value=/Users/kyle/app.properties)
{"outcome" => "success"}
[standalone@localhost:9990 /] 

Create the properties file

• Location is same as “app.config” system property.

hoge=fuge

pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.nailedtothex</groupId>
    <artifactId>config</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>war</packaging>

    <properties>
        <maven.compiler.source>1.7</maven.compiler.source>
        <maven.compiler.target>1.7</maven.compiler.target>
        <failOnMissingWebXml>false</failOnMissingWebXml>
    </properties>

    <dependencies>
        <dependency>
            <groupId>javax</groupId>
            <artifactId>javaee-api</artifactId>
            <version>7.0</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>commons-configuration</groupId>
            <artifactId>commons-configuration</artifactId>
            <version>1.10</version>
        </dependency>
    </dependencies>
</project>

Create an application-scoped bean

package org.nailedtothex.config;

import javax.annotation.PostConstruct;
import javax.enterprise.context.ApplicationScoped;
import javax.inject.Named;

import org.apache.commons.configuration.Configuration;
import org.apache.commons.configuration.ConfigurationException;
import org.apache.commons.configuration.PropertiesConfiguration;
import org.apache.commons.configuration.reloading.FileChangedReloadingStrategy;

@Named
@ApplicationScoped
public class AppConfigBean {

    private Configuration config;

    @PostConstruct
    void init() throws ConfigurationException {
        String path = System.getProperty("app.config");
        PropertiesConfiguration propConfig = new PropertiesConfiguration(path); 
        propConfig.setReloadingStrategy(new FileChangedReloadingStrategy());
        config = propConfig;
    }

    public Configuration getConfig(){
        return config;
    }
}

• If you want to read the config at startup of application, I recommend that use of @javax.ejb.Singleton and @javax.ejb.Startup[2].

Create a servlet

• Get a configuration variable from singleton bean.

package org.nailedtothex.config;

import java.io.IOException;
import java.io.PrintWriter;

import javax.inject.Inject;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

@WebServlet(urlPatterns = "/*")
public class HelloServlet extends HttpServlet {
    private static final long serialVersionUID = 1L;
    @Inject
    private AppConfigBean bean;

    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        try (PrintWriter pw = response.getWriter()) {
            pw.write(bean.getConfig().getString("hoge"));
        }
    }
}

Access via browser

• It needs only very small and clean code.

Check runtime reloading

Edit app.properties:

kyle-no-MacBook:~ kyle$ echo hoge=FUGE > app.properties
kyle-no-MacBook:~ kyle$ 

Reload browser:

• I just reloaded the browser and I didn't do any other operations, but config was updated immediately.

Collaboration with CDI

• There's a way to inject properties to CDI managed bean at [2]. that must be useful.

XMLConfiguration

• XMLConfiguration is good for hierarchical configuration.
• It depended on Commons Collection so don't forget to add the dependency.

pom.xml

        <dependency>
            <groupId>commons-configuration</groupId>
            <artifactId>commons-configuration</artifactId>
            <version>1.10</version>
        </dependency>
        <dependency>
            <groupId>commons-collections</groupId>
            <artifactId>commons-collections</artifactId>
            <version>3.2.1</version>
        </dependency>

XmlTest.java

package org.nailedtothex.config;

import java.util.Iterator;
import java.util.List;

import org.apache.commons.configuration.XMLConfiguration;

public class XmlTest {

    public static void main(String[] args) throws Exception {
        XMLConfiguration config = new XMLConfiguration();
        config.load(XmlTest.class.getResourceAsStream("/config.xml"));

        String backColor = config.getString("colors.background");
        String textColor = config.getString("colors.text");
        String linkNormal = config.getString("colors.link[@normal]");
        String defColor = config.getString("colors.default");
        int rowsPerPage = config.getInt("rowsPerPage");
        List<Object> buttons = config.getList("buttons.name");

        System.out.println(backColor);
        System.out.println(textColor);
        System.out.println(linkNormal);
        System.out.println(defColor);
        System.out.println(rowsPerPage);
        System.out.println(buttons);

        System.out.println();

        Object o;
        Iterator it = config.getKeys("colors.");
        while(it.hasNext()){
            System.out.println(it.next());
        }
    }
}

config.xml

<?xml version="1.0" encoding="UTF-8"?>
<gui-definition>
  <colors>
    <background>#808080</background>
    <text>#000000</text>
    <header>#008000</header>
    <link normal="#000080" visited="#800080"/>
    <default>${colors.header}</default>
  </colors>
  <rowsPerPage>15</rowsPerPage>
  <buttons>
    <name>OK,Cancel,Help</name>
  </buttons>
  <numberFormat pattern="###\,###.##"/>
</gui-definition>

Log

#808080
#000000
#000080
#008000
15
[OK, Cancel, Help]

colors.background
colors.text
colors.header
colors.link[@normal]
colors.link[@visited]
colors.default

References

1. Commons Configuration - Java Configuration API
2. CDIでアプリケーション設定をインジェクション - 見習いプログラミング日記
3. Apache commonsが便利な件（commons-configuration編） - 都元ダイスケ IT-PRESS
4. Commons Configuration - Hierarchical configurations and XML Howto

Environment

• WireMock 1.4.3
• HttpClient 4.3.2
• Oracle JDK7u51

Why need it?

• Sometimes website scraping stops at unexpected error like socket timeout or internal server error.
• We prefer to retry several times at such occasion.
• So we have to develop some retrying mechanism, and we have to create a test and create the mock of server.
  • So in this article, I'm going to try to create the mock with WireMock.

Requirements for client

• Retry if processing fails, for 3 times.
• Retry when timeout occurred.
  • SocketTimeoutException
• Retry when server returned status code 5xx

Test cases

1. Server returns code 200 without any problems.
2. Server returns code 500 at first, then client will retry, server returns code 200.
3. Server returns code 500 forever, then client will retry 3 times and give up.
4. Server delays the response, then client will regard as timeout and retry, then server returns code 200.
5. Server delays the response forever, then client will retry 3 times and give up.

Resources

pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.nailedtothex</groupId>
    <artifactId>wiremock</artifactId>
    <version>0.0.1-SNAPSHOT</version>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.7</maven.compiler.source>
        <maven.compiler.target>1.7</maven.compiler.target>
    </properties>

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.11</version>
            <scope>test</scope>
        </dependency>
        <dependency>
          <groupId>org.hamcrest</groupId>
          <artifactId>hamcrest-core</artifactId>
          <version>1.3</version>
          <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.github.tomakehurst</groupId>
            <artifactId>wiremock</artifactId>
            <version>1.43</version>
            <classifier>standalone</classifier>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>*</groupId>
                    <artifactId>*</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
            <version>4.3.2</version>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>fluent-hc</artifactId>
            <version>4.3.2</version>
        </dependency>
    </dependencies>

</project>

RetryableHttpFetcher.java

package org.nailedtothex.wiremock;

import java.io.IOException;
import java.util.logging.Level;
import java.util.logging.Logger;

import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpRequestRetryHandler;
import org.apache.http.client.HttpResponseException;
import org.apache.http.client.ServiceUnavailableRetryStrategy;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.protocol.HttpContext;
import org.apache.http.util.EntityUtils;

public class RetryableHttpFetcher {

    private static final Logger log = Logger.getLogger(RetryableHttpFetcher.class.getName());

    // these parameters would be better to retrieve through JNDI or any other mechanism
    private int MAX_RETRIES = 3;
    private int RETRY_INTERVAL = 1000;
    private int TIMEOUT = 500;

    private final ServiceUnavailableRetryStrategy MY_SERVICE_UNAVAILABLE_RETRY_STRATEGY = new ServiceUnavailableRetryStrategy() {
        @Override
        public boolean retryRequest(HttpResponse response, int executionCount,
                org.apache.http.protocol.HttpContext context) {

            boolean rc = response.getStatusLine().getStatusCode() >= 500 && executionCount <= MAX_RETRIES;

            log.log(Level.INFO,
                    "retryRequest(): returning={0}, statusCode={1}, executionCount={2}, maxRetries={3}, interval={4}",
                    new Object[] { rc, response.getStatusLine().getStatusCode(), executionCount, MAX_RETRIES,
                            RETRY_INTERVAL });

            return rc;
        }

        @Override
        public long getRetryInterval() {
            return RETRY_INTERVAL;
        }
    };

    private final HttpRequestRetryHandler MY_HTTP_REQUEST_RETRY_HANDLER = new HttpRequestRetryHandler() {

        @Override
        public boolean retryRequest(IOException e, int executionCount, HttpContext context) {
            log.log(Level.INFO, "retryRequest(): exception={0}, executionCount={1}, maxRetries={2}",
                    new Object[] { e.getClass(), executionCount, MAX_RETRIES });

            if (executionCount > MAX_RETRIES) {
                log.log(Level.INFO, "give up: {0}", executionCount);
                return false;
            }

            if (e instanceof java.net.SocketTimeoutException) {
                log.log(Level.INFO, "retry: {0}", e.getMessage());
                return true;
            }

            log.log(Level.INFO, "not retry: {0}", e.getMessage());
            return false;
        }
    };

    private final RequestConfig MY_REQUEST_CONFIG = RequestConfig.custom()
            .setConnectionRequestTimeout(TIMEOUT)
            .setConnectTimeout(TIMEOUT)
            .setSocketTimeout(TIMEOUT)
            .build();

    public String fetchAsString(String url) throws ClientProtocolException, IOException {

        try (CloseableHttpClient client = HttpClientBuilder.create()
                .setDefaultRequestConfig(MY_REQUEST_CONFIG)
                .setRetryHandler(MY_HTTP_REQUEST_RETRY_HANDLER)
                .setServiceUnavailableRetryStrategy(MY_SERVICE_UNAVAILABLE_RETRY_STRATEGY)
                .build()) {

            try (CloseableHttpResponse res = client.execute(new HttpGet(url))) {
                if (res.getStatusLine().getStatusCode() >= 400) {
                    throw new HttpResponseException(res.getStatusLine().getStatusCode(), res.getStatusLine()
                            .getReasonPhrase());
                }
                return EntityUtils.toString(res.getEntity());
            }
        }

    }
}

RetryableHttpFetcherTest.java

package org.nailedtothex.wiremock;

import static com.github.tomakehurst.wiremock.client.WireMock.*;
import static org.hamcrest.CoreMatchers.*;
import static org.junit.Assert.*;

import java.net.SocketTimeoutException;

import org.apache.http.client.HttpResponseException;
import org.junit.Before;
import org.junit.Rule;
import org.junit.Test;

import com.github.tomakehurst.wiremock.junit.WireMockRule;
import com.github.tomakehurst.wiremock.stubbing.Scenario;

public class RetryableHttpFetcherTest {

    @Rule
    public WireMockRule wireMockRule = new WireMockRule(18089);

    private RetryableHttpFetcher instance;

    @Before
    public void init() {
        instance = new RetryableHttpFetcher();
    }

    @Test
    public void test1_ok() throws Exception {
        stubFor(get(urlEqualTo("/hoge.txt")).willReturn(
                aResponse().withStatus(200).withHeader("Content-Type", "text/plain").withBody("OK")));
        String expected = "OK";

        String actual = instance.fetchAsString("http://localhost:18089/hoge.txt");

        assertThat(actual, is(expected));
    }

    @Test
    public void test2_retryAt500() throws Exception {
        stubFor(get(urlEqualTo("/500")).inScenario("retry at 500")
                .whenScenarioStateIs(Scenario.STARTED)
                .willSetStateTo("one time requested")
                .willReturn(aResponse().withBody("error").withStatus(500)));

        stubFor(get(urlEqualTo("/500")).inScenario("retry at 500")
                .whenScenarioStateIs("one time requested")
                .willReturn(aResponse().withBody("OK").withStatus(200)));

        String actual = instance.fetchAsString("http://localhost:18089/500");

        assertThat(actual, is("OK"));
    }

    @Test(expected = HttpResponseException.class)
    public void test3_retryAt500GiveUp() throws Exception {
        stubFor(get(urlEqualTo("/500"))
                .willReturn(aResponse().withBody("500").withStatus(500)));

        instance.fetchAsString("http://localhost:18089/500");
    }

    @Test
    public void test4_retryAtTimeout() throws Exception {
        stubFor(get(urlEqualTo("/timeout")).inScenario("retrying")
                .whenScenarioStateIs(Scenario.STARTED)
                .willSetStateTo("one time requested")
                .willReturn(aResponse().withBody("error").withStatus(500).withFixedDelay(3000)));
        stubFor(get(urlEqualTo("/timeout")).inScenario("retrying")
                .whenScenarioStateIs("one time requested")
                .willReturn(aResponse().withBody("OK").withStatus(200)));

        String actual = instance.fetchAsString("http://localhost:18089/timeout");
        assertThat(actual, is("OK"));
    }

    @Test(expected = SocketTimeoutException.class)
    public void test5_retryAtTimeoutGiveUp() throws Exception {
        stubFor(get(urlEqualTo("/timeout"))
                .willReturn(aResponse().withBody("timeout").withStatus(500).withFixedDelay(Integer.MAX_VALUE)));
        instance.fetchAsString("http://localhost:18089/timeout");
    }

    @Test(expected = HttpResponseException.class)
    public void notFound() throws Exception {
        instance.fetchAsString("http://localhost:18089/NOT_FOUND");
    }
}

Test logs

• All tests were passed.

test1_ok

2 26, 2014 11:34:22 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$1 retryRequest
情報: retryRequest(): returning=false, statusCode=200, executionCount=1, maxRetries=3, interval=1,000

test2_retryAt500

2 26, 2014 11:35:32 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$1 retryRequest
情報: retryRequest(): returning=true, statusCode=500, executionCount=1, maxRetries=3, interval=1,000
2 26, 2014 11:35:33 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$1 retryRequest
情報: retryRequest(): returning=false, statusCode=200, executionCount=2, maxRetries=3, interval=1,000

test3_retryAt500GiveUp

2 26, 2014 11:35:51 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$1 retryRequest
情報: retryRequest(): returning=true, statusCode=500, executionCount=1, maxRetries=3, interval=1,000
2 26, 2014 11:35:52 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$1 retryRequest
情報: retryRequest(): returning=true, statusCode=500, executionCount=2, maxRetries=3, interval=1,000
2 26, 2014 11:35:53 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$1 retryRequest
情報: retryRequest(): returning=true, statusCode=500, executionCount=3, maxRetries=3, interval=1,000
2 26, 2014 11:35:54 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$1 retryRequest
情報: retryRequest(): returning=false, statusCode=500, executionCount=4, maxRetries=3, interval=1,000

test4_retryAtTimeout

2 26, 2014 11:36:12 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$2 retryRequest
情報: retryRequest(): exception=class java.net.SocketTimeoutException, executionCount=1, maxRetries=3
2 26, 2014 11:36:12 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$2 retryRequest
情報: retry: Read timed out
2 26, 2014 11:36:12 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$1 retryRequest
情報: retryRequest(): returning=false, statusCode=200, executionCount=1, maxRetries=3, interval=1,000

test5_retryAtTimeoutGiveUp

2 26, 2014 11:36:39 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$2 retryRequest
情報: retryRequest(): exception=class java.net.SocketTimeoutException, executionCount=1, maxRetries=3
2 26, 2014 11:36:39 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$2 retryRequest
情報: retry: Read timed out
2 26, 2014 11:36:40 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$2 retryRequest
情報: retryRequest(): exception=class java.net.SocketTimeoutException, executionCount=2, maxRetries=3
2 26, 2014 11:36:40 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$2 retryRequest
情報: retry: Read timed out
2 26, 2014 11:36:40 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$2 retryRequest
情報: retryRequest(): exception=class java.net.SocketTimeoutException, executionCount=3, maxRetries=3
2 26, 2014 11:36:40 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$2 retryRequest
情報: retry: Read timed out
2 26, 2014 11:36:41 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$2 retryRequest
情報: retryRequest(): exception=class java.net.SocketTimeoutException, executionCount=4, maxRetries=3
2 26, 2014 11:36:41 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$2 retryRequest
情報: give up: 4

References

1. HttpClient 4 Cookbook