Kohei Nozaki's blog 

Deploying with Jenkins Publish Over SSH Plugin

Posted on Friday Feb 28, 2014 at 09:28AM in Jenkins

Environment

  • WildFly 8.0.0.Final
  • Publish Over SSH Plugin 1.11
  • Jenkins 1.551
  • OS X 10.9.2

Requirements

  • Resources are available in git repository
  • The job is parametarized and can specify the tag to be processed
  • The job will build a WAR and deploy it to the remote application server through ssh

Install the plugin

  • Install “Publish Over SSH Plugin 1.11” at Plug-in page.

Create a key-pair

kyle-no-MacBook:~ jenkins$ whoami
jenkins
kyle-no-MacBook:~ jenkins$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/Users/Shared/Jenkins/.ssh/id_rsa): 
Created directory '/Users/Shared/Jenkins/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /Users/Shared/Jenkins/.ssh/id_rsa.
Your public key has been saved in /Users/Shared/Jenkins/.ssh/id_rsa.pub.
The key fingerprint is:
[...] 
The key's randomart image is:
[...]
kyle-no-MacBook:~ jenkins$ ls -l .ssh
total 16
-rw-------  1 jenkins  jenkins  1679 Feb 28 11:55 id_rsa
-rw-r--r--  1 jenkins  jenkins   411 Feb 28 11:55 id_rsa.pub
kyle-no-MacBook:~ jenkins$

Put the public-key to the server

kyle-osxserver:.ssh kyle$ cat >> authorized_keys << EOF
> ssh-rsa [...] jenkins@kyle-no-MacBook.local
> EOF

Configure

  1. Click “Manage Jenkins”
  2. Click “Configure System”
  3. Go to “Publish over SSH” section
  4. Enter “/Users/Shared/Jenkins/.ssh/id_rsa” to “Path to Key”
  5. Click “Add” at “SSH Servers”
  6. Enter any logical name to “Name”
  7. Enter IP Address or Hostname of the server to “Hostname”
  8. Enter the user name to login to “Username”
  9. Enter any directory to “Remote Directory”
  10. Click “Test Configuration”
  11. Click “Save” at bottom of the page

Create a job

  1. Create or copy a job that can build the WAR correctly.
  2. As I wrote in How to specify a Git tag to be processed, make a job to can specify a tag to be processed.
  3. Click “Add post-build action”
  4. Click “Send build artifacts over SSH”
  5. Enter “Source files”
  6. Enter “Remove prefix”
  7. Enter “Exec command”

WildFly deploy command example:

/Users/kyle/wildfly-8.0.0.Final/bin/jboss-cli.sh --connect --controller=localhost:49990 --command="deploy hoge-0.0.1-SNAPSHOT.war --force"

Create a tag

kyle-no-MacBook:stock kyle$ git tag v0.1
kyle-no-MacBook:stock kyle$ git tag
v0.1
kyle-no-MacBook:stock kyle$ git show v0.1
commit 87a93c8039bd77b8eb8cbf8fbb522705c6451f1e
[...]

Run

Run the job that created with the parameter of name of tag.

  1. Click “Build with Parameters”

  2. Select a tag to be processed and Click “Build”

Log

...
[JENKINS] Archiving /Users/Shared/Jenkins/Home/jobs/HogeDeploy/workspace/hoge/pom.xml to org.nailedtothex/hoge/0.0.1-SNAPSHOT/hoge-0.0.1-SNAPSHOT.pom
[JENKINS] Archiving /Users/Shared/Jenkins/Home/jobs/HogeDeploy/workspace/hoge/target/hoge-0.0.1-SNAPSHOT.war to org.nailedtothex/hoge/0.0.1-SNAPSHOT/hoge-0.0.1-SNAPSHOT.war
channel stopped
SSH: Connecting from host [kyle-no-MacBook.local]
SSH: Connecting with configuration [osxserver] ...
SSH: EXEC: STDOUT/STDERR from command [/Users/kyle/wildfly-8.0.0.Final/bin/jboss-cli.sh --connect --controller=localhost:49990 --command="deploy /Users/kyle/hoge-0.0.1-SNAPSHOT.war --force"] ...
SSH: EXEC: completed after 4,357 ms
SSH: Disconnecting configuration [osxserver] ...
SSH: Transferred 1 file(s)
Email was triggered for: Always
Sending email for trigger: Always
Sending email to: kyle@example.com
Finished: SUCCESS

References

  1. Publish Over - Jenkins - Jenkins Wiki
  2. jenkinsによるWebSphereへのEARファイルデプロイ - 遅れてやってきたプログラマーの小言
  3. Jenkins、Webから再起動する
  4. Git - タグ
  5. Deploy to WildFly using jboss-cli (Tech Tip #11) | Miles to go 2.0 … | Planet JBoss Community

Comments [1]

Tags: jenkins

Integration testing on an application server with Maven

Posted on Thursday Feb 27, 2014 at 11:50AM in Technology

For a example of integration testing, I will try to test a Stateless Session Bean through Remote Interface.

Environment

  • WildFly 8.0.0.Final
  • Apache Maven 3.1.1
  • Oracle JDK7u51

How does the test case work?

  1. Run Unit Testing JUnit classes
  2. Start the application server
  3. Deploy the application to the application server
  4. Run Integration Testing JUnit classes
    • Remote EJB invocation testing
  5. Undeploy the application
  6. Stop the application server

Resources of the sample project

  • Whole resources are available in GitHub

pom.xml

  • 2 profiles are declared.

production

  • It's just a skeleton. additional configuration are required here if you want to deploy to production environment using this pom.xml.

integration-test

  • We can fire the integration-test by command “mvn -P integration-test verify”
  • WildFly installation at “/Users/kyle/apps2/wildfly-8.0.0.Final” is assumed.
  • It has a declaration of port-offset 30000 because I'm already using 8080 or 9990 ports for development.
    • Consequently, this pom.xml fires WildFly at port 38080 and 39990.

Test target classes

Hoge.java

  • Just a POJO class.
  • Supposed to be tested at unit test phase.

HigeImpl.java

  • Stateless Session Bean class which implements Remote Interface.
  • Supposed to be tested at integration-test phase.

JUnit Test classes

HogeTest.java

  • Just a plain JUnit class.
  • Supposed to be excluded at integration-test phase.

HigeTestIT.java

  • There are some EJB lookup codes, and JNDI reference name declared here.
  • Supposed to be excluded at unit-test phase.

jndi.properties

  • Some JNDI remote lookup configurations here.
  • If you want to run this project at a application server except WildFly, then you need to edit this.
  • Port number 8080 is supposed to be used with the application server which integrated with IDE.
    • When kicked by Maven, it will be override by system property (related logics are available at HigeTestIT.java and pom.xml).
    • Consequently, both IDE and Maven can run the test case.

Log

kyle-no-MacBook:it kyle$ mvn -P integration-test verify
Picked up _JAVA_OPTIONS: -Dfile.encoding=UTF-8
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building it 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ it ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 0 resource
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ it ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ it ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ it ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.16:test (default-test) @ it ---
[INFO] Surefire report directory: /Users/kyle/Documents/workspace/it/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Picked up _JAVA_OPTIONS: -Dfile.encoding=UTF-8
Running org.nailedtothex.it.HogeTest
***THIS IS A UNIT TEST***
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec - in org.nailedtothex.it.HogeTest

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] --- maven-war-plugin:2.2:war (default-war) @ it ---
[INFO] Packaging webapp
[INFO] Assembling webapp [it] in [/Users/kyle/Documents/workspace/it/target/it-0.0.1-SNAPSHOT]
[INFO] Processing war project
[INFO] Copying webapp resources [/Users/kyle/Documents/workspace/it/src/main/webapp]
[INFO] Webapp assembled in [23 msecs]
[INFO] Building war: /Users/kyle/Documents/workspace/it/target/it-0.0.1-SNAPSHOT.war
[INFO] 
[INFO] --- wildfly-maven-plugin:1.0.1.Final:start (wildfly-run) @ it ---
[INFO] JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre
[INFO] JBOSS_HOME=/Users/kyle/apps2/wildfly-8.0.0.Final

[INFO] Server is starting up.
Picked up _JAVA_OPTIONS: -Dfile.encoding=UTF-8
2 27, 2014 4:07:33 午後 org.xnio.Xnio <clinit>
INFO: XNIO version 3.2.0.Final
2 27, 2014 4:07:34 午後 org.xnio.nio.NioXnio <clinit>
INFO: XNIO NIO Implementation Version 3.2.0.Final
2 27, 2014 4:07:34 午後 org.jboss.remoting3.EndpointImpl <clinit>
INFO: JBoss Remoting version 4.0.0.Final
16:07:34,277 INFO  [org.jboss.modules] (main) JBoss Modules version 1.3.0.Final
16:07:34,482 INFO  [org.jboss.msc] (main) JBoss MSC version 1.2.0.Final
16:07:34,542 INFO  [org.jboss.as] (MSC service thread 1-6) JBAS015899: WildFly 8.0.0.Final "WildFly" starting
16:07:35,326 INFO  [org.jboss.as.server] (Controller Boot Thread) JBAS015888: Creating http management service using socket-binding (management-http)
16:07:35,340 INFO  [org.xnio] (MSC service thread 1-15) XNIO version 3.2.0.Final
16:07:35,346 INFO  [org.xnio.nio] (MSC service thread 1-15) XNIO NIO Implementation Version 3.2.0.Final
16:07:35,366 INFO  [org.jboss.as.security] (ServerService Thread Pool -- 46) JBAS013171: Activating Security Subsystem
16:07:35,369 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 33) JBAS010280: Activating Infinispan subsystem.
16:07:35,378 INFO  [org.jboss.as.naming] (ServerService Thread Pool -- 41) JBAS011800: Activating Naming Subsystem
16:07:35,382 INFO  [org.jboss.as.security] (MSC service thread 1-16) JBAS013170: Current PicketBox version=4.0.20.Final
16:07:35,383 INFO  [org.jboss.as.webservices] (ServerService Thread Pool -- 50) JBAS015537: Activating WebServices Extension
16:07:35,384 INFO  [org.jboss.as.jsf] (ServerService Thread Pool -- 39) JBAS012615: Activated the following JSF Implementations: [main]
16:07:35,409 INFO  [org.wildfly.extension.undertow] (ServerService Thread Pool -- 49) JBAS017502: Undertow 1.0.0.Final starting
16:07:35,409 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-6) JBAS017502: Undertow 1.0.0.Final starting
16:07:35,419 INFO  [org.jboss.remoting] (MSC service thread 1-15) JBoss Remoting version 4.0.0.Final
16:07:35,420 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 28) JBAS010403: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3)
16:07:35,430 INFO  [org.jboss.as.connector.logging] (MSC service thread 1-12) JBAS010408: Starting JCA Subsystem (IronJacamar 1.1.3.Final)
16:07:35,436 INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-14) JBAS010417: Started Driver service with driver-name = h2
16:07:35,449 INFO  [org.jboss.as.naming] (MSC service thread 1-8) JBAS011802: Starting Naming Service
16:07:35,449 INFO  [org.jboss.as.mail.extension] (MSC service thread 1-12) JBAS015400: Bound mail session [java:jboss/mail/Default]
16:07:35,498 INFO  [org.wildfly.extension.undertow] (ServerService Thread Pool -- 49) JBAS017527: Creating file handler for path /Users/kyle/apps2/wildfly-8.0.0.Final/welcome-content
16:07:35,518 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-4) JBAS017525: Started server default-server.
16:07:35,532 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-4) JBAS017531: Host default-host starting
16:07:35,588 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-9) JBAS017519: Undertow HTTP listener default listening on /127.0.0.1:38080
16:07:35,786 INFO  [org.jboss.as.server.deployment.scanner] (MSC service thread 1-5) JBAS015012: Started FileSystemDeploymentService for directory /Users/kyle/apps2/wildfly-8.0.0.Final/standalone/deployments
16:07:35,795 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-11) JBAS010400: Bound data source [java:jboss/datasources/ExampleDS]
16:07:36,025 INFO  [org.jboss.ws.common.management] (MSC service thread 1-10) JBWS022052: Starting JBoss Web Services - Stack CXF Server 4.2.3.Final
16:07:36,065 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015961: Http management interface listening on http://127.0.0.1:39990/management
16:07:36,066 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015951: Admin console listening on http://127.0.0.1:39990
16:07:36,066 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: WildFly 8.0.0.Final "WildFly" started in 2067ms - Started 183 of 232 services (80 services are lazy, passive or on-demand)
[INFO] 
[INFO] >>> wildfly-maven-plugin:1.0.1.Final:deploy (wildfly-run) @ it >>>
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ it ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 0 resource
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ it ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ it ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ it ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.16:test (default-test) @ it ---
[INFO] Skipping execution of surefire because it has already been run for this configuration
[INFO] 
[INFO] --- maven-war-plugin:2.2:war (default-war) @ it ---
[INFO] Packaging webapp
[INFO] Assembling webapp [it] in [/Users/kyle/Documents/workspace/it/target/it-0.0.1-SNAPSHOT]
[INFO] Processing war project
[INFO] Copying webapp resources [/Users/kyle/Documents/workspace/it/src/main/webapp]
[INFO] Webapp assembled in [5 msecs]
[INFO] Building war: /Users/kyle/Documents/workspace/it/target/it-0.0.1-SNAPSHOT.war
[INFO] 
[INFO] <<< wildfly-maven-plugin:1.0.1.Final:deploy (wildfly-run) @ it <<<
[INFO] 
[INFO] --- wildfly-maven-plugin:1.0.1.Final:deploy (wildfly-run) @ it ---
16:07:39,415 INFO  [org.jboss.as.repository] (management-handler-thread - 3) JBAS014900: Content added at location /Users/kyle/apps2/wildfly-8.0.0.Final/standalone/data/content/63/45e9720d69aeb3604f94f608a25f9d036229a2/content
16:07:39,427 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-6) JBAS015876: Starting deployment of "it.war" (runtime-name: "it.war")
16:07:39,564 INFO  [org.jboss.weld.deployer] (MSC service thread 1-15) JBAS016002: Processing weld deployment it.war
16:07:39,604 INFO  [org.hibernate.validator.internal.util.Version] (MSC service thread 1-15) HV000001: Hibernate Validator 5.0.3.Final
16:07:39,654 INFO  [org.jboss.as.ejb3.deployment.processors.EjbJndiBindingsDeploymentUnitProcessor] (MSC service thread 1-15) JNDI bindings for session bean named HigeImpl in deployment unit deployment "it.war" are as follows:

    java:global/it/HigeImpl!org.nailedtothex.it.Hige
    java:app/it/HigeImpl!org.nailedtothex.it.Hige
    java:module/HigeImpl!org.nailedtothex.it.Hige
    java:jboss/exported/it/HigeImpl!org.nailedtothex.it.Hige
    java:global/it/HigeImpl
    java:app/it/HigeImpl
    java:module/HigeImpl

16:07:39,741 INFO  [org.jboss.weld.deployer] (MSC service thread 1-11) JBAS016005: Starting Services for CDI deployment: it.war
16:07:39,762 INFO  [org.jboss.weld.Version] (MSC service thread 1-11) WELD-000900: 2.1.2 (Final)
16:07:39,788 INFO  [org.jboss.weld.deployer] (MSC service thread 1-9) JBAS016008: Starting weld service for deployment it.war
16:07:40,560 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-11) JBAS017534: Registered web context: /it
16:07:40,596 INFO  [org.jboss.as.server] (management-handler-thread - 3) JBAS018559: Deployed "it.war" (runtime-name : "it.war")
[INFO] 
[INFO] --- maven-failsafe-plugin:2.16:integration-test (default) @ it ---
[INFO] Failsafe report directory: /Users/kyle/Documents/workspace/it/target/failsafe-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Picked up _JAVA_OPTIONS: -Dfile.encoding=UTF-8
Running org.nailedtothex.it.HigeTestIT
2 27, 2014 4:07:41 午後 org.xnio.Xnio <clinit>
INFO: XNIO version 3.2.0.Final
2 27, 2014 4:07:41 午後 org.xnio.nio.NioXnio <clinit>
INFO: XNIO NIO Implementation Version 3.2.0.Final
2 27, 2014 4:07:41 午後 org.jboss.remoting3.EndpointImpl <clinit>
INFO: JBoss Remoting version 4.0.0.Final
2 27, 2014 4:07:41 午後 org.jboss.ejb.client.remoting.VersionReceiver handleMessage
INFO: EJBCLIENT000017: Received server version 2 and marshalling strategies [river]
2 27, 2014 4:07:41 午後 org.jboss.ejb.client.remoting.RemotingConnectionEJBReceiver associate
INFO: EJBCLIENT000013: Successful version handshake completed for receiver context EJBReceiverContext{clientContext=org.jboss.ejb.client.EJBClientContext@6d3a3c8e, receiver=Remoting connection EJB receiver [connection=org.jboss.ejb.client.remoting.ConnectionPool$PooledConnection@a5dc6a8,channel=jboss.ejb,nodename=kyle-no-macbook]} on channel Channel ID b8296d49 (outbound) of Remoting connection 09445fbe to localhost/127.0.0.1:38080
2 27, 2014 4:07:41 午後 org.jboss.ejb.client.EJBClient <clinit>
INFO: JBoss EJB Client version 2.0.0.Final
***THIS IS A INTEGRATION TEST***
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.574 sec - in org.nailedtothex.it.HigeTestIT

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] --- wildfly-maven-plugin:1.0.1.Final:undeploy (wildfly-stop) @ it ---
16:07:41,535 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-14) JBAS017535: Unregistered web context: /it
16:07:41,547 INFO  [org.jboss.weld.deployer] (MSC service thread 1-10) JBAS016009: Stopping weld service for deployment it.war
16:07:41,562 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-1) JBAS015877: Stopped deployment it.war (runtime-name: it.war) in 30ms
16:07:41,578 INFO  [org.jboss.as.repository] (management-handler-thread - 4) JBAS014901: Content removed from location /Users/kyle/apps2/wildfly-8.0.0.Final/standalone/data/content/63/45e9720d69aeb3604f94f608a25f9d036229a2/content
16:07:41,578 INFO  [org.jboss.as.server] (management-handler-thread - 4) JBAS018558: Undeployed "it.war" (runtime-name: "it.war")
[INFO] 
[INFO] --- wildfly-maven-plugin:1.0.1.Final:shutdown (wildfly-stop) @ it ---
16:07:41,603 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-16) JBAS017532: Host default-host stopping
16:07:41,605 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-12) JBAS010409: Unbound data source [java:jboss/datasources/ExampleDS]
16:07:41,611 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-10) JBAS017521: Undertow HTTP listener default suspending
16:07:41,612 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-10) JBAS017520: Undertow HTTP listener default stopped, was bound to /127.0.0.1:38080
16:07:41,616 INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-2) JBAS010418: Stopped Driver service with driver-name = h2
16:07:41,617 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-6) JBAS017506: Undertow 1.0.0.Final stopping
16:07:41,620 INFO  [org.jboss.as] (MSC service thread 1-7) JBAS015950: WildFly 8.0.0.Final "WildFly" stopped in 10ms
[INFO] 
[INFO] --- maven-failsafe-plugin:2.16:verify (default) @ it ---
[INFO] Failsafe report directory: /Users/kyle/Documents/workspace/it/target/failsafe-reports
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10.129s
[INFO] Finished at: Thu Feb 27 16:07:42 JST 2014
[INFO] Final Memory: 13M/245M
[INFO] ------------------------------------------------------------------------
kyle-no-MacBook:it kyle$

References

  1. Maven Failsafe Plugin - Introduction
  2. JUnitのCategoryさんとMavenのintegration-testでの実行 - 日々常々

Comments [0]

Tags: test

Job testing with remote EJB invocation

Posted on Wednesday Feb 26, 2014 at 04:50PM in Technology

I introduced that a way to test a JSR352 job with Arquillian, but I guess it's might too complex for testing of a JSR352 job, so I'm going to try to test them without Arquillian.

Environment

  • WildFly 8.0.0.Final
  • Oracle JDK7u51

Sample project

How does it work?

  1. JUnit test class lookups javax.batch.operations.JobOperator instance from Remote WildFly through Remote EJB Interface.
  2. Test class kicks the job.
  3. Test class will wait till the job finished.
  4. Assert BatchStatus.

How do I run on other application servers?

Log of WildFly

17:06:18,214 INFO  [stdout] (batch-batch - 3) hello

Local log

2 26, 2014 5:14:39 午後 org.xnio.Xnio <clinit>
INFO: XNIO version 3.2.0.Final
2 26, 2014 5:14:40 午後 org.xnio.nio.NioXnio <clinit>
INFO: XNIO NIO Implementation Version 3.2.0.Final
2 26, 2014 5:14:40 午後 org.jboss.remoting3.EndpointImpl <clinit>
INFO: JBoss Remoting version 4.0.0.Final
2 26, 2014 5:14:40 午後 org.jboss.ejb.client.remoting.VersionReceiver handleMessage
INFO: EJBCLIENT000017: Received server version 2 and marshalling strategies [river]
2 26, 2014 5:14:40 午後 org.jboss.ejb.client.remoting.RemotingConnectionEJBReceiver associate
INFO: EJBCLIENT000013: Successful version handshake completed for receiver context EJBReceiverContext{clientContext=org.jboss.ejb.client.EJBClientContext@2db9e6d7, receiver=Remoting connection EJB receiver [connection=org.jboss.ejb.client.remoting.ConnectionPool$PooledConnection@7e244b5,channel=jboss.ejb,nodename=kyle-no-macbook]} on channel Channel ID c584a9f3 (outbound) of Remoting connection 09168b43 to localhost/127.0.0.1:8080
2 26, 2014 5:14:40 午後 org.jboss.ejb.client.EJBClient <clinit>
INFO: JBoss EJB Client version 2.0.0.Final

Remarks

  • It's much faster than Arquillian Remoting on my environment.

Comments [0]

Tags: jbatch

Manage application config with Commons Configuration

Posted on Wednesday Feb 26, 2014 at 01:53PM in Technology

Environment

  • Commons Configuration 1.10
  • WildFly 8.0.0.Final
  • Oracle JDK7u51

Why go this way?

Various portable ways are available for manage configurations of the Java EE application. such as:

  • Configuration table in DB
    • I guess it's not bad, but I feel that is bit tedious work to coding.
    • There's a way to use this mechanism via Commons Configuration, but I haven't tried it yet.
  • Custom JNDI resources
    • I thought that it's most efficient at first, but negative points here:
      • Badness of lookup performance.
      • Editing JNDI resources is very annoying.

So sometimes I'm thinking of any better way, and I found Commons Configurations[1]. notable capabilities of Commons Configuration are:

  • Simple API
  • Auto-reloading and saving mechanism provided
  • Support of various formats and placement styles

So I'd try it.

Prepare sample project and related resources

Create a system property

  • Create a system property which specifies the path of configuration file.
  • I'm using WildFly so go with jboss-cli:
[standalone@localhost:9990 /] /system-property=app.config:add(value=/Users/kyle/app.properties)
{"outcome" => "success"}
[standalone@localhost:9990 /]

Create the properties file

  • Location is same as “app.config” system property.
hoge=fuge

pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.nailedtothex</groupId>
    <artifactId>config</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>war</packaging>

    <properties>
        <maven.compiler.source>1.7</maven.compiler.source>
        <maven.compiler.target>1.7</maven.compiler.target>
        <failOnMissingWebXml>false</failOnMissingWebXml>
    </properties>

    <dependencies>
        <dependency>
            <groupId>javax</groupId>
            <artifactId>javaee-api</artifactId>
            <version>7.0</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>commons-configuration</groupId>
            <artifactId>commons-configuration</artifactId>
            <version>1.10</version>
        </dependency>
    </dependencies>
</project>

Create an application-scoped bean

package org.nailedtothex.config;

import javax.annotation.PostConstruct;
import javax.enterprise.context.ApplicationScoped;
import javax.inject.Named;

import org.apache.commons.configuration.Configuration;
import org.apache.commons.configuration.ConfigurationException;
import org.apache.commons.configuration.PropertiesConfiguration;
import org.apache.commons.configuration.reloading.FileChangedReloadingStrategy;

@Named
@ApplicationScoped
public class AppConfigBean {

    private Configuration config;

    @PostConstruct
    void init() throws ConfigurationException {
        String path = System.getProperty("app.config");
        PropertiesConfiguration propConfig = new PropertiesConfiguration(path); 
        propConfig.setReloadingStrategy(new FileChangedReloadingStrategy());
        config = propConfig;
    }

    public Configuration getConfig(){
        return config;
    }
}
  • If you want to read the config at startup of application, I recommend that use of @javax.ejb.Singleton and @javax.ejb.Startup[2].

Create a servlet

  • Get a configuration variable from singleton bean.
package org.nailedtothex.config;

import java.io.IOException;
import java.io.PrintWriter;

import javax.inject.Inject;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

@WebServlet(urlPatterns = "/*")
public class HelloServlet extends HttpServlet {
    private static final long serialVersionUID = 1L;
    @Inject
    private AppConfigBean bean;

    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        try (PrintWriter pw = response.getWriter()) {
            pw.write(bean.getConfig().getString("hoge"));
        }
    }
}

Access via browser

  • It needs only very small and clean code.

Check runtime reloading

Edit app.properties:

kyle-no-MacBook:~ kyle$ echo hoge=FUGE > app.properties
kyle-no-MacBook:~ kyle$

Reload browser:

  • I just reloaded the browser and I didn't do any other operations, but config was updated immediately.

Collaboration with CDI

  • There's a way to inject properties to CDI managed bean at [2]. that must be useful.

XMLConfiguration

  • XMLConfiguration is good for hierarchical configuration.
  • It depended on Commons Collection so don't forget to add the dependency.

pom.xml

        <dependency>
            <groupId>commons-configuration</groupId>
            <artifactId>commons-configuration</artifactId>
            <version>1.10</version>
        </dependency>
        <dependency>
            <groupId>commons-collections</groupId>
            <artifactId>commons-collections</artifactId>
            <version>3.2.1</version>
        </dependency>

XmlTest.java

package org.nailedtothex.config;

import java.util.Iterator;
import java.util.List;

import org.apache.commons.configuration.XMLConfiguration;

public class XmlTest {

    public static void main(String[] args) throws Exception {
        XMLConfiguration config = new XMLConfiguration();
        config.load(XmlTest.class.getResourceAsStream("/config.xml"));

        String backColor = config.getString("colors.background");
        String textColor = config.getString("colors.text");
        String linkNormal = config.getString("colors.link[@normal]");
        String defColor = config.getString("colors.default");
        int rowsPerPage = config.getInt("rowsPerPage");
        List<Object> buttons = config.getList("buttons.name");

        System.out.println(backColor);
        System.out.println(textColor);
        System.out.println(linkNormal);
        System.out.println(defColor);
        System.out.println(rowsPerPage);
        System.out.println(buttons);

        System.out.println();

        Object o;
        Iterator it = config.getKeys("colors.");
        while(it.hasNext()){
            System.out.println(it.next());
        }
    }
}

config.xml

<?xml version="1.0" encoding="UTF-8"?>
<gui-definition>
  <colors>
    <background>#808080</background>
    <text>#000000</text>
    <header>#008000</header>
    <link normal="#000080" visited="#800080"/>
    <default>${colors.header}</default>
  </colors>
  <rowsPerPage>15</rowsPerPage>
  <buttons>
    <name>OK,Cancel,Help</name>
  </buttons>
  <numberFormat pattern="###\,###.##"/>
</gui-definition>

Log

#808080
#000000
#000080
#008000
15
[OK, Cancel, Help]

colors.background
colors.text
colors.header
colors.link[@normal]
colors.link[@visited]
colors.default

References

  1. Commons Configuration - Java Configuration API
  2. CDIでアプリケーション設定をインジェクション - 見習いプログラミング日記
  3. Apache commonsが便利な件（commons-configuration編） - 都元ダイスケ IT-PRESS
  4. Commons Configuration - Hierarchical configurations and XML Howto

Comments [0]

Tags: javaee

Mocking a unstable HTTP server with WireMock

Posted on Wednesday Feb 26, 2014 at 10:12AM in Technology

Environment

  • WireMock 1.4.3
  • HttpClient 4.3.2
  • Oracle JDK7u51

Why need it?

  • Sometimes website scraping stops at unexpected error like socket timeout or internal server error.
  • We prefer to retry several times at such occasion.
  • So we have to develop some retrying mechanism, and we have to create a test and create the mock of server.
    • So in this article, I'm going to try to create the mock with WireMock.

Requirements for client

  • Retry if processing fails, for 3 times.
  • Retry when timeout occurred.
    • SocketTimeoutException
  • Retry when server returned status code 5xx

Test cases

  1. Server returns code 200 without any problems.
  2. Server returns code 500 at first, then client will retry, server returns code 200.
  3. Server returns code 500 forever, then client will retry 3 times and give up.
  4. Server delays the response, then client will regard as timeout and retry, then server returns code 200.
  5. Server delays the response forever, then client will retry 3 times and give up.

Resources

pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.nailedtothex</groupId>
    <artifactId>wiremock</artifactId>
    <version>0.0.1-SNAPSHOT</version>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.7</maven.compiler.source>
        <maven.compiler.target>1.7</maven.compiler.target>
    </properties>

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.11</version>
            <scope>test</scope>
        </dependency>
        <dependency>
          <groupId>org.hamcrest</groupId>
          <artifactId>hamcrest-core</artifactId>
          <version>1.3</version>
          <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.github.tomakehurst</groupId>
            <artifactId>wiremock</artifactId>
            <version>1.43</version>
            <classifier>standalone</classifier>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>*</groupId>
                    <artifactId>*</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
            <version>4.3.2</version>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>fluent-hc</artifactId>
            <version>4.3.2</version>
        </dependency>
    </dependencies>

</project>

RetryableHttpFetcher.java

package org.nailedtothex.wiremock;

import java.io.IOException;
import java.util.logging.Level;
import java.util.logging.Logger;

import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpRequestRetryHandler;
import org.apache.http.client.HttpResponseException;
import org.apache.http.client.ServiceUnavailableRetryStrategy;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.protocol.HttpContext;
import org.apache.http.util.EntityUtils;

public class RetryableHttpFetcher {

    private static final Logger log = Logger.getLogger(RetryableHttpFetcher.class.getName());

    // these parameters would be better to retrieve through JNDI or any other mechanism
    private int MAX_RETRIES = 3;
    private int RETRY_INTERVAL = 1000;
    private int TIMEOUT = 500;

    private final ServiceUnavailableRetryStrategy MY_SERVICE_UNAVAILABLE_RETRY_STRATEGY = new ServiceUnavailableRetryStrategy() {
        @Override
        public boolean retryRequest(HttpResponse response, int executionCount,
                org.apache.http.protocol.HttpContext context) {

            boolean rc = response.getStatusLine().getStatusCode() >= 500 && executionCount <= MAX_RETRIES;

            log.log(Level.INFO,
                    "retryRequest(): returning={0}, statusCode={1}, executionCount={2}, maxRetries={3}, interval={4}",
                    new Object[] { rc, response.getStatusLine().getStatusCode(), executionCount, MAX_RETRIES,
                            RETRY_INTERVAL });

            return rc;
        }

        @Override
        public long getRetryInterval() {
            return RETRY_INTERVAL;
        }
    };

    private final HttpRequestRetryHandler MY_HTTP_REQUEST_RETRY_HANDLER = new HttpRequestRetryHandler() {

        @Override
        public boolean retryRequest(IOException e, int executionCount, HttpContext context) {
            log.log(Level.INFO, "retryRequest(): exception={0}, executionCount={1}, maxRetries={2}",
                    new Object[] { e.getClass(), executionCount, MAX_RETRIES });

            if (executionCount > MAX_RETRIES) {
                log.log(Level.INFO, "give up: {0}", executionCount);
                return false;
            }

            if (e instanceof java.net.SocketTimeoutException) {
                log.log(Level.INFO, "retry: {0}", e.getMessage());
                return true;
            }

            log.log(Level.INFO, "not retry: {0}", e.getMessage());
            return false;
        }
    };

    private final RequestConfig MY_REQUEST_CONFIG = RequestConfig.custom()
            .setConnectionRequestTimeout(TIMEOUT)
            .setConnectTimeout(TIMEOUT)
            .setSocketTimeout(TIMEOUT)
            .build();

    public String fetchAsString(String url) throws ClientProtocolException, IOException {

        try (CloseableHttpClient client = HttpClientBuilder.create()
                .setDefaultRequestConfig(MY_REQUEST_CONFIG)
                .setRetryHandler(MY_HTTP_REQUEST_RETRY_HANDLER)
                .setServiceUnavailableRetryStrategy(MY_SERVICE_UNAVAILABLE_RETRY_STRATEGY)
                .build()) {

            try (CloseableHttpResponse res = client.execute(new HttpGet(url))) {
                if (res.getStatusLine().getStatusCode() >= 400) {
                    throw new HttpResponseException(res.getStatusLine().getStatusCode(), res.getStatusLine()
                            .getReasonPhrase());
                }
                return EntityUtils.toString(res.getEntity());
            }
        }

    }
}

RetryableHttpFetcherTest.java

package org.nailedtothex.wiremock;

import static com.github.tomakehurst.wiremock.client.WireMock.*;
import static org.hamcrest.CoreMatchers.*;
import static org.junit.Assert.*;

import java.net.SocketTimeoutException;

import org.apache.http.client.HttpResponseException;
import org.junit.Before;
import org.junit.Rule;
import org.junit.Test;

import com.github.tomakehurst.wiremock.junit.WireMockRule;
import com.github.tomakehurst.wiremock.stubbing.Scenario;

public class RetryableHttpFetcherTest {

    @Rule
    public WireMockRule wireMockRule = new WireMockRule(18089);

    private RetryableHttpFetcher instance;

    @Before
    public void init() {
        instance = new RetryableHttpFetcher();
    }

    @Test
    public void test1_ok() throws Exception {
        stubFor(get(urlEqualTo("/hoge.txt")).willReturn(
                aResponse().withStatus(200).withHeader("Content-Type", "text/plain").withBody("OK")));
        String expected = "OK";

        String actual = instance.fetchAsString("http://localhost:18089/hoge.txt");

        assertThat(actual, is(expected));
    }

    @Test
    public void test2_retryAt500() throws Exception {
        stubFor(get(urlEqualTo("/500")).inScenario("retry at 500")
                .whenScenarioStateIs(Scenario.STARTED)
                .willSetStateTo("one time requested")
                .willReturn(aResponse().withBody("error").withStatus(500)));

        stubFor(get(urlEqualTo("/500")).inScenario("retry at 500")
                .whenScenarioStateIs("one time requested")
                .willReturn(aResponse().withBody("OK").withStatus(200)));

        String actual = instance.fetchAsString("http://localhost:18089/500");

        assertThat(actual, is("OK"));
    }

    @Test(expected = HttpResponseException.class)
    public void test3_retryAt500GiveUp() throws Exception {
        stubFor(get(urlEqualTo("/500"))
                .willReturn(aResponse().withBody("500").withStatus(500)));

        instance.fetchAsString("http://localhost:18089/500");
    }

    @Test
    public void test4_retryAtTimeout() throws Exception {
        stubFor(get(urlEqualTo("/timeout")).inScenario("retrying")
                .whenScenarioStateIs(Scenario.STARTED)
                .willSetStateTo("one time requested")
                .willReturn(aResponse().withBody("error").withStatus(500).withFixedDelay(3000)));
        stubFor(get(urlEqualTo("/timeout")).inScenario("retrying")
                .whenScenarioStateIs("one time requested")
                .willReturn(aResponse().withBody("OK").withStatus(200)));

        String actual = instance.fetchAsString("http://localhost:18089/timeout");
        assertThat(actual, is("OK"));
    }

    @Test(expected = SocketTimeoutException.class)
    public void test5_retryAtTimeoutGiveUp() throws Exception {
        stubFor(get(urlEqualTo("/timeout"))
                .willReturn(aResponse().withBody("timeout").withStatus(500).withFixedDelay(Integer.MAX_VALUE)));
        instance.fetchAsString("http://localhost:18089/timeout");
    }

    @Test(expected = HttpResponseException.class)
    public void notFound() throws Exception {
        instance.fetchAsString("http://localhost:18089/NOT_FOUND");
    }
}

Test logs

  • All tests were passed.

test1_ok

2 26, 2014 11:34:22 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$1 retryRequest
情報: retryRequest(): returning=false, statusCode=200, executionCount=1, maxRetries=3, interval=1,000

test2_retryAt500

2 26, 2014 11:35:32 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$1 retryRequest
情報: retryRequest(): returning=true, statusCode=500, executionCount=1, maxRetries=3, interval=1,000
2 26, 2014 11:35:33 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$1 retryRequest
情報: retryRequest(): returning=false, statusCode=200, executionCount=2, maxRetries=3, interval=1,000

test3_retryAt500GiveUp

2 26, 2014 11:35:51 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$1 retryRequest
情報: retryRequest(): returning=true, statusCode=500, executionCount=1, maxRetries=3, interval=1,000
2 26, 2014 11:35:52 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$1 retryRequest
情報: retryRequest(): returning=true, statusCode=500, executionCount=2, maxRetries=3, interval=1,000
2 26, 2014 11:35:53 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$1 retryRequest
情報: retryRequest(): returning=true, statusCode=500, executionCount=3, maxRetries=3, interval=1,000
2 26, 2014 11:35:54 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$1 retryRequest
情報: retryRequest(): returning=false, statusCode=500, executionCount=4, maxRetries=3, interval=1,000

test4_retryAtTimeout

2 26, 2014 11:36:12 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$2 retryRequest
情報: retryRequest(): exception=class java.net.SocketTimeoutException, executionCount=1, maxRetries=3
2 26, 2014 11:36:12 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$2 retryRequest
情報: retry: Read timed out
2 26, 2014 11:36:12 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$1 retryRequest
情報: retryRequest(): returning=false, statusCode=200, executionCount=1, maxRetries=3, interval=1,000

test5_retryAtTimeoutGiveUp

2 26, 2014 11:36:39 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$2 retryRequest
情報: retryRequest(): exception=class java.net.SocketTimeoutException, executionCount=1, maxRetries=3
2 26, 2014 11:36:39 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$2 retryRequest
情報: retry: Read timed out
2 26, 2014 11:36:40 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$2 retryRequest
情報: retryRequest(): exception=class java.net.SocketTimeoutException, executionCount=2, maxRetries=3
2 26, 2014 11:36:40 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$2 retryRequest
情報: retry: Read timed out
2 26, 2014 11:36:40 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$2 retryRequest
情報: retryRequest(): exception=class java.net.SocketTimeoutException, executionCount=3, maxRetries=3
2 26, 2014 11:36:40 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$2 retryRequest
情報: retry: Read timed out
2 26, 2014 11:36:41 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$2 retryRequest
情報: retryRequest(): exception=class java.net.SocketTimeoutException, executionCount=4, maxRetries=3
2 26, 2014 11:36:41 午前 org.nailedtothex.wiremock.RetryableHttpFetcher$2 retryRequest
情報: give up: 4

References

  1. HttpClient 4 Cookbook

Comments [2]

Tags: test

About Me

Java backend engineer obsessed with test driven development. Apache Roller committer. kohei at apache dot org

Archives

« February 2014 »
SunMonTueWedThuFriSat
      
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
 
       
Today

Hot tags

Here are the most frequently used tags in the blog. Hover over a tag to see the number of posts, click a tag to see the most recent posts with the tag.

Recent entries

Recent comments

Twitter