Kohei Nozaki's blog

Multiple deployment use mode is implemented to jberetweb

2015-01-15T12:06:17+00:00

Now jberetweb can operate (start, stop, etc) distributed batches in multiple deployment. to enable multiple deployment use mode, suppress -DjobOperator.jndi and specify -DjobOperator.name=${facade-class-name} (e.g. JobOperatorFacade) in mvn option when you build it.

In multiple deployment use mode, "App Name" column will be added for grasp where is deployment of each job comprehensively. actions such as restart, stop will be executed through lookup of appropriate remote EJB interface in according to "App Name".

Also in Start Job window, "App Name" can be specified. this will lookup appropriate remote interface accordingly too.

Please refer README.md for more information.

Restarting a stopped job with jberetweb

2015-01-09T12:30:49+00:00

Assume that there is a job execution which is STOPPED on the Job Executions table like following. let's see a drop-down list at right end of the row. you can take some actions for each job executions with this "Action" drop-down list.

This drop-down list contains some actions like following (for STOPPED one).

There are 3 actions available. "Restart" means restarting this execution as a new execution but inherit job instance from previous one. "Abandon" means abandoning this execution as won't be used again. after abandoning, restarting will be impossible. "Re-execute as new instance" means opening "Start Job" window with inheriting both of job name and job parameters of selected instance. also "Stop" is available for running executions.

After selection of Restart, pop-up window will be appeared like following.

This window looks similar to "Start Job" window, but Job Name editing is prohibited because this is about to restarting of previous execution. you can edit Job Parameters as well as regular job execution but initial value of them will be inherited if they exists in previous execution.

How to make jberetweb to invoke BatchRuntime.getJobOperator() directly

2015-01-07T10:32:52+00:00

While I prefer to deploy both of jberetweb.war and your batch application archive independently and invoke JobOperator through remote EJB invocation, there is an another option which make jberetweb to invoke BatchRuntime.getJobOperator() directly instead of remote EJB lookup.

Just suppress mvn parameter "-DjobOperator.jndi" when you build your jberetweb WAR (please refer README.md), then jberetweb starts invoke BatchRuntime directly instead of using remote interface. of course, then you have to embed or merge your batch application archive and jberetweb into one WAR or EAR.

I don't recommend this way because this makes deployment time of your application longer due to JSF and JPA initializing. and you have to care about consistency for both of JSF and JPA configuration between your application and jberetweb. but it might be a solution where remote EJB invocation is not allowed by some security reasons (EJBAccessException might be occured when jberetweb tries to lookup remote JobOperator interface on such environment).

Index needed on jobexecutionid column in step_execution table

2015-01-05T12:56:03+00:00

Today I deployed new jberetweb to my production system which have 1 million rows in step_execution table, and felt jberetweb slow. I looked it and found the cause is sequential scan.

jbatch=# explain analyze select * from step_execution where jobexecutionid = 384846;
                                                    QUERY PLAN                                                    
------------------------------------------------------------------------------------------------------------------
 Seq Scan on step_execution  (cost=0.00..36817.53 rows=4 width=626) (actual time=101.046..101.047 rows=2 loops=1)
   Filter: (jobexecutionid = 384846)
   Rows Removed by Filter: 1102459
 Total runtime: 101.074 ms

So I created an index on jobexecutionid column in step_execution table, then jberetweb starts running fast.

jbatch=# create index step_execution_jobexecutionid_idx on step_execution (jobexecutionid);
CREATE INDEX
jbatch=# explain analyze select * from step_execution where jobexecutionid = 384846;
                                                                     QUERY PLAN                                                                     
----------------------------------------------------------------------------------------------------------------------------------------------------
 Index Scan using step_execution_jobexecutionid_idx on step_execution  (cost=0.43..8.50 rows=4 width=626) (actual time=0.054..0.056 rows=2 loops=1)
   Index Cond: (jobexecutionid = 384846)
 Total runtime: 0.113 ms

Maybe more indexes are needed for default JBeret schema if I implement some filtering function to jberetweb.

jberetweb TODOs

2015-01-04T13:37:56+00:00

Just for memo. I think this list could be moved to GitHub's issue tracker.

~~Operation support for various application archive / remote interface~~
Save preferences such as visible columns to cookie
Change some of Integer id columns to Long
Refreshing of execution detail part
Variable / closable table area
In-cell editing of a value of job parameter table
Add clear button to job parameter table
Improve pagination for larger data set
- Jump to specified execution id or page number directly
- Clickable page numbers (1 2 3 4 5...)
- Selection of number of rows in single page
Job schedule managing (like cron) with EJB programmatic timer
Job name completion (with JobOperator#getJobNames() ?)
Filtering / Sorting
- Show BatchStatus=FAILED only etc
- Sorting in ascending order
Covering omitted information
- job_execution.exitstatus, job_execution.restartposition etc
- Checkpointinfo would be useful. just pass it to String.valueOf() after deserialize
Testing against MySQL, Derby, H2, newer WildFly etc
Row highlighting/coloring for various statuses (STARTING/STARTED/STOPPING/STOPPED/ABANDONED). currently cares COMPLETED/FAILED only.
Refactoring (especially I think current CSS/HTML design is terrible due to my bad in client side web technology)
Use of REST API (would following spec of BatchEE be good?)

Job operation features are implemented to jberetweb

2015-01-04T12:26:41+00:00

Now jberetweb is able to operate JobOperator interface through remote EJB invocation, so finally jberetweb can start batch jobs. the Start Job window popups when "Start Job" anchor is clicked on the upper right corner. here's a screenshot:

To use this feature, some preparation are needed. first, you have to expose a remote EJB interface of javax.batch.operations.JobOperator from your batch application archive. an example of simplest one is available here. also entire of the project is here.

You can just put this class to any package in your batch application archive. after that, you can see some notification of JNDI names in your WildFly console at every deployment like this:

21:06:45,501 INFO  [org.jboss.as.ejb3.deployment.processors.EjbJndiBindingsDeploymentUnitProcessor] (MSC service thread 1-10) JNDI bindings for session bean named JobOperatorFacade in deployment unit deployment "jbatchtest-1.0-SNAPSHOT.war" are as follows:

java:global/jbatchtest-1.0-SNAPSHOT/JobOperatorFacade!javax.batch.operations.JobOperator
java:app/jbatchtest-1.0-SNAPSHOT/JobOperatorFacade!javax.batch.operations.JobOperator
java:module/JobOperatorFacade!javax.batch.operations.JobOperator
java:jboss/exported/jbatchtest-1.0-SNAPSHOT/JobOperatorFacade!javax.batch.operations.JobOperator
java:global/jbatchtest-1.0-SNAPSHOT/JobOperatorFacade
java:app/jbatchtest-1.0-SNAPSHOT/JobOperatorFacade
java:module/JobOperatorFacade

Then save first one of yours ("java:global/jbatchtest-1.0-SNAPSHOT/JobOperatorFacade!javax.batch.operations.JobOperator" for example), then put that string to a build parameter of jberetweb (after "-DjobOperator.jndi="). please refer "How to use" section of README.md for build instruction.

Other operations such as restart, stop, abandon are implemented too so I will write more about it later. jberetweb can be obtained from GitHub.

jberetweb, JBeret job repository viewer

2015-01-01T16:01:50+00:00

What are JBeret and jberetweb?

JBeret is the out-of-the-box JSR352 JBatch implementation of the WildFly application server. It manages its statuses of jobs and steps in several types of data storage, like RDBMS, which is called repository. But it has no management console or even a standardized way to see it efficiently (I've been watching the repository through SQL!). So, I have created jberetweb as a small web application which shows the JDBC job repository of JBeret.

jberetweb can be obtained at GitHub.

What can be done with jberetweb?

It shows recent job executions with their names, the start/end time and the batch status on the table in the upper half of the page. These rows are clickable. Additional information, like job parameters and step executions, will be shown when any row of the job executions table is clicked. Execution exception data will be shown if the step has failed because an exception occurred. Any problematic rows, such as a failed execution or step, are highlighted. Thanks to JSF's partial rendering and Ajax, paging and operations are fast.

How do I install it?

As I described at README.md in GitHub repository of it, you have to clone and build it with mvn yourself, and some configuration is needed for WildFly before deploying the WAR.

Create a database on PostgreSQL (jberetweb should run on any other RDBMS, but I haven't tested yet)
Register a XA data source for the job repository
Register JNDI name of the JDBC job repository
Set the job repository type as JDBC
Define JSF project stage to JNDI

Configuration procedures with jboss-cli should be like this:

batch
xa-data-source add \
      --name=JBatchDS \
      --driver-name=postgresql \
      --jndi-name=java:jboss/jdbc/JBatchDS \
      --user-name=postgres \
      --password=***
/subsystem=datasources/xa-data-source="JBatchDS"/xa-datasource-properties=ServerName:add(value="localhost")
/subsystem=datasources/xa-data-source="JBatchDS"/xa-datasource-properties=PortNumber:add(value="5432")
/subsystem=datasources/xa-data-source="JBatchDS"/xa-datasource-properties=DatabaseName:add(value="jbatch")
run-batch
/subsystem=batch/job-repository=jdbc:write-attribute(name=jndi-name, value=java:jboss/jdbc/JBatchDS)
/subsystem=batch:write-attribute(name=job-repository-type, value=jdbc)
/subsystem=naming/binding=java\:\/env\/jsf\/ProjectStage:add(binding-type=simple,value=Development,class=java.lang.String)

NOTE:

JBeret creates the schema automatically if any tables aren't found, so make sure the database user can execute DDLs.
Use XA datasource for both the job repository and your application database.

Here's some related pointers:

I would be grateful for your feedback because it's my first software which is public on GitHub.