WildFly8でPostgreSQLをジョブレポジトリにしてみる
TweetPosted on Thursday Feb 13, 2014 at 07:26AM in Technology
WildFly8でJDBCジョブレポジトリを使ってみるでH2を使ってやってみたけど、PostgreSQLでやってみると、XAデータソースとかではまったのでメモしておく
環境
- WildFly8.0.0.Final
- postgresql-9.3-1100.jdbc41.jar
- Oracle JDK7u51
- PostgreSQL 9.2.4
- OS X 10.9.1
postgresql.confを編集する
max_prepared_transactionsを増やす
デフォルトだと、バッチ実行時にこういう例外が出て死ぬ。
10:56:53,218 WARN [com.arjuna.ats.jta] (batch-batch - 1) ARJUNA016041: prepare on < formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff0a00010b:-79718ed6:52fc262f:17, node_name=1, branch_uid=0:ffff0a00010b:-79718ed6:52fc262f:1b, subordinatenodename=null, eis_name=java:jboss/jdbc/JBatchDS > (XAResourceWrapperImpl@4a5bf320[xaResource=org.jboss.jca.adapters.jdbc.xa.XAManagedConnection@3c38e6e8 pad=false overrideRmValue=null productName=PostgreSQL productVersion=9.2.4 jndiName=java:jboss/jdbc/JBatchDS]) failed with exception XAException.XAER_RMERR: org.postgresql.xa.PGXAException: トランザクションの準備エラー
at org.postgresql.xa.PGXAConnection.prepare(PGXAConnection.java:313)
at org.jboss.jca.adapters.jdbc.xa.XAManagedConnection.prepare(XAManagedConnection.java:330)
at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.prepare(XAResourceWrapperImpl.java:169)
at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelPrepare(XAResourceRecord.java:210) [narayana-jts-jacorb-5.0.0.Final.jar:5.0.0.Final (revision: 9aa71)]
at com.arjuna.ats.arjuna.coordinator.BasicAction.doPrepare(BasicAction.java:2586) [narayana-jts-jacorb-5.0.0.Final.jar:5.0.0.Final (revision: 9aa71)]
at com.arjuna.ats.arjuna.coordinator.BasicAction.doPrepare(BasicAction.java:2536) [narayana-jts-jacorb-5.0.0.Final.jar:5.0.0.Final (revision: 9aa71)]
at com.arjuna.ats.arjuna.coordinator.BasicAction.prepare(BasicAction.java:2097) [narayana-jts-jacorb-5.0.0.Final.jar:5.0.0.Final (revision: 9aa71)]
at com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1481) [narayana-jts-jacorb-5.0.0.Final.jar:5.0.0.Final (revision: 9aa71)]
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:96) [narayana-jts-jacorb-5.0.0.Final.jar:5.0.0.Final (revision: 9aa71)]
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) [narayana-jts-jacorb-5.0.0.Final.jar:5.0.0.Final (revision: 9aa71)]
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1166) [narayana-jts-jacorb-5.0.0.Final.jar:5.0.0.Final (revision: 9aa71)]
at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [narayana-jts-jacorb-5.0.0.Final.jar:5.0.0.Final (revision: 9aa71)]
at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75)
at org.jberet.runtime.runner.ChunkRunner.readProcessWriteItems(ChunkRunner.java:323) [jberet-core-1.0.0.Final.jar:1.0.0.Final]
at org.jberet.runtime.runner.ChunkRunner.run(ChunkRunner.java:193) [jberet-core-1.0.0.Final.jar:1.0.0.Final]
at org.jberet.runtime.runner.StepExecutionRunner.runBatchletOrChunk(StepExecutionRunner.java:204) [jberet-core-1.0.0.Final.jar:1.0.0.Final]
at org.jberet.runtime.runner.StepExecutionRunner.run(StepExecutionRunner.java:131) [jberet-core-1.0.0.Final.jar:1.0.0.Final]
at org.jberet.runtime.runner.CompositeExecutionRunner.runStep(CompositeExecutionRunner.java:162) [jberet-core-1.0.0.Final.jar:1.0.0.Final]
at org.jberet.runtime.runner.CompositeExecutionRunner.runFromHeadOrRestartPoint(CompositeExecutionRunner.java:88) [jberet-core-1.0.0.Final.jar:1.0.0.Final]
at org.jberet.runtime.runner.JobExecutionRunner.run(JobExecutionRunner.java:58) [jberet-core-1.0.0.Final.jar:1.0.0.Final]
at org.wildfly.jberet.services.BatchEnvironmentService$WildFlyBatchEnvironment$1.run(BatchEnvironmentService.java:149) [wildfly-jberet-8.0.0.Final.jar:8.0.0.Final]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_51]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_51]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_51]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_51]
at org.jboss.threads.JBossThread.run(JBossThread.java:122)
Caused by: org.postgresql.util.PSQLException: ERROR: prepared transactions are disabled
ヒント: Set max_prepared_transactions to a nonzero value.
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2161)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1890)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:560)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:403)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:331)
at org.postgresql.xa.PGXAConnection.prepare(PGXAConnection.java:301)
... 26 more
[4]によると、何やらpostgresql.confの設定を変更せねばならんらしい。デフォルトが0で無効になっているらしい[5]。
#max_prepared_transactions = 0 # zero disables the feature
# (change requires restart)
とりあえず20に増やすとOKっぽい
max_prepared_transactions = 20 # zero disables the feature
100にしてみたら、こんな感じで起動しなかった。何か設定が必要なのだろう
Feb 13 11:17:30 kyle-no-MacBook.local postgres[23553]: [1-1] 2014-02-13 11:17:30 JST FATAL: could not create shared memory segment: Invalid argument Feb 13 11:17:30 kyle-no-MacBook.local postgres[23553]: [1-2] 2014-02-13 11:17:30 JST DETAIL: Failed system call was shmget(key=5432001, size=36937728, 03600). Feb 13 11:17:30 kyle-no-MacBook.local postgres[23553]: [1-3] 2014-02-13 11:17:30 JST HINT: This error usually means that PostgreSQL's request for a shared memory segment exceeded your kernel's SHMMAX parameter. You can either reduce the request size or reconfigure the kernel with larger SHMMAX. To reduce the request size (currently 36937728 bytes), reduce PostgreSQL's shared memory usage, perhaps by reducing shared_buffers or max_connections. Feb 13 11:17:30 kyle-no-MacBook.local postgres[23553]: [1-4] If the request size is already small, it's possible that it is less than your kernel's SHMMIN parameter, in which case raising the request size or reconfiguring SHMMIN is called for. Feb 13 11:17:30 kyle-no-MacBook.local postgres[23553]: [1-5] The PostgreSQL documentation contains more information about shared memory configuration.
再起動
sudo launchctl stop com.edb.launchd.postgresql-9.2 sudo launchctl start com.edb.launchd.postgresql-9.2
ジョブレポジトリ用のXAデータソースを定義する
jboss-cliを使う。こういう感じ
batch
xa-data-source add \
--name=JBatchDS \
--driver-name=postgresql \
--jndi-name=java:jboss/jdbc/JBatchDS \
--user-name=postgres \
--password=***
/subsystem=datasources/xa-data-source="JBatchDS"/xa-datasource-properties=ServerName:add(value="localhost")
/subsystem=datasources/xa-data-source="JBatchDS"/xa-datasource-properties=PortNumber:add(value="5432")
/subsystem=datasources/xa-data-source="JBatchDS"/xa-datasource-properties=DatabaseName:add(value="jbatch")
run-batch
接続テストとかその他のデータソース関連操作はWildFly - CLIでデータソースを定義するに書いたのも何かの参考になるかも
アプリ用のXAデータソースを定義する
アプリ用の方は普通のデータソースでもよいのかしら？わからん。ジョブレポジトリ用と同じ感じです。
ジョブレポジトリを変更
WildFly8でJDBCジョブレポジトリを使ってみるで書いたのと同じ感じなので省略。
バッチを動かしてみる
Chunk方式のStepを使ってみるで作ったバッチを動かしてみると、Repository用のDDLが勝手に流されて必要なテーブルやシーケンスを勝手に作ってくれる。ログを見ているとこういうのが流れている
2014-02-13 09:57:02,584 INFO [org.jberet] (pool-2-thread-1) JBERET000021: About to initialize batch job repository with ddl-file: sql/jberet-postgresql.ddl for database PostgreSQL
Repositoryのテーブル一覧を見てみる
jbatch=# \d
List of relations
Schema | Name | Type | Owner
--------+------------------------------------+----------+----------
public | job_execution | table | postgres
public | job_execution_jobexecutionid_seq | sequence | postgres
public | job_instance | table | postgres
public | job_instance_jobinstanceid_seq | sequence | postgres
public | partition_execution | table | postgres
public | step_execution | table | postgres
public | step_execution_stepexecutionid_seq | sequence | postgres
(7 rows)
jbatch=#
Repositoryのテーブルに入っているレコードも見てみる
FAILEDになっているのはXAデータソースがらみではまって死んだところ。いろいろと設定を加えて再実行した結果COMPLETEDが1件できた
jbatch=# select * from job_execution;
jobexecutionid | jobinstanceid | version | createtime | starttime | endtime | lastupdatedtime | batchstatus | exitstatus | jobparameters | restartposition
----------------+---------------+---------+-------------------------+-------------------------+-------------------------+-------------------------+-------------+------------+---------------+-----------------
1 | 1 | | 2014-02-13 09:57:02.685 | 2014-02-13 09:57:02.685 | 2014-02-13 09:57:02.999 | 2014-02-13 09:57:02.999 | FAILED | FAILED | divide = 2 +|
| | | | | | | | | |
2 | 2 | | 2014-02-13 09:58:59.033 | 2014-02-13 09:58:59.033 | 2014-02-13 09:58:59.058 | 2014-02-13 09:58:59.058 | FAILED | FAILED | divide = 2 +|
| | | | | | | | | |
3 | 3 | | 2014-02-13 10:01:44.972 | 2014-02-13 10:01:44.972 | 2014-02-13 10:01:44.994 | 2014-02-13 10:01:44.994 | FAILED | FAILED | divide = 2 +|
| | | | | | | | | |
4 | 4 | | 2014-02-13 10:56:52.962 | 2014-02-13 10:56:52.962 | 2014-02-13 10:56:53.231 | 2014-02-13 10:56:53.231 | FAILED | FAILED | divide = 2 +|
| | | | | | | | | |
5 | 5 | | 2014-02-13 11:18:51.947 | 2014-02-13 11:18:51.947 | 2014-02-13 11:18:52.28 | 2014-02-13 11:18:52.28 | COMPLETED | COMPLETED | divide = 2 +|
| | | | | | | | | |
(5 rows)
jbatch=# select * from job_instance;
jobinstanceid | version | jobname | applicationname
---------------+---------+---------+-----------------
1 | | chunk | jbatchtest
2 | | chunk | jbatchtest
3 | | chunk | jbatchtest
4 | | chunk | jbatchtest
5 | | chunk | jbatchtest
(5 rows)
jbatch=# select * from step_execution;
stepexecutionid | jobexecutionid | version | stepname | starttime | endtime | batchstatus | exitstatus | executionexception | persistentuserdata | readcount | writecount | commitcount | rollbackcount | readskipcount | processskipcount | filtercount | writeskipcount | readercheckpointinfo | writercheckpointinfo
-----------------+----------------+---------+----------+-------------------------+-------------------------+-------------+------------+--------------------+--------------------+-----------+------------+-------------+---------------+---------------+------------------+-------------+----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------
1 | 1 | | doChunk | 2014-02-13 09:57:02.726 | | STARTED | | | | | | | | | | | | |
2 | 2 | | doChunk | 2014-02-13 09:58:59.036 | | STARTED | | | | | | | | | | | | |
3 | 3 | | doChunk | 2014-02-13 10:01:44.974 | | STARTED | | | | | | | | | | | | |
4 | 4 | | doChunk | 2014-02-13 10:56:52.98 | | STARTED | | | | | | | | | | | | |
5 | 5 | | doChunk | 2014-02-13 11:18:51.965 | 2014-02-13 11:18:52.276 | COMPLETED | COMPLETED | | | 10 | 10 | 4 | 0 | 0 | 0 | 0 | 0 | \xaced0005737200116a6176612e6c616e672e496e746567657212e2a0a4f781873802000149000576616c7565787200106a6176612e6c616e672e4e756d62657286ac951d0b94e08b02000078700000000a | \xaced0005737200116a6176612e6c616e672e496e746567657212e2a0a4f781873802000149000576616c7565787200106a6176612e6c616e672e4e756d62657286ac951d0b94e08b020000787000000004
(5 rows)
jbatch=# select * from partition_execution ;
partitionexecutionid | stepexecutionid | version | batchstatus | exitstatus | executionexception | persistentuserdata | readercheckpointinfo | writercheckpointinfo
----------------------+-----------------+---------+-------------+------------+--------------------+--------------------+----------------------+----------------------
(0 rows)
jbatch=#
アプリのテーブルを見てみる
まあ普通
jbatch=# \c jbatcharts You are now connected to database "jbatcharts" as user "kyle". jbatcharts=# select * from chunkoutputitem ; id | result ----+-------- 0 | 0 1 | 5 2 | 10 3 | 15 4 | 20 5 | 25 6 | 30 7 | 35 8 | 40 9 | 45 (10 rows) jbatcharts=#
備考
勝手に作られるRepositoryにはインデックスがほとんどないので、履歴が増えてきたときに備えていくつか作っておいたほうが良いような気もする。
参考文献
Tags: jbatch
