Chunk方式のStepでJDBCを使ってみる
TweetPosted on Thursday Feb 13, 2014 at 07:25AM in Technology
Chunk方式のStepを使ってみるではJPAだったが、同じ事をJDBCでやってみる。一応大量のデータを想定した感じに作ってみる。実際に大量のデータでの実験はしてないけど。
JSR 352 Implementation - Connection close probl… | Communityによると、ここに書いてあるような実装はよろしくないようです。SELECT文の発行などは全て同一トランザクション内で完結する必要があるらしい。open()で取得するのはPKだけにして、実際のデータはreadItem()で別途取得するようにするなどした方が良いかも
トランザクション境界を跨いでJDBCの資源（Connection, Statement, ResultSet等）を使う場合は，非JTAデータソースを使いましょう．詳細はこちらを参照．また，このエントリで出している例のようにopen()でConnection, PreparedStatementを作るのはよろしくないようです．writeItems()内で，つどつど作りましょう．
環境・前提条件
Chunk方式のStepを使ってみると同じ。この記事で作ったプロジェクトや資源がすでに存在するものとする
準備
- 資源の詳細はこのあたり
- エンティティ、テストデータは流用する
- @ResourceでDataSourceを注入するのでweb.xmlとjboss-web.xmlを置いてある
バッチを構成する資源
chunkjdbc.xml
- ジョブXML。artifact以外は前回と同じ
ChunkJDBCItemReader.java
- SELECT文でOFFSET使ってるのでPostgreSQL以外の場合は適当に変えてやる
ChunkJDBCItemProcessor.java
- 一応コピーして作ったけど完全に前回と同じ
- DTO作るのが面倒ならResultSetを受け取ってもよかったかも
ChunkJDBCItemWriter.java
- PreparedStatement#addBatch()とexecuteBatch()を使ってみた
- この例は誤っていて，open()でConnection, PreparedStatementを作るのはよろしくないようです．writeItems()内で，つどつど作りましょう．
テスト用資源
ChunkJDBCJobTest.java
- テストクラス。ジョブの名前以外前回と同じ
実行してみる
ログはこんな感じ。実行結果もテーブル出力も特に問題なさげ
14:41:53,140 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemProcessor] (batch-batch - 7) postConstruct(): divide=2 14:41:53,140 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) open(): checkpoint=null, index starts at=0 14:41:53,142 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) open(): checkpoint=null, index starts at=0 14:41:53,142 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): index=0 14:41:53,143 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): returning=ChunkInputItem [id=0, input=0, processed=false] 14:41:53,143 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemProcessor] (batch-batch - 7) processItem(): input=ChunkInputItem [id=0, input=0, processed=false], output=ChunkOutputItem [id=0, result=0] 14:41:53,143 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) checkpointInfo(): returns=1 14:41:53,143 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): index=1 14:41:53,143 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): returning=ChunkInputItem [id=1, input=10, processed=false] 14:41:53,143 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemProcessor] (batch-batch - 7) processItem(): input=ChunkInputItem [id=1, input=10, processed=false], output=ChunkOutputItem [id=1, result=5] 14:41:53,143 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) checkpointInfo(): returns=2 14:41:53,143 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): index=2 14:41:53,143 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): returning=ChunkInputItem [id=2, input=20, processed=false] 14:41:53,144 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemProcessor] (batch-batch - 7) processItem(): input=ChunkInputItem [id=2, input=20, processed=false], output=ChunkOutputItem [id=2, result=10] 14:41:53,144 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) checkpointInfo(): returns=3 14:41:53,144 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): index=0 14:41:53,144 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): item=ChunkOutputItem [id=0, result=0] 14:41:53,144 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): item=ChunkOutputItem [id=1, result=5] 14:41:53,144 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): item=ChunkOutputItem [id=2, result=10] 14:41:53,151 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): ps#executeBatch() result=1 14:41:53,152 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): ps#executeBatch() result=1 14:41:53,152 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): ps#executeBatch() result=1 14:41:53,152 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) checkpointInfo(): returns=3 14:41:53,152 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) checkpointInfo(): returns=1 14:41:53,154 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) checkpointInfo(): returns=3 14:41:53,160 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): index=3 14:41:53,160 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): returning=ChunkInputItem [id=3, input=30, processed=false] 14:41:53,160 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemProcessor] (batch-batch - 7) processItem(): input=ChunkInputItem [id=3, input=30, processed=false], output=ChunkOutputItem [id=3, result=15] 14:41:53,161 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) checkpointInfo(): returns=4 14:41:53,161 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): index=4 14:41:53,161 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): returning=ChunkInputItem [id=4, input=40, processed=false] 14:41:53,161 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemProcessor] (batch-batch - 7) processItem(): input=ChunkInputItem [id=4, input=40, processed=false], output=ChunkOutputItem [id=4, result=20] 14:41:53,161 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) checkpointInfo(): returns=5 14:41:53,161 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): index=5 14:41:53,161 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): returning=ChunkInputItem [id=5, input=50, processed=false] 14:41:53,161 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemProcessor] (batch-batch - 7) processItem(): input=ChunkInputItem [id=5, input=50, processed=false], output=ChunkOutputItem [id=5, result=25] 14:41:53,161 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) checkpointInfo(): returns=6 14:41:53,161 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): index=1 14:41:53,161 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): item=ChunkOutputItem [id=3, result=15] 14:41:53,162 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): item=ChunkOutputItem [id=4, result=20] 14:41:53,162 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): item=ChunkOutputItem [id=5, result=25] 14:41:53,168 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): ps#executeBatch() result=1 14:41:53,168 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): ps#executeBatch() result=1 14:41:53,168 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): ps#executeBatch() result=1 14:41:53,168 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) checkpointInfo(): returns=6 14:41:53,169 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) checkpointInfo(): returns=2 14:41:53,170 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) checkpointInfo(): returns=6 14:41:53,171 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): index=6 14:41:53,171 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): returning=ChunkInputItem [id=6, input=60, processed=false] 14:41:53,171 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemProcessor] (batch-batch - 7) processItem(): input=ChunkInputItem [id=6, input=60, processed=false], output=ChunkOutputItem [id=6, result=30] 14:41:53,171 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) checkpointInfo(): returns=7 14:41:53,171 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): index=7 14:41:53,171 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): returning=ChunkInputItem [id=7, input=70, processed=false] 14:41:53,171 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemProcessor] (batch-batch - 7) processItem(): input=ChunkInputItem [id=7, input=70, processed=false], output=ChunkOutputItem [id=7, result=35] 14:41:53,171 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) checkpointInfo(): returns=8 14:41:53,171 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): index=8 14:41:53,171 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): returning=ChunkInputItem [id=8, input=80, processed=false] 14:41:53,171 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemProcessor] (batch-batch - 7) processItem(): input=ChunkInputItem [id=8, input=80, processed=false], output=ChunkOutputItem [id=8, result=40] 14:41:53,172 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) checkpointInfo(): returns=9 14:41:53,172 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): index=2 14:41:53,172 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): item=ChunkOutputItem [id=6, result=30] 14:41:53,172 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): item=ChunkOutputItem [id=7, result=35] 14:41:53,172 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): item=ChunkOutputItem [id=8, result=40] 14:41:53,173 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): ps#executeBatch() result=1 14:41:53,173 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): ps#executeBatch() result=1 14:41:53,173 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): ps#executeBatch() result=1 14:41:53,173 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) checkpointInfo(): returns=9 14:41:53,173 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) checkpointInfo(): returns=3 14:41:53,174 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) checkpointInfo(): returns=9 14:41:53,175 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): index=9 14:41:53,175 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): returning=ChunkInputItem [id=9, input=90, processed=false] 14:41:53,175 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemProcessor] (batch-batch - 7) processItem(): input=ChunkInputItem [id=9, input=90, processed=false], output=ChunkOutputItem [id=9, result=45] 14:41:53,175 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) checkpointInfo(): returns=10 14:41:53,175 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): index=10 14:41:53,175 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) readItem(): returning=null 14:41:53,175 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): index=3 14:41:53,175 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): item=ChunkOutputItem [id=9, result=45] 14:41:53,176 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) writeItems(): ps#executeBatch() result=1 14:41:53,176 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) checkpointInfo(): returns=10 14:41:53,176 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) checkpointInfo(): returns=4 14:41:53,177 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) checkpointInfo(): returns=10 14:41:53,178 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) close() 14:41:53,178 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) close() 14:41:53,178 WARN [com.arjuna.ats.jta] (batch-batch - 7) ARJUNA016087: TransactionImple.delistResource - unknown resource 14:41:53,178 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemReader] (batch-batch - 7) close() 14:41:53,178 FINE [org.nailedtothex.jbatch.example.chunkjdbc.ChunkJDBCItemWriter] (batch-batch - 7) close()
備考
- 定型コードが多い。SQL文とバインド変数を外出しすれば、かなり共通化できるような気もするが…。CDIをうまく使えばなんとかなりそうな気もするけど。そのうちまたいろいろ試してみる。
- closeするところはDbUtils.closeQuietly()使えばちょっと楽になりそう
参考文献
Tags: jbatch
GitHubのプロジェクトをインポートしてみる
TweetPosted on Tuesday Feb 11, 2014 at 03:13PM in Technology
GitHubにプロジェクトを上げてみるでpushしたプロジェクトをEclipseでImportしてみる
環境
- Eclipse Kepler SR1
準備
- 手元の端末にある資源を削除しておく
手順
[1]と殆ど同じ。もはやただの記録
File→Import
Projects from Gitを選んでNext
Clone URIを選んでNext
GitHubのRepositoryの画面からURLをコピーしてくる。ここのクリップボード的なボタンを押すとコピーされる
URIに貼付けてNext
Next
ローカルの資源の置き場所を入力してNext
Import exisiting projectsを選んでNext
Finish
インポートしたプロジェクトが現れる
空のフォルダを作る
エラー的なものが出ている。どうやら空のフォルダがgitに上がってないようだ。
ゆえに手作業で作る。何か間違っているような気もするが。
kyle-no-MacBook:git3 kyle$ mkdir -p hellogithub/src/main/resources kyle-no-MacBook:git3 kyle$ mkdir -p hellogithub/src/test/java kyle-no-MacBook:git3 kyle$ mkdir -p hellogithub/src/test/resources
作ったらプロジェクトをリフレッシュ
エラーが消える 
編集→Commit and Pushしてみる
編集する。hogeを追加
プロジェクトを右クリック→Team→Commit
コミットメッセージ入れてCommit and Push
OK
GitHubの画面で編集内容を確認
参考文献
Tags: eclipse
GitHubで遊ぶ
TweetPosted on Tuesday Feb 11, 2014 at 11:16AM in Technology
環境
- git version 1.8.3.4 (Apple Git-47)
- OS X 10.9.1
前提条件
GitHubのアカウント登録とssh公開鍵登録は終わっているものとする。
参考文献を見ると専用の鍵ペア作ったりしてたけど面倒なので適当なid_rsa.pubを放り込んどいた。
Repositoryを作る
[2]に書いてあるけど
ホーム画面でNew Repositoryを押す
Repository nameとDescriptionを適当に埋めてCreate repositoryを押す
つくられた
CLIからいじってみる
cloneしてみる
kyle-no-MacBook:gitprac2 kyle$ git clone git@github.com:lbtc-xxx/hello.git Cloning into 'hello'... Warning: Permanently added the RSA host key for IP address '192.30.252.129' to the list of known hosts. warning: You appear to have cloned an empty repository. Checking connectivity... done kyle-no-MacBook:gitprac2 kyle$ ls -ld hello drwxr-xr-x+ 3 kyle staff 102 2 11 11:28 hello kyle-no-MacBook:gitprac2 kyle$ ls -lR hello total 80 -rwxr-xr-x+ 1 kyle staff 452 2 11 11:28 applypatch-msg.sample -rwxr-xr-x+ 1 kyle staff 896 2 11 11:28 commit-msg.sample -rwxr-xr-x+ 1 kyle staff 189 2 11 11:28 post-update.sample -rwxr-xr-x+ 1 kyle staff 398 2 11 11:28 pre-applypatch.sample -rwxr-xr-x+ 1 kyle staff 1704 2 11 11:28 pre-commit.sample -rwxr-xr-x+ 1 kyle staff 1348 2 11 11:28 pre-push.sample -rwxr-xr-x+ 1 kyle staff 4951 2 11 11:28 pre-rebase.sample -rwxr-xr-x+ 1 kyle staff 1239 2 11 11:28 prepare-commit-msg.sample -rwxr-xr-x+ 1 kyle staff 3611 2 11 11:28 update.sample hello/.git/info: total 8 -rw-r--r--+ 1 kyle staff 240 2 11 11:28 exclude hello/.git/objects: total 0 drwxr-xr-x+ 2 kyle staff 68 2 11 11:28 info drwxr-xr-x+ 2 kyle staff 68 2 11 11:28 pack hello/.git/objects/info: hello/.git/objects/pack: hello/.git/refs: total 0 drwxr-xr-x+ 2 kyle staff 68 2 11 11:28 heads drwxr-xr-x+ 2 kyle staff 68 2 11 11:28 tags hello/.git/refs/heads: hello/.git/refs/tags: kyle-no-MacBook:gitprac2 kyle$ %%%
よくわからないけど何かgitのディレクトリ構造っぽいものができている。
commitしてみる
kyle-no-MacBook:gitprac2 kyle$ cd hello kyle-no-MacBook:hello kyle$ echo hoge > README.md kyle-no-MacBook:hello kyle$ git add README.md kyle-no-MacBook:hello kyle$ git commit -m "first commit" [master (root-commit) 30ec784] first commit 1 file changed, 1 insertion(+) create mode 100644 README.md kyle-no-MacBook:hello kyle$
この時点ではGitHub側の画面をリロードしても資源が上がったりはしていない。たぶんpushってのをやらないとだめなんだろう。
pushしてみる
kyle-no-MacBook:hello kyle$ git push warning: push.default is unset; its implicit value is changing in Git 2.0 from 'matching' to 'simple'. To squelch this message and maintain the current behavior after the default changes, use: git config --global push.default matching To squelch this message and adopt the new behavior now, use: git config --global push.default simple See 'git help config' and search for 'push.default' for further information. (the 'simple' mode was introduced in Git 1.7.11. Use the similar mode 'current' instead of 'simple' if you sometimes use older versions of Git) Warning: Permanently added the RSA host key for IP address '192.30.252.130' to the list of known hosts. No refs in common and none specified; doing nothing. Perhaps you should specify a branch such as 'master'. fatal: The remote end hung up unexpectedly error: failed to push some refs to 'git@github.com:lbtc-xxx/hello.git' kyle-no-MacBook:hello kyle$
よくわからないけどおこられた。[4]を見ると何やらパラメータが必要っぽいのでよくわからないけど追加してみる
kyle-no-MacBook:hello kyle$ git push origin master Counting objects: 3, done. Writing objects: 100% (3/3), 214 bytes | 0 bytes/s, done. Total 3 (delta 0), reused 0 (delta 0) To git@github.com:lbtc-xxx/hello.git * [new branch] master -> master kyle-no-MacBook:hello kyle$
GitHubのgitのコマンド例が書かれたページをリロードしてみると 
何かがアップロードされた。うむ
編集してadd→commit→pushしてみる
kyle-no-MacBook:hello kyle$ echo hogehoge >> README.md kyle-no-MacBook:hello kyle$ git add README.md kyle-no-MacBook:hello kyle$ git commit -m "second commit" [master 79e339f] second commit 1 file changed, 1 insertion(+) kyle-no-MacBook:hello kyle$ git push origin master Counting objects: 5, done. Writing objects: 100% (3/3), 249 bytes | 0 bytes/s, done. Total 3 (delta 0), reused 0 (delta 0) To git@github.com:lbtc-xxx/hello.git 30ec784..79e339f master -> master kyle-no-MacBook:hello kyle$
うむ。
参考文献
Tags: git
GitHubにプロジェクトを上げてみる
TweetPosted on Tuesday Feb 11, 2014 at 10:49AM in Technology
環境
- Eclipse Kelper SR1
- OS X 10.9.1
前提条件
- GitHubのアカウント登録とssh公開鍵登録は終わっているものとする。GitHubで遊ぶでやった感じ。
- ローカルリポジトリへのcommitが出来る状態になっているプロジェクトが存在するものとする。ローカルで一人用Gitを使ってみるでやった感じ。
準備
プロジェクトの確認
GitHubにリポジトリを作る
GitHubで遊ぶでやった感じでGitHubにリポジトリを作る。
Push
Team→Remote→Push
準備で作ったリポジトリのURIをコピーして貼付けてNext
Source refからmaster [branch]を選んでAdd Specを押す
Finish
何やら処理が行われてこんなのが表示される
GitHubのリポジトリを確認してみる
それっぽいのが出来ているようだ
参考文献
Tags: eclipse
Chunk方式のStepで例外発生時にリトライさせてからスキップさせてみる
TweetPosted on Monday Feb 10, 2014 at 06:57PM in Technology
Chunk方式のStepで例外発生時にスキップさせてみる でリトライとスキップをそれぞれ別々に試してみたが、ここではリトライを試み、それでも成功しない場合は、そのデータをスキップして処理を継続させる、というのをやってみる
環境・前提条件
- Chunk方式のStepで例外発生時にスキップさせてみると同じ。この記事で作ったプロジェクトや資源がすでに存在するものとする
JSR352仕様を確認する
[1]に以下のようにある
8.2.1.4.3 Retry and Skip the Same Exception
When the same exception is specified as both retryable and skippable, retryable takes precedence over skippable during regular processing of the chunk. While the chunk is retrying, skippable takes precedence over retryable since the exception is already being retried.
This allows an exception to initially be retried for the entire chunk and then skipped if it recurs. When retrying with default retry behavior (see section 8.2.1.4.4) the skips can occur for individual items, since the retry is done with an item-count of 1.
リトライの回数は1回で固定になるようだ。失敗後は1回はリトライするが、それでも駄目ならスキップとなる。retry-limitを指定してみたが、ここでリトライ回数を指定するために使われる値ではないようだ。
バッチを作る
仕様
- 入力、出力、処理内容は基本的にChunk方式のStepで例外発生時にスキップさせてみるでやったのと同じ
- 6件目のデータ処理時にItemReaderでRuntimeExceptionが起こる。リトライが試みられたあとさらに例外が起こり、スキップされる
- RuntimeExceptionは以下の両方に設定してある
- skippable-exception-classes
- retryable-exception-classes
- RuntimeExceptionは以下の両方に設定してある
資源
資源はこのへんにまとめて全部ある
- バッチ本体
- テスト
動かしてみる
ログ
16:36:59,072 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) open(): checkpoint=null, index starts at=0
16:36:59,076 DEBUG [org.hibernate.SQL] (batch-batch - 5) select chunkinput0_.id as id1_0_, chunkinput0_.input as input2_0_, chunkinput0_.processed as processe3_0_ from ChunkInputItem chunkinput0_ order by chunkinput0_.id
16:36:59,078 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) open(): checkpoint=null, index starts at=0
16:36:59,079 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): index=0, failAt=5
16:36:59,079 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): returning=ChunkInputItem [id=0, input=0, processed=false]
16:36:59,079 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemProcessor] (batch-batch - 5) processItem(): input=ChunkInputItem [id=0, input=0, processed=false], output=ChunkOutputItem [id=0, result=0]
16:36:59,079 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=1
16:36:59,079 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): index=1, failAt=5
16:36:59,080 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): returning=ChunkInputItem [id=1, input=10, processed=false]
16:36:59,080 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemProcessor] (batch-batch - 5) processItem(): input=ChunkInputItem [id=1, input=10, processed=false], output=ChunkOutputItem [id=1, result=5]
16:36:59,080 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=2
16:36:59,080 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): index=2, failAt=5
16:36:59,080 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): returning=ChunkInputItem [id=2, input=20, processed=false]
16:36:59,080 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemProcessor] (batch-batch - 5) processItem(): input=ChunkInputItem [id=2, input=20, processed=false], output=ChunkOutputItem [id=2, result=10]
16:36:59,080 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=3
16:36:59,080 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) writeItems(): index=0
16:36:59,080 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) writeItems(): item=ChunkOutputItem [id=0, result=0]
16:36:59,081 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) writeItems(): item=ChunkOutputItem [id=1, result=5]
16:36:59,081 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) writeItems(): item=ChunkOutputItem [id=2, result=10]
16:36:59,081 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=3
16:36:59,081 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) checkpointInfo(): returns=1
16:36:59,083 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=3
16:36:59,086 DEBUG [org.hibernate.SQL] (batch-batch - 5) insert into ChunkOutputItem (result, id) values (?, ?)
16:36:59,087 DEBUG [org.hibernate.SQL] (batch-batch - 5) insert into ChunkOutputItem (result, id) values (?, ?)
16:36:59,088 DEBUG [org.hibernate.SQL] (batch-batch - 5) insert into ChunkOutputItem (result, id) values (?, ?)
16:36:59,116 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): index=3, failAt=5
16:36:59,117 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): returning=ChunkInputItem [id=3, input=30, processed=false]
16:36:59,117 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemProcessor] (batch-batch - 5) processItem(): input=ChunkInputItem [id=3, input=30, processed=false], output=ChunkOutputItem [id=3, result=15]
16:36:59,117 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=4
16:36:59,117 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): index=4, failAt=5
16:36:59,117 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): returning=ChunkInputItem [id=4, input=40, processed=false]
16:36:59,117 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemProcessor] (batch-batch - 5) processItem(): input=ChunkInputItem [id=4, input=40, processed=false], output=ChunkOutputItem [id=4, result=20]
16:36:59,117 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=5
16:36:59,118 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): index=5, failAt=5
16:36:59,118 WARNING [org.nailedtothex.jbatch.example.chunkretry.MyRetryReadListener] (batch-batch - 5) onRetryReadException(): java.lang.RuntimeException: 5
at org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader.readItem(ChunkRetrySkipItemReader.java:43) [classes:]
at org.jberet.runtime.runner.ChunkRunner.readItem(ChunkRunner.java:343) [jberet-core-1.0.1.Beta-SNAPSHOT.jar:1.0.1.Beta-SNAPSHOT]
at org.jberet.runtime.runner.ChunkRunner.readProcessWriteItems(ChunkRunner.java:288) [jberet-core-1.0.1.Beta-SNAPSHOT.jar:1.0.1.Beta-SNAPSHOT]
at org.jberet.runtime.runner.ChunkRunner.run(ChunkRunner.java:190) [jberet-core-1.0.1.Beta-SNAPSHOT.jar:1.0.1.Beta-SNAPSHOT]
at org.jberet.runtime.runner.StepExecutionRunner.runBatchletOrChunk(StepExecutionRunner.java:204) [jberet-core-1.0.1.Beta-SNAPSHOT.jar:1.0.1.Beta-SNAPSHOT]
at org.jberet.runtime.runner.StepExecutionRunner.run(StepExecutionRunner.java:131) [jberet-core-1.0.1.Beta-SNAPSHOT.jar:1.0.1.Beta-SNAPSHOT]
at org.jberet.runtime.runner.CompositeExecutionRunner.runStep(CompositeExecutionRunner.java:162) [jberet-core-1.0.1.Beta-SNAPSHOT.jar:1.0.1.Beta-SNAPSHOT]
at org.jberet.runtime.runner.CompositeExecutionRunner.runFromHeadOrRestartPoint(CompositeExecutionRunner.java:88) [jberet-core-1.0.1.Beta-SNAPSHOT.jar:1.0.1.Beta-SNAPSHOT]
at org.jberet.runtime.runner.JobExecutionRunner.run(JobExecutionRunner.java:58) [jberet-core-1.0.1.Beta-SNAPSHOT.jar:1.0.1.Beta-SNAPSHOT]
at org.wildfly.jberet.services.BatchEnvironmentService$WildFlyBatchEnvironment$1.run(BatchEnvironmentService.java:149) [wildfly-jberet-8.0.0.Final.jar:8.0.0.Final]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_51]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_51]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_51]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_51]
at org.jboss.threads.JBossThread.run(JBossThread.java:122)
16:36:59,119 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=6
16:36:59,119 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) close()
16:36:59,119 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) close()
16:36:59,119 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) open(): checkpoint=3, index starts at=3
16:36:59,120 DEBUG [org.hibernate.SQL] (batch-batch - 5) select chunkinput0_.id as id1_0_, chunkinput0_.input as input2_0_, chunkinput0_.processed as processe3_0_ from ChunkInputItem chunkinput0_ order by chunkinput0_.id
16:36:59,122 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) open(): checkpoint=1, index starts at=1
16:36:59,122 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=3
16:36:59,122 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): index=3, failAt=5
16:36:59,123 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): returning=ChunkInputItem [id=3, input=30, processed=false]
16:36:59,123 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemProcessor] (batch-batch - 5) processItem(): input=ChunkInputItem [id=3, input=30, processed=false], output=ChunkOutputItem [id=3, result=15]
16:36:59,123 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=4
16:36:59,123 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) writeItems(): index=1
16:36:59,123 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) writeItems(): item=ChunkOutputItem [id=3, result=15]
16:36:59,123 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=4
16:36:59,123 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) checkpointInfo(): returns=2
16:36:59,125 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=4
16:36:59,125 DEBUG [org.hibernate.SQL] (batch-batch - 5) insert into ChunkOutputItem (result, id) values (?, ?)
16:36:59,152 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): index=4, failAt=5
16:36:59,152 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): returning=ChunkInputItem [id=4, input=40, processed=false]
16:36:59,152 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemProcessor] (batch-batch - 5) processItem(): input=ChunkInputItem [id=4, input=40, processed=false], output=ChunkOutputItem [id=4, result=20]
16:36:59,153 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=5
16:36:59,153 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) writeItems(): index=2
16:36:59,153 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) writeItems(): item=ChunkOutputItem [id=4, result=20]
16:36:59,153 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=5
16:36:59,153 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) checkpointInfo(): returns=3
16:36:59,155 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=5
16:36:59,155 DEBUG [org.hibernate.SQL] (batch-batch - 5) insert into ChunkOutputItem (result, id) values (?, ?)
16:36:59,183 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): index=5, failAt=5
16:36:59,183 WARNING [org.nailedtothex.jbatch.example.chunkskip.MySkipReadListener] (batch-batch - 5) onSkipReadItem(): java.lang.RuntimeException: 5
at org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader.readItem(ChunkRetrySkipItemReader.java:43) [classes:]
at org.jberet.runtime.runner.ChunkRunner.readItem(ChunkRunner.java:343) [jberet-core-1.0.1.Beta-SNAPSHOT.jar:1.0.1.Beta-SNAPSHOT]
at org.jberet.runtime.runner.ChunkRunner.readProcessWriteItems(ChunkRunner.java:288) [jberet-core-1.0.1.Beta-SNAPSHOT.jar:1.0.1.Beta-SNAPSHOT]
at org.jberet.runtime.runner.ChunkRunner.run(ChunkRunner.java:190) [jberet-core-1.0.1.Beta-SNAPSHOT.jar:1.0.1.Beta-SNAPSHOT]
at org.jberet.runtime.runner.StepExecutionRunner.runBatchletOrChunk(StepExecutionRunner.java:204) [jberet-core-1.0.1.Beta-SNAPSHOT.jar:1.0.1.Beta-SNAPSHOT]
at org.jberet.runtime.runner.StepExecutionRunner.run(StepExecutionRunner.java:131) [jberet-core-1.0.1.Beta-SNAPSHOT.jar:1.0.1.Beta-SNAPSHOT]
at org.jberet.runtime.runner.CompositeExecutionRunner.runStep(CompositeExecutionRunner.java:162) [jberet-core-1.0.1.Beta-SNAPSHOT.jar:1.0.1.Beta-SNAPSHOT]
at org.jberet.runtime.runner.CompositeExecutionRunner.runFromHeadOrRestartPoint(CompositeExecutionRunner.java:88) [jberet-core-1.0.1.Beta-SNAPSHOT.jar:1.0.1.Beta-SNAPSHOT]
at org.jberet.runtime.runner.JobExecutionRunner.run(JobExecutionRunner.java:58) [jberet-core-1.0.1.Beta-SNAPSHOT.jar:1.0.1.Beta-SNAPSHOT]
at org.wildfly.jberet.services.BatchEnvironmentService$WildFlyBatchEnvironment$1.run(BatchEnvironmentService.java:149) [wildfly-jberet-8.0.0.Final.jar:8.0.0.Final]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_51]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_51]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_51]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_51]
at org.jboss.threads.JBossThread.run(JBossThread.java:122)
16:36:59,185 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=6
16:36:59,185 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): index=6, failAt=5
16:36:59,186 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): returning=ChunkInputItem [id=6, input=60, processed=false]
16:36:59,186 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemProcessor] (batch-batch - 5) processItem(): input=ChunkInputItem [id=6, input=60, processed=false], output=ChunkOutputItem [id=6, result=30]
16:36:59,186 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=7
16:36:59,186 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): index=7, failAt=5
16:36:59,187 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): returning=ChunkInputItem [id=7, input=70, processed=false]
16:36:59,187 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemProcessor] (batch-batch - 5) processItem(): input=ChunkInputItem [id=7, input=70, processed=false], output=ChunkOutputItem [id=7, result=35]
16:36:59,187 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=8
16:36:59,187 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): index=8, failAt=5
16:36:59,187 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): returning=ChunkInputItem [id=8, input=80, processed=false]
16:36:59,187 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemProcessor] (batch-batch - 5) processItem(): input=ChunkInputItem [id=8, input=80, processed=false], output=ChunkOutputItem [id=8, result=40]
16:36:59,187 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=9
16:36:59,188 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) writeItems(): index=3
16:36:59,188 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) writeItems(): item=ChunkOutputItem [id=6, result=30]
16:36:59,188 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) writeItems(): item=ChunkOutputItem [id=7, result=35]
16:36:59,189 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) writeItems(): item=ChunkOutputItem [id=8, result=40]
16:36:59,189 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=9
16:36:59,189 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) checkpointInfo(): returns=4
16:36:59,191 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=9
16:36:59,191 DEBUG [org.hibernate.SQL] (batch-batch - 5) insert into ChunkOutputItem (result, id) values (?, ?)
16:36:59,192 DEBUG [org.hibernate.SQL] (batch-batch - 5) insert into ChunkOutputItem (result, id) values (?, ?)
16:36:59,193 DEBUG [org.hibernate.SQL] (batch-batch - 5) insert into ChunkOutputItem (result, id) values (?, ?)
16:36:59,221 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): index=9, failAt=5
16:36:59,221 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): returning=ChunkInputItem [id=9, input=90, processed=false]
16:36:59,221 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemProcessor] (batch-batch - 5) processItem(): input=ChunkInputItem [id=9, input=90, processed=false], output=ChunkOutputItem [id=9, result=45]
16:36:59,221 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=10
16:36:59,221 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): index=10, failAt=5
16:36:59,221 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) readItem(): returning=null
16:36:59,221 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) writeItems(): index=4
16:36:59,222 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) writeItems(): item=ChunkOutputItem [id=9, result=45]
16:36:59,222 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=10
16:36:59,222 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) checkpointInfo(): returns=5
16:36:59,223 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) checkpointInfo(): returns=10
16:36:59,224 DEBUG [org.hibernate.SQL] (batch-batch - 5) insert into ChunkOutputItem (result, id) values (?, ?)
16:36:59,252 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) close()
16:36:59,269 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) close()
16:36:59,270 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemReader] (batch-batch - 5) close()
16:36:59,270 FINE [org.nailedtothex.jbatch.example.chunkretryskip.ChunkRetrySkipItemWriter] (batch-batch - 5) close()
Repository
job_execution
jbatch=# select * from job_execution order by jobexecutionid desc limit 1;
jobexecutionid | jobinstanceid | version | createtime | starttime | endtime | lastupdatedtime | batchstatus | exitstatus | jobparameters | restartposition
----------------+---------------+---------+-------------------------+-------------------------+-------------------------+-------------------------+-------------+------------+----------------------+-----------------
127 | 122 | | 2014-02-15 16:36:59.068 | 2014-02-15 16:36:59.068 | 2014-02-15 16:36:59.272 | 2014-02-15 16:36:59.272 | COMPLETED | COMPLETED | itemReaderFailAt = 5+|
| | | | | | | | | divide = 2 +|
| | | | | | | | | |
(1 row)
step_execution
jbatch=# select * from step_execution where jobexecutionid =127;
stepexecutionid | jobexecutionid | version | stepname | starttime | endtime | batchstatus | exitstatus | executionexception | persistentuserdata | readcount | writecount | commitcount | rollbackcount | readskipcount | processskipcount | filtercount | writeskipcount | readercheckpointinfo | writercheckpointinfo
-----------------+----------------+---------+----------+------------------------+------------------------+-------------+------------+--------------------+--------------------+-----------+------------+-------------+---------------+---------------+------------------+-------------+----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------
142 | 127 | | doChunk | 2014-02-15 16:36:59.07 | 2014-02-15 16:36:59.27 | COMPLETED | COMPLETED | | | 11 | 9 | 5 | 1 | 1 | 0 | 0 | 0 | \xaced0005737200116a6176612e6c616e672e496e746567657212e2a0a4f781873802000149000576616c7565787200106a6176612e6c616e672e4e756d62657286ac951d0b94e08b02000078700000000a | \xaced0005737200116a6176612e6c616e672e496e746567657212e2a0a4f781873802000149000576616c7565787200106a6176612e6c616e672e4e756d62657286ac951d0b94e08b020000787000000005
(1 row)
動作の流れを見てみる
ざっくり
- 3件目までは普通に完走
- 6件目で例外が起きて、バッファリングされていた4, 5件目の処理結果は破棄
- item-count=1に設定されリトライが行われる
- 4, 5件目が1件ずつ処理されて結果がDBに書き込まれる
- 6件目で2回目の例外が起きる。6件目はあきらめてスキップされる
- 7件目以降が1件ずつ処理されて完走
備考
- リトライの回数を指定できると嬉しいような気はするがまあこれはこれでいいのかも
- アプリ側でリトライの回数を数えて、あきらめる時はスキップ用例外を投げる、みたいな感じで制御してもいいかも
参考文献
Tags: jbatch
