Kohei Nozaki's blog 

Splitで並列処理してみる


Posted on Saturday Feb 15, 2014 at 08:19PM in Technology


「Partitioned Chunk Processingで遊ぶ」で使ったPartitionは一つの処理を手分けしてやる仕組みだが、splitは異なる処理を並列で行う仕組み。簡単なバッチを作って遊んでみる。

環境・前提条件

バッチを作ってみる

仕様

  • 一番高レベルの要素は2つ。順番に動く
    1. split0 (Split)
    2. nextOfSplit (Step)
  • split0にはflowが3つある
    • 全てのflowは1つのstepを持つ
    • 使っているartifactは全て同じ。単純なBatchletで、パラメータで与えられたミリ秒数だけThread.sleep()する
    • それぞれ1, 2, 3秒スリープする
  • nextOfSplitは特に何もしない。ちゃんとsplit0内のflowが全部終わってから遷移するか確かめるために置いた

資源

資源はこのへんにまとめて全部ある

動かしてみる

ログ

21:38:24,423 FINE  [org.nailedtothex.jbatch.example.split.SleepBatchlet] (batch-batch - 5) entering process(): stepName=1sec, sleepInMills=1,000
21:38:24,423 FINE  [org.nailedtothex.jbatch.example.split.SleepBatchlet] (batch-batch - 8) entering process(): stepName=3sec, sleepInMills=3,000
21:38:24,422 FINE  [org.nailedtothex.jbatch.example.split.SleepBatchlet] (batch-batch - 7) entering process(): stepName=2sec, sleepInMills=2,000
21:38:25,424 FINE  [org.nailedtothex.jbatch.example.split.SleepBatchlet] (batch-batch - 5)  exiting process(): stepName=1sec, sleepInMills=1,000
21:38:26,423 FINE  [org.nailedtothex.jbatch.example.split.SleepBatchlet] (batch-batch - 7)  exiting process(): stepName=2sec, sleepInMills=2,000
21:38:27,424 FINE  [org.nailedtothex.jbatch.example.split.SleepBatchlet] (batch-batch - 8)  exiting process(): stepName=3sec, sleepInMills=3,000
21:38:27,430 FINE  [org.nailedtothex.jbatch.example.split.SleepBatchlet] (batch-batch - 4) entering process(): stepName=nextOfSplit, sleepInMills=null
21:38:27,430 FINE  [org.nailedtothex.jbatch.example.split.SleepBatchlet] (batch-batch - 4)  exiting process(): stepName=nextOfSplit, sleepInMills=null

Repository

job_execution

jbatch=# select * from job_execution order by jobexecutionid desc limit 1;
 jobexecutionid | jobinstanceid | version |       createtime        |        starttime        |         endtime         |     lastupdatedtime     | batchstatus | exitstatus | jobparameters | restartposition 
----------------+---------------+---------+-------------------------+-------------------------+-------------------------+-------------------------+-------------+------------+---------------+-----------------
            129 |           124 |         | 2014-02-15 21:38:24.415 | 2014-02-15 21:38:24.415 | 2014-02-15 21:38:27.433 | 2014-02-15 21:38:27.433 | COMPLETED   | COMPLETED  |               | 
(1 row)

step_execution

jbatch=# select * from step_execution where jobexecutionid =129 order by stepexecutionid;
 stepexecutionid | jobexecutionid | version |  stepname   |        starttime        |         endtime         | batchstatus | exitstatus | executionexception | persistentuserdata | readcount | writecount | commitcount | rollbackcount | readskipcount | processskipcount | filtercount | writeskipcount | readercheckpointinfo | writercheckpointinfo 
-----------------+----------------+---------+-------------+-------------------------+-------------------------+-------------+------------+--------------------+--------------------+-----------+------------+-------------+---------------+---------------+------------------+-------------+----------------+----------------------+----------------------
             147 |            129 |         | 1sec        | 2014-02-15 21:38:24.42  | 2014-02-15 21:38:25.424 | COMPLETED   | SUCCESS    |                    |                    |         0 |          0 |           0 |             0 |             0 |                0 |           0 |              0 |                      | 
             148 |            129 |         | 2sec        | 2014-02-15 21:38:24.42  | 2014-02-15 21:38:26.424 | COMPLETED   | SUCCESS    |                    |                    |         0 |          0 |           0 |             0 |             0 |                0 |           0 |              0 |                      | 
             149 |            129 |         | 3sec        | 2014-02-15 21:38:24.42  | 2014-02-15 21:38:27.424 | COMPLETED   | SUCCESS    |                    |                    |         0 |          0 |           0 |             0 |             0 |                0 |           0 |              0 |                      | 
             150 |            129 |         | nextOfSplit | 2014-02-15 21:38:27.428 | 2014-02-15 21:38:27.431 | COMPLETED   | SUCCESS    |                    |                    |         0 |          0 |           0 |             0 |             0 |                0 |           0 |              0 |                      | 
(4 rows)

備考

  • ちゃんとパラで走っているみたい
  • [1]によると、いずれかのFlowが死ぬと、親のSplitも死ぬ
  • Flowは複数のStepを1つにまとめる場合に使う。うまく使うとStep間の遷移をすっきり表現できる気がする。Splitの子要素として以外でも使えるようだ

参考文献

  1. JSR-000352 Batch Applications for the Java Platform - Final Release



No one has commented yet.

Leave a Comment

HTML Syntax: NOT allowed