Spring Batch in Action

505 Pages·2011·11.2 MB·English

IN ACTION
Arnaud Cogoluègnes
Thierry Templier
Gary Gregory
Olivier Bazoud
M A N N I N G
Spring Batch in Action

Spring Batch in Action
ARNAUD COGOLUEGNES
THIERRY TEMPLIER
GARY GREGORY
OLIVIER BAZOUD
MANNING
SHELTER ISLAND
For online information and ordering of this and other Manning books, please visit
www.manning.com. The publisher offers discounts on this book when ordered in quantity.
For more information, please contact
Special Sales Department
Manning Publications Co.
20 Baldwin Road
PO Box 261
Shelter Island, NY 11964
Email: [email protected]
©2012 by Manning Publications Co. All rights reserved.
No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in
any form or by means electronic, mechanical, photocopying, or otherwise, without prior written
permission of the publisher.
Many of the designations used by manufacturers and sellers to distinguish their products are
claimed as trademarks. Where those designations appear in the book, and Manning
Publications was aware of a trademark claim, the designations have been printed in initial caps
or all caps.
Recognizing the importance of preserving what has been written, it is Manning’s policy to have
the books we publish printed on acid-free paper, and we exert our best efforts to that end.
Recognizing also our responsibility to conserve the resources of our planet, Manning books are
printed on paper that is at least 15 percent recycled and processed without the use of elemental
chlorine.
Manning Publications Co. Development editor: Cynthia Kane
20 Baldwin Road Copyeditor: Linda Kern
PO Box 261 Proofreader: Katie Tennant
Shelter Island, NY 11964 Typesetter: Gordan Salinovic
Cover designer: Marija Tudor
ISBN 9781935182955
Printed in the United States of America
1 2 3 4 5 6 7 8 9 10 – MAL – 16 15 14 13 12 11
brief contents
P 1 B ..................................................................1
ART ACKGROUND
1 ■ Introducing Spring Batch 3
2 ■ Spring Batch concepts 32
P 2 C S B ......................................................51
ART ORE PRING ATCH
3 ■ Batch configuration 53
4 ■ Running batch jobs 87
5 ■ Reading data 117
6 ■ Writing data 157
7 ■ Processing data 193
8 ■ Implementing bulletproof jobs 223
9 ■ Transaction management 251
P 3 A S B .............................................275
ART DVANCED PRING ATCH
10 ■ Controlling execution 277
11 ■ Enterprise integration 306
12 ■ Monitoring jobs 345
13 ■ Scaling and parallel processing 373
14 ■ Testing batch applications 407
v

contents
foreword xv
preface xvii
acknowledgments xviii
about this book xx
about the cover illustration xxvi
P 1 B ......................................................1
ART ACKGROUND
1 Introducing Spring Batch 3
1.1 What are batch applications? 4
1.2 Meet Spring Batch 5
Robustness and reliability 6 ■ Scaling strategies 7
1.3 Introducing the case study 8
The online store application 8 ■ Why build an online store with
batch jobs? 9 ■ Why use batch processes? 9 ■ The import product
use case 10
1.4 Reading and writing the product data 11
Anatomy of the read-write step 11 ■ Reading a flat file 14
Implementing a database item writer 17 ■ Configuring a database
item writer 18 ■ Configuring the read-write step 18
vii
viii CONTENTS
1.5 Decompressing the input file with a tasklet 20
Why compress the file? 20 ■ Implementing the decompression
tasklet 20 ■ Configuring the tasklet 22
1.6 Testing the batch process 23
Setting up the test infrastructure 23 ■ Leveraging SpEL for
configuration 25 ■ Writing the test for the job 26
1.7 Skipping incorrect lines instead of failing 28
1.8 Summary 31
2 Spring Batch concepts 32
2.1 The batch domain language 33
Why use a domain language? 33 ■ Main components of the
domain language 34 ■ How Spring Batch interacts with the
outside world 35
2.2 The Spring Batch infrastructure 36
Launching jobs and storing job metadata 36 ■ Configuring the
Spring Batch infrastructure in a database 37
2.3 Anatomy of a job 41
Modeling jobs with steps 42 ■ Running job instances and job
executions 46
2.4 Summary 50
P 2 C S B .........................................51
ART ORE PRING ATCH
3 Batch configuration 53
3.1 The Spring Batch XML vocabulary 54
Using the Spring Batch XML namespace 54 ■ Spring Batch XML
features 56
3.2 Configuring jobs and steps 57
Job entities hierarchy 57 ■ Configuring jobs 58 ■ Configuring
steps 60 ■ Configuring tasklets and chunks 61 ■ Configuring
transactions 69
3.3 Configuring the job repository 72
Choosing a job repository 72 ■ Specifying job repository parameters 73
3.4 Advanced configuration topics 75
Using step scope 75 ■ Leveraging SpEL 76 ■ Using listeners to
provide additional processing 78 ■ Configuration inheritance 83
3.5 Summary 86
CONTENTS ix
4 Running batch jobs 87
4.1 Launching concepts 88
Introducing the Spring Batch launcher API 88 ■ Synchronous vs.
asynchronous launches 89 ■ Overview of launching solutions 90
4.2 Launching from the command line 92
Using Spring Batch’s command-line job runner 92
4.3 Job schedulers 97
Using cron 98 ■ Using the Spring scheduler 99
4.4 Launching from a web application 103
Embedding Spring Batch in a web application 104 ■ Launching a
job with an HTTP request 105
4.5 Stopping jobs gracefully 109
Stopping a job for the operator 110 ■ Stopping a job for the
application developer 113
4.6 Summary 116
5 Reading data 117
5.1 Data reading concepts 118
5.2 Reading flat files 119
Configuring the FlatFileItemReader class 121 ■ Introducing the
DefaultLineMapper class 125 ■ Using the DefaultLineMapper
class 126 ■ Extracting character-separated fields 126
Creating objects from fields 128 ■ Reading JSON 130
Multiline records 132 ■ Reading heterogonous records 133
5.3 Reading XML files 135
5.4 Reading file sets 138
5.5 Reading from relational databases 139
Using JDBC item readers 139 ■ Using ORM item readers 148
5.6 Using other input sources 151
Services as input 151 ■ Reading from JMS 154
5.7 Implementing custom readers 155
5.8 Summary 156
6 Writing data 157
6.1 Data-writing concepts 158
6.2 Writing files 159
Writing flat files 159 ■ Writing XML files 173 ■ Writing file
sets 178