Table Of ContentCONTENTS
About the Author
Disclaimer
Preface
Acknowledgement
About this book
Who should read this BOOK?
Chapter1: KAFKA Overview & Architecture
Chapter 2: Zookeeper Overview and Architecture
Chapter 3: Set Up Zookeeper and Kafka
Chapter 4: Setup Kafka on standalone windows
Chapter 5:Pega(7.4/8.x)- Kafka Functioning
Chapter 6:Pega: Kafka Operations
Chapter 7:Pega Decision Data Flow Services
Chapter 8:Pega decision DataSet?
Chapter 9:Pega Decision DataFlow?
Chapter 10:Things to Remember
Chapter 11: Pega DataFlow Runtime Issues
Chapter 12:UseCases
Chapter 13:Pega Producer- Stream message through Data Set
Chapter 14: Pega consumer-Read Multiple Messages through Data Set
Chapter 15: Pega consumer-Read Single/Latest Message through Data Flow
Chapter 16:Pega consumer-Read Multiple Messages through Data Flow
Chapter17:Monitoring Data Flow Service through Agent at run time?
Conclusion
Apendix
ABOUT THE AUTHOR
N ikhil Garge is a Lead Pega Architect who has
graduated with distinction from Bangalore university
in Karnataka, India. He holds master’s degree in
Computer Science. Nikhil has worked as a consultant for the past 15 years on
many challenging projects across the globe. His expertise ranges from
software engineering, design, development, and architecture to Pega
applications and its various frameworks, such as Customer Process
Management (CPM), Smart Investigate (SI), Smart Dispute (SD), decisioning
and marketing. His roles and responsibilities include providing Pega
solutions to various complex problems that many companies in the industry
face today to help optimize their business processes and keep their portfolios
running smoothly and efficiently.
Nikhil has spent years travelling around the world, and has been to many
parts of Europe, Asia and North America before finally settling down in the
United States with his wife, Snehal, and his son, Aarush. Nikhil loves
spending his free time with family, exploring nature, hiking, and engaging in
late night philosophical discussions with his many friends.
DISCLAIMER
The publisher and the author are providing this book and its contents solely
based on industry experience . The publisher and the author disclaim all such
representations and warranties, including but not limited to warranties for a
particular purpose. In addition, the publisher and the author assume no
responsibility for errors, inaccuracies, omissions, or any other inconsistencies
herein.
The content of this book is for informational purposes only and is not
intended to be verified by authantic source such as Apache kafka, Pega
Systems. You understand that this book is not intended as a substitute for
consultation with a licensed practitioner. The purpose of this book is to put
forward the experience for building the concepts and step by step procedures
w.r.t Pega and kafka. The use of this book implies your acceptance of this
disclaimer.
The publisher and the author make no guarantees concerning the level of
success you may experience by following the advice and strategies contained
in this book, and you accept the risk that results will differ for each
individual. The testimonials and examples provided in this book are solely
based on industry experience and and no company or client information is
disclosed here.
PREFACE
The greatest compliment you can give an author of a technical book is
“This is the book I wish I had when I got started with this subject.”
This is the goal i set for myself when i started writing this book. We looked
back at our experience working in Pega technology, writing Kafka, running
Kafka in production, and helping many companies use Pega & Kafka to build
software architectures and manage their data pipelines and we asked
ourselves, “What are the most useful things we can share with new users to
take them from beginner to experts?” This book is a reflection of the work we
do every day: buid solutions using Pega, run Apache Kafka and help others
use it in the best ways.
We included what we believe you need to know in order to successfully run
Apache Kafka in production and build robust and performant Pega
integration on top of it.I have highlighted the popular use cases: stream-
processing applications, and large-scale data pipelines. This book also
focuses on making the book general and comprehensive enough so it will be
useful to anyone working with Pega & Kafka.It cover practical matters such
as how to install and configure Zookeeper,Kafka,building Pega- Kafka
integration and explore functioning of Kafka 's delightful architecture details:
replication factor,how data is stored in Kafka topic., Broker
details.Knowledge of Kafka's design and internals is not only fun read for
those interested in distributed systems, but it is also incredibly useful for
those who are seeking to make informed decisions when they deploy Pega
Kafka use case in production and design applications that use Kafka along
with Pega.The better you understand how Kafka and Pega works together, the
more you can make informed decisions regarding the many trade-offs that are
involved in engineering.
One of the problems in software engineering is that there is always more than
one way to do anything. Platforms such as Apache Kafka and Pega provide
plenty of flexibility, which is great for experts but makes for a steep learning
curve for beginners.In this book my focus on: how to stream events from
Pega application to Kafka, how to read events from kafka into Pega
application and process, how to create and monitor Dataflows real time and
what are key things to remember, How the data is stored into kafka through
dataset and how it is read from Kafka through dataset.
This book will give you confidence to stream and read events using Pega
technology. By end of this book, you will be expert in building Pega-Kafka
integration. Above all, we hope this book act as a springboard for a broader
conversation about how we, as software Pega engineers should work with
events.
ACKNOWLEDGEMENT
Iwould like to thank my wife Snehal and my son Aarush for their
continuous support through the long process of writing this book, as well as
my parents for lifelong encouragement. And many thanks to my mentors,
Project directors Shastri V Subramanyam, Shailendra Bade, Nikhil Daxini for
believing in me and giving opportunity to explore and help other teams to
configure and build Pega-Kafka Integration.
ABOUT THIS BOOK
Stream events to Kafka is commonly used in today’s information
technology world as data is flowing in and out through systems in various
industries like banking, healthcare, CRM, sales etc. Key factor of information
technology is data analytics, data cleansing, real time data monitoring etc.
This book explains the Zookeeper and Kafka concepts, architecture,
installation and setup, How messages are stored in Kafka, about Pega dataset
and Dataflow, how messages can be streamed efficiently into kafka and read
from Kafka through Pega BPM application. Also, it explains about the key
configuration required to make this integration successful.
WHO SHOULD READ THIS
BOOK?
Pega Stream Events In Action -Real time streaming with Kafka: The
Definitive Guide is written for software engineers who develop
applications using Pega that use Kafka’s APIs and who install, configure,
tune, and monitor Pega and Kafka in across environments. I also wrote the
book with data architects and data engineers in mind—those responsible for
designing and building an organization’s entire data infrastructure. Chapeters
1 and 2 are to build the cocept about Kafka and zookeeper functioning.Other
chapters content, are geared toward Pega developers and architects. Those
chapters assume that the reader is familiar with the basics of Pega, including
topics such as exception handling and concurrency. Other chapters, especially
chapters 3 and 4, assume the reader has some experience running windows
and some familiarity with storage and network configuration in Window. The
rest of the book discusses Kafka and software architectures in more general
terms and does not assume special knowledge.
Here are the detail roapmap.
