Table Of ContentCode Connected Volume 1
Pieter Hintjens
Code Connected Volume 1
Learning ZeroMQ
Professional Edition for C/C++
To my children Noémie, Freeman, and Gregor
Code Connected Volume 1
Copyright © 2010-2013 Pieter Hintjens
First Printing: January 6th, 2013
Licensed under CC-BY-SA-3.0
Published by iMatix Corporation
“ Code Connected” is a trademark of Pieter Hintjens. “ ØMQ” , “ ØMQ” ,
“ ZeroMQ” , and the ØMQ logo are trademarks of iMatix Corporation.
Edited by Andy Oram at O'Reilly
This text originated as “ ZeroMQ - The Guide” at http://zguide.zeromq.org,
with examples from the ØMQ community in C, C++, C#, Clojure, Delphi,
Erlang, Go, Haskell, Haxe, Java, Lisp, Lua, Node.js, Objective-C, Perl, PHP,
Python, Ruby, Scala, Tcl, and other languages.
ISBN 978-1481262651
Contents
7 Preface
7 ØMQ in a Hundred Words
7 How It Began
7 The Zen of Zero
8 How This Book Came To Be
9 Audience
9 Code Connected
11 Chapter 1. Basics
11 Fixing the World
12 Starting Assumptions
13 Getting the Examples
13 Ask and Ye Shall Receive
16 A Minor Note on Strings
17 Version Reporting
18 Getting the Message Out
22 Divide and Conquer
26 Programming with ØMQ
28 Why We Needed ØMQ
32 Socket Scalability
32 Upgrading from ØMQ v2.2 to ØMQ v3.2
34 Warning: Unstable Paradigms!
35 Chapter 2. Sockets and Patterns
35 The Socket API
40 Messaging Patterns
56 Handling Errors and ETERM
60 Handling Interrupt Signals
62 Detecting Memory Leaks
63 Multithreading with ØMQ
67 Signaling Between Threads (PAIR Sockets)
69 Node Coordination
72 Zero-Copy
73 Pub-Sub Message Envelopes
75 High-Water Marks
76 Missing Message Problem Solver
v
vi Code Connected Volume 1
79 Chapter 3. Advanced Request-Reply Patterns
79 The Request-Reply Mechanisms
83 Request-Reply Combinations
86 Exploring ROUTER Sockets
88 The Load Balancing Pattern
98 A High-Level API for ØMQ
107 The Asynchronous Client/Server Pattern
111 Worked Example: Inter-Broker Routing
135 Chapter 4. Reliable Request-Reply Patterns
135 What is “ Reliability” ?
136 Designing Reliability
137 Client-Side Reliability (Lazy Pirate Pattern)
141 Basic Reliable Queuing (Simple Pirate Pattern)
144 Robust Reliable Queuing (Paranoid Pirate Pattern)
152 Heartbeating
155 Contracts and Protocols
156 Service-Oriented Reliable Queuing (Majordomo Pattern)
178 Asynchronous Majordomo Pattern
184 Service Discovery
185 Idempotent Services
186 Disconnected Reliability (Titanic Pattern)
198 High-Availability Pair (Binary Star Pattern)
214 Brokerless Reliability (Freelance Pattern)
234 Conclusion
235 Chapter 5. Advanced Pub-Sub Patterns
235 Pros and Cons of Pub-Sub
237 Pub-Sub Tracing (Espresso Pattern)
239 Last Value Caching
244 Slow Subscriber Detection (Suicidal Snail Pattern)
247 High-Speed Subscribers (Black Box Pattern)
249 Reliable Pub-Sub (Clone Pattern)
311 Postface
311 Tales from Out There
312 How This Book Happened
314 Removing Friction
315 Licensing
316 Code Connected Volume 2
Preface
ØMQ in a Hundred Words
ØMQ (also known as ZeroMQ, ØMQ, or zmq) looks like an embeddable networking
library but acts like a concurrency framework. It gives you sockets that carry atomic
messages across various transports like in-process, inter-process, TCP, and multicast.
You can connect sockets N-to-N with patterns like fan-out, pub-sub, task distribution,
and request-reply. It’ s fast enough to be the fabric for clustered products. Its
asynchronous I/O model gives you scalable multicore applications, built as
asynchronous message-processing tasks. It has a score of language APIs and runs on
most operating systems. ØMQ is from iMatix1 and is LGPLv3 open source.
How It Began
We took a normal TCP socket, injected it with a mix of radioactive isotopes stolen
from a secret Soviet atomic research project, bombarded it with 1950-era cosmic rays,
and put it into the hands of a drug-addled comic book author with a badly-disguised
fetish for bulging muscles clad in spandex (see figure 1). Yes, ØMQ sockets are the
world-saving superheroes of the networking world.
The Zen of Zero
The Ø in ØMQ is all about
tradeoffs. On the one hand this
strange name lowers ØMQ’ s
visibility on Google and
Twitter. On the other hand it
annoys the heck out of some
Danish folk who write us things
like “ ØMG røtfl” , and “ Ø is not
a funny looking zero!” and
“ Rødgrød med Fløde!” , which is
apparently an insult that means
“ may your neighbours be the
direct descendants of Grendel!”
Seems like a fair trade.
1 http://www.imatix.com
7
8 Code Connected Volume 1 - Preface
Originally the zero in ØMQ was meant as “ zero broker” and (as close to) “ zero
latency” (as possible). Since then, it has come to encompass different goals: zero
administration, zero cost, zero waste. More generally, “ zero” refers to the culture of
minimalism that permeates the project. We add power by removing complexity rather
than by exposing new functionality.
How This Book Came To Be
In the summer of 2010, ØMQ was still a little-known niche library described by its
rather terse reference manual and a living but sparse wiki. Martin Sustrik and myself
were sitting in the bar of the Hotel Kyjev in Bratislava plotting how to make ØMQ
more widely popular. Martin had written most of the ØMQ code, and I’ d put up the
funding and organized the community. Over some Zlaty Bazants, we agreed that ØMQ
needed a new, simpler web site and a basic guide for new users.
Martin collected some ideas for topics to explain. I’ d never written a line of ØMQ
code before this, so it became a live learning documentary. As I worked through simple
examples to more complex ones, I tried to answer many of the questions I’ d seen on the
mailing list. Because I’ d been building large-scale architectures for 30 years, there were
a lot of problems at which I was keen to throw ØMQ. Amazingly the results were
mostly simple and elegant, even when working in C. I felt a pure joy learning ØMQ and
using it to solve real problems, which brought me back to programming after a few
years' pause. And often, not knowing how it was “ supposed” to be done, we improved
ØMQ as we went along.
From the start, I wanted the ØMQ guide to be a community project, so I put it onto
GitHub and let others contribute with pull requests. This was considered a radical, even
vulgar approach by some. We came to a division of labor: I’ d do the writing and make
the original C examples, and others would help fix the text and translate the examples
into other languages.
This worked better than I dared hope. You can now find all the examples in several
languages and many in a dozen languages. It’ s a kind of programming language Rosetta
stone and a valuable outcome in itself. We set up a high score: reach 80% translation
and your language got its own Guide. PHP, Python, Lua, and Haxe reached this goal.
People asked for PDFs, and we created those. People asked for ebooks, and got those.
About a hundred people contributed to the examples to date.
The book, in its on-line version “ The Guide” , achieved its goal of popularizing ØMQ.
The style pleases most and annoys some, which is how it should be. In December 2010,
my work on ØMQ and this guide stopped, as I found myself going through late-stage
cancer, heavy surgery, and six months of chemotherapy. When I picked up work again
in mid-2011, it was to start using ØMQ in anger for one of the largest use cases
imaginable: on the mobile phones and tablets of the world’ s biggest electronics
company.
But the goal of the ØMQ book was, from the start, a printed work. So it was exciting
to get an email from Bill Lubanovic in January 2012 introducing me to his editor, Andy
Code Connected Volume 1 - Preface 9
Oram, at O’ Reilly, suggesting a ØMQ book. Of course! Where do I sign? How much do
I have to pay? Oh, I get money for this? All I have to do is finish it?
Of course as soon as O’ Reilly announced a ØMQ book, other publishers started
sending out emails to potential authors. You’ ll probably see a rash of ØMQ books
coming out next year. That’ s good. Our niche library has hit the mainstream and
deserves its six inches of shelf space. My apologies to the other ØMQ authors. We’ ve set
the bar horribly high, and my advice is to make your books complementary. Perhaps
focus on a specific language, platform, or pattern.
This is the magic and power of communities: be the first community in a space, stay
healthy, and you own that space for ever.
Audience
This book is written for professional programmers who want to learn how to make
the massively distributed software that will dominate the future of computing. We
assume you can read C code, because most of the examples here are in C even though
ØMQ is used in many languages. We assume you care about scale, because ØMQ solves
that problem above all others. We assume you need the best possible results with the
least possible cost, because otherwise you won’ t appreciate the trade-offs that ØMQ
makes. Other than that basic background, we try to present all the concepts in
networking and distributed computing you will need to use ØMQ.
Code Connected
This book, “ Code Connected Volume 1” , is the first volume covering the theme of
how to build distributed software over ØMQ. In this book, you’ ll learn how to use the
ØMQ library. We’ ll cover the basics, the API, the different socket types and how they
work, reliability, and a host of patterns you can use in your applications. You’ ll get the
best results by working through the examples and text from start to end.
This is the Professional Edition for C/C++ developers. But whatever your
programming language, the patterns and techniques that Code Connected explains are
relevant to you. Get the examples online in C, C++, C#, Clojure, Delphi, Erlang, Go,
Haskell, Haxe, Java, Lisp, Lua, Node.js, Objective-C, Perl, PHP, Python, Ruby, Scala,
and Tcl.
