ZeroMQ: Messaging for Many Applications

762 Pages·2013·6.87 MB·English

by  Pieter Hintjens

ZeroMQ
Pieter Hintjens
Beijing • Cambridge • Farnham • Köln • Sebastopol • Tokyo
Download from Wow! eBook
To Noémie, Freeman, and Gregor.
Special Upgrade Offer
If you purchased this ebook directly from oreilly.com, you have the following
benefits:
DRM-free ebooks—use your ebooks across devices without restrictions or
limitations
Multiple formats—use on your laptop, tablet, or phone
Lifetime access, with free updates
Dropbox syncing—your files, anywhere
If you purchased this ebook from another retailer, you can upgrade your ebook to
take advantage of all these benefits for just $4.99. Click here to access your
ebook upgrade.
Please note that upgrade offers are not available from sample content.
Preface
ØMQ in a Hundred Words
ØMQ (also known as ZeroMQ, 0MQ, or zmq) looks like an embeddable
networking library, but acts like a concurrency framework. It gives you sockets
that carry atomic messages across various transports, like in-process, inter-
process, TCP, and multicast. You can connect sockets N-to-N with patterns like
fan-out, pub-sub, task distribution, and request-reply. It’s fast enough to be the
fabric for clustered products. Its asynchronous I/O model gives you scalable
multicore applications, built as asynchronous message-processing tasks. It has a
score of language APIs and runs on most operating systems. ØMQ is from
iMatix and is LGPLv3 open source.
The Zen of Zero
The Ø in ØMQ is all about trade-offs. On the one hand, this strange name lowers
ØMQ’s visibility on Google and Twitter. On the other hand, it annoys the heck
out of some Danish folk who write us things like “ØMG røtfl”, and “Ø is not a
funny-looking zero!” and “Rødgrød med Fløde!” (which is apparently an insult
that means “May your neighbours be the direct descendants of Grendel!”).
Seems like a fair trade.
Originally, the zero in ØMQ was meant to signify “zero broker” and (as close to)
“zero latency” (as possible). Since then, it has come to encompass different
goals: zero administration, zero cost, zero waste. More generally, “zero” refers to
the culture of minimalism that permeates the project. We add power by removing
complexity rather than by exposing new functionality.
How This Book Came to Be
In the summer of 2010, ØMQ was still a little-known niche library described by
its rather terse reference manual and a living but sparse wiki. Martin Sustrik and
I were sitting in the bar of the Hotel Kyjev in Bratislava plotting how to make
ØMQ more widely popular. Martin had written most of the ØMQ code, and I’d
put up the funding and organized the community. Over some Zlatý Bažant, we
agreed that ØMQ needed a new, simpler website and a basic guide for new users.
Martin collected some ideas for topics to explain. I’d never written a line of
ØMQ code before this, so it became a live learning documentary. As I worked
through simple examples to more complex ones, I tried to answer many of the
questions I’d seen on the mailing list. Because I’d been building large-scale
architectures for 30 years, there were a lot of problems I was keen to throw
ØMQ at. Amazingly, the results were mostly simple and elegant, even when
working in C. I felt a pure joy learning ØMQ and using it to solve real problems,
which brought me back to programming after a few years’ pause. And often, not
knowing how it was “supposed” to be done, we improved ØMQ as we went
along.
From the start, I wanted the guide to be a community project, so I put it onto
GitHub and let others contribute with pull requests. This was considered a
radical, even vulgar approach by some. We came to a division of labor: I’d do
the writing and make the original C examples, and others would help fix the text
and translate the examples into other languages.
This worked better than I dared hope. You can now find all the examples in
several languages, and many in a dozen languages. It’s a kind of programming
language Rosetta Stone, and a valuable outcome in itself. We set up a high score:
reach 80% translation and your language gets its own guide. PHP, Python, Lua,
and Haxe reached this goal. People asked for PDFs, and we created those.
People asked for ebooks, and got those. About a hundred people have
contributed to the guide to date.
The guide achieved its goal of popularizing ØMQ. The style pleases most and
annoys some, which is how it should be. In December 2010, my work on ØMQ
and the guide stopped, as I found myself going through late-stage cancer, heavy
surgery, and six months of chemotherapy. When I picked up work again in mid-
2011, it was to start using ØMQ in anger for one of the largest use-cases
imagineable: on the mobile phones and tablets of the world’s biggest electronics
company.
But the goal of the guide was, from the start, a printed book. So it was exciting
to get an email from Bill Lubanovic in January 2012, introducing me to his
editor, Andy Oram, at O’Reilly, suggesting a ØMQ book. “Of course!” I said.
Where do I sign? How much do I have to pay? Oh, I get money for this? All I
have to do is finish it?”
Of course, as soon as O’Reilly announced a ØMQ book, other publishers started
sending out emails to potential authors. You’ll probably see a rash of ØMQ
books coming out next year. That’s good. Our niche library has hit the
mainstream and deserves its six inches of shelf space. My apologies to the other
ØMQ authors. We’ve set the bar horribly high, and my advice is to make your
books complementary. Perhaps focus on a specific language, platform, or
pattern.
This is the magic and power of communities: be the first community in a space,
stay healthy, and you own that space for ever.
Audience
This book is written for professional programmers who want to learn how to
make the massively distributed software that will dominate the future of
computing. We assume you can read C code, because most of the examples here
are in C (even though ØMQ is used in many languages). We assume you care
about scale, because ØMQ solves that problem above all others. We assume you
need the best possible results with the least possible cost, because otherwise you
won’t appreciate the trade-offs that ØMQ makes. Other than that basic
background, we try to present all the concepts in networking and distributed
computing you will need to use ØMQ.