As far as I was aware there was a patch like that available for the ZMQ 3.0 but was nixed for 3.1, same with a patch to deal with the issue of 1000 clients on an XREP socket, where the first 999 disappear and thus don't need to have their messages processed and should just be dropped on the XREP socket... that was added in 3.0 and then reverted in 3.1.
I've worked around a lot of issues in ZeroMQ with retries, heartbeats, and stuff like that, but it just feels kludgy and like I am writing code that should be part of the library. I've looked at adding the functionality myself from the old patchsets that were available and neither were straight forward to implement. The code (at least ZeroMQ) is somewhat of a mess and difficult to follow/understand and I am saying that as a C++ programmer working on large enterprise applications.
Yep. The patches were reverted because they've broker wire-level compatibility with older versions of 0MQ. The good news is that I protocol versioning will be available in next release of crossroads.io and thus the patches like that can be applied without breaking backward compatibility (the library will just speak two different protocol versions).
As for the code quality, I would say the code is complex to the extent where it's almost impossible to maintain and improve it. (In case of need here's the overview of the architecture: http://www.aosabook.org/en/zeromq.html) However, it's not a mess in sense of being lousy. I've spent literally months of my time cleaning it up. Although it may have got worse since I've started working on the fork.
I can understand the need for them to be reverted, although I honestly think that the patches provided functionality that should have already existed. Worst case the patched could have stayed and they could have been made available through certain flags (much like the HWM, LWM, and others).
I wouldn't say that the code quality is lousy, au contraire I think overall the code quality is very good, just very difficult to figure out where everything goes. I found it difficult to "brain" map it such that I could get proficient at following what was going on while reading different sections of the code.
Is there any effort under way in crossroads to make the notification level triggered and not just edge triggered? For example sticking ZeroMQ into a libev loop is a pain because if two messages arrive at the same time I only get notified once and I have to loop through and zmq_read(), which if the ZeroMQ backend is sending really fast means I now starve all my other sockets!
As for the options: Yes. There's patch that supports protocol versioning (version=socket option) already applied to crossroads.io. That in turn makes it possible to improve the protocol without breaking backward compatibility.
As for code complexity: It's hard to brain-map it for me as well, which is pretty alarming given that I am the author :) I am working on major rewrite of the codebase (simplification of the threading model) but that'll take some time to finish.
As for level-triggering, that's a problem with POSIX, not 0MQ. There's no way in POSIX to simulate a file descriptor in the user space. The closest you can get is eventfd() which is a) Linux-specific b) doesn't allow to signal both !POLLIN and !POLLOUT at the same time :(
I've worked around a lot of issues in ZeroMQ with retries, heartbeats, and stuff like that, but it just feels kludgy and like I am writing code that should be part of the library. I've looked at adding the functionality myself from the old patchsets that were available and neither were straight forward to implement. The code (at least ZeroMQ) is somewhat of a mess and difficult to follow/understand and I am saying that as a C++ programmer working on large enterprise applications.