-
-
Notifications
You must be signed in to change notification settings - Fork 115
Description
Is your feature request related to a problem?
Currently, taskiq provides support for a ZeroMQ broker. However, the existing ZMQ implementation comes with certain architectural limitations, specifically regarding seamless asynchronous integration, dynamic worker discovery, and the complexity of managing socket lifecycles when nodes randomly disconnect or scale.
For high-performance, brokerless distributed systems, developers often need a lighter, more modern alternative to handle messaging topologies without the boilerplate and fragility of raw ZMQ socket management inside asyncio loops.
Describe the solution you'd like
I would like to propose adding NNG (Nanomsg Next Generation) broker support to taskiq, potentially utilizing Python bindings like pynng.
NNG is a modern, lightweight rewrite of Nanomsg that addresses many of the scalability and state-handling issues found in older ZMQ architectures. A dedicated NNG broker would provide:
- Clean, native
asynciointegration out-of-the-box. - Decoupled dialers and listeners, meaning far more robust reconnection patterns when workers crash or restart.
- A clean, lightweight environment for fast inter-process and network communications, ideal for modern microservice architectures.
Describe alternatives you've considered
- Using the existing ZeroMQ broker: This currently feels restrictive due to the boilerplate required to safely manage central proxy topologies and the lack of robust, automated reconnect handling in complex asynchronous contexts.
- Writing a custom broker from scratch: While possible, NNG is a standard, highly efficient protocol. Having official or community-supported integration in the
taskiqecosystem would greatly benefit developers building fast, lightweight asynchronous services.
Additional context
Adding an NNG backend (e.g., taskiq-nng) would be a huge win for developers looking for reliability and performance in their message queues without deploying heavy infrastructure like Redis, Kafka or NATS, keeping the technology stack lean and efficient.
Which component would this affect?
Broker