I have recently upgraded my Autobahn/Twisted libraries, and I am now using Crossbar as my WAMP router.
As part of the upgrade, my publishing code now is run by Crossbar as an ApplicationSession instance.
I use WAMP to support some real-time web applications, and it supports a fair number of users. This is a Pub/Sub model, also using AutobahnJS libraries for the client.
I am currently having a problem where my application is eventually no longer able to publish messages to the crossbar router for delivery. Clients are still able to connect and subscribe to topics no problem, they just stop getting updates due to the inability to publish.
If it makes any difference, this seems to always occur when I get up in between 20,000 to 30,000 client connections. I am publishing an average of 1500 messages per second.
The basic publishing code is as follows:
def on_message(self, headers, message):
STOMP message handler, when I get a message from the queue, dispatch it
to my users
self.log.debug(‘received a STOMP message %s’, message)
try: response = json.loads(message) destination = '%s.%s' % ( response['topic'], response['destination_id'] ) self.socket.publish( destination, json.dumps(response) ) except Exception as e: self.log.error(str(e))
str(e) is an empty string, I am working right now to gather any useful information from the exception,
I have made sure to follow all of the production Crossbar.io configuration suggestions, and the server itself is not under heavy load when this occurs.
Is this expected behavior, have I just hit some kind of known limit? I was hoping to support more throughput per instance, but I can certainly load balance if that is the answer. What can I do to prevent, or recover from this when it happens?