Hi Tobias et al,
It’s been a while since I was active on here. I’m happy to share that my engineering team has migrated from our previous custom pubsub transport layer (Tornado + redis) to Autobahn (AutobahnJS and AutobahnPython) and in the following months we’ve seen a noticeable improvement in performance. Our throughput is at least 3500% greater with Autobahn, measured in messages delivered per second without performance degradation.
I’ve taken notes on a handful of issues, questions, and feedback that I’d like to share in the near future. For now, I wanted to focus on a particular issue I recently discovered.
In the JS client, autobahn.Connection.close doesn’t immediately close the socket when there is an active session but instead sends a GOODBYE request in Session.leave. It only closes the socket once it gets a GOODBYE acknowledgement from the server (relevant code: https://github.com/tavendo/AutobahnJS/blob/v0.9.4/package/lib/connection.js#L351). This leads to several unexpected behaviors:
A) RPC callbacks from calls that were made before the close request could be invoked after connection.close() is called.
B) PubSub subscription callbacks can be invoked after connection.close() is called.
Because there’s no guarantee when the Autobahn server will process the client’s GOODBYE request relative to other messages intended for the client, the client may or may not observe either of these behaviors. Both of these are race conditions and have caused errors in our application. In our case, here’s what is happening:
We call connection.close() if the Autobahn client stops getting responses to its keepalive messages after 30 seconds. We implement these keepalives as RPCs though it would be nice if Autobahn implemented client/server keep-alive itself, as I’m sure other developers would benefit from it.
We then immediately create a new Connection object, call open on it, resubscribe to all relevant topics, and kick off a new round of RPC keepalives.
In some cases the original Autobahn client (the one we called close on in #1) regains connectivity and suddenly receives a whole bunch of RPC callbacks and subscription callbacks. That’s because the server hasn’t yet processed its GOODBYE request. This causes all sorts of havoc for our application since we now have two Autobahn connections (the “zombie” connection in #1 that temporarily came back from the dead then gets terminated by the server, and the new connection we created in #2).
A few questions/comments:
Would it be possible to prevent these behaviors in the JS library? It would be nice for the Session to invalidate any incoming messages if self._goodbye_sent is True.
At the very least, it would be helpful to mention these race conditions in the documentation so other developers don’t encounter errors from false assumptions like I did.
In the meantime, do you have suggestions on the best way to force a hard disconnect to eliminate the possibility of these behaviors? In this case, the client would close its socket immediately without waiting for a GOODBYE acknowledgement. My initial implementation involves calling connection.close() followed by connection._transport.close() but I’m wondering if there’s a better way.