autobahn websocket pubsub publish in a while loop

#1

I am using the autobahn websocket with the pubsub.

I have a service that gets data and collects it and puts it in a topic in Redis

I then want to listen for new messages and then publish to a topic with the autobahn pubsub. It works fine when the function exits, but when I put it in a for loop (constantly listen for new redis messages) It does not publish anything

I think its because this is blocking the reactor to actually publish the data. Iv tried looking into deferToThread but that doesnt work.

I was wondering does anyone know how i could basically go about publishing data in a while loop to the autobahn topics.

Thanks

0 Likes

#2

I am using the autobahn websocket with the pubsub.

I have a service that gets data and collects it and puts it in a topic
in Redis

I then want to listen for new messages and then publish to a topic with
the autobahn pubsub. It works fine when the function exits, but when I
put it in a for loop (constantly listen for new redis messages) It does
not publish anything

I think its because this is blocking the reactor to actually publish the

Yep, that's likely the cause of why it's not working. Don't block the reactor.

data. Iv tried looking into deferToThread but that doesnt work.

That should work, if you also to callFromThread for doing the dispatch/publish of the event

http://twistedmatrix.com/documents/13.2.0/core/howto/threading.html

An alternative would be to look for a non-blocking Twisted style client library for Redis.

/Tobias

···

Am 19.12.2013 23:22, schrieb Aditya Joshi:

0 Likes

#3

Thanks for the advise,

I am still having trouble with it.

Can you write a small snippet like:

def infinitLoop():

while True:

self.publish(“a”,“asdf”)

and how to use that ^ and not block the reactor

Thanks!

···

On Thursday, December 19, 2013 2:25:43 PM UTC-8, Tobias Oberstein wrote:

Am 19.12.2013 23:22, schrieb Aditya Joshi:

I am using the autobahn websocket with the pubsub.

I have a service that gets data and collects it and puts it in a topic

in Redis

I then want to listen for new messages and then publish to a topic with

the autobahn pubsub. It works fine when the function exits, but when I

put it in a for loop (constantly listen for new redis messages) It does

not publish anything

I think its because this is blocking the reactor to actually publish the

Yep, that’s likely the cause of why it’s not working. Don’t block the
reactor.

data. Iv tried looking into deferToThread but that doesnt work.

That should work, if you also to callFromThread for doing the
dispatch/publish of the event

http://twistedmatrix.com/documents/13.2.0/core/howto/threading.html

An alternative would be to look for a non-blocking Twisted style client
library for Redis.

/Tobias

0 Likes

#4

actually :
def infinitLoop():

while True:

msg = redis.pubsub.listen() <-- blocking

self.publish(“a”,msg)

···

On Thursday, December 19, 2013 2:34:23 PM UTC-8, Aditya Joshi wrote:

Thanks for the advise,

I am still having trouble with it.

Can you write a small snippet like:

def infinitLoop():

while True:

self.publish(“a”,“asdf”)

and how to use that ^ and not block the reactor

Thanks!

On Thursday, December 19, 2013 2:25:43 PM UTC-8, Tobias Oberstein wrote:

Am 19.12.2013 23:22, schrieb Aditya Joshi:

I am using the autobahn websocket with the pubsub.

I have a service that gets data and collects it and puts it in a topic

in Redis

I then want to listen for new messages and then publish to a topic with

the autobahn pubsub. It works fine when the function exits, but when I

put it in a for loop (constantly listen for new redis messages) It does

not publish anything

I think its because this is blocking the reactor to actually publish the

Yep, that’s likely the cause of why it’s not working. Don’t block the
reactor.

data. Iv tried looking into deferToThread but that doesnt work.

That should work, if you also to callFromThread for doing the
dispatch/publish of the event

http://twistedmatrix.com/documents/13.2.0/core/howto/threading.html

An alternative would be to look for a non-blocking Twisted style client
library for Redis.

/Tobias

0 Likes

#5

Try:

reactor.callFromThread(self.publish, "a", msg)

···

Am 19.12.2013 23:37, schrieb Aditya Joshi:

actually :
def infinitLoop():
     while True:
       msg = redis.pubsub.listen() <-- blocking
         self.publish("a",msg)

0 Likes

#6

Thanks so much.

Worked like a charm :slight_smile:

···

On Thursday, December 19, 2013 2:42:59 PM UTC-8, Tobias Oberstein wrote:

Am 19.12.2013 23:37, schrieb Aditya Joshi:

actually :

def infinitLoop():

 while True:
   msg = redis.pubsub.listen() <-- blocking
     self.publish("a",msg)

Try:

reactor.callFromThread(self.publish, “a”, msg)

0 Likes