Distribute pubsub among many processes

#1

I’ve been trying CrossbarIO and I could not see clearly how to solve this specific scenario:

  1. a twisted based network process that should receive some notifications such as read the database again, stop one of its protocols. It produces some notifications.

  2. a WSGI application that could send messages to 1 (this should be implemented with corssbar WSGI host service).

  3. pubsub that send the last “known” value upon subscriptions. This will interact directly with browser. WAMP.

I see that 2 to 3 could use the Push Service. But I don’t see how to spread notifications from 1 to 3. RPC?

If I had more processes like 1, that are twisted based, though they must run in a different address space, for some reason. Should I have some sort of proxy in 3?

Thanks.

0 Likes

#2

I've been trying CrossbarIO and I could not see clearly how to solve
this specific scenario:

1) a twisted based network process that should receive some
notifications such as read the database again, stop one of its
protocols. It produces some notifications.

You can run an AutobahnPython based WAMP client component together with the other network stuff you want to do in 1 process. The WAMP client component can connect to Crossbar.io, can be accessed from your "other network stuff" and publish regular WAMP events.

As a bonus, you can have Crossbar.io start and monitor above process as a "guest worker".

2) a WSGI application that could send messages to 1 (this should be
implemented with corssbar WSGI host service).

The Crossbar.io WSGI service (http://crossbar.io/docs/WSGI-Host-Service/) is a convenience mechanism to start and monitor a regular WSGI app from and under Crossbar.io.

However, this service (currently) does not expose WAMP client functionality.

You can still achive this today. Here is how.

Using AutobahnPython and Twisted, you write a program that runs a WAMP client connecting to the router plus a Twisted WSGI container. The only thing to take care of is when publishing from WSGI code.

The WAMP component will run on the main thread while the WSGI code will run on a background thread, and hence you will need to use callFromThread.

http://twistedmatrix.com/documents/current/api/twisted.internet.interfaces.IReactorThreads.callFromThread.html

When wrapping a regular WSGI application (which is blocking since it is WSGI), and running this WSGI under Twisted, Twisted will run the WSGI code on a background worker thread pool.

Depending on what you actually want to achieve with WSGI, you can also go fully async, e.g. using Klein, a Flask clone:

http://tavendo.com/blog/post/mixing-web-and-wamp-code-with-twisted-klein/

···

Am 25.11.2014 03:22, schrieb Nahuel Defossé:

3) pubsub that send the last "known" value upon subscriptions. This will
interact directly with browser. WAMP.

I see that 2 to 3 could use the Push Service. But I don't see how to
spread notifications from 1 to 3. RPC?
If I had more processes like 1, that are twisted based, though they must
run in a different address space, for some reason. Should I have some
sort of proxy in 3?

Thanks.

--
You received this message because you are subscribed to the Google
Groups "Crossbar" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to crossbario+...@googlegroups.com
<mailto:crossbario+...@googlegroups.com>.
To post to this group, send email to cross...@googlegroups.com
<mailto:cross...@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/crossbario/0f380eab-ccbc-42cb-9243-fbd77de64e5f%40googlegroups.com
<https://groups.google.com/d/msgid/crossbario/0f380eab-ccbc-42cb-9243-fbd77de64e5f%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

0 Likes