Testing PubSub

#1

I have an Autobahn WAMP server which uses PubSub very heavily.

I’ve developed a simple testing script: It starts up the server, and two clients: One to call RPCs and verify the output, the other to receive and verify the PubSubs. (All three on the same reactor.)

As the server gets more and more complicated and sending more and more PubSubs, I’ve started encountering a problem: Some of the PubSubs are received when the next RPC call is made, causing the tests to fail. To combat this, I’ve modified the tester to send several junk RPC calls after the one intended for testing, giving Twisted the chance to return to the event loop and send/receive the PubSubs.

However, this is just an ugly hack, and is in no way an ideal solution. Any ideas for a better and more permanent fix for this?

0 Likes

#2

Hi Theron,

sorry for late reply ..

I have an Autobahn WAMP server which uses PubSub very heavily.

I've developed a simple testing script: It starts up the server, and
two clients: One to call RPCs and verify the output, the other to
receive and verify the PubSubs. (All three on the same reactor.)

Could you put up a test script that reproduces the problem somewhere .. I will have a deep look into, since should there be an issue, this is bad and needs to be addressed ASAP.

/Tobias

As the server gets more and more complicated and sending more and more
PubSubs, I've started encountering a problem: Some of the PubSubs are
received when the *next* RPC call is made, causing the tests to fail.
  To combat this, I've modified the tester to send several junk RPC
calls after the one intended for testing, giving Twisted the chance to
return to the event loop and send/receive the PubSubs.

Also: please report your exact environment: OS, Py, Twisted (including which reactor you are actually running on), Autobahn for both client and server as well as what networks sits in between (loopback, LAN ..)

However, this is just an ugly hack, and is in no way an ideal solution.
  Any ideas for a better and more permanent fix for this?

If I can verify the issue, we'll address it with high-prio. This must not happen. We won't give in to any hacks as above ..

···

Am 28.09.2013 20:39, schrieb Theron Luhn:

--
You received this message because you are subscribed to the Google
Groups "Autobahn" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to autobahnws+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

0 Likes

#3

Actually, I found the problem this morning. I just hadn’t read the docs carefully enough: WampProtocol.dispatch() returns a Deferred, which I didn’t realize, so I wasn’t yielding it to @inlineCallbacks. So RPC call would return and the dispatch would send whenever the event loop got around to it, which, in the case of many dispatches being sent, was too late.

When I get a chance I’ll refactor the code to do fix this and let you know how it goes.

Sorry for the scare :slight_smile:

···

— Theron

On Fri, Oct 4, 2013 at 2:20 PM, Tobias Oberstein tobias.o...@gmail.com wrote:

Hi Theron,

sorry for late reply …

Am 28.09.2013 20:39, schrieb Theron Luhn:

I have an Autobahn WAMP server which uses PubSub very heavily.

I’ve developed a simple testing script: It starts up the server, and

two clients: One to call RPCs and verify the output, the other to

receive and verify the PubSubs. (All three on the same reactor.)

Could you put up a test script that reproduces the problem somewhere … I will have a deep look into, since should there be an issue, this is bad and needs to be addressed ASAP.

/Tobias

As the server gets more and more complicated and sending more and more

PubSubs, I’ve started encountering a problem: Some of the PubSubs are

received when the next RPC call is made, causing the tests to fail.

To combat this, I’ve modified the tester to send several junk RPC

calls after the one intended for testing, giving Twisted the chance to

return to the event loop and send/receive the PubSubs.

Also: please report your exact environment: OS, Py, Twisted (including which reactor you are actually running on), Autobahn for both client and server as well as what networks sits in between (loopback, LAN …)

However, this is just an ugly hack, and is in no way an ideal solution.

Any ideas for a better and more permanent fix for this?

If I can verify the issue, we’ll address it with high-prio. This must not happen. We won’t give in to any hacks as above …

You received this message because you are subscribed to the Google

Groups “Autobahn” group.

To unsubscribe from this group and stop receiving emails from it, send

an email to autobahnws+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

0 Likes

#4

Actually, I found the problem this morning. I just hadn't read the docs
carefully enough: WampProtocol.dispatch() returns a Deferred, which I
didn't realize, so I wasn't yielding it to @inlineCallbacks. So RPC
call would return and the dispatch would send whenever the event loop
got around to it, which, in the case of many dispatches being sent, was
too late.

When I get a chance I'll refactor the code to do fix this and let you
know how it goes.

Sorry for the scare :slight_smile:

Puh;) No problem!

/Tobias

···

Am 04.10.2013 22:18, schrieb Theron Luhn:

� Theron

On Fri, Oct 4, 2013 at 2:20 PM, Tobias Oberstein > <tobias.o...@gmail.com <mailto:tobias.o...@gmail.com>> wrote:

    Hi Theron,

    sorry for late reply ..

    Am 28.09.2013 20:39, schrieb Theron Luhn:

        I have an Autobahn WAMP server which uses PubSub very heavily.

        I've developed a simple testing script: It starts up the
        server, and
        two clients: One to call RPCs and verify the output, the other to
        receive and verify the PubSubs. (All three on the same reactor.)

    Could you put up a test script that reproduces the problem somewhere
    .. I will have a deep look into, since should there be an issue,
    this is bad and needs to be addressed ASAP.

    /Tobias

        As the server gets more and more complicated and sending more
        and more
        PubSubs, I've started encountering a problem: Some of the
        PubSubs are
        received when the *next* RPC call is made, causing the tests to
        fail.

           To combat this, I've modified the tester to send several junk RPC
        calls after the one intended for testing, giving Twisted the
        chance to
        return to the event loop and send/receive the PubSubs.

    Also: please report your exact environment: OS, Py, Twisted
    (including which reactor you are actually running on), Autobahn for
    both client and server as well as what networks sits in between
    (loopback, LAN ..)

        However, this is just an ugly hack, and is in no way an ideal
        solution.
           Any ideas for a better and more permanent fix for this?

    If I can verify the issue, we'll address it with high-prio. This
    must not happen. We won't give in to any hacks as above ..

        --
        You received this message because you are subscribed to the Google
        Groups "Autobahn" group.
        To unsubscribe from this group and stop receiving emails from
        it, send

        an email to autobahnws+unsubscribe@__googlegroups.com
        <mailto:autobahnws%2...@googlegroups.com>.
        For more options, visit
        https://groups.google.com/__groups/opt_out
        <https://groups.google.com/groups/opt_out>.

0 Likes

#5

A note: It appears dispatch() returns None (rather than Deferred) in Autobahn v0.5.14 (which, as of Oct 4, 2013, is the latest version on PyPI/pip) and earlier. It does return Deferred in v0.6.2, which is the current version of the source code on GitHub.

Any chance of a PyPI update soon? Installing from the source isn’t difficult, but pip is just so convenient :slight_smile:

···

— Theron

On Fri, Oct 4, 2013 at 3:20 PM, Tobias Oberstein tobias.o...@gmail.com wrote:

Am 04.10.2013 22:18, schrieb Theron Luhn:

Actually, I found the problem this morning. I just hadn’t read the docs

carefully enough: WampProtocol.dispatch() returns a Deferred, which I

didn’t realize, so I wasn’t yielding it to @inlineCallbacks. So RPC

call would return and the dispatch would send whenever the event loop

got around to it, which, in the case of many dispatches being sent, was

too late.

When I get a chance I’ll refactor the code to do fix this and let you

know how it goes.

Sorry for the scare :slight_smile:

Puh;) No problem!

/Tobias

— Theron

On Fri, Oct 4, 2013 at 2:20 PM, Tobias Oberstein

<tobias.o...@gmail.com mailto:tobias.oberstein@gmail.com> wrote:

Hi Theron,



sorry for late reply ..



Am 28.09.2013 20:39, schrieb Theron Luhn:



    I have an Autobahn WAMP server which uses PubSub very heavily.



    I've developed a simple testing script:  It starts up the

    server, and

    two clients:  One to call RPCs and verify the output, the other to

    receive and verify the PubSubs.  (All three on the same reactor.)





Could you put up a test script that reproduces the problem somewhere

.. I will have a deep look into, since should there be an issue,

this is bad and needs to be addressed ASAP.



/Tobias





    As the server gets more and more complicated and sending more

    and more

    PubSubs, I've started encountering a problem:  Some of the

    PubSubs are

    received when the *next* RPC call is made, causing the tests to

    fail.



       To combat this, I've modified the tester to send several junk RPC

    calls after the one intended for testing, giving Twisted the

    chance to

    return to the event loop and send/receive the PubSubs.





Also: please report your exact environment: OS, Py, Twisted

(including which reactor you are actually running on), Autobahn for

both client and server as well as what networks sits in between

(loopback, LAN ..)







    However, this is just an ugly hack, and is in no way an ideal

    solution.

       Any ideas for a better and more permanent fix for this?





If I can verify the issue, we'll address it with high-prio. This

must not happen. We won't give in to any hacks as above ..





    --

    You received this message because you are subscribed to the Google

    Groups "Autobahn" group.

    To unsubscribe from this group and stop receiving emails from

    it, send

an email to autobahnws+unsubscribe@__googlegroups.com

    <mailto:autobahnws%2Bunsu...@googlegroups.com>.

    For more options, visit

    [https://groups.google.com/__groups/opt_out](https://groups.google.com/__groups/opt_out)

    <[https://groups.google.com/groups/opt_out](https://groups.google.com/groups/opt_out)>.

You received this message because you are subscribed to a topic in the Google Groups “Autobahn” group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/autobahnws/CNwBItHW6cw/unsubscribe.

To unsubscribe from this group and all its topics, send an email to autobahnws+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

0 Likes

#6

Yep: I am preparing to release over weekend …

···

— Theron

On Fri, Oct 4, 2013 at 3:20 PM, Tobias Oberstein tobias.o...@gmail.com wrote:

Am 04.10.2013 22:18, schrieb Theron Luhn:

Actually, I found the problem this morning. I just hadn’t read the docs

carefully enough: WampProtocol.dispatch() returns a Deferred, which I

didn’t realize, so I wasn’t yielding it to @inlineCallbacks. So RPC

call would return and the dispatch would send whenever the event loop

got around to it, which, in the case of many dispatches being sent, was

too late.

When I get a chance I’ll refactor the code to do fix this and let you

know how it goes.

Sorry for the scare :slight_smile:

Puh;) No problem!

/Tobias

— Theron

On Fri, Oct 4, 2013 at 2:20 PM, Tobias Oberstein

<tobias.o...@gmail.com mailto:tobias.oberstein@gmail.com> wrote:

Hi Theron,



sorry for late reply ..



Am 28.09.2013 20:39, schrieb Theron Luhn:



    I have an Autobahn WAMP server which uses PubSub very heavily.



    I've developed a simple testing script:  It starts up the

    server, and

    two clients:  One to call RPCs and verify the output, the other to

    receive and verify the PubSubs.  (All three on the same reactor.)





Could you put up a test script that reproduces the problem somewhere

.. I will have a deep look into, since should there be an issue,

this is bad and needs to be addressed ASAP.



/Tobias





    As the server gets more and more complicated and sending more

    and more

    PubSubs, I've started encountering a problem:  Some of the

    PubSubs are

    received when the *next* RPC call is made, causing the tests to

    fail.



       To combat this, I've modified the tester to send several junk RPC

    calls after the one intended for testing, giving Twisted the

    chance to

    return to the event loop and send/receive the PubSubs.





Also: please report your exact environment: OS, Py, Twisted

(including which reactor you are actually running on), Autobahn for

both client and server as well as what networks sits in between

(loopback, LAN ..)







    However, this is just an ugly hack, and is in no way an ideal

    solution.

       Any ideas for a better and more permanent fix for this?





If I can verify the issue, we'll address it with high-prio. This

must not happen. We won't give in to any hacks as above ..





    --

    You received this message because you are subscribed to the Google

    Groups "Autobahn" group.

    To unsubscribe from this group and stop receiving emails from

    it, send

an email to autobahnws+unsubscribe@__googlegroups.com

    <mailto:autobahnws%2Bunsu...@googlegroups.com>.

    For more options, visit

    [https://groups.google.com/__groups/opt_out](https://groups.google.com/__groups/opt_out)

    <[https://groups.google.com/groups/opt_out](https://groups.google.com/groups/opt_out)>.

You received this message because you are subscribed to a topic in the Google Groups “Autobahn” group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/autobahnws/CNwBItHW6cw/unsubscribe.

To unsubscribe from this group and all its topics, send an email to autobahnws+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

0 Likes