Object changes notification: how to send message to the client right after subscription?

#1

Hi.

I am trying to implement some object changes notification progated to the client.

So, there is object on server side with a state defined by the values of some properties.

Web browser UI wants to fetch object’s current state and after that subscribe to changes.

Currently, I’m trying to do it like that:

  1. Client subscribes to the [changes] topic of URI identifying specific object, e.g. http://objects/{id}/changes

  2. First message sent to the client have to be whole object state (big dict with all properties and values);

  3. Further messages can contain small dicts with values changed only;

Questions

···

=========

  1. Is my way of solving this kind of task (getting changes notifications) using WAMP correct?

Or you can advice some more convenient pattern / scheme for solution?

  1. The problem is I cannot sent first message from Subscription Handler (defined by @wamp.exportSub) directly - by WampServerProtocol.dispatch(),

because its responsibility to return boolean value = decision to subscribe this client or not.

Currently, I am scheduling sending first message 1 second by reactor.callLater(1, …) - it works, but looks a little weird…

and I don’t want clients to wait based on some delay constant taken “from the sky”.

Is there a more convenient/right way to reliably dispatch a message(s) from server to the specific client, right after subscription?

Thanks,

Alex.

0 Likes

#2

Hi.

I am trying to implement some object changes notification progated to
the client.

So, there is object on server side with a state defined by the values of
some properties.
Web browser UI wants to fetch object's current state and after that
subscribe to changes.

Currently, I'm trying to do it like that:

1. Client subscribes to the [changes] topic of URI identifying specific
object, e.g. http://objects/{id}/changes

2. First message sent to the client have to be whole object state (big
dict with all properties and values);

3. Further messages can contain small dicts with values changed only;

Questions

1. Is my way of solving this kind of task (getting changes
notifications) using WAMP correct?
     Or you can advice some more convenient pattern / scheme for solution?

A recurring pattern we use in applications is:

Have RPC endpoints for get (all current attributes) and modify (attributes of) a domain object. Plus delete and create object.

Within the modify implementation, dispatch events with delta information (onModify).

When a client connects, it subscribes to onCreate, onModify, onDelete topics, and retrieves initial object.

2. The problem is I cannot sent first message from Subscription Handler
(defined by @wamp.exportSub) directly - by WampServerProtocol.dispatch(),
     because its responsibility to return boolean value = decision to
subscribe this client or not.
     Currently, I am scheduling sending first message 1 second by
reactor.callLater(1, ...) - it works, but looks a little weird..
     and I don't want clients to wait based on some delay constant taken
"from the sky".

     Is there a more convenient/right way to reliably dispatch a
message(s) from server to the specific client, right after subscription?

To dispatch an event to a single client you can restrict the receivers (subscribed) by setting eligible to that single client (WAMP sessionID).

The custom subscription handler is intended only for implementing custom logic that decides if the client should actually be subscribed or not.

You cannot (currently) dispatch to that (the subscribing) client before the custom subscription handler has returned.

We could either allow the custom sub/pub handlers to not only return a truth value, but instead return a callable which is then called after the internal machinery has been setup, and hence you can dispatch therein.

Or we could have onAfterSubAllowed, onAfterSubDenied, .. hooks. The latter would match the ones we already have for RPC (onAfterCallSuccess etc)

/Tobias

···

Am 20.06.2013 13:42, schrieb Alexander Petrov:

Thanks,
Alex.

--
You received this message because you are subscribed to the Google
Groups "Autobahn" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to autobahnws+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

0 Likes

#3

A recurring pattern we use in applications is:

Have RPC endpoints for get (all current attributes) and modify
(attributes of) a domain object. Plus delete and create object.

Within the modify implementation, dispatch events with delta information
(onModify).

When a client connects, it subscribes to onCreate, onModify, onDelete
topics, and retrieves initial object.

Yes. I also used such approach a little. It has API simplicity as advantage, but some additional processing logic of object state have to be done at client side if more strict “consistency” between server and clients is needed. I mean following.

After subscription to onModify but before initial state have arrived, there can be [modify] events already passed to the client. Some of them contain deltas before initial state to be retrieved and they need to be ignored, and some of them have to be applied to the object after successful RPC-get. This can be tracked by using some revision/sequence numbers inside object (something a like mechanics of _changes stream in CouchDB if you’re familiar).

If modifications are happening rarely, or they are not very “valuable” or “important” then we can ignore/loose some of them… I am suppose probability of situation described above is rather low. So, in previous project I simply decided to do not think about it (and used RPC + Pub/Sub api you described). Now I want to implement some hardware monitoring solution, where some parameters values changes are not very valuable (can be lost “a little”) and for some I want to make sure if client reliably connected then it will recieve all notifications and will have a state consistent with hardware monitored.

So, I can implement some buffers/queues of changes inside client, and after recieving object state by RPC synchronize it with changes recieved.

Or I thought if I can use some of the properties of WAMP itself - may be there will be no need in “buffers” or additional synchronization.

I assumed that WAMP guarantees that order of messages published to the topic is the same as the order of messages recieved by client subscribed (some of them can be lost, e.g. due to connection problem, but them cannot be mixed/permuted).

Q: BTW, can you confirm that am I right in that about WAMP / Autobahn impl.?

So, currently I am trying to implement / probe such API:

RPC: get / modify / create / delete

Topics to subscribe:

  • onCreate

  • onChange (and there will be at least 2 types of changes: “full”, “delta”)

delete notification can be separate topic onDelete, or it can be implemented as a 3rd type of change event;

It would be interesting for me if you try to share you thoughts on this approach.

Anyway, thanks for previous answer and for the software. :slight_smile:

To dispatch an event to a single client you can restrict the receivers
(subscribed) by setting eligible to that single client (WAMP sessionID).

The custom subscription handler is intended only for implementing custom
logic that decides if the client should actually be subscribed or not.

You cannot (currently) dispatch to that (the subscribing) client before
the custom subscription handler has returned.

We could either allow the custom sub/pub handlers to not only return a
truth value, but instead return a callable which is then called after
the internal machinery has been setup, and hence you can dispatch therein.

Or we could have onAfterSubAllowed, onAfterSubDenied, … hooks. The
latter would match the ones we already have for RPC (onAfterCallSuccess etc)

Yes, I have used using [eligible] parameter.

First approach with callable returned IMHO seems more Twisted :slight_smile:

But personally I like the consistency with other parts of the API.

It would be good to have some kind onAfterSubAllowed, … .

BTW, looking now into sources once more, there is not exposed to the API documented method WampServerFactory.onClientSubscribed - it seems that I can use it for the purpose (dispatching message right after subscription), am I? Will test tomorrow. But anyway from design point of view it will be more correct to have sub event handlers on the protocol level.

Thank you,

Alex.

0 Likes

#4

If modifications are happening rarely

Sorry for my english)), thought wasn’t expressed correctly.

I meaned there if client needs to observe some parameter value much more rarely, than this parameter changes (frequency of observation << frequency of changes)…

In such case we can simply lost some change, and wait for next fresh value…

0 Likes

#5

Hello Alex,

    A recurring pattern we use in applications is:

    Have RPC endpoints for get (all current attributes) and modify
    (attributes of) a domain object. Plus delete and create object.

    Within the modify implementation, dispatch events with delta
    information
    (onModify).

    When a client connects, it subscribes to onCreate, onModify, onDelete
    topics, and retrieves initial object.

Yes. I also used such approach a little. It has API simplicity as
advantage, but some additional processing logic of object state have to
be done at client side if more strict "consistency" between server and
clients is needed. I mean following.

After subscription to onModify but before initial state have arrived,
there can be [modify] events already passed to the client. Some of them

You are correct, this can happen .. though it's probably rare.

It is undesirable, I agree. I have no simple solution. We happen to live with this "small probabability" for our own apps ..

contain deltas before initial state to be retrieved and they need to be
ignored, and some of them have to be applied to the object after
successful RPC-get. This can be tracked by using some revision/sequence
numbers inside object (something a like mechanics of _changes stream in
CouchDB if you're familiar).

Yes, that would work. A client subscribes. The events contain an increasing "object change sequence" number. The client issues the inital "getObjectState" RPC, which also contains the sequence number. Now the client can just ignore all events with sequence number smaller than the one that was returned from the RPC.

If modifications are happening rarely, or they are not very "valuable"
or "important" then we can ignore/loose some of them... I am suppose
probability of situation described above is rather low. So, in previous
project I simply decided to do not think about it (and used RPC +
Pub/Sub api you described). Now I want to implement some hardware
monitoring solution, where some parameters values changes are not very
valuable (can be lost "a little") and for some I want to make sure if
client reliably connected then it will recieve all notifications and
will have a state consistent with hardware monitored.

So, I can implement some buffers/queues of changes inside client, and
after recieving object state by RPC synchronize it with changes recieved.
Or I thought if I can use some of the properties of WAMP itself - may be
there will be no need in "buffers" or additional synchronization.

I assumed that WAMP guarantees that order of messages published to the
topic is the same as the order of messages recieved by client subscribed
(some of them can be lost, e.g. due to connection problem, but them
cannot be mixed/permuted).

The order of events on some topic received by a given client is preserved for all events published to that topic _from a given publisher_.

When 2 publishers publish events to the same topic:

Publisher 1: P1E1, P1E2, P1E3, ..
Publisher 2: P2E1, P2E2, P2E3, ..

a subscriber might see

P1E1, P1E2, P1E3, P2E1, P2E2, P2E3, ..
or
P1E1, P2E1, P1E2, P1E3, P2E2, P2E3, ..

but never

P1E2, P1E1, P1E3, P2E1, P2E2, P2E3, ..

Further, since WAMP runs over WS which runs over TCP, if a subscriber
has received

P1E3

then all previous events will have arrived also (no loss), since the transport is reliable and ordered.

In general, I'd say the WAMP spec and the Autobahn docs both clearly lack in exactly describing these things.

This also applies to ordering between "publish" and "RPC call". The RPC returns are asynch by nature, so ordering here is a different matter.

Q: BTW, can you confirm that am I right in that about WAMP / Autobahn impl.?

So, currently I am trying to implement / probe such API:
RPC: get / modify / create / delete
Topics to subscribe:
  - onCreate
  - onChange (and there will be at least 2 types of changes: "full",
"delta")
delete notification can be separate topic onDelete, or it can be
implemented as a 3rd type of change event;

Sorry, could you explain the sequence of actions/events a freshly connected client will do?

It would be interesting for me if you try to share you thoughts on this
approach.

Anyway, thanks for previous answer and for the software. :slight_smile:

    To dispatch an event to a single client you can restrict the receivers
    (subscribed) by setting eligible to that single client (WAMP
    sessionID).

    The custom subscription handler is intended only for implementing
    custom
    logic that decides if the client should actually be subscribed or not.

    You cannot (currently) dispatch to that (the subscribing) client before
    the custom subscription handler has returned.

    We could either allow the custom sub/pub handlers to not only return a
    truth value, but instead return a callable which is then called after
    the internal machinery has been setup, and hence you can dispatch
    therein.

    Or we could have onAfterSubAllowed, onAfterSubDenied, .. hooks. The
    latter would match the ones we already have for RPC
    (onAfterCallSuccess etc)

Yes, I have used using [eligible] parameter.

First approach with callable returned IMHO seems more Twisted :slight_smile:

But personally I like the consistency with other parts of the API.
It would be good to have some kind onAfterSubAllowed, ... .

BTW, looking now into sources once more, there is not exposed to the API
documented method WampServerFactory.onClientSubscribed - it seems that I
can use it for the purpose (dispatching message right after
subscription), am I? Will test tomorrow. But anyway from design point of
view it will be more correct to have sub event handlers on the protocol
level.

Yes, I fully agree. onAfterSubAllowed etc on protocol instance seems right and consistent.

WampServerFactory.onClientSubscribed
WampServerFactory.onClientUnsubscribed

Ah, right, yes, you can use those .. sorry, the WAMP class docs need more love ..

/Tobias

···

Am 21.06.2013 11:19, schrieb Alexander Petrov:

Thank you,
Alex.

--
You received this message because you are subscribed to the Google
Groups "Autobahn" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to autobahnws+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

0 Likes