WAMP and serialization of Python class

#1

Hi,

I tried to create a simple example where I register and call a procedure that has an instance of a class as argument (see code pasted at bottom). It fails with the following message:

./client3.py session ready
procedure registered
could not call procedure: Unable to serialize WAMP application payload (<__main__.SomeClass instance at 0x1f86878> is not JSON serializable)

Should serialization of this simple object work out of the box or do I need to do more? What is the right approach for this? (I started specializing JsonSerializer, but that resulted in further problems.)


Thanks
Florian


#!/usr/bin/python
from autobahn.twisted.wamp import ApplicationSession
from twisted.internet.defer import inlineCallbacks
from autobahn.twisted.util import sleep
class SomeClass:
def __init__(self,name):
self.name = name
class MyComponent(ApplicationSession):
@inlineCallbacks
def onJoin(self, details):
print("session ready")
def printName(obj):
print(obj.name)
try:
yield self.register(printName, u'com.myapp.printname')
print("procedure registered")
except Exception as e:
print("could not register procedure: {0}".format(e))
sleep(1)
try:
someObj = SomeClass("Oberstet")
yield self.call(u'com.myapp.printname', someObj)
print("procedure called")
except Exception as e:
print("could not call procedure: {0}".format(e))
from autobahn.twisted.wamp import ApplicationRunner
runner = ApplicationRunner(url = "ws://localhost:8080/ws1", realm = "realm1")
runner.run(MyComponent)
0 Likes

#2

I’m facing the same issue and am a little further in the research process. It’s my understanding that this is the expected behavior. To send classes over the WebSockets, you need a different serializer like JsonPickle. See as reference:

···

On Friday, December 19, 2014 7:41:51 AM UTC-5, Florian Conrady wrote:

Hi,

I tried to create a simple example where I register and call a procedure that has an instance of a class as argument (see code pasted at bottom). It fails with the following message:

./client3.py session ready
procedure registered
could not call procedure: Unable to serialize WAMP application payload (<__main__.SomeClass instance at 0x1f86878> is not JSON serializable)


Should serialization of this simple object work out of the box or do I need to do more? What is the right approach for this? (I started specializing JsonSerializer, but that resulted in further problems.)



Thanks
Florian



#!/usr/bin/python
from autobahn.twisted.wamp import ApplicationSession
from twisted.internet.defer import inlineCallbacks
from autobahn.twisted.util import sleep
class SomeClass:
def __init__(self,name):
[self.name](http://self.name) = name
class MyComponent(
   ApplicationSession):
@inlineCallbacks
def onJoin(self, details):
print("session ready")
def printName(obj):
print([obj.name](http://obj.name)
      )
try:
yield self.register(printName, u'com.myapp.printname')
print("procedure registered")
except Exception as e:
print("could not register procedure: {0}".format(e))
sleep(1)
try:
someObj = SomeClass("Oberstet")
yield self.call(u'com.myapp.         printname', someObj)
print("procedure called")
except Exception as e:
print("could not call procedure: {0}".format(e))
from autobahn.twisted.wamp import ApplicationRunner
runner = ApplicationRunner(url = "ws://localhost:8080/ws1", realm = "realm1")
runner.run(MyComponent)
0 Likes

#3

Hi Clayton,

I just finished writing code that seems to solve the issue for me and I did use jsonpickle for it. (I hadn’t yet read your reply that suggested jsonpickle as well!) I wrapped the register and call methods of the WAMP application session. In the register method shown below I first “serialize” the method to be registered, i.e. I transform the method with Python object parameter into a function with a JSON string as parameter. Then I register this “serialized” version of the method with WAMP. In the call method I first convert the object parameter into a JSON string and then pass it to the WAMP call method. I pasted part of the code below to explain the gist of it. (Let me know if you need more details.)

Thanks

Florian

    @inlineCallbacks
def register(self, instance, method):
uri = self.getUri(instance, method)
def serialize(func):
def wrapper(jsonString):
# Deserialize Json string to Python object.
obj = jsonpickle.decode(jsonString)
return func(obj)
return wrapper
print("Trying to register %s." % uri)
yield self.getSession().register(serialize(method), uri)
print("Registered %s." % uri)
@inlineCallbacks
def call(self, methodUri, parameter, callback):
jsonString = jsonpickle.encode(parameter)
result = yield self.getSession().call(methodUri, jsonString)
callback(result)
···

On Saturday, December 20, 2014 6:58:22 PM UTC+2, Clayton Daley wrote:

I’m facing the same issue and am a little further in the research process. It’s my understanding that this is the expected behavior. To send classes over the WebSockets, you need a different serializer like JsonPickle. See as reference:

On Friday, December 19, 2014 7:41:51 AM UTC-5, Florian Conrady wrote:

Hi,

I tried to create a simple example where I register and call a procedure that has an instance of a class as argument (see code pasted at bottom). It fails with the following message:

./client3.py session ready
procedure registered
could not call procedure: Unable to serialize WAMP application payload (<__main__.SomeClass instance at 0x1f86878> is not JSON serializable)


Should serialization of this simple object work out of the box or do I need to do more? What is the right approach for this? (I started specializing JsonSerializer, but that resulted in further problems.)



Thanks
Florian



#!/usr/bin/python
from autobahn.twisted.wamp import ApplicationSession
from twisted.internet.defer import inlineCallbacks
from autobahn.twisted.util import sleep
class SomeClass:
def __init__(self,name):
[self.name](http://self.name) = name
class MyComponent(
   ApplicationSession):
@inlineCallbacks
def onJoin(self, details):
print("session ready")
def printName(obj):
print([obj.name](http://obj.name)
      )
try:
yield self.register(printName, u'com.myapp.printname')
print("procedure registered")
except Exception as e:
print("could not register procedure: {0}".format(e))
sleep(1)
try:
someObj = SomeClass("Oberstet")
yield self.call(u'com.myapp.         printname', someObj)
print("procedure called")
except Exception as e:
print("could not call procedure: {0}".format(e))
from autobahn.twisted.wamp import ApplicationRunner
runner = ApplicationRunner(url = "ws://localhost:8080/ws1", realm = "realm1")
runner.run(MyComponent)
0 Likes

#4

In addition to wrapping the Autobahn register and call methods I also tried specializing the serialize / unserialize method in autobahn/wamp/serializer.py. However, in that case, my code fails. The connection is dropped for some reason.

  $ ./client4.py WAMP-over-WebSocket transport lost: wasClean = False, code = 1006, reason = 'connection was closed uncleanly (I failed the WebSocket connection by dropping the TCP connection)'
Traceback (most recent call last):
File "/home/florian/Downloads/autobahn-0.9.4-2/autobahn/    wamp/websocket.py", line 75, in onClose
self._session.onClose(wasClean)
AttributeError: WampWebSocketClientProtocol instance has no attribute '_session'

I pasted the complete code of the script below. When I modify the JsonObjectSerializer in the Autobahn source code itself, I do not get this error message. So there must be something wrong in the way I am specializing it. Could someone give me a hint at what should be done differently? Note that in the script I have not yet changed anything about the serialization. At this point JsonPickleSerializer should behave identically to JsonSerializer.

Thanks

Florian

#!/usr/bin/python

···

############################################################

Specializing serialization

############################################################
from autobahn.wamp.serializer import *
from autobahn.wamp.interfaces import IObjectSerializer, ISerializer
class JsonPickleObjectSerializer:
BINARY = False
def init(self, batched = False):
“”"
Ctor.
:param batched: Flag that controls whether serializer operates in batched mode.
:type batched: bool
“”"
self._batched = batched
def serialize(self, obj):
“”"
Implements :func:autobahn.wamp.interfaces.IObjectSerializer.serialize
“”"
_dumps = lambda obj: json.dumps(obj, separators = (’,’,’:’), ensure_ascii = False)
s = _dumps(obj)
if isinstance(s, six.text_type):
s = s.encode(‘utf8’)
if self._batched:
return s + b’\30’
else:
return s
def unserialize(self, payload):
“”"
Implements :func:autobahn.wamp.interfaces.IObjectSerializer.unserialize
“”"
if self._batched:
chunks = payload.split(b’\30’)[:-1]
else:
chunks = [payload]
if len(chunks) == 0:
raise Exception(“batch format error”)
_loads = json.loads
return [_loads(data.decode(‘utf8’)) for data in chunks]
IObjectSerializer.register(JsonPickleObjectSerializer)
class JsonPickleSerializer(Serializer):
SERIALIZER_ID = “jsonpickle”
MIME_TYPE = “application/json”
def init(self, batched = False):
“”"
Ctor.
:param batched: Flag to control whether to put this serialized into batched mode.
:type batched: bool
“”"
Serializer.init(self, JsonPickleObjectSerializer(batched = batched))
if batched:
self.SERIALIZER_ID = “jsonpickle.batched”
ISerializer.register(JsonPickleSerializer)
############################################################

Application component

############################################################
from autobahn.twisted.wamp import ApplicationSession
from twisted.internet.defer import inlineCallbacks
from autobahn.twisted.util import sleep
class SomeClass:
def init(self,name):
self.name = name
class MyComponent(ApplicationSession):
@inlineCallbacks
def onJoin(self, details):
print(“session ready”)

Method with string parameter.

def printName(name):
print(name)
try:
yield self.register(printName, u’com.myapp.printname’)
print(“procedure registered”)
except Exception as e:
print(“could not register procedure: {0}”.format(e))
sleep(1)
try:
someName = “Tobias”
yield self.call(u’com.myapp.printname’, someName)
print(“procedure called”)
except Exception as e:
print(“could not call procedure: {0}”.format(e))

Method with object parameter.

def printObjectName(obj):
print(obj.name)
try:
yield self.register(printObjectName, u’com.myapp.printobjectname’)
print(“procedure registered”)
except Exception as e:
print(“could not register procedure: {0}”.format(e))
sleep(1)
try:
someObj = SomeClass(“Oberstein”)
yield self.call(u’com.myapp.printobjectname’, someObj)
print(“procedure called”)
except Exception as e:
print(“could not call procedure: {0}”.format(e))
############################################################

Running client

############################################################
from autobahn.twisted.wamp import ApplicationSessionFactory
from autobahn.twisted.websocket import WampWebSocketClientFactory
from autobahn.twisted.choosereactor import install_reactor
from twisted.internet.endpoints import clientFromString
from twisted.internet.defer import inlineCallbacks
from autobahn.wamp import types
config = types.ComponentConfig(realm = “realm1”)
sessionFactory = ApplicationSessionFactory(config)
sessionFactory.session = MyComponent
sessionFactory.sessionInstance = None
serializers = []
serializers.append(JsonPickleSerializer()) #JsonSerializer()) JsonPickleSerializer())
transportFactory = WampWebSocketClientFactory(sessionFactory, “ws://localhost:8080/ws1”, serializers = serializers, debug = False, debug_wamp = True)
reactor = install_reactor()
client = clientFromString(reactor, “tcp:localhost:8080”)
client.connect(transportFactory)
reactor.run()


On Monday, December 22, 2014 2:50:12 AM UTC+2, Florian Conrady wrote:
> Hi Clayton,
> 

> I just finished writing code that seems to solve the issue for me and I did use jsonpickle for it. (I hadn't yet read your reply that suggested jsonpickle as well!) I wrapped the register and call methods of the WAMP application session. In the register method shown below I first "serialize" the method to be registered, i.e. I transform the method with Python object parameter into a function with a JSON string as parameter. Then I register this "serialized" version of the method with WAMP. In the call method I first convert the object parameter into a JSON string and then pass it to the WAMP call method. I pasted part of the code below to explain the gist of it. (Let me know if you need more details.)

> 

> Thanks

> Florian

> 

> ```
>     @inlineCallbacks
> def register(self, instance, method):
> uri = self.getUri(instance, method)
> def serialize(func):
> def wrapper(jsonString):
> # Deserialize Json string to Python object.
> obj = jsonpickle.decode(jsonString)
> return func(obj)
> return wrapper
> print("Trying to register %s." % uri)
> yield self.getSession().register(        serialize(method), uri)
> print("Registered %s." % uri)
> @inlineCallbacks
> def call(self, methodUri, parameter, callback):
> jsonString = jsonpickle.encode(parameter)
> result = yield self.getSession().call(        methodUri, jsonString)
> callback(result)
> ```

> 

> 
> 
> On Saturday, December 20, 2014 6:58:22 PM UTC+2, Clayton Daley wrote:
> > I'm facing the same issue and am a little further in the research process.  It's my understanding that this is the expected behavior.  To send classes over the WebSockets, you need a different serializer like JsonPickle.  See as reference:
> > - [https://groups.google.com/forum/#!searchin/autobahnws/serialize/autobahnws/ZcUUQ4oqxgo/tPVULR-TtlQJ](https://groups.google.com/forum/#!searchin/autobahnws/serialize/autobahnws/ZcUUQ4oqxgo/tPVULR-TtlQJ)
> > - [https://github.com/tavendo/AutobahnPython/issues/163](https://github.com/tavendo/AutobahnPython/issues/163)
> > Unfortunately, I haven't found an example of how to actually implement it so I'm optimistic that someone can provide this to us both (even better if it were available out-of-the-box).

> > 

> > 

> > 
> > On Friday, December 19, 2014 7:41:51 AM UTC-5, Florian Conrady wrote:
> > > Hi,
> > > 

> > > I tried to create a simple example where I register and call a procedure that has an instance of a class as argument (see code pasted at bottom). It fails with the following message:

> > > 

> > > ```
> > > ./client3.py session ready
> > > procedure registered
> > > could not call procedure: Unable to serialize WAMP application payload (<__main__.SomeClass instance at 0x1f86878> is not JSON serializable)
> > > ```
> > > ```
> > > 
> > > 
> > > ```
> > > ```
> > > Should serialization of this simple object work out of the box or do I need to do more? What is the right approach for this? (I started specializing JsonSerializer, but that resulted in further problems.)
> > > 
> > > ```
> > > ```
> > > 
> > > 
> > > ```
> > > ```
> > > Thanks
> > > ```
> > > ```
> > > Florian
> > > ```
> > > ```
> > > 
> > > 
> > > ```
> > > 

> > > ```
> > > 
> > > #!/usr/bin/python
> > > from autobahn.twisted.wamp import ApplicationSession
> > > from twisted.internet.defer import inlineCallbacks
> > > from autobahn.twisted.util import sleep
> > > class SomeClass:
> > > def __init__(self,name):
> > > [self.name](http://self.name) = name
> > > class MyComponent(
> > >    ApplicationSession):
> > > @inlineCallbacks
> > > def onJoin(self, details):
> > > print("session ready")
> > > def printName(obj):
> > > print([obj.name](http://obj.name)
> > >       )
> > > try:
> > > yield self.register(printName, u'com.myapp.printname')
> > > print("procedure registered")
> > > except Exception as e:
> > > print("could not register procedure: {0}".format(e))
> > > sleep(1)
> > > try:
> > > someObj = SomeClass("Oberstet")
> > > yield self.call(u'com.myapp.         printname', someObj)
> > > print("procedure called")
> > > except Exception as e:
> > > print("could not call procedure: {0}".format(e))
> > > from autobahn.twisted.wamp import ApplicationRunner
> > > runner = ApplicationRunner(url = "ws://localhost:8080/ws1", realm = "realm1")
> > > runner.run(MyComponent)
> > > ```

> > > 

> > > 

> > > 

> > > 

> > > 

> > > 

> > > 

> > > 

> > > 

> > > 

> > > 

> > > 

> > > 

> > >

</details>
0 Likes

#5

Hi Florian,

WAMP is language agnostic. That means, given a WAMP procedure, any WAMP client (in any language) is supposed to be able to call the procedure.

That being said, it should still be possible to do if you insist;)

Don’t follow the overrriding “call, register, …” approach … this leads to insanity. Your approach with JsonPickleObjectSerializer is the right one.

With your code, change this

SERIALIZER_ID = “jsonpickle”

and leave it at "json".

a) the output of jsonpickle is still JSON (not MsgPack or anything)
b) no WAMP router will understand how to treat "jsonpickle" serialization
c) the fact that the JSON is "special" (e.g. contains Python classnames) is irrelevant to a WAMP router

Of course to make it work, you'll then need to use jsonpickle.encode etc in your code below.

Cheers,
/Tobias

<details class='elided'>
<summary title='Show trimmed content'>&#183;&#183;&#183;</summary>

Am Freitag, 19. Dezember 2014 13:41:51 UTC+1 schrieb Florian Conrady:
> Hi,
> 

> I tried to create a simple example where I register and call a procedure that has an instance of a class as argument (see code pasted at bottom). It fails with the following message:

> 

> ```
> ./client3.py session ready
> procedure registered
> could not call procedure: Unable to serialize WAMP application payload (<__main__.SomeClass instance at 0x1f86878> is not JSON serializable)
> ```
> ```
> 
> 
> ```
> ```
> Should serialization of this simple object work out of the box or do I need to do more? What is the right approach for this? (I started specializing JsonSerializer, but that resulted in further problems.)
> 
> ```
> ```
> 
> 
> ```
> ```
> Thanks
> ```
> ```
> Florian
> ```
> ```
> 
> 
> ```
> 

> ```
> 
> #!/usr/bin/python
> from autobahn.twisted.wamp import ApplicationSession
> from twisted.internet.defer import inlineCallbacks
> from autobahn.twisted.util import sleep
> class SomeClass:
> def __init__(self,name):
> [self.name](http://self.name) = name
> class MyComponent(
>    ApplicationSession):
> @inlineCallbacks
> def onJoin(self, details):
> print("session ready")
> def printName(obj):
> print([obj.name](http://obj.name)
>       )
> try:
> yield self.register(printName, u'com.myapp.printname')
> print("procedure registered")
> except Exception as e:
> print("could not register procedure: {0}".format(e))
> sleep(1)
> try:
> someObj = SomeClass("Oberstet")
> yield self.call(u'com.myapp.         printname', someObj)
> print("procedure called")
> except Exception as e:
> print("could not call procedure: {0}".format(e))
> from autobahn.twisted.wamp import ApplicationRunner
> runner = ApplicationRunner(url = "ws://localhost:8080/ws1", realm = "realm1")
> runner.run(MyComponent)
> ```

> 

> 

> 

> 

> 

> 

> 

> 

> 

> 

> 

> 

> 

>

</details>
0 Likes

#6

Tobias,

I understand how this approach aligns with platform independence. At the same time,

  • If I want to use Autobahn in a single-language environment, it should be easy to drop in a serializer that minimizes complexity. Better for the community if I’m using autobahn for everything I write… rather than switching to something that’s simpler for a single-language implementation.
  • For Python (at least), it might be better to use a “safe” serializer like serpent (https://pypi.python.org/pypi/serpent). Incidentally, this already includes support for Python, Java and C#/.NET.
  • When it comes to some packages and types, it’d be nice to have a library of (community contributed) plugins that serialize (and de-serialize) specific objects (or packages) in one or more languages. For example, Python’s timedelta is unsupported by JsonSerialize but is generated by MySQLdb and supported by PyMongo. Certainly, anything supported widely in Mongo clients could be duplicated in Autobahn.
    Especially on this last point, I’m not clear if this is easy (but not documented) or difficult. Certainly would be nice to know (and have some pointers to best practices) so users like Florian and I can implement the capabilities correctly and easily share with other users who may be interested.

Clayton

···

On Tuesday, December 30, 2014 8:25:33 AM UTC-5, Tobias Oberstein wrote:

Hi Florian,

WAMP is language agnostic. That means, given a WAMP procedure, any WAMP client (in any language) is supposed to be able to call the procedure.

That being said, it should still be possible to do if you insist;)

Don’t follow the overrriding “call, register, …” approach … this leads to insanity. Your approach with JsonPickleObjectSerializer is the right one.

With your code, change this

SERIALIZER_ID = “jsonpickle”

and leave it at "json".

a) the output of jsonpickle is still JSON (not MsgPack or anything)
b) no WAMP router will understand how to treat "jsonpickle" serialization
c) the fact that the JSON is "special" (e.g. contains Python classnames) is irrelevant to a WAMP router

Of course to make it work, you'll then need to use jsonpickle.encode etc in your code below.

Cheers,
/Tobias

Am Freitag, 19. Dezember 2014 13:41:51 UTC+1 schrieb Florian Conrady:
> Hi,
> 

I tried to create a simple example where I register and call a procedure that has an instance of a class as argument (see code pasted at bottom). It fails with the following message:

./client3.py session ready
procedure registered
could not call procedure: Unable to serialize WAMP application payload (<__main__.SomeClass instance at 0x1f86878> is not JSON serializable)


Should serialization of this simple object work out of the box or do I need to do more? What is the right approach for this? (I started specializing JsonSerializer, but that resulted in further problems.)



Thanks
Florian



#!/usr/bin/python
from autobahn.twisted.wamp import ApplicationSession
from twisted.internet.defer import inlineCallbacks
from autobahn.twisted.util import sleep
class SomeClass:
def __init__(self,name):
[self.name](http://self.name) = name
class MyComponent(
   ApplicationSession):
@inlineCallbacks
def onJoin(self, details):
print("session ready")
def printName(obj):
print([obj.name](http://obj.name)
      )
try:
yield self.register(printName, u'com.myapp.printname')
print("procedure registered")
except Exception as e:
print("could not register procedure: {0}".format(e))
sleep(1)
try:
someObj = SomeClass("Oberstet")
yield self.call(u'com.myapp.         printname', someObj)
print("procedure called")
except Exception as e:
print("could not call procedure: {0}".format(e))
from autobahn.twisted.wamp import ApplicationRunner
runner = ApplicationRunner(url = "ws://localhost:8080/ws1", realm = "realm1")
runner.run(MyComponent)
0 Likes

#7

Hi Clayton,

> I understand how this approach aligns with platform independence. At

the same time,

  * If I want to use Autobahn in a single-language environment, it
    should be easy to drop in a serializer that minimizes complexity.
    Better for the community if I'm using autobahn for everything I
    write... rather than switching to something that's simpler for a
    single-language implementation.

If you want language specific, transparent type marshalling (like Python serpent) or stuff like transparent object remoting (like CORBA or Java RMI), then WAMP probably isn't for you.

WAMP was deliberately designed _not_ following these approaches. We like to keep stuff simple and flexible.

  * For Python (at least), it might be better to use a "safe" serializer
    like serpent (https://pypi.python.org/pypi/serpent). Incidentally,
    this already includes support for Python, Java and C#/.NET.

This seems to use a proprietory, Python specific serialization format.

This cannot be used with WAMP, since WAMP currently only specifies JSON and MsgPack on the wire.

We won't add language specific serializers to WAMP as this does not serve the WAMP ecosystem, but fragments it.

  * When it comes to some packages and types, it'd be nice to have a
    library of (community contributed) plugins that serialize (and
    de-serialize) specific objects (or packages) in one or more
    languages. For example, Python's timedelta is unsupported by
    JsonSerialize but is generated by MySQLdb and supported by PyMongo.
      Certainly, anything supported widely in Mongo clients could be
    duplicated in Autobahn.

Especially on this last point, I'm not clear if this is easy (but not
documented) or difficult. Certainly would be nice to know (and have
some pointers to best practices) so users like Florian and I can
implement the capabilities correctly and easily share with other users
who may be interested.

Sure. You can create a custom JSON serializer that is capable of serializing Python timedelta. It will need to have means to differentiate normal strings from timedelta strings for deserialization.

You would start creating custom versions of

https://github.com/tavendo/AutobahnPython/blob/master/autobahn/autobahn/wamp/serializer.py#L206

and plug that into the respective WAMP transport.

Cheers,
/Tobias

···

Clayton

On Tuesday, December 30, 2014 8:25:33 AM UTC-5, Tobias Oberstein wrote:

    Hi Florian,

    WAMP is language agnostic. That means, given a WAMP procedure, _any_
    WAMP client (in any language) is supposed to be able to call the
    procedure.

    That being said, it should still be possible to do if you insist;)

    Don't follow the overrriding "call, register, .." approach .. this
    leads to insanity. Your approach with JsonPickleObjectSerializer is
    the right one.

    With your code, change this

    SERIALIZER_ID = "jsonpickle"

    and leave it at "json".

    a) the output of jsonpickle is still JSON (not MsgPack or anything)
    b) no WAMP router will understand how to treat "jsonpickle"
    serialization
    c) the fact that the JSON is "special" (e.g. contains Python
    classnames) is irrelevant to a WAMP router

    Of course to make it work, you'll then need to use jsonpickle.encode
    etc in your code below.

    Cheers,
    /Tobias

    Am Freitag, 19. Dezember 2014 13:41:51 UTC+1 schrieb Florian Conrady:

        Hi,

        I tried to create a simple example where I register and call a
        procedure that has an instance of a class as argument (see code
        pasted at bottom). It fails with the following message:

        ./client3.py
        session ready
        procedure registered
        could not call procedure: Unable to serialize WAMP application payload (<__main__.SomeClass instance at 0x1f86878> is not JSON serializable)

        Should serialization of this simple object work out of the box or do I need to do more? What is the right approach for this? (I started specializingJsonSerializer, but that resulted in further problems.)

        Thanks

        Florian

        #!/usr/bin/python

        from autobahn.twisted.wamp import ApplicationSession
        from twisted.internet.defer import inlineCallbacks
        from autobahn.twisted.util import sleep

        class SomeClass:
            def __init__(self,name):
               self.name <http://self.name> = name

        class MyComponent(ApplicationSession):

            @inlineCallbacks
            def onJoin(self, details):
               print("session ready")

               def printName(obj):
                  print(obj.name <http://obj.name>)

               try:
                  yield self.register(printName, u'com.myapp.printname')
                  print("procedure registered")
               except Exception as e:
                  print("could not register procedure: {0}".format(e))

               sleep(1)

               try:
                  someObj = SomeClass("Oberstet")
                  yield self.call(u'com.myapp.printname', someObj)
                  print("procedure called")
               except Exception as e:
                  print("could not call procedure: {0}".format(e))

        from autobahn.twisted.wamp import ApplicationRunner

        runner = ApplicationRunner(url = "ws://localhost:8080/ws1", realm = "realm1")
        runner.run(MyComponent)

--
You received this message because you are subscribed to the Google
Groups "Autobahn" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to autobahnws+...@googlegroups.com
<mailto:autobahnws+...@googlegroups.com>.
To post to this group, send email to autob...@googlegroups.com
<mailto:autob...@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/autobahnws/cc026982-b503-4003-a907-a57c1a4e6484%40googlegroups.com
<https://groups.google.com/d/msgid/autobahnws/cc026982-b503-4003-a907-a57c1a4e6484%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

0 Likes

#8

Obviously, this is your thing… but I always assume that the amount of support (financial and volunteer, as well as commercial revenue) for a project is roughly proportional to the number of users.

  • Telling the large fraction of users – who are building a single language system – to go somewhere else for ease of use seems counter-productive. I figure it actually invites fragmentation by encouraging the development/ enhancement of competitive platforms.
  • Your JsonPickle response seems to suggest that the only barrier is a serializer… so the rest of the platform (and any future improvements) would be shared by the entire community.

Sure. You can create a custom JSON serializer that is capable of serializing Python timedelta. It will need to have means to differentiate normal strings from timedelta strings [empahsis added] for deserialization.

Agreed. Perhaps a generic (at least standard to WAMP) solution to this problem would be warranted. My impression is the following (but you know your architecture better to criticize):

  • I believe Florian and I are really concerned about serializing args and kwargs.
  • The current architecture only serializes once… on the complete message. Changing the message-level serializer works, but it’s massive overkill. Every receiver must use that substitute serializer… which means that the serializer must exist on every supported platform.
  • It would be better to implement a set of (optional) serializer for args/kwargs. Recurse through dicts and lists. Skip JSON native types. Add some metadata identifying the serializer (None for native, string identifying other serialized).
···

On Wed, Dec 31, 2014 at 6:35 AM, Tobias Oberstein tobias.o...@gmail.com wrote:

Hi Clayton,

I understand how this approach aligns with platform independence. At

the same time,

  • If I want to use Autobahn in a single-language environment, it

    should be easy to drop in a serializer that minimizes complexity.

    Better for the community if I’m using autobahn for everything I

    write… rather than switching to something that’s simpler for a

    single-language implementation.

If you want language specific, transparent type marshalling (like Python serpent) or stuff like transparent object remoting (like CORBA or Java RMI), then WAMP probably isn’t for you.

WAMP was deliberately designed not following these approaches. We like to keep stuff simple and flexible.

  • For Python (at least), it might be better to use a “safe” serializer

    like serpent (https://pypi.python.org/pypi/serpent). Incidentally,

    this already includes support for Python, Java and C#/.NET.

This seems to use a proprietory, Python specific serialization format.

This cannot be used with WAMP, since WAMP currently only specifies JSON and MsgPack on the wire.

We won’t add language specific serializers to WAMP as this does not serve the WAMP ecosystem, but fragments it.

  • When it comes to some packages and types, it’d be nice to have a

    library of (community contributed) plugins that serialize (and

    de-serialize) specific objects (or packages) in one or more

    languages. For example, Python’s timedelta is unsupported by

    JsonSerialize but is generated by MySQLdb and supported by PyMongo.

    Certainly, anything supported widely in Mongo clients could be
    

    duplicated in Autobahn.

Especially on this last point, I’m not clear if this is easy (but not

documented) or difficult. Certainly would be nice to know (and have

some pointers to best practices) so users like Florian and I can

implement the capabilities correctly and easily share with other users

who may be interested.

Sure. You can create a custom JSON serializer that is capable of serializing Python timedelta. It will need to have means to differentiate normal strings from timedelta strings for deserialization.

You would start creating custom versions of

https://github.com/tavendo/AutobahnPython/blob/master/autobahn/autobahn/wamp/serializer.py#L206

and plug that into the respective WAMP transport.

Cheers,

/Tobias

Clayton

On Tuesday, December 30, 2014 8:25:33 AM UTC-5, Tobias Oberstein wrote:

Hi Florian,



WAMP is language agnostic. That means, given a WAMP procedure, _any_

WAMP client (in any language) is supposed to be able to call the

procedure.



That being said, it should still be possible to do if you insist;)



Don't follow the overrriding "call, register, .." approach .. this

leads to insanity. Your approach with JsonPickleObjectSerializer is

the right one.



With your code, change this



SERIALIZER_ID = "jsonpickle"



and leave it at "json".



a) the output of jsonpickle is still JSON (not MsgPack or anything)

b) no WAMP router will understand how to treat "jsonpickle"

serialization

c) the fact that the JSON is "special" (e.g. contains Python

classnames) is irrelevant to a WAMP router



Of course to make it work, you'll then need to use jsonpickle.encode

etc in your code below.



Cheers,

/Tobias



Am Freitag, 19. Dezember 2014 13:41:51 UTC+1 schrieb Florian Conrady:



    Hi,



    I tried to create a simple example where I register and call a

    procedure that has an instance of a class as argument (see code

    pasted at bottom). It fails with the following message:



    ./client3.py

    session ready

    procedure registered

    could not call procedure: Unable to serialize WAMP application payload (<__main__.SomeClass instance at 0x1f86878> is not JSON serializable)

Should serialization of this simple object work out of the box or do I need to do more? What is the right approach for this? (I started specializingJsonSerializer, but that resulted in further problems.)

    Thanks



    Florian







    #!/usr/bin/python



    from autobahn.twisted.wamp import ApplicationSession

    from twisted.internet.defer import inlineCallbacks

    from autobahn.twisted.util import sleep



    class SomeClass:

        def __init__(self,name):

           [self.name](http://self.name)  <[http://self.name](http://self.name)>  = name



    class MyComponent(ApplicationSession):



        @inlineCallbacks

        def onJoin(self, details):

           print("session ready")



           def printName(obj):

              print([obj.name](http://obj.name)  <[http://obj.name](http://obj.name)>)



           try:

              yield self.register(printName, u'com.myapp.printname')

              print("procedure registered")

           except Exception as e:

              print("could not register procedure: {0}".format(e))



           sleep(1)



           try:

              someObj = SomeClass("Oberstet")

              yield self.call(u'com.myapp.printname', someObj)

              print("procedure called")

           except Exception as e:

              print("could not call procedure: {0}".format(e))





    from autobahn.twisted.wamp import ApplicationRunner



    runner = ApplicationRunner(url = "ws://localhost:8080/ws1", realm = "realm1")

    runner.run(MyComponent)

You received this message because you are subscribed to the Google

Groups “Autobahn” group.

To unsubscribe from this group and stop receiving emails from it, send

an email to autobahnws+unsubscribe@googlegroups.com

mailto:autobahnws+unsub...@googlegroups.com.

To post to this group, send email to autob...@googlegroups.com

mailto:autobahnws@googlegroups.com.

To view this discussion on the web visit

https://groups.google.com/d/msgid/autobahnws/cc026982-b503-4003-a907-a57c1a4e6484%40googlegroups.com

<https://groups.google.com/d/msgid/autobahnws/cc026982-b503-4003-a907-a57c1a4e6484%40googlegroups.com?utm_medium=email&utm_source=footer>.

For more options, visit https://groups.google.com/d/optout.

You received this message because you are subscribed to a topic in the Google Groups “Autobahn” group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/autobahnws/6Pk0c8JiSn0/unsubscribe.

To unsubscribe from this group and all its topics, send an email to autobahnws+unsubscribe@googlegroups.com.

To post to this group, send email to autob...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/autobahnws/54A3DF91.1060804%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

Clayton Daley

0 Likes

#9

Hi Clayton,

happy new year to you also!

I reply not inline, because I first need to understand better: it might be we are not on the same track yet.

···

==

We should differentiate the following in the discussion:

1. the WAMP serialization format at the wire-level
2. the serializers in WAMP client libraries (and routers)

The WAMP spec currently defines two serialization formats for 1):

* JSON (with an added/optional convention for binary data)
* MsgPack

==

jsonpickle (https://jsonpickle.github.io/) is a serializer library for Python that produces _standard_ JSON.

However, the JSON produces will follow certain conventions to add Python language specific information:

$ python
Python 2.7.9 (default, Dec 10 2014, 12:24:55) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import jsonpickle
>>> class Foo:
... pass
...
>>> f = Foo()
>>> jsonpickle.encode(f)
'{"py/object": "__main__.Foo"}'

Any WAMP router will happily process messages which have stuff serialized using this serializer in args or kwargs.

Any WAMP client library (supporting JSON) will be able to process such messages .. it just won't interpret the "magic attributes" like "py/object" specially.

The question of how to hook jsonpickle into AutobahnPython is totally specific to AutobahnPython, and this should be possible today.

If you want that, and can't get it working, come back, we'll get it done.

Note: I still don't think it's wise to restrict your choices (language) by using above, but if you are doing a Python-only app it'll work, and it should be a developer choice to do so.

==

Serpent (https://github.com/irmen/Serpent) is both a serialization format _and_ a serialization library (for Java, .NET, and Python https://pypi.python.org/pypi/serpent)

The serialization format doesn't seem to be documented, but it is definitely not JSON, but something "custom":

$ python
Python 2.7.9 (default, Dec 10 2014, 12:24:55) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import serpent
>>> class Foo:
... pass
...
>>> f = Foo()
>>> serpent.dumps(f)
"# serpent utf-8 python2.6\n{'__class__':'Foo'}"

Are you suggesting a WAMP based system should be able to use this custom serialization format?

a) using this serialization format for the complete WAMP messages: I' not convinced this makes sense, since the format is totally proprietory and not even documented. Routers would need to fully implement the format to do their job.

b) using this seriazation format for args/kwargs only within WAMP messages (but not for the "surrounding" stuff in WAMP messages): this is also problematic for different reasons:

- a WAMP router is supposed to be able to transparently translate between different serialization formats, so clients using different serialization formats are shielded from this.

- a WAMP router is supposed to ensure anything forwarded strictly follows the serialization format used to protect innocent client libraries from being attacked (security). if the router doesn't understand args/kwargs serialization format, it no longer can provide that service

- a WAMP router might be configured to do "schema validation" on app payload (args/kwargs) - this is actually already partially implemented in Crossbar.io. E.g. you can require the payload of a WAMP procedure follow a schema, and the router will validate, and deny the forwarding of a call if the payload doesn't match

==

Cheers,
/Tobias

Am 31.12.2014 um 23:29 schrieb Clayton Daley:

Obviously, this is your thing... but I always assume that the amount of
support (financial and volunteer, as well as commercial revenue) for a
project is roughly proportional to the number of users.

  * Telling the large fraction of users -- who are building a single
    language system -- to go somewhere else for ease of use seems
    counter-productive. I figure it actually invites fragmentation by
    encouraging the development/ enhancement of competitive platforms.
  * Your JsonPickle response seems to suggest that the only barrier is a
    serializer... so the rest of the platform (and any future
    improvements) would be shared by the entire community.

> Sure. You can create a custom JSON serializer that is capable of
serializing Python timedelta. It will need to have *means to
differentiate normal strings from timedelta strings [empahsis added]*
for deserialization.

Agreed. Perhaps a generic (at least standard to WAMP) solution to this
problem would be warranted. My impression is the following (but you
know your architecture better to criticize):

  * I believe Florian and I are really concerned about serializing args
    and kwargs.
  * The current architecture only serializes once... on the complete
    message. Changing the message-level serializer works, but it's
    massive overkill. Every receiver must use that substitute
    serializer... which means that the serializer must exist on every
    supported platform.
  * It would be better to implement a set of (optional) serializer for
    args/kwargs. Recurse through dicts and lists. Skip JSON native
    types. Add some metadata identifying the serializer (None for
    native, string identifying other serialized).

Clayton Daley

On Wed, Dec 31, 2014 at 6:35 AM, Tobias Oberstein > <tobias.o...@gmail.com <mailto:tobias.o...@gmail.com>> wrote:

    Hi Clayton,

    > I understand how this approach aligns with platform independence. At

        the same time,

           * If I want to use Autobahn in a single-language environment, it
             should be easy to drop in a serializer that minimizes
        complexity.
             Better for the community if I'm using autobahn for everything I
             write... rather than switching to something that's simpler
        for a
             single-language implementation.

    If you want language specific, transparent type marshalling (like
    Python serpent) or stuff like transparent object remoting (like
    CORBA or Java RMI), then WAMP probably isn't for you.

    WAMP was deliberately designed _not_ following these approaches. We
    like to keep stuff simple and flexible.

           * For Python (at least), it might be better to use a "safe"
        serializer
             like serpent (https://pypi.python.org/pypi/__serpent
        <https://pypi.python.org/pypi/serpent>). Incidentally,
             this already includes support for Python, Java and C#/.NET.

    This seems to use a proprietory, Python specific serialization format.

    This cannot be used with WAMP, since WAMP currently only specifies
    JSON and MsgPack on the wire.

    We won't add language specific serializers to WAMP as this does not
    serve the WAMP ecosystem, but fragments it.

           * When it comes to some packages and types, it'd be nice to
        have a
             library of (community contributed) plugins that serialize (and
             de-serialize) specific objects (or packages) in one or more
             languages. For example, Python's timedelta is unsupported by
             JsonSerialize but is generated by MySQLdb and supported by
        PyMongo.
               Certainly, anything supported widely in Mongo clients
        could be
             duplicated in Autobahn.

        Especially on this last point, I'm not clear if this is easy
        (but not
        documented) or difficult. Certainly would be nice to know (and have
        some pointers to best practices) so users like Florian and I can
        implement the capabilities correctly and easily share with other
        users
        who may be interested.

    Sure. You can create a custom JSON serializer that is capable of
    serializing Python timedelta. It will need to have means to
    differentiate normal strings from timedelta strings for deserialization.

    You would start creating custom versions of

    https://github.com/tavendo/__AutobahnPython/blob/master/__autobahn/autobahn/wamp/__serializer.py#L206
    <https://github.com/tavendo/AutobahnPython/blob/master/autobahn/autobahn/wamp/serializer.py#L206>

    and plug that into the respective WAMP transport.

    Cheers,
    /Tobias

        Clayton

        On Tuesday, December 30, 2014 8:25:33 AM UTC-5, Tobias Oberstein > wrote:

             Hi Florian,

             WAMP is language agnostic. That means, given a WAMP
        procedure, _any_
             WAMP client (in any language) is supposed to be able to
        call the
             procedure.

             That being said, it should still be possible to do if you
        insist;)

             Don't follow the overrriding "call, register, .." approach
        .. this
             leads to insanity. Your approach with
        JsonPickleObjectSerializer is
             the right one.

             With your code, change this

             SERIALIZER_ID = "jsonpickle"

             and leave it at "json".

             a) the output of jsonpickle is still JSON (not MsgPack or
        anything)
             b) no WAMP router will understand how to treat "jsonpickle"
             serialization
             c) the fact that the JSON is "special" (e.g. contains Python
             classnames) is irrelevant to a WAMP router

             Of course to make it work, you'll then need to use
        jsonpickle.encode
             etc in your code below.

             Cheers,
             /Tobias

             Am Freitag, 19. Dezember 2014 13:41:51 UTC+1 schrieb > Florian Conrady:

                 Hi,

                 I tried to create a simple example where I register and
        call a
                 procedure that has an instance of a class as argument
        (see code
                 pasted at bottom). It fails with the following message:

                 ./client3.py
                 session ready
                 procedure registered
                 could not call procedure: Unable to serialize WAMP
        application payload (<__main__.SomeClass instance at 0x1f86878>
        is not JSON serializable)

                 Should serialization of this simple object work out of
        the box or do I need to do more? What is the right approach for
        this? (I started specializingJsonSerializer, but that resulted
        in further problems.)

                 Thanks

                 Florian

                 #!/usr/bin/python

                 from autobahn.twisted.wamp import ApplicationSession
                 from twisted.internet.defer import inlineCallbacks
                 from autobahn.twisted.util import sleep

                 class SomeClass:
                     def __init__(self,name):
        self.name <http://self.name> <http://self.name> = name

                 class MyComponent(__ApplicationSession):

                     @inlineCallbacks
                     def onJoin(self, details):
                        print("session ready")

                        def printName(obj):
                           print(obj.name <http://obj.name>
        <http://obj.name>)

                        try:
                           yield self.register(printName,
        u'com.myapp.printname')
                           print("procedure registered")
                        except Exception as e:
                           print("could not register procedure:
        {0}".format(e))

                        sleep(1)

                        try:
                           someObj = SomeClass("Oberstet")
                           yield self.call(u'com.myapp.__printname',
        someObj)
                           print("procedure called")
                        except Exception as e:
                           print("could not call procedure: {0}".format(e))

                 from autobahn.twisted.wamp import ApplicationRunner

                 runner = ApplicationRunner(url =
        "ws://localhost:8080/ws1", realm = "realm1")
                 runner.run(MyComponent)

        --
        You received this message because you are subscribed to the Google
        Groups "Autobahn" group.
        To unsubscribe from this group and stop receiving emails from
        it, send
        an email to autobahnws+unsubscribe@__googlegroups.com
        <mailto:autobahnws%2...@googlegroups.com>
        <mailto:autobahnws+_...@googlegroups.com
        <mailto:autobahnws%2...@googlegroups.com>>.
        To post to this group, send email to autob...@googlegroups.com
        <mailto:autob...@googlegroups.com>
        <mailto:autob...@__googlegroups.com
        <mailto:autob...@googlegroups.com>>.
        To view this discussion on the web visit
        https://groups.google.com/d/__msgid/autobahnws/cc026982-__b503-4003-a907-a57c1a4e6484%__40googlegroups.com
        <https://groups.google.com/d/msgid/autobahnws/cc026982-b503-4003-a907-a57c1a4e6484%40googlegroups.com>
        <https://groups.google.com/d/__msgid/autobahnws/cc026982-__b503-4003-a907-a57c1a4e6484%__40googlegroups.com?utm_medium=__email&utm_source=footer
        <https://groups.google.com/d/msgid/autobahnws/cc026982-b503-4003-a907-a57c1a4e6484%40googlegroups.com?utm_medium=email&utm_source=footer>>.
        For more options, visit https://groups.google.com/d/__optout
        <https://groups.google.com/d/optout>.

    --
    You received this message because you are subscribed to a topic in
    the Google Groups "Autobahn" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/__topic/autobahnws/6Pk0c8JiSn0/__unsubscribe
    <https://groups.google.com/d/topic/autobahnws/6Pk0c8JiSn0/unsubscribe>.
    To unsubscribe from this group and all its topics, send an email to
    autobahnws+unsubscribe@__googlegroups.com
    <mailto:autobahnws%2...@googlegroups.com>.
    To post to this group, send email to autob...@googlegroups.com
    <mailto:autob...@googlegroups.com>.
    To view this discussion on the web visit
    https://groups.google.com/d/__msgid/autobahnws/54A3DF91.__1060804%40gmail.com
    <https://groups.google.com/d/msgid/autobahnws/54A3DF91.1060804%40gmail.com>.

    For more options, visit https://groups.google.com/d/__optout
    <https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google
Groups "Autobahn" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to autobahnws+...@googlegroups.com
<mailto:autobahnws+...@googlegroups.com>.
To post to this group, send email to autob...@googlegroups.com
<mailto:autob...@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/autobahnws/CACKEAnVCh592WY1Hn07eDkJE7D%3DigXu6jtAJue65WtdBQGp-7w%40mail.gmail.com
<https://groups.google.com/d/msgid/autobahnws/CACKEAnVCh592WY1Hn07eDkJE7D%3DigXu6jtAJue65WtdBQGp-7w%40mail.gmail.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

0 Likes

#10

[Apologize for the duplicate… forgot to trim signature so I deleted my last post off the list]

Thanks for your thoughtful reply and happy new year to you too. There are a lot of moving pieces, but my perfect world looks something like:

  • I have some channels that are Python-only. I’d prefer to use a minimum-friction (Python only is fine) serializer so I don’t have to think about the types/objects in function signatures.
  • JsonPickle has the right behavior. Python consumers (who share JsonPickle) will deserialize these objects without breaking my line of communication with others. It’s my problem to only send serialized objects to receivers that understand them.
  • However, Pickle is a recognized security vulnerability. I’d rather use something “safer” like Serpent in case one of the nodes in the system is breached.
  • I will definitely have some Python-JS channels and may add some Python-PHP channels.
  • If I ever want to transfer an object across one of these wires, I’d like a simple way to add support on both ends. I have to write the classes at both ends, but I’d rather not have to wrap every arg and every return that involves the object.
  • Something like the “py/object” syntax (e.g. “ab/” for autobahn/) would be fine for a representation, but I’d rather configure it in one place and have it automatically applied to any instance that crosses the wire.
    It still seems to me like args/kwargs, the return, and both ends of the pub/sub are natural places to handle non-JSON types/classes. I think an ordered list of serializers has the right feel to it (iterate until one handles the type). The first can be a “serializer” that simply skips native types. I could follow that with custom Serializer for types supported in multiple platforms. Finally, I can toss in a catch-all serializer for Python classes for low friction.

If your only problem with Serpent is the binary (rather than string) representation, that’s probably easy to fix. The standard is open-source and I’ve found Irmen accessible. Besides that, it seems like serpent is just as safe as a programmer who custom-serializes their own objects. If the receiver doesn’t have serpent, they’ll just see a meaningless representation.

Anyway… that’s my mindset.

Clayton

[trimmed]

0 Likes

#11

Hi Clayton,

1)
yes, both Pickle _and_ jsonpickle are a security vulnerability. e.g. read here

https://blog.nelhage.com/2011/03/exploiting-pickle/

adjusted to jsonpickle: https://gist.github.com/oberstet/3723147344967b9794c0

So while you should (technically) be able to use jsonpickle as a custom serializer with AutobahnPython, I would _strongly_ recommend against doing this (at least outside a strictly controlled environment).

2)
I have now tried it out, and I don't think it works like you expect:

https://gist.github.com/oberstet/6df18395c1c173a1cf89

As you can see, it'll happily serialize objects like timestamp or DemoClass instances, but it will _not_ recreate instances of those classes.

So it (I guess) only solves 1 aspect of the 2 you are after: it can serialize any Python object transparently, but _not_ the other direction.

Note that any system that is capable of doing the latter is likely a security vulnerability. Please correct me if you think I am wrong!

Serpent: it's also an adhoc, non-standard serialization format. And I have a problem with this, yes. It's not about binary/text.

Cheers,
/Tobias

···

Am 07.01.2015 um 00:59 schrieb Clayton Daley:

[Apologize for the duplicate... forgot to trim signature so I deleted my
last post off the list]

Thanks for your thoughtful reply and happy new year to you too. There
are a lot of moving pieces, but my perfect world looks something like:

  * I have some channels that are Python-only. I'd prefer to use a
    minimum-friction (Python only is fine) serializer so I don't have to
    think about the types/objects in function signatures.
      o JsonPickle has the right behavior. Python consumers (who share
        JsonPickle) will deserialize these objects without breaking my
        line of communication with others. It's my problem to only send
        serialized objects to receivers that understand them.
      o However, Pickle is a recognized security vulnerability. I'd
        rather use something "safer" like Serpent in case one of the
        nodes in the system is breached.
  * I will definitely have some Python-JS channels and may add some
    Python-PHP channels.
      o If I ever want to transfer an object across one of these wires,
        I'd like a simple way to add support on both ends. I have to
        write the classes at both ends, but I'd rather not have to wrap
        every arg and every return that involves the object.
      o Something like the "py/object" syntax (e.g. "ab/<serializer>"
        for autobahn/<serializer>) would be fine for a representation,
        but I'd rather configure it in one place and have it
        automatically applied to any instance that crosses the wire.

It still seems to me like args/kwargs, the return, and both ends of the
pub/sub are natural places to handle non-JSON types/classes. I think an
ordered list of serializers has the right feel to it (iterate until one
handles the type). The first can be a "serializer" that simply skips
native types. I could follow that with custom Serializer for types
supported in multiple platforms. Finally, I can toss in a catch-all
serializer for Python classes for low friction.

If your only problem with Serpent is the binary (rather than string)
representation, that's probably easy to fix. The standard is open-source
and I've found Irmen accessible. Besides that, it seems like serpent is
just as safe as a programmer who custom-serializes their own objects. If
the receiver doesn't have serpent, they'll just see a meaningless
representation.

Anyway... that's my mindset.

Clayton
[trimmed]

--
You received this message because you are subscribed to the Google
Groups "Autobahn" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to autobahnws+...@googlegroups.com
<mailto:autobahnws+...@googlegroups.com>.
To post to this group, send email to autob...@googlegroups.com
<mailto:autob...@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/autobahnws/07af9de5-cf4d-49a8-9292-63482f65f397%40googlegroups.com
<https://groups.google.com/d/msgid/autobahnws/07af9de5-cf4d-49a8-9292-63482f65f397%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

0 Likes

#12

Hi Tobias,

thanks for your reply!

WAMP is language agnostic. That means, given a WAMP procedure, any WAMP client (in any language) is supposed to be able to call the procedure. <<

I realized later that jsonpickle is too Python specific for our purpose. We need communication between Java, Python and JavaScript.

With your code, change this

SERIALIZER_ID = “jsonpickle”

and leave it at "json". <<

I tried this and it did work! Now I know how to specialize Serializer. :-)

>> Don't follow the overrriding "call, register, .." approach .. this leads to insanity. <<

Why is making a custom Serializer better than overriding "call, register, ..."? Because then you have to write it only once?

We would like to go beyond the provided serializer, since we want to support simple nested classes of the type

class NestedClass:

def __init__(self,name,id,anotherObj):

[self.name](http://self.name/) = name

[self.id](http://self.id/) = id

self.anotherObj = anotherObj

class AnotherClass:

def __init__(self,whatever):

self.whatever = whatever

anotherObj = AnotherClass("Bla")

nestedObj = NestedClass("Bla",123456,anotherObj)

The challenge is that this is ambiguous when deserializing. The code would not be able to tell whether a JSON dictionary should become a Python object of some user-defined class or simply a Python dictionary.

To deal with this we are thinking of the following approach:

1. Arguments can be provided as is (e.g. of type NestedClass above) and will be converted to JSON using

json.dumps(obj, default=lambda o: o.__dict__)

2. Return values are produced by

json.loads(jsonString)

and are Python built-ins. When it is a dictionary, it is up to the user of our code to decide whether to convert it into an instance of a user-defined class or leave it as a dictionary.

Does this make sense?

Thanks

Florian

<details class='elided'>
<summary title='Show trimmed content'>&#183;&#183;&#183;</summary>

On Tuesday, December 30, 2014 at 8:25:33 AM UTC-5, Tobias Oberstein wrote:
> Hi Florian,
> 
> WAMP is language agnostic. That means, given a WAMP procedure, _any_ WAMP client (in any language) is supposed to be able to call the procedure.
> 
> That being said, it should still be possible to do if you insist;)
> 
> Don't follow the overrriding "call, register, .." approach .. this leads to insanity. Your approach with JsonPickleObjectSerializer is the right one.
> 
> With your code, change this
> 
> ```
> ```
> SERIALIZER_ID = "jsonpickle"
> 
> 
> ```
> and leave it at "json".
> 
> a) the output of jsonpickle is still JSON (not MsgPack or anything)
> b) no WAMP router will understand how to treat "jsonpickle" serialization
> c) the fact that the JSON is "special" (e.g. contains Python classnames) is irrelevant to a WAMP router
> 
> Of course to make it work, you'll then need to use jsonpickle.encode etc in your code below.
> 
> Cheers,
> /Tobias
> 
> Am Freitag, 19. Dezember 2014 13:41:51 UTC+1 schrieb Florian Conrady:
> > Hi,
> > 

> > I tried to create a simple example where I register and call a procedure that has an instance of a class as argument (see code pasted at bottom). It fails with the following message:

> > 

> > ```
> > ./client3.py session ready
> > procedure registered
> > could not call procedure: Unable to serialize WAMP application payload (<__main__.SomeClass instance at 0x1f86878> is not JSON serializable)
> > ```
> > ```
> > 
> > 
> > ```
> > ```
> > Should serialization of this simple object work out of the box or do I need to do more? What is the right approach for this? (I started specializing JsonSerializer, but that resulted in further problems.)
> > 
> > ```
> > ```
> > 
> > 
> > ```
> > ```
> > Thanks
> > ```
> > ```
> > Florian
> > ```
> > ```
> > 
> > 
> > ```
> > 

> > ```
> > 
> > #!/usr/bin/python
> > from autobahn.twisted.wamp import ApplicationSession
> > from twisted.internet.defer import inlineCallbacks
> > from autobahn.twisted.util import sleep
> > class SomeClass:
> > def __init__(self,name):
> > [self.name](http://self.name) = name
> > class MyComponent(
> >    ApplicationSession):
> > @inlineCallbacks
> > def onJoin(self, details):
> > print("session ready")
> > def printName(obj):
> > print([obj.name](http://obj.name)
> >       )
> > try:
> > yield self.register(printName, u'com.myapp.printname')
> > print("procedure registered")
> > except Exception as e:
> > print("could not register procedure: {0}".format(e))
> > sleep(1)
> > try:
> > someObj = SomeClass("Oberstet")
> > yield self.call(u'com.myapp.         printname', someObj)
> > print("procedure called")
> > except Exception as e:
> > print("could not call procedure: {0}".format(e))
> > from autobahn.twisted.wamp import ApplicationRunner
> > runner = ApplicationRunner(url = "ws://localhost:8080/ws1", realm = "realm1")
> > runner.run(MyComponent)
> > ```

> > 

> > 

> > 

> > 

> > 

> > 

> > 

> > 

> > 

> > 

> > 

> > 

> > 

> >

</details>
0 Likes