autobahn.wamp.exception.SerializationError: Unable to serialize WAMP application payload ()

#1

When I add a component in the .crossbar/config.json and start crossbar, I face that behavior once the data have been retrieve by the publisher.

autobahn.wamp.exception.SerializationError: Unable to serialize WAMP application payload (can’t serialize time.struct_time(tm_year=2015, tm_mon=3, tm_mday=2, tm_hour=11, tm_min=53, tm_sec=0, tm_wday=0, tm_yday=61, tm_isdst=0))

When I dont add the component to .crossbar/config and start crossbar, then start the publisher all went well…

Do you know a trick that fixes that behavior or do you need the piece of code that raise that error ?

0 Likes

#2

once notice :
I install crossbar that way :

pip install crossbar

then

pip install autobahn[twisted,accelerate,compress,serialization]

To enable serialization

···

Le lundi 2 mars 2015 14:39:16 UTC+1, foxmask a écrit :

When I add a component in the .crossbar/config.json and start crossbar, I face that behavior once the data have been retrieve by the publisher.

autobahn.wamp.exception.SerializationError: Unable to serialize WAMP application payload (can’t serialize time.struct_time(tm_year=2015, tm_mon=3, tm_mday=2, tm_hour=11, tm_min=53, tm_sec=0, tm_wday=0, tm_yday=61, tm_isdst=0))

When I dont add the component to .crossbar/config and start crossbar, then start the publisher all went well…

Do you know a trick that fixes that behavior or do you need the piece of code that raise that error ?

0 Likes

#3

The error says it all: time.struct_time cannot be serialized to JSON.

/Tobias

···

Am 02.03.2015 um 14:39 schrieb foxmask:

When I add a component in the .crossbar/config.json and start crossbar,
I face that behavior once the data have been retrieve by the publisher.

autobahn.wamp.exception.SerializationError: Unable to serialize WAMP
application payload (can't serialize time.struct_time(tm_year=2015,
tm_mon=3, tm_mday=2, tm_hour=11, tm_min=53, tm_sec=0, tm_wday=0,
tm_yday=61, tm_isdst=0))

0 Likes

#4

I’ve well understood that

but with crossbar 0.10.1 and autobahn 0.10.0 I dont have any issue like that .

This occurs with crossbar 0.10.2 + autobahn 0.10.1

···

Le lundi 2 mars 2015 16:04:48 UTC+1, Tobias Oberstein a écrit :

Am 02.03.2015 um 14:39 schrieb foxmask:

When I add a component in the .crossbar/config.json and start crossbar,

I face that behavior once the data have been retrieve by the publisher.

autobahn.wamp.exception.SerializationError: Unable to serialize WAMP

application payload (can’t serialize time.struct_time(tm_year=2015,

tm_mon=3, tm_mday=2, tm_hour=11, tm_min=53, tm_sec=0, tm_wday=0,

tm_yday=61, tm_isdst=0))

The error says it all: time.struct_time cannot be serialized to JSON.

/Tobias

0 Likes

#5

I've well understood that

but with crossbar 0.10.1 and autobahn 0.10.0 I dont have any issue like
that .

I doubt that.

time.struct_time can't be serialized .. and this did not change over AutobahnPython versions.

oberstet@thinkpad-t430s:~/scm/scratchbox$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> ts = time.localtime()
>>> type(ts)
<type 'time.struct_time'>
>>> import json
>>> json.dumps(ts)
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "/usr/lib/python2.7/json/__init__.py", line 243, in dumps
     return _default_encoder.encode(obj)
   File "/usr/lib/python2.7/json/encoder.py", line 207, in encode
     chunks = self.iterencode(o, _one_shot=True)
   File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode
     return _iterencode(o, 0)
   File "/usr/lib/python2.7/json/encoder.py", line 184, in default
     raise TypeError(repr(o) + " is not JSON serializable")
TypeError: time.struct_time(tm_year=2015, tm_mon=3, tm_mday=2, tm_hour=16, tm_min=59, tm_sec=56, tm_wday=0, tm_yday=61, tm_isdst=0) is not JSON serializable
>>>

/Tobias

···

Am 02.03.2015 um 16:27 schrieb foxmask:

This occurs with crossbar 0.10.2 + autobahn 0.10.1

Le lundi 2 mars 2015 16:04:48 UTC+1, Tobias Oberstein a écrit :

    Am 02.03.2015 um 14:39 schrieb foxmask:
     > When I add a component in the .crossbar/config.json and start
    crossbar,
     > I face that behavior once the data have been retrieve by the
    publisher.
     >
     > autobahn.wamp.exception.SerializationError: Unable to serialize WAMP
     > application payload (can't serialize time.struct_time(tm_year=2015,
     > tm_mon=3, tm_mday=2, tm_hour=11, tm_min=53, tm_sec=0, tm_wday=0,
     > tm_yday=61, tm_isdst=0))

    The error says it all: time.struct_time cannot be serialized to JSON.

    /Tobias

--
You received this message because you are subscribed to the Google
Groups "Autobahn" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to autobahnws+...@googlegroups.com
<mailto:autobahnws+...@googlegroups.com>.
To post to this group, send email to autob...@googlegroups.com
<mailto:autob...@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/autobahnws/4cbcc608-0e80-4ccf-b719-44a5d5afcc2d%40googlegroups.com
<https://groups.google.com/d/msgid/autobahnws/4cbcc608-0e80-4ccf-b719-44a5d5afcc2d%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

0 Likes

#6

Ok …
So :

I’ve a publisher that get feeds rss url (and date of the last handling) from the database
the url is then loaded by feedparser ; and for each item I check the last time the handling has been done
when one item has not been handled I add it to a dict and returnValue() on it.
then for each item I publish it.
on the other side the subcriber receive the stuff.
with crossbar0.10.1 / autobahn 0.10.0 all of this is ok

you can have a look at the 2 components if you want :

  1. https://github.com/foxmask/wamp-th/blob/master/components/th_rss/back.py
  2. https://github.com/foxmask/wamp-th/blob/master/components/th_evernote/front.py
    on the front.py line 255 I added print(data) and as you can see the date (published propery) of the RSS is well retrieved
    2015-03-02 17:12:26+0100 [WampWebSocketClientProtocol,client] *************************************
    2015-03-02 17:12:26+0100 [WampWebSocketClientProtocol,client] Internet
    2015-03-02 17:12:26+0100 [WampWebSocketClientProtocol,client]
    2015-03-02 17:12:26+0100 [WampWebSocketClientProtocol,client] Flux de Strict minimum
    2015-03-02 17:12:26+0100 [WampWebSocketClientProtocol,client] {u’summary_detail’: {u’base’: u’http://blog.timothe-leroy.com/feed/tumblog/’, u’type’: u’text/html’, u’value’: u’

    http://www.gameblog.fr/news/36845-ouya-le-coup-de-gueule-du-president-de-mo5

    \n\t… et je partage totalement son avis

    Laisser un commentaire

    ’, u’language’: None}, u’published_parsed’: {u’tm_sec’: 0, u’n_sequence_fields’: 9, u’n_unnamed_fields’: 0, u’tm_mday’: 18, u’tm_isdst’: 0, u’tm_year’: 2013, u’tm_mon’: 7, u’tm_yday’: 199, u’tm_hour’: 9, u’n_fields’: 9, u’tm_wday’: 3, u’tm_min’: 47}, u’links’: [{u’href’: u’http://www.gameblog.fr/news/36845-ouya-le-coup-de-gueule-du-president-de-mo5’, u’type’: u’text/html’, u’rel’: u’alternate’}], u’author’: u’Tim’, u’title’: u’Point de vue du pr\xe9sident de MO5 sur la console Ouya’, u’slash_comments’: u’0’, u’comments’: u’http://blog.timothe-leroy.com/point-de-vue-du-president-de-mo5-sur-la-console-ouya/#comments’, u’summary’: u’

    http://www.gameblog.fr/news/36845-ouya-le-coup-de-gueule-du-president-de-mo5

    \n\t… et je partage totalement son avis

    Laisser un commentaire

    ’, u’content’: [{u’base’: u’http://blog.timothe-leroy.com/feed/tumblog/’, u’type’: u’text/html’, u’value’: u’

    http://www.gameblog.fr/news/36845-ouya-le-coup-de-gueule-du-president-de-mo5

    \n\t

    … et je partage totalement son avis

    \n

    Laisser un commentaire

    ’, u’language’: None}], u’guidislink’: False, u’title_detail’: {u’base’: u’http://blog.timothe-leroy.com/feed/tumblog/’, u’type’: u’text/plain’, u’value’: u’Point de vue du pr\xe9sident de MO5 sur la console Ouya’, u’language’: None}, u’link’: u’http://www.gameblog.fr/news/36845-ouya-le-coup-de-gueule-du-president-de-mo5’, u’authors’: [{}], u’author_detail’: {u’name’: u’Tim’}, u’wfw_commentrss’: u’http://blog.timothe-leroy.com/point-de-vue-du-president-de-mo5-sur-la-console-ouya/feed/’, u’id’: u’http://www.gameblog.fr/news/36845-ouya-le-coup-de-gueule-du-president-de-mo5’, u’tags’: [{u’term’: u’Best of’, u’scheme’: None, u’label’: None}], u’published’: u’Thu, 18 Jul 2013 09:47:00 +0000’}

So, my understanding is ; as the data I retrieve from back.py are exchange to front.py and can be displayed ; there is no problem of serialization, isnt it ?

Or am I in a special case that bypass the serialization issue ?

···

Le lundi 2 mars 2015 17:00:42 UTC+1, Tobias Oberstein a écrit :

Am 02.03.2015 um 16:27 schrieb foxmask:

I’ve well understood that

but with crossbar 0.10.1 and autobahn 0.10.0 I dont have any issue like

that .

I doubt that.

time.struct_time can’t be serialized … and this did not change over
AutobahnPython versions.

oberstet@thinkpad-t430s:~/scm/scratchbox$ python

Python 2.7.6 (default, Mar 22 2014, 22:59:56)

[GCC 4.8.2] on linux2

Type “help”, “copyright”, “credits” or “license” for more information.

import time

ts = time.localtime()

type(ts)

<type ‘time.struct_time’>

import json

json.dumps(ts)

Traceback (most recent call last):

File “”, line 1, in

File “/usr/lib/python2.7/json/init.py”, line 243, in dumps

 return _default_encoder.encode(obj)

File “/usr/lib/python2.7/json/encoder.py”, line 207, in encode

 chunks = self.iterencode(o, _one_shot=True)

File “/usr/lib/python2.7/json/encoder.py”, line 270, in iterencode

 return _iterencode(o, 0)

File “/usr/lib/python2.7/json/encoder.py”, line 184, in default

 raise TypeError(repr(o) + " is not JSON serializable")

TypeError: time.struct_time(tm_year=2015, tm_mon=3, tm_mday=2,
tm_hour=16, tm_min=59, tm_sec=56, tm_wday=0, tm_yday=61, tm_isdst=0) is
not JSON serializable

/Tobias

This occurs with crossbar 0.10.2 + autobahn 0.10.1

Le lundi 2 mars 2015 16:04:48 UTC+1, Tobias Oberstein a écrit :

Am 02.03.2015 um 14:39 schrieb foxmask:
 > When I add a component in the .crossbar/config.json and start
crossbar,
 > I face that behavior once the data have been retrieve by the
publisher.
 >
 > autobahn.wamp.exception.SerializationError: Unable to serialize WAMP
 > application payload (can't serialize time.struct_time(tm_year=2015,
 > tm_mon=3, tm_mday=2, tm_hour=11, tm_min=53, tm_sec=0, tm_wday=0,
 > tm_yday=61, tm_isdst=0))
The error says it all: time.struct_time cannot be serialized to JSON.
/Tobias

You received this message because you are subscribed to the Google

Groups “Autobahn” group.

To unsubscribe from this group and stop receiving emails from it, send

an email to autobahnws+...@googlegroups.com

mailto:autobahnws+unsub...@googlegroups.com.

To post to this group, send email to auto...@googlegroups.com

mailto:autob...@googlegroups.com.

To view this discussion on the web visit

https://groups.google.com/d/msgid/autobahnws/4cbcc608-0e80-4ccf-b719-44a5d5afcc2d%40googlegroups.com

<https://groups.google.com/d/msgid/autobahnws/4cbcc608-0e80-4ccf-b719-44a5d5afcc2d%40googlegroups.com?utm_medium=email&utm_source=footer>.

For more options, visit https://groups.google.com/d/optout.

0 Likes

#7

The same happens with RPC calls and this is bad because then the caller gets no error response either, and is left waiting forever. I consider this a major bug in Autobahn. If serialization fails, the caller should get a response saying that the call failed because the result couldn’t be serialized – or something similar.

maanantai 2. maaliskuuta 2015 15.39.16 UTC+2 foxmask kirjoitti:

···

When I add a component in the .crossbar/config.json and start crossbar, I face that behavior once the data have been retrieve by the publisher.

autobahn.wamp.exception.SerializationError: Unable to serialize WAMP application payload (can’t serialize time.struct_time(tm_year=2015, tm_mon=3, tm_mday=2, tm_hour=11, tm_min=53, tm_sec=0, tm_wday=0, tm_yday=61, tm_isdst=0))

When I dont add the component to .crossbar/config and start crossbar, then start the publisher all went well…

Do you know a trick that fixes that behavior or do you need the piece of code that raise that error ?

0 Likes

#8

Hi Alex,

> The same happens with RPC calls and this is bad because then the caller

gets no error response either, and is left waiting forever. I consider
this a major bug in Autobahn. If serialization fails, the caller should
get a response saying that the call failed because the result couldn't
be serialized -- or something similar.

The serialization already fails in the client library of the callee. But yes, this doesn't lead to an error being sent to the peer. This is a bug, I agree

https://github.com/tavendo/AutobahnPython/issues/349

Further, even with this, we need proper timeouts inside the router already (an invocation of an endpoint might not come back for other reasons ..).

Cheers,
/Tobias

0 Likes

#9

So am i wrong ? If so how to handle such data ? If not is that a bug ?

0 Likes

#10

Don’t try to send data that contains struct_times. Alternatively you can customize the msgpack serializer to serialize those to either strings or custom types. I’ve done something similar in my own app and I made a post about it just a while ago, though it never got any replies: https://groups.google.com/forum/#!topic/autobahnws/T8i-ldq-Piw
To integrate this with autobahn, you need code like this:


def patch_autobahn_msgpack():
from types import ModuleType
from autobahn.wamp import serializer
    serializer.msgpack = ModuleType('msgpack')
    serializer.msgpack.packb = partial(packb, default=default)
    serializer.msgpack.unpackb = partial(unpackb, ext_hook=ext_hook)

maanantai 2. maaliskuuta 2015 21.17.31 UTC+2 foxmask kirjoitti:

<details class='elided'>
<summary title='Show trimmed content'>&#183;&#183;&#183;</summary>

> So am i wrong ? If so how to handle such data ? If not is that a bug ?

</details>
0 Likes

#11

Don't try to send data that contains struct_times. Alternatively you can

Exactly. In the case of struct_times, just convert it to a string according to a standard (like ISO8601)

>>> import time
>>> ts = time.gmtime()
>>> time.strftime("%Y-%m-%dT%H:%M:%SZ", ts)
'2015-03-02T20:10:27Z'

customize the msgpack serializer to serialize those to either strings or
custom types. I've done something similar in my own app and I made a
post about it just a while ago, though it never got any
replies: https://groups.google.com/forum/#!topic/autobahnws/T8i-ldq-Piw

Sorry, I am still catching up with mailing list posts (answered a bunch recently).

Custom serialization to string works, but is of limited value, since the deserialization will not recreate the original types.

Custom serialization to custom types is only possible with msgpack, and won't work in general, since a router will need to reserialize to other formats (JSON) transparently.

E.g. a browser speaking WAMP-over-WebSocket/JSON to a C++ client speaking WAMP-over-RawSocket/MsgPack.

···

Am 02.03.2015 um 21:00 schrieb Alex Grönholm:

To integrate this with autobahn, you need code like this:

defpatch_autobahn_msgpack():

     fromtypesimportModuleType

     fromautobahn.wampimportserializer
     serializer.msgpack = ModuleType('msgpack')
     serializer.msgpack.packb = partial(packb,default=default)
     serializer.msgpack.unpackb = partial(unpackb,ext_hook=ext_hook)

maanantai 2. maaliskuuta 2015 21.17.31 UTC+2 foxmask kirjoitti:

    So am i wrong ? If so how to handle such data ? If not is that a bug ?

--
You received this message because you are subscribed to the Google
Groups "Autobahn" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to autobahnws+...@googlegroups.com
<mailto:autobahnws+...@googlegroups.com>.
To post to this group, send email to autob...@googlegroups.com
<mailto:autob...@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/autobahnws/a2399929-4d3a-4c01-a293-46d9459fedf0%40googlegroups.com
<https://groups.google.com/d/msgid/autobahnws/a2399929-4d3a-4c01-a293-46d9459fedf0%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

0 Likes

#12

Hi
I shouldn’t have to do this as the date and time I get come :

  1. from a column of a table of my database which is in expected format
  2. from RSS data (usually the published node) which are also in the good format.

The fact is ; it’s not my code the play with the datetime format

And once again I dont have this behavior with autobahn 0.10.0 / crossbar 0.10.1 ; all work fine.

Regards

···

Le lundi 2 mars 2015 21:14:08 UTC+1, Tobias Oberstein a écrit :

Am 02.03.2015 um 21:00 schrieb Alex Grönholm:

Don’t try to send data that contains struct_times. Alternatively you can

Exactly. In the case of struct_times, just convert it to a string
according to a standard (like ISO8601)

import time

ts = time.gmtime()

time.strftime("%Y-%m-%dT%H:%M:%SZ", ts)

‘2015-03-02T20:10:27Z’

customize the msgpack serializer to serialize those to either strings or

custom types. I’ve done something similar in my own app and I made a

post about it just a while ago, though it never got any

replies: https://groups.google.com/forum/#!topic/autobahnws/T8i-ldq-Piw

Sorry, I am still catching up with mailing list posts (answered a bunch
recently).

Custom serialization to string works, but is of limited value, since the
deserialization will not recreate the original types.

Custom serialization to custom types is only possible with msgpack, and
won’t work in general, since a router will need to reserialize to other
formats (JSON) transparently.

E.g. a browser speaking WAMP-over-WebSocket/JSON to a C++ client
speaking WAMP-over-RawSocket/MsgPack.

To integrate this with autobahn, you need code like this:

defpatch_autobahn_msgpack():

 fromtypesimportModuleType
 fromautobahn.wampimportserializer
 serializer.msgpack = ModuleType('msgpack')
 serializer.msgpack.packb = partial(packb,default=default)
 serializer.msgpack.unpackb = partial(unpackb,ext_hook=ext_hook)

maanantai 2. maaliskuuta 2015 21.17.31 UTC+2 foxmask kirjoitti:

So am i wrong ? If so how to handle such data ? If not is that a bug ?

You received this message because you are subscribed to the Google

Groups “Autobahn” group.

To unsubscribe from this group and stop receiving emails from it, send

an email to autobahnws+...@googlegroups.com

mailto:autobahnws+unsub...@googlegroups.com.

To post to this group, send email to auto...@googlegroups.com

mailto:autob...@googlegroups.com.

To view this discussion on the web visit

https://groups.google.com/d/msgid/autobahnws/a2399929-4d3a-4c01-a293-46d9459fedf0%40googlegroups.com

<https://groups.google.com/d/msgid/autobahnws/a2399929-4d3a-4c01-a293-46d9459fedf0%40googlegroups.com?utm_medium=email&utm_source=footer>.

For more options, visit https://groups.google.com/d/optout.

0 Likes