Help - Critical issue with Safari ..

#1

Hi, I’ve just run into a bit of a show-stopper, if anyone has any ideas they would be very gratefully received.

My application is loading PDF files out of a MongoDB GridFS … base64 encoding them, then making them available to Javascript clients via RPC. I’ve been testing on Chrome/Firefox and not had any problems … now I have my first Safari users and it just “doesn’t work”.

Specifically, the client makes the call, it sits there for maybe 30 seconds, then the websocket connection closes.

Not a vast amount of debug available, just the socket close message.

What’s worse … sometimes it works, maybe 50-50. So I can retry “the same document”, and it works second time around. Sometimes.

My problem; it works fine in Chrome/Firefox, and I just can’t see a “pattern” in Safari.

I was thinking issues with message size (typically 100k-300k) but that wouldn’t sit with the document working “sometimes”.

Can anyone think of any Mac/Safari issues that might cause a websocket transaction randomly not to complete, then close the socket after a timeout?

Server code looks like this; (note; the “re” is to fix a bug in Safari’s “atob” function)

@wamp.register(u'nac.rpc.get.doc')
def nac_rpc_get_doc(self,params,details):
    try:
        user = self.getSession(details)
        if not user: return self.fail('no such user')
        gridid = params.get('id',None)
        gridid = ObjectId(gridid)
        if not gridid: return self.fail('invalid document id')
        record = self.gridfs.find_one({'_id':gridid})
        if not record: return self.fail('document is missing')
        blob = self.gridfs.get(record._id).read()
        blob = blob.encode('base64')
        pattern = re.compile(r'\s+')
        blob = re.sub(pattern, '', blob)
        return self.ok({'pdf':blob})
    except:
        log.err()

``

Client code looks like this;

function success(data) {
    self.render(data);
};
console.log("[PDF] Fetching content with key (",this.ident,")");
rpc.call('nac.rpc.get.doc',{'id':this.ident},success,failure);

Just for completeness,rpc.call looks like this; (session is an autobahn session)

call: function(topic,args,success,failure) {

session.call(topic,[args],{},{disclose_me:true}).then(success,failure);

}

``

When it fails, “render” is never reached … (there’s a console.log on the first line) …

No errors logged on the server … it’s not rocket science, well … not for Chrome / Firefox … :frowning:

Any ideas?

(and for context, I then feed the data into PDF.js to view the file)

0 Likes

#2

Hi Gareth,

I'm not aware of any specific issue with (modern) Safari.

To track down the issue, here are a couple of ideas:

1) Modify your code for

def nac_rpc_get_doc(self,params,details):
      return u"test"

and see if that works across all browser.

2) Enable wire-logging in AutobahnJS

Add

AUTOBAHN_DEBUG = true;

_before_ importing AutobahnJS, like here

https://github.com/crossbario/crossbarexamples/blob/master/hello/python/web/index.html#L6

3) Enable wire-logging in CB (using trunk):

crossbar start --loglevel=trace

This will spit out an insane amount of log, but you should be able to track down where the call (or it's result) "is lost":

* caller -> CB (CALL)
* CB -> callee (INVOCATION)
* callee -> CB (YIELD)
* CB -> caller (CALL RESULT

4) Different network scenarios

Try running CB / Safari on same PC and via LAN.

···

---

Then, given results of above quick checks, we should be able to track this down further.

Cheers,
/Tobias

Am 30.01.2016 um 00:22 schrieb Gareth Bult:

Hi, I've just run into a bit of a show-stopper, if anyone has any ideas
they would be very gratefully received.

My application is loading PDF files out of a MongoDB GridFS .. base64
encoding them, then making them available to Javascript clients via RPC.
I've been testing on Chrome/Firefox and not had any problems .. now I
have my first Safari users and it just "doesn't work".

Specifically, the client makes the call, it sits there for maybe 30
seconds, then the websocket connection closes.
Not a vast amount of debug available, just the socket close message.

What's worse .. sometimes it works, maybe 50-50. So I can retry "the
same document", and it works second time around. Sometimes.

My problem; it works fine in Chrome/Firefox, and I just can't see a
"pattern" in Safari.
I was thinking issues with message size (typically 100k-300k) but that
wouldn't sit with the document working "sometimes".

Can anyone think of any Mac/Safari issues that might cause a websocket
transaction randomly not to complete, then close the socket after a timeout?

Server code looks like this; (note; the "re" is to fix a bug in Safari's
"atob" function)

>
@wamp.register(u'nac.rpc.get.doc')
defnac_rpc_get_doc(self,params,details):
try:
             user =self.getSession(details)
ifnotuser:returnself.fail('no such user')
             gridid =params.get('id',None)
             gridid =ObjectId(gridid)
ifnotgridid:returnself.fail('invalid document id')
             record =self.gridfs.find_one({'_id':gridid})
ifnotrecord:returnself.fail('document is missing')
             blob =self.gridfs.get(record._id).read()
             blob =blob.encode('base64')
             pattern =re.compile(r'\s+')
             blob =re.sub(pattern,'',blob)
returnself.ok({'pdf':blob})
except:
             log.err()
>

Client code looks like this;

>
functionsuccess(data){
self.render(data);
};
     console.log("[PDF] Fetching content with key (",this.ident,")");
     rpc.call('nac.rpc.get.doc',{'id':this.ident},success,failure);

Justforcompleteness,rpc.call looks like this; (session is an autobahn
session)

call: function(topic,args,success,failure) {
     session.call(topic,[args],{},{disclose_me:true}).then(success,failure);
}
>
When it fails, "render" is never reached .. (there's a console.log on
the first line) ...

No errors logged on the server ... it's not rocket science, well .. not
for Chrome / Firefox .. :frowning:

Any ideas?

(and for context, I then feed the data into PDF.js to view the file)

--
You received this message because you are subscribed to the Google
Groups "Crossbar" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to crossbario+...@googlegroups.com
<mailto:crossbario+...@googlegroups.com>.
To post to this group, send email to cross...@googlegroups.com
<mailto:cross...@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/crossbario/28c11fbd-89d0-4b5e-837f-362fba3d5b3e%40googlegroups.com
<https://groups.google.com/d/msgid/crossbario/28c11fbd-89d0-4b5e-837f-362fba3d5b3e%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

0 Likes

#3

Ok, thanks for that, debugging in progress … first thing I’ve found - this issue is “wss” only, seems to work fine on “ws” !!

···

On Friday, 29 January 2016 23:22:28 UTC, Gareth Bult wrote:

Hi, I’ve just run into a bit of a show-stopper, if anyone has any ideas they would be very gratefully received.

My application is loading PDF files out of a MongoDB GridFS … base64 encoding them, then making them available to Javascript clients via RPC. I’ve been testing on Chrome/Firefox and not had any problems … now I have my first Safari users and it just “doesn’t work”.

Specifically, the client makes the call, it sits there for maybe 30 seconds, then the websocket connection closes.

Not a vast amount of debug available, just the socket close message.

What’s worse … sometimes it works, maybe 50-50. So I can retry “the same document”, and it works second time around. Sometimes.

My problem; it works fine in Chrome/Firefox, and I just can’t see a “pattern” in Safari.

I was thinking issues with message size (typically 100k-300k) but that wouldn’t sit with the document working “sometimes”.

Can anyone think of any Mac/Safari issues that might cause a websocket transaction randomly not to complete, then close the socket after a timeout?

Server code looks like this; (note; the “re” is to fix a bug in Safari’s “atob” function)

@wamp.register(u'nac.rpc.get.doc')
def nac_rpc_get_doc(self,params,details):
    try:
        user = self.getSession(details)
        if not user: return self.fail('no such user')
        gridid = params.get('id',None)
        gridid = ObjectId(gridid)
        if not gridid: return self.fail('invalid document id')
        record = self.gridfs.find_one({'_id':gridid})
        if not record: return self.fail('document is missing')
        blob = self.gridfs.get(record._id).read()
        blob = blob.encode('base64')
        pattern = re.compile(r'\s+')
        blob = re.sub(pattern, '', blob)
        return self.ok({'pdf':blob})
    except:
        log.err()

``

Client code looks like this;

function success(data) {
    self.render(data);
};
console.log("[PDF] Fetching content with key (",this.ident,")");
rpc.call('nac.rpc.get.doc',{'id':this.ident},success,failure);

Just for completeness,rpc.call looks like this; (session is an autobahn session)

call: function(topic,args,success,failure) {

session.call(topic,[args],{},{disclose_me:true}).then(success,failure);

}

``

When it fails, “render” is never reached … (there’s a console.log on the first line) …

No errors logged on the server … it’s not rocket science, well … not for Chrome / Firefox … :frowning:

Any ideas?

(and for context, I then feed the data into PDF.js to view the file)

0 Likes

#4

Ok, here’s my summary so far;

Server: Crossbar 0.11.1

Server: Autobahn 0.10.9
Client: AutobahnJS 0.9.9

Code seems to work perfectly on Chrome/Firefox, issue below when using Safari.

If I switch to NON-SSL, it works perfectly on all three (!)

I’m doing “everything” over the WSS connection, and it’s “just” the large transaction (PDF) that are failing, and it doesn’t fail consistently, maybe 100% for 20 secs, then 60% for 30 secs, then 80% for 20 secs - if I repeatedly load stuff.

On the server …

[Controller 18529 crossbar.router.router.Router]^[[39m Router.process: WAMP PUBLISH Message … (this message contains the actual PDF content, which is logged)

[Router 18534 crossbar.router.router.Router]^[[39m Validate ‘call_result’ for ‘nac.rpc.get.doc’

[Controller 18529 crossbar.router.router.Router]^[[39m Router.process: WAMP PUBLISH Message (request = 678, …, args = (u"Validate ‘call_result’ …

[Router 18534 crossbar.router.protocol.WampWebSocketServerFactory]^[[39m Connection to/from tcp4:xxx.xxx.xx.xx:50043 was lost in a non-clean fashion: Connection lost

[Controller 18529 crossbar.router.router.Router]^[[39m Router.process: WAMP PUBLISH Message (request = 679, … args = (u’Connection to/from tcp4:xxx.xxx.xx.xx:50043 was lost in a non-clean fashion: Connection lost’,)

On the client;

[Log] Call (nac.rpc.get.doc) Args> – {id: “56ab7ebde82c5c2228c8e784”} (ionman.js, line 241)
[Log] [48, 8295663626354688, {disclose_me: true}, “nac.rpc.get.doc”, [{id: “56ab7ebde82c5c2228c8e784”}], {}] (6) (autobahn.min.js, line 32)
[Log] WebSocket transport send – “[48,8295663626354688,{“disclose_me”:true},“nac.rpc.get.doc”,[{“id”:“56ab7ebde82c5c2228c8e784”}],{}]” (autobahn.min.js, line 32)
[Log] [close] - – “Session closed” (ionman.js, line 29)
[Log] > Reason: – “lost” (ionman.js, line 211)

… I read this as; “the client is crashing while receiving data”.

… without resorting to reading code that is going to take me weeks to get into, my gut is saying “buffer overrun somewhere in autobahn SSL code that is only fatal on Safari” …

(but I’m likely completely wrong … :slight_smile: )

Any ideas from this?

Regards,

Gareth.

0 Likes

#5

... I read this as; "the client is crashing while receiving data".
... without resorting to reading code that is going to take me weeks to
get into, my gut is saying "buffer overrun somewhere in autobahn SSL
code that is only fatal on Safari" ...

A buffer overflow in AutobahnPython or Crossbar.io is unlikely for a simple reason: there is no C code at all in these libraries;)

The SSL support of these libraries is based on OpenSSL (which is C code of course), and used via PyOpenSSL (a wrapper on top of the native code).

The OpenSSL library itself, and the PyOpenSSL wrapper run native code - and hence these in principle can trigger classic C issues.

I think that's unlikely too. It's more likely a Safari issue. But further investigation is needed.

What kind of network connection is there between Safari and CB? Any firewalls, proxies, etc?

What TLS ciphers are negotiated when running with Safari? (Try fiddling with CB permissible ciphers configuration).

Have you enabled WebSocket compression? (If so, disable that in CB).

Did you try IE10+? (If 3 browsers work, and one doesn't, that would be a further hint that it's really Safari related)

Cheers,
/Tobias

0 Likes

#6

Hi,

I appreciate the unlikely nature of the fault … that said, from the logs you will appreciate that there’s little scope for the issue being in the ‘application’ … (?)

So;

Proxies; the site I’m testing against is a standard VM running on Digital Ocean.

Firewall looks like this;

interface eth0 public
protection strong
server icmp accept
server https accept
server tinc accept
server smtp accept
client all accept

``

Client(s) are just standard home-user broadband, I have 1 x Random Mac user, + I have a Mac here … both show identical problems.

This is my CB SSL config;

     "endpoint": {
        "type": "tcp",
        "port": 8443,
        "tls": {
          "key" : "server_key.pem",
          "certificate" : "server_cert.pem",
          "dhparam": "dhparam.pem"
        }
      },

``

So I’m just using the CD default for ciphers … the only thing the the log file that looks relevant is;

Ok, OpenSSL is using ECDH elliptic curve prime256v1

``

Compression is not enabled.

I know it sounds mad, but I don’t actually have access to Internet Explorer … I’m one of those wierdo’s who refuses to allow “Windows” on their network … :slight_smile:

Caveat; I have seen “occasional” issues displaying PDF’s on Chrome that I have attributed to a coding bug that I’ve not yet found, but it’s very occasional

and I’ve not been able to reproduce it. However, I am starting to wonder whether it’s a generic problem that’s just hitting Safari harder than other browsers …

??

I’m not really up on SSL ciphers, could you recommend one that’s least likely to cause a problem?

Gareth.

···

On Sunday, 31 January 2016 08:59:27 UTC, Tobias Oberstein wrote:

… I read this as; “the client is crashing while receiving data”.

… without resorting to reading code that is going to take me weeks to

get into, my gut is saying "buffer overrun somewhere in autobahn SSL

code that is only fatal on Safari" …

A buffer overflow in AutobahnPython or Crossbar.io is unlikely for a
simple reason: there is no C code at all in these libraries;)

The SSL support of these libraries is based on OpenSSL (which is C code
of course), and used via PyOpenSSL (a wrapper on top of the native code).

The OpenSSL library itself, and the PyOpenSSL wrapper run native code -
and hence these in principle can trigger classic C issues.

I think that’s unlikely too. It’s more likely a Safari issue. But
further investigation is needed.

What kind of network connection is there between Safari and CB? Any
firewalls, proxies, etc?

What TLS ciphers are negotiated when running with Safari? (Try fiddling
with CB permissible ciphers configuration).

Have you enabled WebSocket compression? (If so, disable that in CB).

Did you try IE10+? (If 3 browsers work, and one doesn’t, that would be a
further hint that it’s really Safari related)

Cheers,

/Tobias

0 Likes