How do cases get closed?

#1

Hi there,

I'm working on ws4py and obviously I'm using Autobahn's great
testsuite. Most of my tests pass but all of them are marked as
uncleanly closed.

What I don't understand is how the client actually close the stream.
Let's take case 1.1.1, it says it should be closed by the client but I
fail to see when that happens. In any case, the connection is
eventually killed and my test marked as unclean.

Some tests on the other hand do use the closeAfter method to
explicitely close the stream after a delay.

Could someone shed a light because this drives me nuts :slight_smile:

Cheers,
- Sylvain

0 Likes

#2

Hello Sylvain,

taking case 1.1.1 as an example, the case sends a text message and then waits 1s before initiating a
WS closing handshake with the peer (the waiting is done because the case expects the text message
to be echo'ed by the peer):

https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/case/case1_1_1.py#L32
=>
https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/fuzzing.py#L267
=>
https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/fuzzing.py#L275

The case will then be marked as "clean close" if the peer responds to the closing handshake
by sending back a close frame and - if the peer is a server - drops the TCP.

This pattern is the same for all cases where the case does something fully valid. When the
case does something violating the protocol or the case is explicitly testing close behavior
things will be different ..

Hope this helps,
Tobias

···

Am 14.12.2011 22:48, schrieb Sylvain Hellegouarch:

Hi there,

I'm working on ws4py and obviously I'm using Autobahn's great
testsuite. Most of my tests pass but all of them are marked as
uncleanly closed.

What I don't understand is how the client actually close the stream.
Let's take case 1.1.1, it says it should be closed by the client but I
fail to see when that happens. In any case, the connection is
eventually killed and my test marked as unclean.

Some tests on the other hand do use the closeAfter method to
explicitely close the stream after a delay.

Could someone shed a light because this drives me nuts :slight_smile:

Cheers,
- Sylvain

0 Likes

#3

Hi Tobias,

Thanks for the swift answer. I guess I’m confused at the fact that the tests fails saying that the server didn’t close the socket in due time. If I look at how long the test took it says: 1001ms. Considering the 1s delay your test use before sending the closing handshake, and the fact that the test says that the server did return the appropriate closing code, I don’t understand the unclean state.

I can see in my server that I reply to the closing handshake as soon as I receive it.

Any idea?

  • Sylvain
···

On Thu, Dec 15, 2011 at 8:19 AM, Tobias Oberstein tobias.o...@gmail.com wrote:

Hello Sylvain,

taking case 1.1.1 as an example, the case sends a text message and then waits 1s before initiating a

WS closing handshake with the peer (the waiting is done because the case expects the text message

to be echo’ed by the peer):

https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/case/case1_1_1.py#L32

=>

https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/fuzzing.py#L267

=>

https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/fuzzing.py#L275

The case will then be marked as “clean close” if the peer responds to the closing handshake

by sending back a close frame and - if the peer is a server - drops the TCP.

This pattern is the same for all cases where the case does something fully valid. When the

case does something violating the protocol or the case is explicitly testing close behavior

things will be different …

Hope this helps,

Tobias

Am 14.12.2011 22:48, schrieb Sylvain Hellegouarch:

Hi there,

I’m working on ws4py and obviously I’m using Autobahn’s great

testsuite. Most of my tests pass but all of them are marked as

uncleanly closed.

What I don’t understand is how the client actually close the stream.

Let’s take case 1.1.1, it says it should be closed by the client but I

fail to see when that happens. In any case, the connection is

eventually killed and my test marked as unclean.

Some tests on the other hand do use the closeAfter method to

explicitely close the stream after a delay.

Could someone shed a light because this drives me nuts :slight_smile:

Cheers,

  • Sylvain

0 Likes

#4

Does your server drop the TCP after having replied to the close
frame sent by the fuzzing client?

To get close clean, the appropriate peer must drop the TCP (there is

text with SHOULD in the spec about that).

The appropriate peer to drop the TCP is the server. Server SHOULD

drop, client SHOULD NOT drop. If thats

violated, case get close unclean.

What does the detail report say in the closing behavior block? In

particular in

wasNotCleanReason
None
When wasClean == False, the reason what
happened.
wasServerConnectionDropTimeout
False
When we are a client, and we expected the
server to drop the TCP, but that didn’t happen in time, this
gets True.

Compare to ..

or reply and attach the detail report HTML so I can have a look ..
···

http://www.tavendo.de/autobahn/testsuite/report/servers/autobahnserver_0_4_3_case_1_1_1.html

Hi Tobias,

    Thanks for the swift answer. I guess I'm confused at the fact

that the tests fails saying that the server didn’t close the
socket in due time. If I look at how long the test took it says:
1001ms. Considering the 1s delay your test use before sending
the closing handshake, and the fact that the test says that the
server did return the appropriate closing code, I don’t
understand the unclean state.

    I can see in my server that I reply to the closing handshake

as soon as I receive it.

Any idea?

  • Sylvain

        On Thu, Dec 15, 2011 at 8:19 AM, Tobias
    

Oberstein tobias.o...@gmail.com
wrote:

        Hello

Sylvain,

        taking case 1.1.1 as an example, the case sends a text

message and then waits 1s before initiating a

        WS closing handshake with the peer (the waiting is done

because the case expects the text message

        to be echo'ed by the peer):



        [https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/case/case1_1_1.py#L32](https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/case/case1_1_1.py#L32)

        =>

        [https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/fuzzing.py#L267](https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/fuzzing.py#L267)

        =>

        [https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/fuzzing.py#L275](https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/fuzzing.py#L275)



        The case will then be marked as "clean close" if the peer

responds to the closing handshake

        by sending back a close frame and - if the peer is a server
  • drops the TCP.

          This pattern is the same for all cases where the case does
    

something fully valid. When the

        case does something violating the protocol or the case is

explicitly testing close behavior

        things will be different ..



        Hope this helps,

        Tobias







        Am 14.12.2011 22:48, schrieb Sylvain Hellegouarch:

Hi there,

          I'm working on ws4py and obviously I'm using Autobahn's

great

          testsuite. Most of my tests pass but all of them are

marked as

          uncleanly closed.



          What I don't understand is how the client actually close

the stream.

          Let's take case 1.1.1, it says it should be closed by the

client but I

          fail to see when that happens. In any case, the connection

is

          eventually killed and my test marked as unclean.



          Some tests on the other hand do use the closeAfter method

to

          explicitely close the stream after a delay.



          Could someone shed a light because this drives me nuts :)



          Cheers,

          - Sylvain

    - Sylvain

    [http://www.defuze.org](http://www.defuze.org)

    [http://twitter.com/lawouach](http://twitter.com/lawouach)
0 Likes

#5

Dang my report is at home and I don’t have it at hand.

Maybe should we agree on what you mean by dropping the connection.

My code does the following:

  1. Receives the closing frame from client

  2. Replies with a closing frame with the same code

  3. Closes the socket using: shutdown(RD), close()

All those are done sequentially but without pause.

Isn’t it the expected behavior?

  • Sylvain
···

On Thu, Dec 15, 2011 at 9:45 AM, Tobias Oberstein tobias.o...@gmail.com wrote:

Does your server drop the TCP after having replied to the close

frame sent by the fuzzing client?

To get close clean, the appropriate peer must drop the TCP (there is

text with SHOULD in the spec about that).

The appropriate peer to drop the TCP is the server. Server SHOULD

drop, client SHOULD NOT drop. If thats

violated, case get close unclean.



What does the detail report say in the closing behavior block? In

particular in

wasNotCleanReason
None
When wasClean == False, the reason what
happened.
wasServerConnectionDropTimeout
False
When we are a client, and we expected the
server to drop the TCP, but that didn’t happen in time, this
gets True.

Compare to ..

http://www.tavendo.de/autobahn/testsuite/report/servers/autobahnserver_0_4_3_case_1_1_1.html

or reply and attach the detail report HTML so I can have a look ..



Am 15.12.2011 09:38, schrieb Sylvain Hellegouarch:

Hi Tobias,

    Thanks for the swift answer. I guess I'm confused at the fact

that the tests fails saying that the server didn’t close the
socket in due time. If I look at how long the test took it says:
1001ms. Considering the 1s delay your test use before sending
the closing handshake, and the fact that the test says that the
server did return the appropriate closing code, I don’t
understand the unclean state.

    I can see in my server that I reply to the closing handshake

as soon as I receive it.

Any idea?

  • Sylvain

        On Thu, Dec 15, 2011 at 8:19 AM, Tobias > > Oberstein <tobias.o...@gmail.com> > >           wrote:
    
        Hello

Sylvain,

        taking case 1.1.1 as an example, the case sends a text

message and then waits 1s before initiating a

        WS closing handshake with the peer (the waiting is done

because the case expects the text message

        to be echo'ed by the peer):



        [https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/case/case1_1_1.py#L32](https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/case/case1_1_1.py#L32)


        =>

        [https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/fuzzing.py#L267](https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/fuzzing.py#L267)

        =>

        [https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/fuzzing.py#L275](https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/fuzzing.py#L275)



        The case will then be marked as "clean close" if the peer

responds to the closing handshake

        by sending back a close frame and - if the peer is a server
  • drops the TCP.

          This pattern is the same for all cases where the case does
    

something fully valid. When the

        case does something violating the protocol or the case is

explicitly testing close behavior

        things will be different ..



        Hope this helps,

        Tobias







        Am 14.12.2011 22:48, schrieb Sylvain Hellegouarch:

Hi there,

          I'm working on ws4py and obviously I'm using Autobahn's

great

          testsuite. Most of my tests pass but all of them are

marked as

          uncleanly closed.



          What I don't understand is how the client actually close

the stream.

          Let's take case 1.1.1, it says it should be closed by the

client but I

          fail to see when that happens. In any case, the connection

is

          eventually killed and my test marked as unclean.



          Some tests on the other hand do use the closeAfter method

to

          explicitely close the stream after a delay.



          Could someone shed a light because this drives me nuts :)



          Cheers,

          - Sylvain

    - Sylvain

    [http://www.defuze.org](http://www.defuze.org)

    [http://twitter.com/lawouach](http://twitter.com/lawouach)

0 Likes

#6

Thats the sequence of things that should happen.

There could probably be 2 things going wrong:

make sure that the close reply actually goes on the wire before the

socket is closed

make sure the socket is really fully closed, probably not only "TCP

half-close" (like closing

the read leg).

If you want to do "TCP half-close", we will probably need to have a

more detailed look.

What Python networking framework are you using?

In any case, it would be interesting to see the case 1.1.1 detail

report … just post it here

when you get back to it ..
···

Am 15.12.2011 09:56, schrieb Sylvain Hellegouarch:

Dang my report is at home and I don’t have it at hand.

    Maybe should we agree on what you mean by dropping the

connection.

My code does the following:

  1. Receives the closing frame from client
  1. Replies with a closing frame with the same code
  1. Closes the socket using: shutdown(RD), close()

All those are done sequentially but without pause.

Isn’t it the expected behavior?

  • Sylvain
      On Thu, Dec 15, 2011 at 9:45 AM, Tobias > Oberstein <tobias.o...@gmail.com> >           wrote:
          Does your server drop

the TCP after having replied to the close frame sent by
the fuzzing client?

          To get close clean, the appropriate peer must drop the TCP

(there is text with SHOULD in the spec about that).

          The appropriate peer to drop the TCP is the server. Server

SHOULD drop, client SHOULD NOT drop. If thats

          violated, case get close unclean.



          What does the detail report say in the closing behavior

block? In particular in

wasNotCleanReason
None
When wasClean == False, the reason what
happened.
wasServerConnectionDropTimeout
False
When we are a client, and we expected the server
to drop the TCP, but that didn’t happen in time,
this gets True.

          Compare to ..

          [http://www.tavendo.de/autobahn/testsuite/report/servers/autobahnserver_0_4_3_case_1_1_1.html](http://www.tavendo.de/autobahn/testsuite/report/servers/autobahnserver_0_4_3_case_1_1_1.html)



          or reply and attach the detail report HTML so I can have a

look …

          Am 15.12.2011 09:38, schrieb Sylvain Hellegouarch:

Hi Tobias,

                  Thanks for the swift answer. I guess I'm

confused at the fact that the tests fails saying
that the server didn’t close the socket in due
time. If I look at how long the test took it says:
1001ms. Considering the 1s delay your test use
before sending the closing handshake, and the fact
that the test says that the server did return the
appropriate closing code, I don’t understand the
unclean state.

                  I can see in my server that I reply to the

closing handshake as soon as I receive it.

Any idea?

  • Sylvain

                      On Thu, Dec 15, 2011 at > > > 8:19 AM, Tobias Oberstein <tobias.o...@gmail.com> > > >                         wrote:
    

Hello Sylvain,

                      taking case 1.1.1 as an example, the case

sends a text message and then waits 1s before
initiating a

                      WS closing handshake with the peer (the

waiting is done because the case expects the
text message

                      to be echo'ed by the peer):



                      [https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/case/case1_1_1.py#L32](https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/case/case1_1_1.py#L32)

                      =>

                      [https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/fuzzing.py#L267](https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/fuzzing.py#L267)

                      =>

                      [https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/fuzzing.py#L275](https://github.com/oberstet/Autobahn/blob/master/lib/python/autobahn/fuzzing.py#L275)



                      The case will then be marked as "clean close"

if the peer responds to the closing handshake

                      by sending back a close frame and - if the

peer is a server - drops the TCP.

                      This pattern is the same for all cases where

the case does something fully valid. When the

                      case does something violating the protocol or

the case is explicitly testing close behavior

                      things will be different ..



                      Hope this helps,

                      Tobias







                      Am 14.12.2011 22:48, schrieb Sylvain > > > > Hellegouarch:

Hi there,

                        I'm working on ws4py and obviously I'm using

Autobahn’s great

                        testsuite. Most of my tests pass but all of

them are marked as

                        uncleanly closed.



                        What I don't understand is how the client

actually close the stream.

                        Let's take case 1.1.1, it says it should be

closed by the client but I

                        fail to see when that happens. In any case,

the connection is

                        eventually killed and my test marked as

unclean.

                        Some tests on the other hand do use the

closeAfter method to

                        explicitely close the stream after a delay.



                        Could someone shed a light because this

drives me nuts :slight_smile:

                        Cheers,

                        - Sylvain

                  - Sylvain

                  [http://www.defuze.org](http://www.defuze.org)

                  [http://twitter.com/lawouach](http://twitter.com/lawouach)

    - Sylvain

    [http://www.defuze.org](http://www.defuze.org)

    [http://twitter.com/lawouach](http://twitter.com/lawouach)
0 Likes

#7
In any case, it would be interesting to see the case 1.1.1 detail

report … just post it here

when you get back to it ..

Here goes the report.

cherrypy_case_1_1_1.html (12.1 KB)

···

0 Likes

#8

could you give me a short description what I need to do to test
ws4py myself … so I can reproduce case 1.1.1?

···

Am 15.12.2011 12:40, schrieb Sylvain Hellegouarch:

        In any case, it would be interesting to see the case 1.1.1

detail report … just post it here

        when you get back to it ..

Here goes the report.

  - Sylvain

  [http://www.defuze.org](http://www.defuze.org)

  [http://twitter.com/lawouach](http://twitter.com/lawouach)
0 Likes

#9

I will only be able to provide you with this this evening as I haven’t comitted some local changes yet.

···

On Thu, Dec 15, 2011 at 2:02 PM, Tobias Oberstein tobias.o...@gmail.com wrote:

could you give me a short description what I need to do to test

ws4py myself … so I can reproduce case 1.1.1?

http://twitter.com/lawouach

0 Likes

#10

Hi Tobias,

···

On Thu, Dec 15, 2011 at 2:04 PM, Sylvain Hellegouarch s...@defuze.org wrote:

On Thu, Dec 15, 2011 at 2:02 PM, Tobias Oberstein tobias.o...@gmail.com wrote:

could you give me a short description what I need to do to test

ws4py myself … so I can reproduce case 1.1.1?

I will only be able to provide you with this this evening as I haven’t comitted some local changes yet.

I’ve pushed my changes to my repository:

https://github.com/Lawouach/WebSocket-for-Python

If you want to try, you’ll need CherryPy 3.2.2 or gevent installed.

The results are there:
http://www.defuze.org/oss/ws4py/testreports/servers/0.1.5/

http://twitter.com/lawouach

0 Likes