"SSL handshake error" causes socket not to close.

[smoku]

When SSL connection have following error: "SSL handshake error (error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol)" socket is not closed, and it can be found via netstat with state: CLOSE_WAIT.

Probably this issue causes also memleak in c2s module.

Also need to be checked if "flash patch" is responsible for that (Stephen Marquard mentioned about "zero byte" problem with that patch).

---

If any SSL handshake error occurs the functions _sx_close and _sx_error are invoked. So the stream switches to state_CLOSING and we have still something to write on the stream.

So sx_can_write will be invoked which finally invokes _sx_ssl_wio which will return -2 (permanent error) because of the previous SSL error for this stream.

Here you see the invocations with the return values:

sx_can_write 
  (-1) _sx_get_pending_write 
    (-2) _sx_chain_io_write 
      (-2) _sx_ssl_wio

The concerning code in function sx_can_write:

ret = _sx_get_pending_write(s);
if (ret < 0) {
/* fatal error */
/* !!! shut down */
return 0;
}   

Nothing is intiated here to close the stream and the underlying connection. Furthermore s2s will constantly have to write something new on this stream because of failing callbacks. But of course this will always fail the same way as described above. These continuos attempts to write will assure that the stream never gets idle. So the connection will never close.

When the peer closes the connection (in my case he does that immediatly after the SSL handshake error), i.e. sends FIN, our os will keep that connection in state CLOSE_WAIT forever.

My patch just puts the stream directly in state CLOSED and tells the app to close the connection. I don't think we have to check if the connection is already in state CLOSING because this error is permanent. So we won't have a chance to read/write in the future.

[smoku]

Patch by Klara Mall <klara.mall@…>

[smoku]

(In [163]) Integrated SSL handshake patch by Klara Mall. Fixes #59