# Problem with SSL connection

## mitek

Hi all,

I've got Gentoo/Apache2.0.58/mod_ssl/PHP5.2.2 installed on server. The clients are using IE/FF browsers with client software written in Flex.

The problem is that some of the server responses are not received back by the client.

Server logs showing that everything is fine - request received, processed and sent back. Client ends up waiting and then after the timeout (30s) giving up.

I installed Wireshark on the client side and looked into network traffic. And this is what happening:

client keeps sending these ones:

TCP      [TCP Dup ACK 12#1] 2312 > https [ACK] Seq=105 Ack=1 Win=17520 Len=0

SSL      [TCP Retransmission] Client Hello

I tried to disable SSLSessionCache(none), the situation became worse, so I re-enabled it.

Could anybody point me where to look at? Thanks in advance!

Cheers,

Dmitri.

----------

## Hu

Large numbers of duplicate acknowledgments or other retransmissions are usually a sign of network problems.  It could be a problem with network hardware, or it could be that a firewall or other routing device in the path is dropping the packets.  As a first step, put net-analyzer/tcpdump on the server side and capture traffic on both the client and server at the same time.  Compare the results to make sure that all packets sent by the server arrived at the client and vice versa.  Since you are seeing retransmissions, I suspect you will find that some packets are not arriving.  If so, then you should start sniffing intermediate devices until you find where the packets are being dropped.  If you are providing this service over a WAN, intermediate sniffing may not be possible (due to lack of control over your ISP's routers).

Is this over a LAN or a WAN?  Do other large transfers between this server and its clients work?  Try using wget to download a large file over HTTP from the server, and then try to upload that file as a POST request.  This should indicate whether the problem is with the SSL or a more fundamental TCP connectivity problem.

What is the output of emerge -p -v dev-libs/openssl?  Do you get any interesting output from using /usr/bin/openssl in s_client mode to perform a raw SSL negotiation?  The goal here is to eliminate the possibility that something is wrong with the HTTP message encapsulated in SSL.

----------

## mitek

Thank you for help, Hu!

Yes, it happens over a WAN connection. It is a dedicated server, located at ISP's premises.

I transferred the site in the non-ssl mode and checked with WireShark - the number of dropped packets is very small. So this problem is only present in SSL mode.

I did the whole update (emerge world) just recently and I don't know which version of openssl I had, but now it's:

[ebuild   R   ] dev-libs/openssl-0.9.8e-r2  USE="(sse2) zlib -bindist -emacs -test" 0 kB

Also it updated apache from 2.0.58 to 2.2.6.

It looks much better now - I haven't seen this error again so far. But I still see a lot of TCP CHECKSUM ERROR.

Also I increased the Apache setting KeepAliveTimeout to 300 sec and improved the performance on the subsequent requests. 

Still analysing the server tcp dumps to identify which side has problem. It's a little bit tricky now as I haven't seen so far lost packets - only packets with checksum error.

Thank you once again!

Cheers,

Dmitri. 

 *Hu wrote:*   

> Large numbers of duplicate acknowledgments or other retransmissions are usually a sign of network problems.  It could be a problem with network hardware, or it could be that a firewall or other routing device in the path is dropping the packets.  As a first step, put net-analyzer/tcpdump on the server side and capture traffic on both the client and server at the same time.  Compare the results to make sure that all packets sent by the server arrived at the client and vice versa.  Since you are seeing retransmissions, I suspect you will find that some packets are not arriving.  If so, then you should start sniffing intermediate devices until you find where the packets are being dropped.  If you are providing this service over a WAN, intermediate sniffing may not be possible (due to lack of control over your ISP's routers).
> 
> Is this over a LAN or a WAN?  Do other large transfers between this server and its clients work?  Try using wget to download a large file over HTTP from the server, and then try to upload that file as a POST request.  This should indicate whether the problem is with the SSL or a more fundamental TCP connectivity problem.
> 
> What is the output of emerge -p -v dev-libs/openssl?  Do you get any interesting output from using /usr/bin/openssl in s_client mode to perform a raw SSL negotiation?  The goal here is to eliminate the possibility that something is wrong with the HTTP message encapsulated in SSL.

 

----------

## Hu

Unfortunately, TCP checksum errors may or may not be an indication of problems.  If you only see the checksum errors when sending and the sent checksum is set to zero, it may not be a problem, as far as I know.  I have seen Windows systems exhibit the behavior of sending a packet with the TCP checksum set to zero when the checksum was being set by the network card, and thus Wireshark captured the packet before it was fully initialized.  I have never been able to run Linux on such a system to see if you get the same behavior.  This behavior only applies for certain network cards that support checksum offloading.

As far as I know, any case where a TCP checksum error is reported and the value that was sent is not zero, is a problem.

----------

## mitek

Hu, here is the sample from dump:

  TCP      1087 > https [ACK] Seq=1 Ack=1 Win=65535 [TCP CHECKSUM INCORRECT] Len=0

    Checksum: 0x40c8 [incorrect, should be 0x7bfa (maybe caused by "TCP checksum offload"?)]

Does it mean that it is not so bad?  

Thank you!

Dmitri.

 *Hu wrote:*   

> Unfortunately, TCP checksum errors may or may not be an indication of problems.  If you only see the checksum errors when sending and the sent checksum is set to zero, it may not be a problem, as far as I know.  I have seen Windows systems exhibit the behavior of sending a packet with the TCP checksum set to zero when the checksum was being set by the network card, and thus Wireshark captured the packet before it was fully initialized.  I have never been able to run Linux on such a system to see if you get the same behavior.  This behavior only applies for certain network cards that support checksum offloading.
> 
> As far as I know, any case where a TCP checksum error is reported and the value that was sent is not zero, is a problem.

 

----------

## Hu

I think that is bad, but I have little experience with systems that report frequent checksum errors, so I may be wrong.  If the peer is accepting the traffic, then the peer is satisfied with the checksum, and it is probably OK.  If the traffic is getting dropped without being processed by the peer, then there is a problem.

----------

