Re: Tuning TCP/IP

From: Rajeev Prabhakar <rprabha01_at_gmail.com>
Date: Sat, 15 Sep 2007 13:57:20 -0400
Message-ID: <2ba656800709151057i5e4680e5x812c9cc592a5359b@mail.gmail.com>

Charles

I found this explanation on a tcp tuning related website. May be, this will help answer some of your questions ?

Solaris 8, 9 and 10 do not take the send window into account. This results in an unexpected behaviour for a bulk transfer from node A to node B when the bandwidth-delay
product<http://kb.pert.geant2.net/PERTKB/BandwidthDelayProduct>is larger than 64KiB and

A's receive buffer (tcp_recv_hiwat) < 64KiB
A's send buffer (tcp_xmit_hiwat) > 64KiB
B's receive buffer > 64KiB

A will not advertize the window scaling option and B will not do so either according to RFC 1323 <ftp://ftp.rfc-editor.org/in-notes/rfc1323.txt>. As a consequence, throughput will be limited by a congestion window of 64KiB.

Large windows can cause large "bursts" of consecutive segments/packets. When there is a bottleneck in the path - because of a slower link or because of cross-traffic - these bursts will fill up buffers in the network device (router or switch) in front of that bottleneck. The larger these bursts, the higher are the risks that this buffer overflows and causes multiple segments to be dropped. So a large window can lead to "sawtooth" behavior and worse link utilisation than with a just-big-enough window where TCP could operate at a steady rate.

Rajeev

On 9/14/07, Charles Schultz <sacrophyte_at_gmail.com> wrote:
>
> I have a document from Sun that references some tcp/ip parameters that
> could be tweaked for better network performance, but has a very obvious
> caveat:
> " *oracle is very sensitive to round-trip performance, and setting
> tcp_xmit_hiwat (and tcp_recv_hiwat) to a value much higher than the default
> causes much longer waiting time.*"
>
> The parameters in question are:
>
> /usr/sbin/ndd -set /dev/tcp tcp_tstamp_if_wscale 1
>
> /usr/sbin/ndd -set /dev/tcp tcp_sack_permitted 2
>
> /usr/sbin/ndd -set /dev/tcp tcp_xmit_hiwat 16777216
>
> /usr/sbin/ndd -set /dev/tcp tcp_recv_hiwat 16777216
>
> /usr/sbin/ndd -set /dev/tcp tcp_cwnd_max 16777216
>
> /usr/sbin/ndd -set /dev/tcp tcp_max_buf 16777216
>

I did some looking around on google, but the only thing I could find
> documented were some suggestions. For example, it is recommend that
> tcp_xmit_hiwat be maxed at 32k for 9i and 64k for 10g. But why? In what ways
> is Oracle "very sensitive to round-trip performance", and why would these
> parameters increase wait time? In talking with my local sysadmin, we have
> come to the conclusion that any outgoing tcp/ip packets that hit a device
> with a lower hiwat (High Watermark) will have be re-chunked into the lower
> number, which may cause some overhead. I am hoping that others out there
> smarter than I might be able to affirm or contradict that hypothesis.
>
> Obviously, we are going to test these parameters. We have already done
> some simple OS testing and found very favorable performance increases. In
> testing an Oracle communication, would kind of wait event would I look for?
> SQL*Net message to and from client?
>
> PS - Obviously, this is a Sun document and they cannot speak for Oracle. I
> am taking their statement with a little bit of salt. =)
>
> --
> Charles Schultz
>

--
http://www.freelists.org/webpage/oracle-l

Received on Sat Sep 15 2007 - 12:57:20 CDT