Some actual packets - DF=0 and DF=1

>> Most hosts send packets with a non-zero Identifier, including those with DF=0.

>> Solved:  This was illusory - the TCP/IP stack was sending larger than MTU packets to the Ethernet driver, because the Ethernet chip (Broadcom BCM5751) can break the oversize TCP packet into smaller separate TCP packets, using "Large Send Offload"
AKA TSO See my message to the RRG list:  http://psg.com/lists/rrg/2008/msg02175.html. My server regularly sends out TCP HTTP packets to web clients way longer than 1500 bytes - up to 8.8k bytes or so, but I can't figure out how it can do this, since the MSS values at the start of the TCP connection were both less than 1500 bytes.

>> Google servers have an MSS of 1430 and (assuming the client's MSS is this or higher) sends only DF=0 packets, launching straight into maximum sized packets of 1470 bytes, with no attempt to do DF=1 RFC 1191 style PMTUD.

Let's have a look at how the Identifier is typically set:


results in a hex dump of the first 16 bytes of packets coming into my home server's DSL line.

     45 = Protocol and header length - every packet is the same.     
     ||
     ||** DiffServ & ECN
     ||||
     |||| LLLL = Total Length
     |||| |  |
     |||| |  | IIII = Identification
     |||| |  | |  |
     4500 007e 1c07 0000 6e11 c13d 4533 f116
         Evil bit__/|\\\
               DF _/| \\\
                   MF  Fragment
                       Offset  

                           Next            
                           Protocol
                       TTL || Checksum
                         \\|| ////
     4500 007e 1c07 0000 6e11 c13d 4533 f116
                                   |||| ||||
                                   Part of
                                   Source Address

The first hex nybble of the Flags (Evil bit, DF, MF and MSB of Fragment Offset is either:

or                    

In several thousand packets I captured, none were fragments.

There were plenty of long packets with DF=0, which surprised me.

More than half the packets were like this:


which I think were mainly from Google.  These are 1452 bytes long (hex 05ac) and have DF=0.  This shows more about the packets:


Here is an edited example:


This is quite interesting in itself.  It seems Google figures it is OK to send out packets of 1452 bytes, expecting the network to fragment them if there are any MTU problems.  

The 1452 bytes probably arises from my ifcg-ppp0 file setting CLAMPMSS=1412.  The 1452 byte packet length is the MSS 1412 plus 20 bytes TCP header plus 20 bytes IP header.

See below where a Google server sends out 1470 byte packets with DF=0.  Presumably they wouldn't do this if there were, in general, any MTU problems.  I guess it saves their servers having to keep a record of recently sent bytes so they can be resent if there is a PTB message.  

See the false-alarm.html page in this directory for the stuff I wrote about this.

What does Google do with a "browser" which can handle jumboframes?  I found it hard to get Google to send anything to my Texas server, from an image search URL - maybe because wget doesn't look like a browser to it.  However, I was able to get large images from Google's press center which was a simple HTTP image download.  I used a www.google.com IP address so I could access the same machine from Australia without any funny DNS stuff pointing me to an Australian server:

 

Here the maximum size packets were DF=0 with a size of 1470 bytes (05be):


I would be surprised if there was a 1500 byte MTU 100Mbps Ethernet link between my server and Google, so my guess is that Google simply sends out DF=0 bytes with a length of 1470, or less according to the TCP MSS reported by the destination host.

Here the the lines from the first packets sent by Google's server when it served the file:


So Google's server goes straight to 1470 bytes, all DF=0 - no trying DF=1 to test the Path MTU as a well-behaved server should.

What TCP MSS did my server provide when it opened the connection?  1460. That would allow for a 1500 byte packet size.  The Google server responded with an MSS of  1430, which was used for the session - hence the 1470 byte packets.

If there is a router between the Google server and the client with an MTU of, for instance, 1400 bytes then the client should be configured to supply an MSS of 1360 bytes, which sets a maximum packet size of 1400 bytes for TCP.

In the future, when more and more clients can be reached with jumboframes (such as packets or 8k bytes) it will become increasingly tempting for Google etc. not to limit their servers to 1470 byte packets.  Then, they will presumably need to do proper PMTUD, since there will be situations where a host gives an 8k or so MSS, but that there a possiblity that some router or data link en-route would have a 1500 MTU.  

But whose problem would this be?  Google's, or the person whose host this is?  It should be Google's problem, for not using RFC 1191 PMTUD.  However if Google and enough other companies do this - just send packets as big as the client's MSS allows - AND if there are problems for some clients, with the bottlenecks being closer to the clients, then it will be the clients' ISPs who probably cop the flak, since Google etc. in their server farms can say it works fine for most people, apart from those at certain ISPs . . .

Here are some lines of packets on my home server  The second column is the size, fourth column the flags.