TCP MSS Question

[NamelessOne23]

Hey guys. I'm new here. I have a question concerning TCP Maximum Segment Size advertising between two endpoints that I wasn't able to find in the RFCs and other documents that I've read.

Say that I have a TCP connection to a server (of any kind) and I advertise an MSS of 800 bytes in my SYN packet and in the server's SYN+ACK it advertises it's own MSS of 1,000 bytes. The lower of the two is supposed to be taken and used, to avoid IP fragmentation, so I could receive up to 800 bytes (the server could send me up to that amount) and the server can receive up to 1,000 bytes. Would it be legal (as per the TCP MSS standard spec) to send the server >800 bytes? IE., say I send the server 900 or 1,000 byte TCP segments but continue to receive the 800 byte limit that I advertised/imposed during the TCP connection setup, would this still be legal since it's within the server's own advertised MSS?

What's unclear to me is whether the MSS is always supposed to apply to both inbound and outbound TCP segment transmissions (ingress & egress/Rx & Tx). While this seems to be the case, I also know that the MSS can differ for an endpoint's inbound and outbound interfaces (ie., they're different, multihomed), so this is why I ask. As long as I send <= 1,000 byte TCP segments, there wouldn't be any fragmentation between us (provided that the path's intermediate routers/hops support up to 1,000 byte segments), and on the return path I'd be able to receive up to my advertised 800 bytes for TCP segments.

From what I've read it sounds like the MSS is a bidirectional/symmetrical value that applies to them both, even though this could be different (variable) in each direction. Other papers I've read and e-books say it only corresponds to how many bytes per TCP segment that can be received without incurring IP fragmentation. If any networking experts could help clarify this I'd really appreciate it. While every TCP/IP stack I know would likely limit my outbound TCP segments to 800 bytes as well, I'm just asking if my scenario would still be technically legal in regard to the TCP MSS spec (and TCP in general), not whether it's actually ever handled like this in various OS TCP/IP stacks.

Thanks in advance!

 

[Gryz]

First question: what problem are you trying to solve ? Are you writing your own TCP implementation ?

Secondly. You talk about "would it be legal to ...". TCP/IP is not a legal system with laws. The ground-rule one should never forget is: "be strict in what you send, be liberal in what you accept".

This rule is really awesome, and needs to be grasped before trying to understand protocols. Older protocols, like old IBM protocols, and the old OSI stack, were made by standards bodies. Competing firms tried to make sure that their implementations would adhere to the standards more than the implementations of their competitors. It was no problem to ignore packets and drop connections from devices made by other companies, just because "the other guys are wrong". They were trying to prove a point ("we are better ! we are more compatible than the other guys, so we won't talk with their devices !"). While the end-result was that for endusers, their products would not inter-operate. Standards wars are the cost of the customer. One of the reasons TCP/IP was so successful was because of the strict/liberal rule.

So imho, yes, I don't see a need for a TCP implementation to throw away packets unneccesarily. If you can accept a packet, and you know how to understand/parse it, do that. No artificial limitations.

On the other hand, if you set the MSS to 800 yourself, then there must be a reason why you do that, right ? Because you know that your interface has a small MTU of 800-something (858 over ethernet I would assume). Why try to send 1000 (+ 56) byte frames out, when you knew you couldn't receive anything over 858 ? If you can receive larger than 858-byte frames, then why advertise a MSS of 858 ?

So for practical reasons, I think you should pick the MSS as the smallest of the two advertised values. If you don't do that, you likely get fragmentation.

With fragmentation stuff usually still works. But you probably get a performance penalty. Path-MTU-discovery might make things even uglier. There is a reason people try to avoid fragmentation.

About multi-homed hosts. Yes, a host can have 2 or more interfaces. But in the TCP/IP world that means that both interfaces get their own and unique IP-address. The TCP connection is between to IP-addresses, not between two hosts. So packets will always arrive on the same incoming interfaces. Will packets always leave on the same interface ? That depends on the implementation. An implementation could set the MSS to a fixed value (e.g. 536), or to the lowest of the MTUs of its interfaces. Or it could use the MTU of the interface on which the IP-address is configured, and then let fragmentation/pmtu take care of it. But I would personally take the lowest value of all interfaces myself. Again, it's an implemantation issue, and any host can do it their own way without breaking RFCs.
(I just saw that in RFC 6691, paragraph 2, it says: "the determination of what MTU value should be used, especially in the case of multi-homed hosts, is beyond the scope of this document". In other words, it is implementation dependent).


I just read https://tools.ietf.org/html/rfc6691
I don't see anywhere where it says that the used MSS must be the minimum of advertised and received MSS. In which RFC did you read that ? Or did you read it in code somewhere ? As I wrote above, it's the sensible thing to do, so I assume implementations do that.

 

[NamelessOne23]

Thanks for the reply. Every implementation I've seen takes the lowest of the two advertised MSS values. Or, if it isn't advertised, then 536 by default unless one end advertises a value even smaller than that. IBM states this in regard to their TCP MSS usage in AIX 6.1...

The TCP protocol includes a mechanism for both ends of a connection to advertise the MSS to be used over the connection when the connection is created. Each end uses the OPTIONS field in the TCP header to advertise a proposed MSS. The MSS that is chosen is the smaller of the values provided by the two ends. If one endpoint does not provide its MSS, then 536 bytes is assumed, which is bad for performance.

http://www-01.ibm.com/support/knowl...m.aix.performance/tcp_max_seg_size_tuning.htm

This is how I've seen all TCP/IP stacks work with mismatched MSS advertisements, where the smaller (lower) is always used and it makes sense.

My question about the legality of my MSS scenario was just asking "Does this violate the TCP standard specification for it, or not?"

I understand about multi-homed hosts having multiple interfaces, each with a different IP address. What I meant there was for routing using different sized MTUs for upstream gateways (first hop, let's say). If I have more than one gateway to route my IP packets and one has an MTU of 1500 and the other an MTU of 1024 then the outbound MSS (due to the MTU being different) could be larger or smaller, depending on which gateway forwarded my TCP/IP packets to set up the connection. Administrative policies, for example, on routers and switches could also limit the effective TCP MSS in either direction with features such as MSS Clamping (mainly used for PPPoE and VPN tunnels).

The RFCs say that the MSS can be independent/different in both directions of the TCP connection. What they didn't elaborate on was whether you can have independent MSS values per direction on each side (in & out). MSS is defined as the largest TCP segment size that can be received by a host but it doesn't say whether it also globally applies to data sent out on that connection. My question is hypothetical, I'm not developing anything, just curious about this. Receiving up to your full MSS TCP segments is the clear part, what isn't clear is if the other end can receive more (due to having a larger advertised MSS), and as a result you sending larger TCP segments to them that exceed your advertised receive MSS but don't exceed THEIR advertised receive MSS.

I don't know of any network stacks that allow this to be done since all of the ones I've tested treat your advertised MSS as a bidirectional value, the same max sized value for all TCP/IP packets both sent and received over a TCP connection. I would like to know if anybody knows of this being done though.

 

[Gryz]

My question about the legality of my MSS scenario was just asking "Does this violate the TCP standard specification for it, or not?"

I don't think so. I've read RFC 6691, which I think is the most recent RFC on the issue. Nowhere does it say anything about sending smaller segments.

What I meant there was for routing using different sized MTUs for upstream gateways (first hop, let's say).

TCP will work if both sides don't include a MSS, ignore the MSS, or violate the MSS. The MSS is only an aid to help improve performance by avoiding fragmentation.

Your example does "per packet load-balancing". Per-packet-load-balancing is bad. For performance reasons. For performance reasons, any host or router implementation should try to send all packets belonging to a specific flow over the exact same path. Load balancing should be done per destination, or per flow. The reason is that if you do per packet load-balancing, packets from the same flow will go over different paths. And can arrive out of order. While the TCP specs says that this should not be a problem, in reality it is a problem. Endpoints do not like reordered packets. Because they need to allocate a lot of buffer space to re-order. Most TCP implementations don't do that. If packets arrive out of order, they will send acks, triggering a resend. Or some even send RSTs. As a result, TCP performance with per packet load-balancing is worse than no load-balancing. And use of bandwidth is less efficient. So while your hypothetical situation is interesting, it should never occur in practice.

MSS is defined as the largest TCP segment size that can be received by a host but it doesn't say whether it also globally applies to data sent out on that connection. My question is hypothetical, I'm not developing anything, just curious about this.

The RFC says enough to build interoperable implementations. It's just smart to not only send segments equal/smaller than the other guy's MSS, but also not larger than your own MSS. It's maybe not explicitly written in the RFC, but when you think about it for 2 minutes, it is clear.

I like that. I like RFCs that give enough information to write interoperable implementations. But that leave enough open so that programmers can invent their own (local) optimizations and improvements. I've improved on RFCs myself in my implementation of certain protocols in the past. And I was really happy that the RFC was not written more precise than it was. A great example is IS-IS versus OSPF. Where OSPF tries to specify all little details and tell implementers exactly what algorithms to use, and what data-structures to keep around. And IS-IS is much more flexible. IS-IS implementations are much more optimized, robust and scalable than OSPF implementations. (Or at least they were when I worked in that field).

I don't know of any network stacks that allow this to be done since all of the ones I've tested treat your advertised MSS as a bidirectional value, the same max sized value for all TCP/IP packets both sent and received over a TCP connection. I would like to know if anybody knows of this being done though.

It's the smart thing to do. If you know you can send more than your own MSS, then why did you set your MSS so low ? If there was an implementation that would send segments bigger than its own MSS, then I would not call it an RFC-violation. But I would certainly call it a bug.

 

[alkemyst]

You can experiment by setting the DNF (do not fragment) bit and testing packet size.