Client: fallback to TCP after X MPC

[matttbe]

MPC packets could be explicitly dropped or reset by firewall somewhere in the network, e.g. with:

iptables -t filter -A FORWARD -p tcp --tcp-option 30 -j DROP

It sounds safer to fallback to TCP after X attempts (defined in a sysctl option?), "just in case".

Later, this could be improved with a cache, see #57.

[matttbe]

From the last weekly meeting:

• it seems really needed because in some places packets having unknown options are dropped: it has been seen (but we don't have figures showing how frequent it is)
• recommendations:
  • try 5 times (when using net.ipv4.tcp_syn_linear_timeouts), then fallback to "plain" TCP
  • use a cache technique to avoid doing that all the time, e.g.:
    • TFO was caching issues per destination, using tcp-metrics
    • but now it just stops for a period of time because "the issues are certainly on the client side"
    • see how tfo_active_disable_times is used: https://elixir.bootlin.com/linux/latest/A/ident/tfo_active_disable_times

[matttbe]

From the last weekly meeting:

• that seems important if MPTCP is used by default on the client side
• but probably hard to have it backported
• important to have a sysctl to control the blackhole + an easy way to check if this feature is available
• fallback cannot be too "optimised"/smart because the userspace can already interract with the MPTCP socket at that point
  • we can change the ULP ops
  • to do something smart, the MPTCP socket should inherit from the TCP one: waste of memory, etc. just for the client side and to support fallbacks, probably not worth it.