===== Master Communications - Protocols and Networks ===== Details for traffic shaping are in [[http://lartc.org]]. Details regarding tcp testing are here: [[http://www.linuxfoundation.org/collaborate/workgroups/networking/tcp_testing]]. ==== Setup of the Linux test platform ==== Network tests are done with a network of Ubuntu Linux computers which are connected via an ethernet switch. ==== Laboratory expirements ==== * [[mscom_lab1|Master COM Lab 1 - Ethernet Measurements]] * [[mscom_lab2|Master COM Lab 2 - tftp Measurements]] ==== TCP Throughput test with iperf ==== Setup a server on one computer iperf -s Run a client on a second computer. The client will send data to the server. iperf -c SERVERIP -i 1 Option -i will show intermediate bandwidth results every second. ==== Add netem network emulation traffic shaping to an ethernet device ==== In the following example an additional 100 ms delay to the outgoing traffic is added. sudo tc qdisc add dev eth0 root netem delay 100ms Show the current traffic shaper tc qdisc show Remove a traffic shaper sudo tc qdisc del dev eth0 root For limiting the bandwidth a token bucket shaper can be used. In the example below the packet is delayed by 100 ms with the netem shaper and the goes to the token bucket shaper. See [[http://lartc.org/howto/lartc.qdisc.classless.html#AEN690]]. sudo tc qdisc add dev eth0 root handle 1: netem delay 100ms sudo tc qdisc add dev eth0 parent 1:1 handle 10: tbf rate 256kbit buffer 1600 limit 300 tc -s qdisc ==== Analyzing and setting ethernet card parameters ==== Ethernet card parameters can be configured with ethtool. fritz@ubuntu:~$ ethtool eth0 Settings for eth0: Supported ports: [ TP ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Supported pause frame use: No Supports auto-negotiation: Yes Advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Advertised pause frame use: No Advertised auto-negotiation: Yes Speed: 1000Mb/s Duplex: Full Port: Twisted Pair PHYAD: 0 Transceiver: internal Auto-negotiation: on MDI-X: off Cannot get wake-on-lan settings: Operation not permitted Current message level: 0x00000007 (7) drv probe link Link detected: yes fritz@ubuntu:~$ ethtool DEVICENAME will show you the current settings of the ethernet card. In the example above the link speed is for example 1GBit/s. In order to modify the speed settings and set it to 100 MBit/s full duplex mode do: sudo ethtool -s eth0 speed 100 duplex full Modern network cards can do TCP segmentation in hardware. This is called TCP segmentation offloading (TSO), which offloads the packet segmentation from the cpu. To show the offload parameters do: fritz@ubuntu:~$ ethtool -k eth0 Offload parameters for eth0: rx-checksumming: off tx-checksumming: on scatter-gather: on tcp-segmentation-offload: on udp-fragmentation-offload: off generic-segmentation-offload: on generic-receive-offload: on large-receive-offload: off rx-vlan-offload: on tx-vlan-offload: on ntuple-filters: off receive-hashing: off fritz@ubuntu:~$ In the example above the tcp segmentation offloading is activated. This will result in wireshark showing longer TCP packets than allowed according to Maximum Transmit Unit size (MTU) which is typically 1500 Bytes. You can check the MTU value with "ifconfig". In order to switch TSO offloading off, you can do sudo ethtool -K eth0 tso off ==== Adding tcpprobe traffic log ==== The tcp_probe module can trace the state of a tcp connection. sudo modprobe tcp_probe port=5001 sudo chmod a+r /proc/net/tcpprobe In order to store the log data in file test.log: cat /proc/net/tcpprobe > test.log tcp_probe only adds a log entry, when the contention window size changes. In order to have a log entry for every received packet sudo modprobe tcp_probe port=5001 full=1 The Ubuntu 12.04 LTS is based on linux kernel 3.11. The tcp_probe kernel module source code can be viewed on the [[https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/net/ipv4/tcp_probe.c?id=v3.11|linux kernel git server]]. To unload the module: sudo modprobe -r tcp_probe The log file from tcp_probe contains lines with the log information 1 2 3 4 5 6 7 8 9 10 7.768727538 192.168.178.33:51715 192.168.178.27:5001 32 0xe656e892 0xe6568e12 16 2147483647 42248 1 1: Time in seconds 2: Source IP:Port 3: Destination IP:Port 4: Packet length in bytes 5: snd_nxt: Sequence Number of the next packet to send 6: snd_una: Sequence Number of the first unacknowledged packet 7: snd_cwnd: Contention Window 8: ssthr: Slow Start Threshold (-1 => not known yet) 9: snd_wnd: Send Window size in Bytes 10: srtt: Averaged round trip time estimation The data from the log file can be plotted with gnu plot gnuplot gnuplot> plot "test.log" using 1:7 title "cwnd", "test.log" using 1:($8 > 20000000 ? 0 : $8) title "ssthr" ==== TCP configuration ==== === Local memory restrictions === Set the maximum window sizes for tcp connections to 512000 Bytes. sudo sh -c "echo 512000 > /proc/sys/net/core/wmem_max" sudo sh -c "echo 512000 > /proc/sys/net/core/wmem_default" sudo sh -c "echo 512000 > /proc/sys/net/core/rmem_max" sudo sh -c "echo 512000 > /proc/sys/net/core/rmem_default" sudo sh -c "echo 512000 512000 512000 > /proc/sys/net/ipv4/tcp_wmem" sudo sh -c "echo 512000 512000 512000 > /proc/sys/net/ipv4/tcp_rmem" === Initial TCP contention window size === The following section shows how to change the initial contention window size to to 1 MSS. fritz@ubuntu:~$ ip route show default via 192.168.178.1 dev eth0 proto static 169.254.0.0/16 dev eth0 scope link metric 1000 192.168.178.0/24 dev eth0 proto kernel scope link src 192.168.178.33 metric 1 fritz@ubuntu:~$ sudo ip route change default via 192.168.178.1 dev eth0 proto static initcwnd 1 fritz@ubuntu:~$ ip route show default via 192.168.178.1 dev eth0 proto static initcwnd 1 169.254.0.0/16 dev eth0 scope link metric 1000 192.168.178.0/24 dev eth0 proto kernel scope link src 192.168.178.33 metric 1 fritz@ubuntu:~$ === TCP Congestion avoidance algorithm selection === The linux kernel provides several algorithms for tcp congestion avoidance. The following code shows how to figure out the available algorithms (here cubic and reno) and set and control the setting. In this example the congestion avoidance algorithm is switched from "cubic" to "reno". fritz$ cd /proc/sys/net/ipv4/ fritz$ cat tcp_available_congestion_control cubic reno fritz$ cat tcp_congestion_control cubic fritz$ sudo sh -c "echo reno > tcp_congestion_control" fritz$ cat tcp_congestion_control reno