Details for traffic shaping are in http://lartc.org.
Details regarding tcp testing are here: http://www.linuxfoundation.org/collaborate/workgroups/networking/tcp_testing.
Network tests are done with a network of Ubuntu Linux computers which are connected via an ethernet switch.
Setup a server on one computer
iperf -s
Run a client on a second computer. The client will send data to the server.
iperf -c SERVERIP -i 1
Option -i will show intermediate bandwidth results every second.
In the following example an additional 100 ms delay to the outgoing traffic is added.
sudo tc qdisc add dev eth0 root netem delay 100ms
Show the current traffic shaper
tc qdisc show
Remove a traffic shaper
sudo tc qdisc del dev eth0 root
For limiting the bandwidth a token bucket shaper can be used. In the example below the packet is delayed by 100 ms with the netem shaper and the goes to the token bucket shaper. See http://lartc.org/howto/lartc.qdisc.classless.html#AEN690.
sudo tc qdisc add dev eth0 root handle 1: netem delay 100ms sudo tc qdisc add dev eth0 parent 1:1 handle 10: tbf rate 256kbit buffer 1600 limit 300 tc -s qdisc
Ethernet card parameters can be configured with ethtool.
fritz@ubuntu:~$ ethtool eth0 Settings for eth0: Supported ports: [ TP ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Supported pause frame use: No Supports auto-negotiation: Yes Advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Advertised pause frame use: No Advertised auto-negotiation: Yes Speed: 1000Mb/s Duplex: Full Port: Twisted Pair PHYAD: 0 Transceiver: internal Auto-negotiation: on MDI-X: off Cannot get wake-on-lan settings: Operation not permitted Current message level: 0x00000007 (7) drv probe link Link detected: yes fritz@ubuntu:~$
ethtool DEVICENAME will show you the current settings of the ethernet card. In the example above the link speed is for example 1GBit/s. In order to modify the speed settings and set it to 100 MBit/s full duplex mode do:
sudo ethtool -s eth0 speed 100 duplex full
Modern network cards can do TCP segmentation in hardware. This is called TCP segmentation offloading (TSO), which offloads the packet segmentation from the cpu. To show the offload parameters do:
fritz@ubuntu:~$ ethtool -k eth0 Offload parameters for eth0: rx-checksumming: off tx-checksumming: on scatter-gather: on tcp-segmentation-offload: on udp-fragmentation-offload: off generic-segmentation-offload: on generic-receive-offload: on large-receive-offload: off rx-vlan-offload: on tx-vlan-offload: on ntuple-filters: off receive-hashing: off fritz@ubuntu:~$
In the example above the tcp segmentation offloading is activated. This will result in wireshark showing longer TCP packets than allowed according to Maximum Transmit Unit size (MTU) which is typically 1500 Bytes. You can check the MTU value with “ifconfig”.
In order to switch TSO offloading off, you can do
sudo ethtool -K eth0 tso off
The tcp_probe module can trace the state of a tcp connection.
sudo modprobe tcp_probe port=5001 sudo chmod a+r /proc/net/tcpprobe
In order to store the log data in file test.log:
cat /proc/net/tcpprobe > test.log
tcp_probe only adds a log entry, when the contention window size changes. In order to have a log entry for every received packet
sudo modprobe tcp_probe port=5001 full=1
The Ubuntu 12.04 LTS is based on linux kernel 3.11. The tcp_probe kernel module source code can be viewed on the linux kernel git server.
To unload the module:
sudo modprobe -r tcp_probe
The log file from tcp_probe contains lines with the log information
1 2 3 4 5 6 7 8 9 10 7.768727538 192.168.178.33:51715 192.168.178.27:5001 32 0xe656e892 0xe6568e12 16 2147483647 42248 1 1: Time in seconds 2: Source IP:Port 3: Destination IP:Port 4: Packet length in bytes 5: snd_nxt: Sequence Number of the next packet to send 6: snd_una: Sequence Number of the first unacknowledged packet 7: snd_cwnd: Contention Window 8: ssthr: Slow Start Threshold (-1 => not known yet) 9: snd_wnd: Send Window size in Bytes 10: srtt: Averaged round trip time estimation
The data from the log file can be plotted with gnu plot
gnuplot gnuplot> plot "test.log" using 1:7 title "cwnd", "test.log" using 1:($8 > 20000000 ? 0 : $8) title "ssthr"
Set the maximum window sizes for tcp connections to 512000 Bytes.
sudo sh -c "echo 512000 > /proc/sys/net/core/wmem_max" sudo sh -c "echo 512000 > /proc/sys/net/core/wmem_default" sudo sh -c "echo 512000 > /proc/sys/net/core/rmem_max" sudo sh -c "echo 512000 > /proc/sys/net/core/rmem_default" sudo sh -c "echo 512000 512000 512000 > /proc/sys/net/ipv4/tcp_wmem" sudo sh -c "echo 512000 512000 512000 > /proc/sys/net/ipv4/tcp_rmem"
The following section shows how to change the initial contention window size to to 1 MSS.
fritz@ubuntu:~$ ip route show default via 192.168.178.1 dev eth0 proto static 169.254.0.0/16 dev eth0 scope link metric 1000 192.168.178.0/24 dev eth0 proto kernel scope link src 192.168.178.33 metric 1 fritz@ubuntu:~$ sudo ip route change default via 192.168.178.1 dev eth0 proto static initcwnd 1 fritz@ubuntu:~$ ip route show default via 192.168.178.1 dev eth0 proto static initcwnd 1 169.254.0.0/16 dev eth0 scope link metric 1000 192.168.178.0/24 dev eth0 proto kernel scope link src 192.168.178.33 metric 1 fritz@ubuntu:~$
The linux kernel provides several algorithms for tcp congestion avoidance. The following code shows how to figure out the available algorithms (here cubic and reno) and set and control the setting. In this example the congestion avoidance algorithm is switched from “cubic” to “reno”.
fritz$ cd /proc/sys/net/ipv4/ fritz$ cat tcp_available_congestion_control cubic reno fritz$ cat tcp_congestion_control cubic fritz$ sudo sh -c "echo reno > tcp_congestion_control" fritz$ cat tcp_congestion_control reno