Network & disk bandwidth limiting on Linux

Recently, in my research, I want to throttle the I/O and network bandwidth of my program so as to show my program is robust in different senarios. In practice, we may also want to limit the resources used by certain programs (rsync, wget, …). Fortunately, this is possible in Linux. Next, I will show how to do it on Ubuntu 14.04. I believe it’s doable on any Linux distributions.

For network bandwidth limiting, we will use a program called trickle.

1. Install trickle:
   
   Shell

   sudo apt-get install trickle

   1	sudo apt-get install trickle

2. Usage of trickle:
   Shell

   trickle -s -u <up link limit in KB/s> -d <down link limit in KB/s> <path-to-your-program> [args...]

   1	trickle -s -u <up link limit in KB/s> -d <down link limit in KB/s> <path-to-your-program> [args...]

3. Example: limit the bandwidth for scp under 10MB/s
   
   Shell

   trickle -s -u 10240 -d 10240 scp ~/graph-data-origin/twitter_rv.net linux15:~/

   1	trickle -s -u 10240 -d 10240 scp ~/graph-data-origin/twitter_rv.net linux15:~/

An alternative for network bandwidth limiting is wondershaper. But I found that tirckle works better.

For disk bandwidth limiting, you can use the cgroup mechanism and its blkio controller.

1. Install cgroup-bin:
   
   Shell

   sudo apt-get install cgroup-bin

   1	sudo apt-get install cgroup-bin

2. Create a cgroup named limit under blkio:
   
   Shell

   sudo cgcreate -t qliu -g blkio:limit

   1	sudo cgcreate -t qliu -g blkio:limit

3. Set the read/write limit to 1 MB/s (see below for explanation)
   
   Shell

   sudo cgset -r blkio.throttle.read_bps_device="8:0 1000000" /limit sudo cgset -r blkio.throttle.write_bps_device="8:0 1000000" /limit

   1 2	sudo cgset -r blkio.throttle.read_bps_device="8:0 1000000" /limit sudo cgset -r blkio.throttle.write_bps_device="8:0 1000000" /limit

   
   Here, the format of blkio.throttle.read_bps_device and blkio.throttle.write_bps_device is
   
   <major>:<minor> <bytes_per_second>

   1	<major>:<minor>  <bytes_per_second>

   
   To find out <major>:<minor> for a certain disk like /dev/sda, we just run
   
   Shell

   $ ls -l /dev/sda brw-rw---- 1 root disk 8, 0 Sep 9 20:36 /dev/sda

   1 2	$ ls -l /dev/sda brw-rw---- 1 root disk 8, 0 Sep  9 20:36 /dev/sda

   
   Then the output disk 8, 0 means <major>:<minor> should be 8:0.
4. Finally, we can limit the disk bandwidth for scp with cgexec (suppose the source file is stored on disk /dev/sda):
   
   Shell

   cgexec -g blkio:limit scp ~/graph-data-origin/twitter_rv.net linux1:~/

   1	cgexec -g blkio:limit scp ~/graph-data-origin/twitter_rv.net linux1:~/