Tower of Babel

Network & disk bandwidth limiting on Linux

Posted onSeptember 9, 2015 by Qin Liu

Recently, in my research, I want to throttle the I/O and network bandwidth of my program so as to show my program is robust in different senarios. In practice, we may also want to limit the resources used by certain programs (rsync, wget, …). Fortunately, this is possible in Linux. Next, I will show how to do it on Ubuntu 14.04. I believe it’s doable on any Linux distributions. For network bandwidth limiting, we will use a program called trickle. Install trickle:

sudo apt-get install trickle

1	sudo apt-get install trickle

Usage of trickle:

trickle -s -u <up link limit in KB/s> -d <down link limit in KB/s> <path-to-your-program> [args...]

1	trickle -s -u <up link limit in KB/s> -d <down link limit in KB/s> <path-to-your-program> [args...]

Example: limit the bandwidth for scp under 10MB/s

trickle -s -u 10240 -d 10240 scp ~/graph-data-origin/twitter_rv.net linux15:~/

1	trickle -s -u 10240 -d 10240 scp ~/graph-data-origin/twitter_rv.net linux15:~/

An alternative for network bandwidth limiting is wondershaper. But I found that tirckle works better. For disk bandwidth limiting, you can use the cgroup[…]

Using git and other commands behind a socks proxy server

Posted onJuly 15, 2014 by Qin Liu

Install tsocks sudo apt-get install tsocks Edit /etc/tsocks.conf sudo vim /etc/tsocks.conf Sample configuration for CSE at CUHK:

local = 137.189.88.0/255.255.252.0 # CSE domains
local = 127.0.0.0/255.255.255.0 # localhost
server = 137.189.90.232 # socks.cse.cuhk.edu.hk
server_type = 5
server_port = 1080

1

2

3

4

5

local = 137.189.88.0/255.255.252.0 # CSE domains

local = 127.0.0.0/255.255.255.0 # localhost

server = 137.189.90.232 # socks.cse.cuhk.edu.hk

server_type = 5

server_port = 1080

Done. Use it with tsocks git pull

Howto – Cross the Great Fire Wall

Posted onFebruary 9, 2013 by Qin Liu

Today is Chinese New Year’s Eve, this most important holiday for Chinese. I came back to my hometown, Fuzhou, Fujian, last night. My flight was delayed for 45mins, which is kind of lucky. In the mainland of China, the government disallows people to access Google, Facebook, Dropbox and many other useful services using the so-called Great Fire Wall (GFW). Before I went to Hong Kong, I was always using goagent to cross the GFW. Before goagent, I have also tried to buy ssh or VPN service in the US. Goagent is based on Google App Engine, and because of some limitations of GAE, you have to install some certificates manully before using goagent, which I think it is nasty and[…]

Install GCC 4.6 on Fedora 12

Posted onJanuary 10, 2013 by Qin Liu

For some reason, I have to install GCC 4.6 on a Fedora 12 server in the department (sadly that I can’t wipe it out and reinstall a latest version of Ubuntu). I believe the following steps [1] are similar in other versions of Fedora or CentOS. First, install necessary libraries:

sudo yum install gcc mpfr-devel libmpc libmpc-devel glibc-devel

1	sudo yum install gcc mpfr-devel libmpc libmpc-devel glibc-devel

Then download the source code, configure, make and install,

cd ~
mkdir build-gcc
cd build-gcc
wget ftp://ftp.dti.ad.jp/pub/lang/gcc/releases/gcc-4.6.3/gcc-4.6.3.tar.gz # download gcc source
tar xzvf gcc-4.6.3.tar.gz
mkdir build
cd build
../gcc-4.6.3/configure --program-suffix=-4.6 --enable-languages=c,c++
make
sudo make install

1

2

3

4

5

6

7

8

9

10

cd ~

mkdir build-gcc

cd build-gcc

wget ftp://ftp.dti.ad.jp/pub/lang/gcc/releases/gcc-4.6.3/gcc-4.6.3.tar.gz # download gcc source

tar xzvf gcc-4.6.3.tar.gz

mkdir build

cd build

../gcc-4.6.3/configure --program-suffix=-4.6 --enable-languages=c,c++

make

sudo make install

The GCC will be installed under /usr/local. You can change the install dir by adding –prefix=PATH_TO_INSTALL_DIR on configuration command. To use GCC 4.6 in cmake, set CC=gcc-4.6 and CXX=g++-4.6. You may encounter a fatal error: gnu/stubs-32.h: No such file or directory. [2] This error message shows up on the 64 bit systems where GCC/UPC multilib feature is enabled, and[…]

Random Thoughts on Setup

Posted onJanuary 9, 2013 by Qin Liu

Recently, I love to read the blog The Setup. It includes many interviews about how people get their work done from web developers, kernel developers, professors, researchers, musicians, politics, poet etc. It’s quite interesting and I can learn some new tools and new tricks to help myself in study and work. According to these interviews, most people use MacOS or Linux while the situation in China is quite different that Windows XP is still dominant. After reading one of those interviews, I have give ArchLinux a try in VirtualBox. I think it takes too much time for me to configure a functional system with GUI. Anyway, I’m fine with Ubuntu now. I’ve transferred from Unity to Gnome3 recently. In the[…]

Chinese Input Method in Ubuntu – SunPinyin

Posted onNovember 16, 2012 by Qin Liu

The default Simplified Chinese input method in Ubuntu is ibus-pinyin, and it really sucks!! SunPinyin is much better. It may be not as good as Sogou or Google pinyin, but it is acceptable… SunPinyin is a statistical language model based Chinese input method, which was firstly developed by Sun Beijing Globalization team, and opensource’d to community with opensolaris project, with LGPLv2 and CDDL dual-licenses. Install it using

sudo apt-get install ibus-sunpinyin

1	sudo apt-get install ibus-sunpinyin

The default keys for prev/next page are not -/+. To change the settings of sunpinyin, you can run

/usr/lib/ibus-sunpinyin/ibus-setup-sunpinyin

1	/usr/lib/ibus-sunpinyin/ibus-setup-sunpinyin

[Share] This is How Winners Are Made – YouTube

Posted onOctober 14, 2012 by Qin Liu

Life is tough, that’s a given. When you stand up, you’re gonna be shoved back down. When you’re down, you’re gonna be stepped on. My advice to you doesn’t come with a lot of bells and whistles. It’s no secret, you’ll fall down, you stumble, you get pushed, you land square on your face. And every time that happens, you get back on your feet. You get up just as fast as you can, no matter how many times you need to do it. Remember this, success has been and continues to be defined as getting up one more time than you’ve been knocked down. If experience has taught me anything, it’s that nothing is free and living ain’t easy.[…]