Boost Server Performance with Gearman, CPU Affinity, and Nginx Worker Binding

Use Gearman to build a Map/Reduce framework and manage all workers with GearmanManager. Start multiple gearman‑manager daemons so that they can run on different CPU cores and fully utilize server resources.

Assume you start 10 gearman‑manager daemons on a 4‑core CPU.

[root@www ~]# ps aux | grep gearman-manager | awk {'print $2;'} | sort -k1,1 | head -3 | xargs -n 1 taskset -cp 0
[root@www ~]# ps aux | grep gearman-manager | awk {'print $2;'} | sort -k1,1 | tail -3 | xargs -n 1 taskset -cp 1
[root@www ~]# ps aux | grep gearman-manager | awk {'print $2;'} | sort -k1,1 | sed -n '4,7p' | xargs -n 1 taskset -cp 2

The commands bind the first three daemons to CPU#0, the middle four to CPU#2, and the last three to CPU#1 (CPU numbering starts at 0).

Background: squeezing the server by running processes on specific CPUs.

Two terms: SMP (Symmetrical Multi‑Processing): A system where multiple CPUs share memory and bus structures. CPU affinity: Also called "CPU亲和力" in Chinese, it binds one or more processes to specific processors in a CMP architecture.

Changing a process's CPU affinity on Linux On Linux, the taskset command can modify affinity. In CentOS it is provided by the util‑linux-2.13-pre7 package; compile and install if needed. To assign CPUs #1, #2, #3 to a running process with PID 12345: [root@www ~]# taskset -cp 1,2,3 12345 For services already running, such as MySQL, you must set affinity at startup:

[root@www ~]# taskset -c 1,2,3 /usr/local/mysql/bin/mysqld_safe &

Other processes can be handled similarly (except Nginx, see below). After setting, use top and press 1 to view CPU usage per core.

Binding Nginx workers to CPUs Nginx allows explicit CPU binding via the worker_cpu_affinity directive in conf/nginx.conf . The worker_processes directive defines how many worker processes to start (default 1).

worker_processes  3;
worker_cpu_affinity 0010 0100 1000;

The masks 0010 , 0100 , and 1000 correspond to CPU cores 2, 3, and 4 respectively. After restarting Nginx, each worker runs on its assigned core.

Deep dive

If you write your own code, you can bind a process to a CPU using the sched_setaffinity system call on Linux.

A child process inherits the CPU affinity of its parent (using taskset to start a process essentially performs a fork+exec with the specified affinity).

@Xitong Linode typically runs only on CPU#0, largely due to Xen configuration. When the kernel supports SMP, the default behavior is to distribute load across all CPUs as much as possible.

Via: http://blogread.cn/it/article/5985?f=sr