How to Split Nginx Architecture into a Scalable Cluster – A Step‑by‑Step Guide

Nginx Architecture Split into a Cluster

06 Nginx Architecture Split into a Cluster

1. Split Database to a Separate Server

1.1 Why Split the Database

Running the entire LNMP stack on a single server can cause OOM, kill MySQL, and slow site access. Splitting the database to an independent server brings three main benefits: relieve web pressure, improve read/write performance, and accelerate user access.

1. Relieve web pressure;

2. Enhance database read/write performance;

3. Increase user access speed;

1.2 Database Split Architecture

1.3 Database Split Environment

Example environment: RockyLinux9 hosts web01.newy.net (nginx+php) at 10.0.0.7/172.16.1.7 and db01.newy.net (MySQL) at 10.0.0.51/172.16.1.51.

1.4 Database Split Practice

Steps to migrate the database:

1. Backup MySQL data on the current web server and copy the dump to the new database server.

2. Install MySQL on the new server and import the dump.

3. Create a remote user with appropriate privileges.

4. Update application configuration to point to the new database host.

[root@web01 ~]# mysqldump -uroot -p'newy.net' -B wordpress zh > app-database.sql
[root@web01 ~]# scp app-database.sql [email protected]:/tmp
[root@db01 ~]# yum install mysql-server -y
[root@db01 ~]# systemctl enable mysqld --now
[root@db01 ~]# mysql -uroot < /tmp/app-database.sql
[root@db01 ~]# CREATE USER 'app'@'%' IDENTIFIED BY 'newy.net';
[root@db01 ~]# GRANT ALL PRIVILEGES ON *.* TO 'app'@'%';
[root@db01 ~]# FLUSH PRIVILEGES;

Finally modify the application’s config files (e.g., wp-config.php) to use the new host 172.16.1.51.

2. Expand Multiple Identical Web Applications

2.1 Why Expand Web Nodes

Running a site on a single server limits concurrent users. Adding more web nodes improves availability, scalability, and response time.

1. A single web node failure brings the whole service down.

2. Multiple web nodes ensure continuous service and high scalability.

3. Multiple nodes reduce user access latency.

2.2 Expansion Architecture

2.3 Expansion Environment

Example hosts: web01.newy.net (10.0.0.7) and web02.newy.net (10.0.0.8) both running nginx+php, sharing the same MySQL server db01.newy.net (10.0.0.51).

2.4 Expansion Practice

Quickly add a new web node:

# Create user and group
[root@web02 ~]# groupadd -g 666 www
[root@web02 ~]# useradd -u 666 -g 666 www
# Install LNP stack
[root@web02 ~]# yum install -y nginx php php-fpm php-cli php-common php-devel php-gd php-mcrypt php-bcmath php-mbstring php-pdo php-xml php-mysqlnd php-opcache php-pecl-zip php-pecl-redis php-pecl-mongodb
# Copy nginx and php config from an existing node
[root@web02 ~]# scp -rp [email protected]:/etc/nginx /etc/
[root@web02 ~]# scp -rp [email protected]:/etc/php-fpm.d /etc/
[root@web02 ~]# scp -rp [email protected]:/etc/php.ini /etc/
# Copy website code
[root@web01 ~]# tar czf code.tar.gz /code
[root@web01 ~]# scp code.tar.gz [email protected]:/tmp
[root@web02 ~]# tar xf /tmp/code.tar.gz -C /
# Enable and start services
[root@web02 ~]# systemctl enable nginx php-fpm --now

3. Split Static Resources to an Independent Server

3.1 Why Split Static Resources

When multiple web nodes exist, uploaded media stored on one node is inaccessible to others. Using NFS shared storage solves this and reduces storage duplication.

1. Guarantees consistent static resources across all web nodes.

2. Saves storage space on each node.

3. Allows code updates without worrying about static assets.

3.2 Static Resource Split Architecture

3.3 Add Shared Storage Environment

Deploy an NFS server (e.g., nfs.newy.net) and export directories /data/blog and /data/zh to the 172.16.1.0/24 network.

[root@nfs ~]# yum install nfs-utils -y
[root@nfs ~]# cat /etc/exports
/data/blog 172.16.1.0/24(rw,sync,all_squash,anonuid=666,anongid=666)
/data/zh   172.16.1.0/24(rw,sync,all_squash,anonuid=666,anongid=666)
[root@nfs ~]# mkdir -p /data/{blog,zh}
[root@nfs ~]# chown -R www.www /data/
[root@nfs ~]# systemctl restart nfs-server

Synchronize existing uploads to the NFS share and mount it on each web node:

[root@web01 ~]# scp -rp /code/wordpress/wp-content/uploads/* [email protected]:/data/blog
[root@web01 ~]# mount -t nfs 172.16.1.22:/data/blog /code/wordpress/wp-content/uploads/
[root@web02 ~]# mount -t nfs 172.16.1.22:/data/blog /code/wordpress/wp-content/uploads/

4. Problem Thinking

4.1 How to Quickly Expand a New Node

1. Prepare LNP environment (manual or Ansible).

2. Copy configuration files and code from any existing node.

3. Mount NFS storage.

4.2 How Multiple Nodes Should Be Accessed

Two approaches:

DNS round‑robin – requires public IPs for each node, no health checks.

Load balancer – nodes stay behind private IPs, health checks and various scheduling algorithms are available.

4.3 Nginx Load Balancing Configuration

Example upstream for two web nodes:

upstream blog {
    server 172.16.1.7:80;
    server 172.16.1.8:80;
}
server {
    listen 80;
    server_name blog.newy.net;
    location / {
        proxy_pass http://blog;
        proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

6. Nginx Load Balancing Session Sharing

6.1 What Is Session Persistence

When a user logs in, the server stores a Session to keep the user in a logged‑in state.

6.2 Why Session Persistence Is Needed

Load‑balancing round‑robin distributes requests across nodes, causing the session to be lost if the next request lands on a different node.

6.3 How to Implement Session Persistence

1. Sticky session (IP_hash).

2. Session replication across nodes.

3. Session persistence in a database.

4. Session sharing via in‑memory store (Redis, Memcached).

5. Cookie‑based routing (e.g., HAProxy).

6.4 Session Persistence Demo

6.4.1 Configure Web Nodes

Install phpMyAdmin on each web node and point it to the remote MySQL server.

# Download and extract phpMyAdmin
[root@web01 ~]# wget https://files.phpmyadmin.net/phpMyAdmin/5.2.1/phpMyAdmin-5.2.1-all-languages.zip
[root@web01 ~]# unzip phpMyAdmin-5.2.1-all-languages.zip -d /code/
[root@web01 ~]# ln -s /code/phpMyAdmin-5.2.1-all-languages/ /code/phpmyadmin
# Configure remote DB host
[root@web01 phpmyadmin]# cp config.sample.inc.php config.inc.php
# Edit config.inc.php to set host to 172.16.1.51

Set up nginx for phpMyAdmin:

server {
    listen 80;
    server_name admin.newy.net;
    root /code/phpmyadmin;
    location / {
        index index.php index.html;
    }
    location ~ \.php$ {
        fastcgi_pass 127.0.0.1:9000;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }
}

6.4.2 Configure Load Balancer

upstream php {
    server 172.16.1.7:80;
    server 172.16.1.8:80;
}
server {
    listen 80;
    server_name php.newy.net;
    location / {
        proxy_pass http://php;
        proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

6.4.3 Install Redis Service

[root@db01 ~]# yum install redis -y
[root@db01 ~]# sed -i 's/^bind .*/bind 127.0.0.1 172.16.1.51/' /etc/redis.conf
[root@db01 ~]# systemctl start redis
[root@db01 ~]# systemctl enable redis

6.4.4 Configure PHP to Use Redis for Sessions

# Edit /etc/php.ini
session.save_handler = redis
session.save_path = "tcp://172.16.1.51:6379"
# Comment out file‑based session settings in php-fpm www.conf
# php_value[session.save_handler] = files
# php_value[session.save_path] = /var/lib/php/session
# Restart php-fpm
[root@web ~]# php-fpm -t
[root@web ~]# systemctl restart php-fpm