How to Split Nginx Architecture into a Scalable Cluster: Database, Web Nodes, and Session Sharing

Nginx Architecture Split into a Cluster

1. Split Database to an Independent Server

1.1 Why split the database

Running the whole LNMP stack on a single server can cause OOM, kill MySQL, and slow website access. Moving MySQL to a dedicated server relieves web pressure, improves read/write performance, and speeds up user access.

Relieve web pressure

Enhance database read/write performance

Increase user access speed

1.2 Database split architecture

1.3 Database split environment

Typical environment: RockyLinux9 hosts, web01.newy.net (nginx+php) at 10.0.0.7 (internal 172.16.1.7) and db01.newy.net (mysql) at 10.0.0.51 (internal 172.16.1.51).

1.4 Database split practice

Steps to migrate MySQL to a new server:

Backup the current MySQL data on the web server and copy the dump to the new DB server.

Install MySQL on the new server and import the dump.

Create a remote user with appropriate privileges.

Update application configuration to point to the new DB host.

[root@web01 ~]# mysqldump -uroot -p'newy.net' -B wordpress zh > app-database.sql
[root@web01 ~]# scp app-database.sql [email protected]:/tmp

[root@db01 ~]# yum install mysql-server -y
[root@db01 ~]# systemctl enable mysqld --now
[root@db01 ~]# mysql -uroot < /tmp/app-database.sql

CREATE USER 'app'@'%' IDENTIFIED BY 'newy.net';
GRANT ALL PRIVILEGES ON *.* TO 'app'@'%';
FLUSH PRIVILEGES;

# Update wp-config.php
define('DB_NAME', 'wordpress');
define('DB_USER', 'app');
define('DB_PASSWORD', 'newy.net');
define('DB_HOST', '172.16.1.51');

2. Expand Multiple Identical Web Applications

2.1 Why add more web nodes

A single web server can handle only limited concurrent users. Adding more web nodes increases capacity, provides fault tolerance, and improves response speed.

Single web node failure leads to service down.

Multiple web nodes ensure high availability.

More nodes improve user access speed.

2.2 Multi‑web node architecture

2.3 Multi‑web node environment

Example hosts: web01.newy.net (nginx+php) at 10.0.0.7 (172.16.1.7) and web02.newy.net (nginx+php) at 10.0.0.8 (172.16.1.8); database servers db01.newy.net at 10.0.0.51 (172.16.1.51).

2.4 Multi‑web node practice

Quickly add a new web02 node using the existing web01 configuration and code.

Install LNP environment (group/user, nginx, php, extensions).

Copy nginx and php configuration files from web01 to web02.

Copy application code to web02.

Start nginx and php‑fpm and enable them at boot.

[root@web02 ~]# groupadd -g666 www
[root@web02 ~]# useradd -u666 -g666 www
[root@web02 ~]# scp -rp [email protected]:/etc/yum.repos.d/nginx.repo /etc/yum.repos.d/
[root@web02 ~]# yum install -y nginx php php-fpm php-cli php-common php-devel php-gd php-mcrypt php-bcmath php-mbstring php-pdo php-xml php-mysqlnd php-opcache php-pecl-zip php-pecl-redis php-pecl-mongodb
[root@web02 ~]# scp -rp [email protected]:/etc/nginx /etc/
[root@web02 ~]# scp -rp [email protected]:/etc/php-fpm.d /etc/
[root@web02 ~]# scp -rp [email protected]:/etc/php.ini /etc/

[root@web01 ~]# tar czf code.tar.gz /code
[root@web01 ~]# scp code.tar.gz [email protected]:/tmp
[root@web02 ~]# tar xf /tmp/code.tar.gz -C /

[root@web02 ~]# systemctl enable nginx php-fpm --now

3. Split Static Resources to an Independent Server

3.1 Why split static resources

When multiple web nodes exist, uploaded files may reside only on one node, making them inaccessible to others. Using NFS shared storage solves this and brings consistency, storage savings, and easier updates.

Ensures static resources are consistent across web nodes.

Saves storage space on each web server.

Allows code‑only updates without moving static files.

3.2 Static‑resource split architecture

3.3 Add shared storage environment

Deploy an NFS server (nfs.newy.net) at 10.0.0.22 (internal 172.16.1.22) and export /data/blog and /data/zh to the 172.16.1.0/24 network.

3.4 Shared storage practice

Install NFS utilities on a dedicated server.

Synchronize existing static files to the NFS export.

Mount the NFS share on each web node.

[root@nfs ~]# yum install nfs-utils -y
[root@nfs ~]# cat /etc/exports
/data/blog 172.16.1.0/24(rw,sync,all_squash,anonuid=666,anongid=666)
/data/zh   172.16.1.0/24(rw,sync,all_squash,anonuid=666,anongid=666)
[root@nfs ~]# mkdir -p /data/{blog,zh}
[root@nfs ~]# chown -R www.www /data/
[root@nfs ~]# systemctl restart nfs-server

[root@web01 ~]# scp -rp /code/wordpress/wp-content/uploads/* [email protected]:/data/blog
[root@web01 ~]# mount -t nfs 172.16.1.22:/data/blog /code/wordpress/wp-content/uploads/
[root@web02 ~]# mount -t nfs 172.16.1.22:/data/blog /code/wordpress/wp-content/uploads/

4. Problem Considerations

4.1 How to quickly expand a new node

Prepare LNP environment (manual or Ansible), copy configuration and code from any existing node, and mount NFS storage.

4.2 How multiple nodes are accessed

Two main methods:

DNS round‑robin (requires public IPs for each node, no health checks).

Load balancer (no public IP needed, provides health checks and various scheduling algorithms).

4.3 Nginx load‑balancing configuration

# blog upstream
upstream blog {
    server 172.16.1.7:80;
    server 172.16.1.8:80;
}
server {
    listen 80;
    server_name blog.newy.net;
    location / {
        proxy_pass http://blog;
        proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

# zh upstream
upstream zh {
    server 172.16.1.7:80;
    server 172.16.1.8:80;
}
server {
    listen 80;
    server_name zh.newy.net;
    location / {
        proxy_pass http://zh;
        proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

5. Nginx Session Persistence

5.1 What is session persistence

When a user logs in, the server stores a Session to keep the user in a logged‑in state.

5.2 Why it is needed

Load‑balancing round‑robin can send subsequent requests to different nodes, causing the session to be missing and forcing the user to log in again.

5.3 How to achieve session persistence

Sticky sessions (IP_hash).

Session replication across nodes.

Session storage in a database.

Session sharing via in‑memory stores like Redis or Memcached.

Cookie‑based load‑balancing (e.g., HAProxy).

5.4 Session persistence demo

5.4.1 Configure web nodes (phpMyAdmin example)

[root@web01 ~]# wget https://files.phpmyadmin.net/phpMyAdmin/5.2.1/phpMyAdmin-5.2.1-all-languages.zip
[root@web01 ~]# unzip phpMyAdmin-5.2.1-all-languages.zip -d /code/
[root@web01 ~]# ln -s /code/phpMyAdmin-5.2.1-all-languages/ /code/phpmyadmin

# Edit /code/phpmyadmin/config.inc.php
$cfg['Servers'][$i]['host'] = '172.16.1.51';

# Nginx config for phpmyadmin
server {
    listen 80;
    server_name admin.newy.net;
    root /code/phpmyadmin;
    location / {
        index index.php index.html;
    }
    location ~ \.php$ {
        fastcgi_pass 127.0.0.1:9000;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }
}

5.4.2 Configure load balancer for php nodes

upstream php {
    server 172.16.1.7:80;
    server 172.16.1.8:80;
}
server {
    listen 80;
    server_name php.newy.net;
    location / {
        proxy_pass http://php;
        proxy_set_header Host $http_host;
    }
}

5.4.3 Deploy Redis for session storage

[root@db01 ~]# yum install redis -y
[root@db01 ~]# sed -i '/^bind/c\bind 127.0.0.1 172.16.1.51' /etc/redis.conf
[root@db01 ~]# systemctl start redis
[root@db01 ~]# systemctl enable redis

5.4.4 Configure PHP to use Redis

# /etc/php.ini
session.save_handler = redis
session.save_path = "tcp://172.16.1.41:6379"
# If Redis requires a password:
#session.save_path = "tcp://172.16.1.41:6379?auth=123"

# Disable file‑based session handling in php‑fpm
# /etc/php-fpm.d/www.conf
;php_value[session.save_handler] = files
;php_value[session.save_path] = /var/lib/php/session

[root@web ~]# php-fpm -t
[root@web ~]# systemctl restart php-fpm