Step‑by‑Step Guide to Deploying MinIO Distributed Object Storage

Overview

MinIO is a high‑performance object storage released under the GNU Affero General Public License v3.0. It is compatible with the Amazon S3 API and can be used to build high‑performance infrastructure for machine‑learning, analytics, and application data workloads.

Official documentation: https://docs.min.io/

Chinese documentation: http://docs.minio.org.cn/docs/

GitHub: https://github.com/minio/minio

Features:

Data protection – Distributed MinIO uses erasure coding to tolerate multiple node failures and bit rot. A distributed deployment requires at least four disks, automatically enabling erasure‑code protection.

High availability – A single‑node MinIO has a single point of failure, but a distributed cluster with N disks remains readable as long as at least N/2 disks are online; writing requires at least N/2+1 disks.

Consistency – MinIO enforces read‑after‑write consistency in both distributed and single‑node modes.

Advantages:

Easy deployment – A single binary (minio) contains everything and runs on many platforms.

Massive storage – Scales by zone and supports objects up to 5 TB.

Low redundancy with high disk‑failure tolerance – Standard redundancy factor is 2 (1 MiB stored uses 2 MiB on disk). Any n/2 failed disks can still read data, and recovery is per‑object, not per‑volume.

Excellent read/write performance

MinIO Basic Concepts

S3

– Simple Storage Service, introduced by Amazon in 2006; the basis of object storage. Object – The basic stored entity in MinIO (file, byte stream, etc.). Bucket – Logical container for objects; data in different buckets is isolated. Drive – Physical disk configured for MinIO; all object data is stored on drives. Set – A collection of drives. In a distributed deployment the system automatically creates one or more sets; each set’s drives are spread across different locations.

Set / Drive relationship

The object ultimately resides on a set.

A set is a group of drives; in the diagram all blue and orange drives form a set.

Erasure Code

Erasure Code (EC) is a data‑protection method that splits data into fragments, adds redundant blocks, encodes them, and stores them in different locations (disks, nodes, or geographic sites).

Three main types in distributed storage: Array Code (RAID5, RAID6), Reed‑Solomon (RS) code, and Low‑Density Parity‑Check (LDPC) code.

EC can reconstruct original data from any n out of n+m fragments, tolerating up to m failed fragments.

MinIO uses Reed‑Solomon code, splitting objects into N/2 data blocks and N/2 parity blocks.

Within a cluster MinIO automatically creates erasure groups (sets); as long as the number of failed disks in a set does not exceed the number of parity disks, data can be recovered.

MinIO Deployment Modes

1) Single‑host, single‑disk mode

MinIO runs on one server with data on a single disk; this mode has a single point of failure and is mainly for development or testing.

2) Single‑host, multiple‑disk mode

MinIO runs on one server but spreads data across multiple (more than four) disks, providing data safety.

3) Multi‑host, multi‑disk (distributed) mode

Most common architecture: multiple servers share the same access and secret keys, data is spread across many disks (no upper limit), and Reed‑Solomon erasure coding provides strong redundancy.

MinIO Environment Deployment (Distributed)

1) Environment preparation

|        主机名        |       IP       |               data               |
|-------------------|-----------------|-----------------------------------|
| local-168-182-110 | 192.168.182.110 | /opt/bigdata/minio/data/export{1,2,3,4} |
| local-168-182-111 | 192.168.182.111 | /opt/bigdata/minio/data/export{1,2,3,4} |
| local-168-182-112 | 192.168.182.112 | /opt/bigdata/minio/data/export{1,2,3,4} |

2) Download

mkdir -p /opt/bigdata/minio ; cd /opt/bigdata/minio
# download RPM package for installation
# wget https://dl.min.io/server/minio/release/linux-amd64/archive/minio-20220802235916.0.0.x86_64.rpm -O minio.rpm
# yum -y install minio.rpm

# download binary package for deployment
wget https://dl.min.io/server/minio/release/linux-amd64/minio
chmod +x minio
export PATH=$PATH:/opt/bigdata/minio
minio --help

3) Add four disks per node

# without reboot, refresh the SCSI bus to detect new disks
for host in $(ls /sys/class/scsi_host) ; do echo "- - -" > /sys/class/scsi_host/$host/scan ; done

lsblk

# format
mkfs.ext4 /dev/sdb
mkfs.ext4 /dev/sdc
mkfs.ext4 /dev/sdd
mkfs.ext4 /dev/sde

# mount
mount /dev/sdb /opt/bigdata/minio/data/export1
mount /dev/sdc /opt/bigdata/minio/data/export2
mount /dev/sdd /opt/bigdata/minio/data/export3
mount /dev/sde /opt/bigdata/minio/data/export4

Disk size must be >1 GB; the example uses four 2 GB disks.

4) Configuration

MinIO defaults to port 9000. In the configuration file you can change it with --address "127.0.0.1:9029". MINIO_ACCESS_KEY: username, minimum 5 characters. MINIO_SECRET_KEY: password, minimum 8 characters; overly simple passwords cause startup failure. --config-dir: directory for cluster configuration files. --address: API port, default 9000. --console-address: web console port, default random.

5) Write startup script (/opt/bigdata/minio/run.sh)

#!/bin/bash
# create log directory
mkdir -p /opt/bigdata/minio/logs
# create data directories on each node
mkdir -p /opt/bigdata/minio/data/export{1,2,3,4}
# create configuration directory
mkdir -p /etc/minio
export MINIO_ROOT_USER=admin
export MINIO_ROOT_PASSWORD=admin123456

# start MinIO on all three machines (distributed mode)
/opt/bigdata/minio/minio server \
  --address 0.0.0.0:9000 \
  --console-address 0.0.0.0:9001 \
  --config-dir /etc/minio \
  http://local-168-182-110/opt/bigdata/minio/data/export1 \
  http://local-168-182-110/opt/bigdata/minio/data/export2 \
  http://local-168-182-110/opt/bigdata/minio/data/export3 \
  http://local-168-182-110/opt/bigdata/minio/data/export4 \
  http://local-168-182-111/opt/bigdata/minio/data/export1 \
  http://local-168-182-111/opt/bigdata/minio/data/export2 \
  http://local-168-182-111/opt/bigdata/minio/data/export3 \
  http://local-168-182-111/opt/bigdata/minio/data/export4 \
  http://local-168-182-112/opt/bigdata/minio/data/export1 \
  http://local-168-182-112/opt/bigdata/minio/data/export2 \
  http://local-168-182-112/opt/bigdata/minio/data/export3 \
  http://local-168-182-112/opt/bigdata/minio/data/export4 > /opt/bigdata/minio/logs/minio_server.log

When copying the script, ensure there is no space after the backslash (\). The directories correspond to separate disks; using a single directory for all four exports will cause errors such as part of root disk, will not be used .

6) Start the service

# reload systemd
systemctl daemon-reload
# start MinIO service
systemctl start minio
# enable auto‑start on boot
systemctl enable minio

Modify or create minio.service to manage the process:

[Unit]
Description=Minio service
Documentation=https://docs.min.io/

[Service]
WorkingDirectory=/opt/bigdata/minio
ExecStart=/opt/bigdata/minio/run.sh
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

chmod +x /usr/lib/systemd/system/minio.service && chmod +x /opt/bigdata/minio/minio && chmod +x /opt/bigdata/minio/run.sh
# copy service and binaries to the other nodes
scp -r /usr/lib/systemd/system/minio.service local-168-182-111:/usr/lib/systemd/system/
scp -r /opt/bigdata/minio local-168-182-111:/opt/bigdata/
scp -r /usr/lib/systemd/system/minio.service local-168-182-112:/usr/lib/systemd/system/
scp -r /opt/bigdata/minio local-168-182-112:/opt/bigdata/

Using Nginx for Load Balancing

Load balancing across MinIO nodes improves availability.

# install nginx
yum install epel-release -y
yum install nginx -y
systemctl start nginx
systemctl enable nginx

Create /etc/nginx/conf.d/minio.conf with the following content:

upstream minio_api {
    server 192.168.182.110:9000;
    server 192.168.182.111:9000;
    server 192.168.182.112:9000;
}

upstream minio_console {
    server 192.168.182.110:9001;
    server 192.168.182.111:9001;
    server 192.168.182.112:9001;
}

server {
    listen 19000;
    server_name 192.168.182.110;
    ignore_invalid_headers off;
    client_max_body_size 0;
    proxy_buffering off;
    location / {
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_connect_timeout 300;
        proxy_http_version 1.1;
        chunked_transfer_encoding off;
        proxy_ignore_client_abort on;
        proxy_pass http://minio_api;
    }
}

server {
    listen 19001;
    server_name 192.168.182.110;
    ignore_invalid_headers off;
    client_max_body_size 0;
    proxy_buffering off;
    location / {
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_connect_timeout 300;
        proxy_http_version 1.1;
        chunked_transfer_encoding off;
        proxy_ignore_client_abort on;
        proxy_pass http://minio_console;
    }
}

# test configuration and reload
nginx -t
nginx -s reload
# or
systemctl reload nginx

Access the load‑balanced endpoints at http://local-168-182-110:19001.

MinIO Client (mc)

Provides UNIX‑like commands (ls, cat, cp, mirror, diff) for both local file systems and S3‑compatible object storage.

Built to be compatible with AWS S3 API and tested against both MinIO and AWS S3.

Use at your own risk with other S3‑compatible services.

1) Download

cd /opt/bigdata/minio/
wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
./mc --help

2) Add a MinIO host

# plain text
./mc config host add minio http://local-168-182-110:19000 admin admin123456

# or interactive (recommended)
./mc config host add minio http://local-168-182-110:19000
Enter Access Key: admin
Enter Secret Key: admin123456

3) Test

./mc admin info minio
# create aliases for convenience
alias minfo='/opt/bigdata/minio/mc admin info'
alias mheal='/opt/bigdata/minio/mc admin heal'

More examples are available in the official documentation: http://docs.minio.org.cn/docs/master/minio-admin-complete-guide

You can also use public cloud object storage such as Tencent Cloud COS or Alibaba Cloud OSS.

The guide ends here; further practical operations will be added later.