Master Linux File Splitting and Merging with split and cat

Because of network transmission limits, large files often need to be split on Linux, transferred in pieces, and then merged.

File Splitting – split

The split command makes splitting large files easy.

Syntax

-a   # set suffix length (default 2: aa, ab...)
-d   # use numeric suffixes
-l   # split by number of lines (default 1000)
-b   # split by bytes (supports units: k/m)
-C   # split by size while preserving line integrity

split [-a] [-d] [-l <lines>] [-b <bytes>] [-C <bytes>] <input_file> <output_prefix>

Examples

# Split by lines
$ split -l 300000 users.sql /data/users_

# Use numeric suffixes
$ split -d -l 300000 users.sql /data/users_

# Split by byte size
$ split -d -b 100m users.sql

For help, run split --help which displays options such as -a, -d, -l, -b, -C, and others.

File Merging – cat

The cat command can concatenate the split pieces back into a single file.

Syntax

-n   # show line numbers
-e   # display $ at end of each line
-t   # show TAB characters (^I)

cat [-n] [-e] [-t] <output_file>

Examples

# Merge files
$ cat /data/users_* > users.sql

Help: cat --help shows options like -A, -b, -e, -E, -n, -s, -t, -T, etc.