Mastering OpenSSL: From SSL/TLS Basics to Practical Encryption Commands

SSL/TLS Overview

SSL (Secure Sockets Layer) and its successor TLS (Transport Layer Security) provide encrypted channels between applications and the transport layer, protecting data from eavesdropping or tampering. TLS adds stronger algorithms and has largely replaced older SSL versions.

Why Encrypt at the Transport Layer

Selective protection for specific applications without encrypting all traffic at lower layers.

End‑to‑end security; network‑ or data‑link‑layer encryption may only protect traffic between two devices or networks.

Data captured at lower layers remains unreadable without the transport‑layer encryption.

OpenSSL Overview

OpenSSL is an open‑source library and command‑line toolkit for handling SSL/TLS protocols and performing cryptographic operations. Most Linux distributions include the openssl binary; its version can be checked with openssl version.

Symmetric Encryption

Encrypt data with a single shared key. The openssl enc utility supports many algorithms.

openssl enc -e -algorithm -a -salt -in input_file -out encrypted_file

Example using 3DES with Base64 output:

openssl enc -e -des3 -a -salt -in file1 -out file1.cipher

Use -salt to add a random salt (generated automatically).

Replace -des3 with other algorithms such as -aes-256-cbc or -des; see man openssl‑enc for details.

The -a flag encodes the ciphertext in Base64.

Symmetric Decryption

openssl enc -d -algorithm -a -salt -in encrypted_file -out decrypted_file

Example (3DES):

openssl enc -d -des3 -a -salt -in file1.cipher -out file1_new

The algorithm must match the one used for encryption.

You will be prompted for the same password used during encryption.

Asymmetric Encryption

Public‑key cryptography uses a key pair: a public key for encryption and a private key for decryption. Encrypting with the public key ensures confidentiality; encrypting with the private key (and decrypting with the public key) provides a digital signature.

Generate a Private Key

openssl genrsa -out private_key.pem

Default key length is 2048 bits (RSA). Longer keys increase security but also processing time.

The key is output in PEM (Base64) format.

Extract the Public Key

openssl rsa -in private_key.pem -pubout -out public_key.pem

Public‑Key Encryption

openssl rsautl -encrypt -pubin -inkey public_key.pem -in plaintext.txt -out ciphertext.bin

Private‑Key Decryption

openssl rsautl -decrypt -inkey private_key.pem -in ciphertext.bin -out recovered.txt

One‑Way Hashing

Hash functions produce a fixed‑size digest that uniquely represents input data; they are irreversible and useful for integrity verification. openssl dgst -algorithm file.txt Example (default SHA‑256): openssl dgst file1 If no algorithm is specified, SHA‑256 is used.

Output is shown in hexadecimal by default.

Generating Encrypted Password Strings

OpenSSL can create password hashes suitable for the useradd command.

openssl passwd -6 mypassword   # SHA‑512 with random salt

openssl passwd -5 mypassword   # SHA‑256 with random salt

openssl passwd -1 mypassword   # MD5 with random salt

Example creating a user with a SHA‑512 password hash:

useradd bob -m -s /bin/bash -p $(openssl passwd -6 redhat)

Generating Random Numbers

openssl rand -hex 16   # 16 random bytes, hex output (32 characters)

openssl rand -base64 16   # 16 random bytes, Base64 output

Base64 Encoding Introduction

Base64 encodes arbitrary binary data using 64 printable ASCII characters, making it safe for text‑only protocols (e.g., early SMTP, HTTP).

Encoding Principle

Take three bytes (24 bits) of input and split them into four groups of six bits.

Convert each six‑bit group to a decimal value and map it to a Base64 character.

Repeat until all input bytes are processed.

If the input length is not a multiple of three, pad with = characters.