10 Powerful cURL Tricks Every Developer Should Know

Why cURL Is a Must‑Know Command‑Line Tool

cURL is a widely used command‑line utility and library for transferring data via URLs. While many only use it for simple downloads or health checks, it offers a rich set of features that can automate complex networking tasks.

Filename Globbing

cURL can expand URL patterns, allowing multiple requests in a single command. Examples:

$ curl -s "https://jsonplaceholder.typicode.com/users/[1-3]" | jq -s .
$ curl -s "https://jsonplaceholder.typicode.com/users/[0-10:2]" | jq -s .
$ curl -s "https://jsonplaceholder.typicode.com/photos/{1,6,35}" | jq -s .
$ curl -s "https://jsonplaceholder.typicode.com/users/[1-3]" -o "file_#1.json"

These commands request a range of URLs, optionally using step values, and can pipe JSON responses to jq for aggregation. Globbing also works with protocol lists, e.g., {http,https}://..., and can combine with output variables to generate numbered filenames like file_1.json, file_2.json, file_3.json.

Using a .curlrc Configuration File

Repeated options such as proxy settings, timeouts, or headers can be stored in ~/.curlrc:

# ~/.curlrc
# Some headers
-H "Upgrade-Insecure-Requests: 1"
-H "Accept-Language: en-US,en;q=0.8"
# Follow redirects
--location

The file is read automatically; you can also specify an alternate file with -K (e.g., curl -K .curlrc https://google.com). Credentials can be kept in a .netrc file and used via --netrc-file .netrc.

Parallel Requests

cURL can issue multiple requests concurrently using --parallel (or -Z) and --parallel-max to limit connections:

$ curl -I --parallel --parallel-immediate --parallel-max 3 --config websites.txt
$ curl -I --parallel --parallel-immediate --parallel-max 3 stackoverflow.com google.com example.com

URLs can be supplied via a config file ( url = "example.com" per line) or directly on the command line.

Output Formatting and Variables

To extract specific data, use the -w option with a format file. Example format ( format.txt) and command:

$ curl --silent --output /dev/null --show-error -w @format.txt http://example.com/
# format.txt
content_type: %{content_type}
response_code: %{response_code}

# From 8.1.0:
url.scheme: %{url.scheme}
url.host: %{url.host}
url.port: %{url.port}
# Header example:
%header{date}

Variables are enclosed in %{...} and can represent response codes, URL components, or header values. Timing metrics (e.g., %{time_total}) enable precise performance measurement.

Testing and Troubleshooting

cURL assists in network debugging. Bind a request to a specific interface:

$ ip link show
$ curl --interface wlp5s0 https://example.com

Force a DNS server:

$ curl --dns-ipv4-addr 1.1.1.1 https://example.com

Test timeouts and capture exit codes:

$ curl --connect-timeout 30 --silent --output /dev/null \
  --show-error -w 'Total time: %{time_total}s
' http://google.com/ || EXIT_CODE=$?
if [ $EXIT_CODE = 28 ]; then
  echo "Cannot connect (timeout)."
else
  echo "Can connect."
fi

Proxy testing uses -x http://proxy.example.com:80.

trurl – URL Parsing Companion

trurl, a sibling project of curl, parses URLs and can output JSON. Installation example:

$ sudo apt-get install libcurl4-openssl-dev
$ git clone https://github.com/curl/trurl.git
$ cd trurl
$ make

Usage examples:

$ trurl --url https://example.com/some/path/to/file.html --get '{path}'
# /some/path/to/file.html
$ trurl --url "https://example.com/?name=hello" --append query=key=value
# https://example.com/?name=hello&key=value
$ trurl --url "https://example.com/?name=hello" --json
# JSON representation of URL components

Sending/Uploading Data

POST JSON data directly:

$ curl -X POST "https://httpbin.org/post" -H "accept: application/json" --json '{"key": "value"}'

Combine jo to build JSON on the fly:

$ jo -p key=value | curl -X POST "https://httpbin.org/post" -H "accept: application/json" --json @-

The --json flag can also read from a file ( --json @data.json).

Supported Protocols

Beyond HTTP/HTTPS, curl supports many protocols such as telnet, IMAP, POP3, SMTP, etc. Example telnet test:

# Same as telnet example.com 1234
$ curl telnet://example.com:1234

Reading email via IMAP:

$ curl --url "imaps://imap.gmail.com:993/Inbox;UID=1" --user "[email protected]:PASSWORD"

Sending email via SMTP:

$ curl smtp://mail.example.com \
  --mail-from "[email protected]" \
  --mail-rcpt "[email protected]" \
  --upload-file message.txt \
  -u "[email protected]:PASSWORD"

Conclusion

The examples above barely scratch the surface of what curl (and its library libcurl) can do. Explore the official documentation and https://everything.curl.dev/ for deeper insights.

This article is a translation of Martin Heinz’s original post: https://martinheinz.dev/blog/113. Copyright belongs to the original author.