Comprehensive Guide to Using cURL for HTTP/HTTPS Requests, File Transfer, and Troubleshooting

Preface

cURL is a powerful command‑line tool for transferring data to or from a server, supporting protocols such as HTTP, HTTPS, FTP, FTPS, SMTP, etc. It can download files and test/debug network services like RESTful APIs.

Basic Usage

The simplest use is to fetch a web page:

curl http://example.com

This prints the HTML content of http://example.com to standard output.

Parameter Details

Key options include:

-X, --request – specify request method (GET, POST, PUT, DELETE, …)

-d, --data – send POST data

-H, --header – add custom HTTP header

-i, --include – include response headers in output

-o, --output – write output to a file

-u, --user – set server authentication

--data-urlencode – URL‑encode data

--form – multipart/form‑data upload

--compressed – request compressed content

--silent – silent mode (no progress bar)

--verbose – detailed communication for debugging

cURL Commands Related to HTTP

Examples of common HTTP operations:

curl -X POST http://example.com/resource

curl -d "param=value" http://example.com/resource  # defaults to POST

curl -H "Content-Type: application/json" -H "Authorization: Bearer token" http://example.com/api

curl -I http://example.com

curl -D headers.txt http://example.com

curl -o response.html http://example.com

curl -u username:password http://example.com/protected

curl -L http://example.com

curl -x http://proxy.example.com:8080 http://target-site.com

curl --insecure https://example.com

curl --silent http://example.com

curl --verbose http://example.com

cURL Commands Related to HTTPS

HTTPS usage is automatic when the URL starts with https:// :

curl https://example.com

To ignore certificate errors (for testing only):

curl --insecure https://example.com

curl -k https://example.com

Client‑certificate authentication:

curl --cert client.pem --key key.pem https://example.com

curl --cert cert-and-key.pem https://example.com

Specify a CA bundle or directory:

curl --cacert ca-bundle.crt https://example.com

curl --capath /etc/ssl/certs https://example.com

Force SSL/TLS even for HTTP URLs:

curl --ssl-reqd http://example.com

Show detailed SSL information:

curl --verbose https://example.com

Set TLS version:

curl --tlsv1.2 https://example.com

Provide password for encrypted client certificates:

curl --cert client.pem --key key.pem --pass mypassword https://example.com

Custom OpenSSL options:

curl --ssl "ALL:!ADH" https://example.com

Show SSL errors and verbose output:

curl --show-error --verbose https://example.com

File Transfer with cURL

Download a file (output to terminal):

curl http://example.com/file.zip

Save to a local file:

curl -o localfile.zip http://example.com/file.zip

Resume interrupted download:

curl -C - -o localfile.zip http://example.com/file.zip

Upload a single file using multipart/form‑data:

curl -F "file=@/path/to/localfile.txt" http://example.com/upload

Specify filename and MIME type:

curl -F "file=@/path/to/localfile.txt;filename=myfile.txt" -F "filetype=application/text" http://example.com/upload

Upload via HTTP PUT:

curl -X PUT -T /path/to/localfile.txt http://example.com/resource

Upload multiple files:

curl -F "files=@/path/to/file1.txt" -F "files=@/path/to/file2.txt" http://example.com/upload

Upload with basic authentication or custom headers:

curl -u username:password -F "file=@/path/to/localfile.txt" http://example.com/upload

curl -H "Authorization: Bearer your_token_here" -F "file=@/path/to/localfile.txt" http://example.com/upload

Set content type for upload:

curl -H "Content-Type: application/octet-stream" -F "file=@/path/to/localfile.txt" http://example.com/upload

Reading Files with cURL

cURL is not intended for direct local file reading, but you can send a local file as POST data:

curl -X POST -H "Content-Type: application/json" --data-binary @data.json https://example.com/api

Or use the file:// protocol to display a local file:

curl file:///path/to/yourfile.txt

Avoiding cURL Errors

1. Verify the URL is correct (protocol, domain, port, path).

2. Handle SSL/TLS certificates: use --insecure for testing, or --cacert to specify a trusted CA.

3. Use the appropriate HTTP method with -X .

4. Provide correct authentication with -u .

5. Add required request headers using -H .

6. Ensure POST data format matches server expectations.

7. Follow redirects with -L .

8. Check network connectivity and proxy settings.

9. Set timeouts using --connect-timeout and --max-time .

10. Use --verbose for detailed output.

11. Encode data correctly with --data-urlencode .

12. Specify a proxy with -x when needed.