Mastering cURL: Syntax, Options, and Automated Log File Upload

Introduction

The author recalls a past project that used cURL to implement FOTA (firmware‑over‑the‑air) for Qualcomm IoT modules and decides to write a tutorial on using cURL for file uploads.

What is cURL?

cURL is a widely used command‑line tool for transferring data with URLs. It was created by Daniel Stenberg in 1998 and supports many protocols (HTTP, FTP, SFTP, etc.) and features such as POST, cookies, authentication, resume, rate limiting, and progress display.

Installation

sudo apt-get install curl

Basic Syntax

curl [options] [URL...]

URL Structure

A URL consists of protocol, hostname, optional port, and path (plus optional parameters, query string, and fragment). Example format:

protocol://hostname[:port]/path[;parameters][?query]#fragment

Common Options (selected)

--anyauth

: let cURL choose any supported authentication method (HTTP/HTTPS). -a, --append: append to a remote file (FTP/SFTP). --basic: use HTTP Basic authentication. --cacert FILE: specify a CA certificate. -E, --cert CERT[:PASSWD]: client certificate. --compressed: request compressed response. -L, --location: follow redirects. -i, --include: include response headers in output. -I, --head: fetch only the headers. -v, --verbose: show detailed request/response communication. --user USER[:PASSWD]: HTTP/FTP authentication. --upload-file FILE ( -T): upload a file. --cookie STRING/FILE: send cookies. --header LINE: add a custom request header.

Exit Codes

cURL returns numeric error codes; the article lists many of them (e.g., 1 = unsupported protocol, 22 = HTTP page not retrieved, 25 = FTP upload denied, 28 = operation timeout, 47 = too many redirects, etc.).

Usage Demonstrations

1. Retrieve Web Page Source

curl www.sohu.com

Use -o filename to save the page.

2. Follow Redirects

curl -L www.sohu.com

3. Show Headers

curl -i www.sohu.com

4. Show Only Headers

curl -I www.sohu.com

5. Verbose Communication

curl -v www.sohu.com

For even more detail, use --trace output.txt or --trace-ascii output.txt.

6. Send Form Data

GET example: curl example.com/form.cgi?data=xxx POST example:

curl -X POST --data "data=xxx" example.com/form.cgi

URL‑encode data with --data-urlencode.

7. HTTP Verbs

curl -X POST www.example.com

8. File Upload

curl --form upload=@localfilename --form press=OK [URL]

9. Set Referer

curl --referer http://www.example.com http://www.example.com

10. Set User‑Agent

curl --user-agent "Mozilla/5.0 (iPhone; …) Safari/6531.22.7" [URL]

11. Cookies

curl --cookie "name=xxx" www.example.com

Save cookies with -c cookie-file and reuse with -b cookie-file.

12. Custom Headers

curl --header "Content-Type:application/json" http://example.com

13. FTP Operations

List directory: curl ftp://host/ -u user:pass Download file: curl -u user:pass ftp://host/file -o file Upload file: curl -u user:pass -T localfile ftp://host/ Delete file:

curl -u user:pass ftp://host/ -X 'DELE path/file'

Automated Log File Upload Program

Functionality

Record the last modification time of a log file.

Every 10 seconds check whether the file changed; if not, sleep.

If changed, copy the log to a timestamped backup.

Upload the backup to an FTP server via cURL.

Delete the backup and repeat.

Key Functions

init()

: obtains initial modification time. check_file_change(): compares current mtime with stored value. file_name_add_time(): creates a copy named t-YYYY-M-D-H-M-S.log. stat() and system() are used for file info and command execution.

Source Code

/*   Copyright (C)   公众号: yikoulinux   */
#include <sys/stat.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <time.h>

typedef struct stat ST;
unsigned long last_mtime;

char name[32] = "user";
char pass[32] = "123456";
char ip[32]   = "192.168.43.117";
char filename[32] = "t.log";
char dstfile[256] = {0};

int init(void) {
    ST status;
    int res = stat(filename, &status);
    if (res == -1) {
        perror("error:open file fail
");
        return 0;
    }
    last_mtime = status.st_mtime;
    printf("init time:%s 
", ctime(&last_mtime));
    return 1;
}

int check_file_change(void) {
    ST status;
    int res = stat(filename, &status);
    if (res == -1) {
        perror("error:open file fail
");
        return 0;
    }
    if (last_mtime == status.st_mtime) {
        printf("file not change
");
        return 0;
    } else {
        printf("file updated
");
        last_mtime = status.st_mtime;
        return 1;
    }
}

void file_name_add_time(void) {
    time_t t = time(NULL);
    struct tm *tblock = localtime(&t);
    sprintf(dstfile, "t-%d-%d-%d-%d-%d-%d.log",
            tblock->tm_year + 1900,
            tblock->tm_mon,
            tblock->tm_mday,
            tblock->tm_hour,
            tblock->tm_min,
            tblock->tm_sec);
    char cmd[1024] = {0};
    sprintf(cmd, "cp %s %s", filename, dstfile);
    system(cmd);
}

int main(void) {
    char cmd[1024] = {0};
    init();
    while (1) {
        if (check_file_change() == 1) {
            file_name_add_time();
            sprintf(cmd, "curl -u %s:%s ftp://%s/ -T %s", name, pass, ip, dstfile);
            system(cmd);
            unlink(dstfile);
        }
        sleep(10);
    }
    return 0;
}

Execution Screenshots

Initial run shows no upload because the log has not changed. After manually appending text to t.log, the program detects the change, creates a timestamped copy, uploads it via cURL, and the file appears on the FTP server.

Additional Notes

Credentials are hard‑coded for simplicity; a real implementation should read them from a configuration file.

The FTP server setup (e.g., using FileZilla) is not covered.

For long‑running services, consider turning the program into a daemon.