How to Uncover Hidden Threats: Mastering Web Log Analysis with Secsoso

Why Analyze Web Logs

As web technologies evolve, websites become critical business services, making their behavior and security paramount. Analyzing web logs helps identify security events, support incident forensics, and understand common business actions.

How to Perform Log Analysis

Web servers generate logs for each request. For example, an Apache access log entry looks like:

114.221.137.87 - - [10/Sep/2019:14:52:28 +0800] "GET /login HTTP/1.1" 200 1068 https://secilog.secisland.com/ Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.105 Safari/537.36 Vivaldi/2.4.1488.40

From such entries we can differentiate normal user requests from malicious scans or attacks by looking for patterns such as numerous 404 responses for non‑existent URLs or suspicious query strings.

Log Analysis Scenarios

Scenario 1: Identify the time window of an intrusion, then examine suspicious logs within that range to pinpoint the attacker and reconstruct the attack flow.

Scenario 2: After an intrusion, attackers often leave backdoor files; locating these files provides clues for deeper analysis.

While experienced engineers can use shell commands, most users benefit from a dedicated tool. The example below uses the free tool Secsoso to demonstrate common queries.

Business Behavior Statistics

Top 10 IPs by Access Count

secsoso 'file=("/export/home/20190613/access.log"," ")|stats count($1) by $1|sort 10 -count_$1'

Result shows IP addresses and their request counts.

Count of Distinct IPs

secsoso 'access.log|rename $1 as ip |stats dc(ip)'

Page‑Specific Access Count

secsoso 'access.log " /nodejs-underscore/?cf_action=sync_comments"|eventcount'

Pages Visited per IP

secsoso 'access.log $7="*.html*" |stats $1 as ip,count($1) as count by $1'

Hourly Access Volume

secsoso 'access.log|eval date=$4.to_date("[dd/MMM/yyyy:HH:mm:ss")|stats count(date) by tspan(date,"1h")'

Browser Fingerprint Filtering

secsoso 'access.log "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9) Gecko Minefield/3.0" $7!="*.js*" $7!="*.css*"|fields $1,$4,$7'

Crawler Statistics

secsoso 'access.log "Googlebot|Baiduspider|Applebot|bingbot|YoudaoBot|Sosospider|MSNBot"|stats dc($1) as count'

IP Segment Access Count

secsoso 'access.log|stats rex($1,"[0-3]+\.[0-9]+\.[0-9]+") as ipper,count(1) as count by ipper|sort -count'

Status Code Distribution

secsoso 'access.log|stats $9 as stat ,count($9) as count by $9'

URL Access Frequency

secsoso 'access.log|eval url=$7.substring(0,POSITION("?",$7))|stats count(url) by url'

Security Statistics

CC Attack Detection (excluding static resources)

secsoso 'access.log $7!="/.*\.js.*|.*\.css.*|.*\.png.*/" |eval date=$4.to_date("[dd/MMM/yyyy:HH:mm:ss")|stats $1 as ip,count(date) as count by tspan(date,"1h"),$1|search count>50'

CC Attack Detection (including static resources)

secsoso 'access.log|eval date=$4.to_date("[dd/MMM/yyyy:HH:mm:ss")|stats $1 as ip,count(date) as count by tspan(date,"1h"),$1|search count>100'

SQL Injection Attempts

secsoso 'access.log "%20select%20|%20and%201=1|%20and%201=2|%20exec|%27exec|information_schema.tables|%20where%20|%20union%20|%20SELECT%20|%2ctable_name%20|cmdshell|%20table_schema"|fields $1,$7'

SQL Injection IP Ranking

secsoso 'access.log "%20select%20|%20and%201=1|%20and%201=2|%20exec|%27exec|information_schema.tables|%20where%20|%20union%20|%20SELECT%20|%2ctable_name%20|cmdshell|%20table_schema"|fields $1,$7|stats $1 as ip,count(1) as count by $1|sort -count'

Sensitive File Scanning

secsoso 'access.log ".zip|.rar|.mdb|.inc|.sql|.config|.bak|/login.inc.php|.svn|/mysql/|config.inc.php|.bak|wwwroot|网站备份|/gf_admin/|/DataBackup/|/Web.config|/web.config|/1.txt|/test.txt"|fields $1,$7'

Vulnerability Exploitation

secsoso 'access.log "struts|jmx-console|ajax_membergroup.php|iis.txt|phpMyAdmin|getWriter|dirContext|phpmyadmin|acunetix.txt|/e/|/SouthidcEditor/|/DatePicker/"|fields $1,$7'

File Inclusion Attacks

secsoso 'access.log "/passwd|%00|/win.ini|/my.ini|/MetaBase.xml|/ServUDaemon.ini|cmd.exe"|fields $1,$7'

Getshell Attacks

secsoso 'access.log " eval|%eval|%execute|%3binsert|%20makewebtaski|/1.asp|/1.jsp|/1.php|/1.aspx|/xiaoma.jsp|/tom.jsp|/py.jsp|/k8cmd.jsp|/k8cmd|/ver007.jsp|/ver008.jsp|/ver007|/ver008|\.aar|%if"|fields $1,$7'

XSS (Cross‑Site Scripting) Attacks

secsoso 'access.log "(S)%3C(S+)%3E|(S)%3C(S+)%2F%3E|(S+)<(S+)>|(S+)<(S+)/>|onerror|onmouse|expression|alert|document.|prompt\()"|fields $1,$7'

Login Password Guessing

Detect IPs with more than ten failed login attempts within five minutes:

secsoso 'access.log $6="*POST" $7="*/login" $9="200"|eval date=$4.to_date("[dd/MMM/yyyy:HH:mm:ss")|stats count($1) as count by tspan(date,"5m"),$1|search count>10|stats sum(count) by $1'

The analysis demonstrates that web logs contain rich information for security monitoring, but effective use requires preserving request parameters (including POST data) and applying systematic queries.