Master Splunk: Install, Configure, and Search Logs on Linux and Windows

Splunk Overview

Splunk is a powerful log analysis platform that indexes raw log data and preserves the original content, enabling fast search across various log formats such as Apache, syslog, Windows Event logs, and more.

Splunk Indexing Scope

Splunk can index logs, configurations, traps, alerts, metrics, and other performance data from applications, servers, and network devices. Data can be ingested from files, network ports, databases, custom APIs, or interfaces without affecting the original data integrity.

Installation

Download the appropriate package from http://www.splunk.com/download after registering an account. For Linux (Red Hat) the installation steps are:

Install the RPM package: # rpm -ivh splunk-4.1.7.95063-linux-2.6-x86_64.rpm Disable SELinux: # setenforce 0 Start Splunk: # /opt/splunk/bin/splunk start Access the web interface at http://localhost.localdomain:8000 and verify that ports 8000 and 8089 are listening (e.g., # netstat -ant).

Configure Automatic Startup

Link the startup script: # ln -s /usr/local/splunk /bin/splunk/etc/rc2.d/S80splunk Register as a service:

# ln -s /usr/local/splunk /bin/splunk/etc/init.d/splunk

System Configuration – Collecting Logs

Examples of log collection configurations:

1. Collect Cisco device logs via Syslog

logging <syslog server IP>
logging trap <severity>

Splunk listens on UDP 514 by default.

2. Collect Linux host logs

# Send syslog to Splunk server
*.<severity> @<syslog server IP>

3. Collect Windows logs via WMI

Configure inputs.conf:

[script://$SPLUNK_HOME\bin\scripts\splunk-wmi.py]
interval = 10
source = wmi
sourcetype = wmi
disabled = 0

Create wmi.conf for each host:

[WMI:<Name>]
server = <Remote Windows IP>
interval = 60
event_log_file = <Event Log Type>
disabled = 0

Set Up Data Indexing

After logging into the Splunk web UI, reset the admin password, choose Chinese language if desired, and add data sources. Select "Local files and directories" and point to /var/log to index common system logs.

Using Splunk Search

Download sample data from http://www.splunk.com/base/images/Tutorial/Sampledata.zip. Add the data via the "Add Data" wizard, selecting the appropriate source type (e.g., access_combined_wcookie).

Key UI components include the match/scan event counts, timeline, field menu, and event viewer.

Search Example – Troubleshooting an IP Issue

To find logs related to IP 10.2.1.44: sourcetype=access_combined_wcookie 10.2.1.44 Refine the query with additional keywords such as purchase:

sourcetype=access_combined_wcookie 10.2.1.44 purchase

Boolean Operators

Exclude successful HTTP responses (status 200) to locate errors:

sourcetype=access_* AND 10.2.1.44 AND purchase NOT 200

All Boolean operators must be uppercase; parentheses can group expressions for complex queries.

Timeline Feature

The timeline visualizes event frequency over time. Selecting a bar zooms into that time window (default 1 minute per bar). Use the zoom controls to expand or contract the view.

Practical Use Cases

Search up OR down to monitor interface status.

Search duplicate to detect IP address conflicts.

Search SYNflood for SYN‑flood attacks.

Search power to find power‑off events.

Search deny to locate dropped packets on core switches.

Query Windows event IDs: EventCode=6005 OR EventCode=6006 to get system boot and shutdown times.

These queries help pinpoint network issues, security incidents, and system health problems.