How to View Hadoop/YARN Application Logs via History Server and Yarn Commands
This guide explains how to retrieve Hadoop/YARN application logs using the History Server UI, Yarn command‑line tools, and direct HDFS log access, including commands for listing applications, fetching specific logs, and locating the remote log directory.
To inspect Hadoop/YARN application logs you can use the History Server UI, which shows completed jobs unless the task exited abnormally.
Alternatively, use Yarn command‑line tools. The command yarn application -list -appStates ALL lists all applications with details such as Application‑Id, User, State, and Tracking URL.
To fetch logs for a specific application, run yarn logs -applicationId application_1500702983136_0002. The output includes stderr, stdout, and syslog sections with timestamps and log contents.
For direct HDFS log access, first check the configuration in
<property>
<name>yarn.nodemanager.remote-app-log-dir</name>
<value>/app-logs</value>
</property>within yarn-site.xml to locate the remote log directory (e.g., /app-logs). Then list files with hdfs dfs -ls /app-logs/hdfs/logs and view specific logs using the same yarn logs -applicationId ... command.
These steps help troubleshoot Hadoop jobs by retrieving execution logs from the History Server, Yarn CLI, or directly from HDFS.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
Big Data Technology & Architecture
Wang Zhiwu, a big data expert, dedicated to sharing big data technology.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.