Step-by-Step Guide to Installing and Configuring Hue on CentOS 7 with Hadoop, Hive, and YARN

This article provides a complete walkthrough for installing and configuring Hue, the Hadoop UI, on a CentOS 7 environment that already runs JDK 1.8, Hadoop 2.7.3, Hive 1.2.2, and Hue 3.10.0.

Environment

centos7 jdk1.8.0_111 Hadoop 2.7.3 Hive 1.2.2 hue-3.10.0

Hue installation

1. Download the Hue package:

https://dl.dropboxusercontent.com/u/730827/hue/releases/3.10.0/hue-3.10.0.tgz

2. Install required system packages:

yum install libffi-devel

yum install gmp-devel

yum install python-devel mysql-devel

yum install ant gcc gcc-c++ rsync krb5-devel mysql openssl-devel cyrus-sasl-devel cyrus-sasl-gssapi sqlite-devel openldap-devel python-simplejson

yum install libtidy libxml2-devel libxslt-devel

yum install python-devel python-simplejson python-setuptools

yum install maven

3. Compile Hue:

tar -xzvf hue-3.10.0.tgz

cd hue-3.10.0

make apps

make install

Integrate Hue with Hadoop

HDFS configuration

<property>
  <name>dfs.webhdfs.enabled</name>
  <value>true</value>
</property>

core-site.xml configuration

<property>
  <name>hadoop.proxyuser.hue.hosts</name>
  <value>*</value>
</property>
<property>
  <name>hadoop.proxyuser.hue.groups</name>
  <value>*</value>
</property>

Distribute the modified hdfs-site.xml and core-site.xml files to all Hadoop worker nodes.

Edit hue-3.10.0/desktop/conf/hue.ini to point to the HDFS cluster:

[hadoop]
  [[hdfs_clusters]]
    [[[default]]]
      # Enter the filesystem uri
      fs_defaultfs=hdfs://localhost:8020
      # Use WebHDFS/HttpFs as the communication mechanism.
      webhdfs_url=http://localhost:50070/webhdfs/v1

YARN configuration

[hadoop]
  [[yarn_clusters]]
    [[[default]]]
      # Enter the host on which you are running the ResourceManager
      resourcemanager_host=localhost
      submit_to=True
      # URL of the ResourceManager API
      resourcemanager_api_url=http://localhost:8088
      # URL of the ProxyServer API
      proxy_api_url=http://localhost:8088
      # URL of the HistoryServer API
      history_server_api_url=http://localhost:19888

Hive integration

[beeswax]
  # Host where HiveServer2 is running.
  hive_server_host=localhost
  # Hive configuration directory, where hive-site.xml is located
  hive_conf_dir=/etc/hive/conf

Modify hive-site.xml as needed, for example:

<property>
  <name>hive.server2.thrift.bind.host</name>
  <value>localhost</value>
  <description>Bind host on which to run the HiveServer2 Thrift service.</description>
</property>

Start Hive services (note the warnings about the Metastore and HiveServer2 ports):

$ bin/hive --service metastore

$ bin/hive --service hiveserver2

Start Hue build/env/bin/supervisor Open a browser and navigate to http://192.168.187.128:8888 to access the Hue web UI and log in.

References

http://www.opexlabs.com/2016/07/20/compiling-hue-centos-7/ http://gethue.com/how-to-configure-hue-in-your-hadoop-cluster/