HDP-hadoop安装部署(一）

一、官方链接

官方搭建文档：https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-installation/content/ch_Getting_Ready.html
离线搭建yum源，包下载列表：https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-installation/content/hdp_25_repositories.html

适合系统：RedHat 7 CentOS 7 Oracle Linux 7

Ambari 2.4.2.0
Base URL    http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.0
Repo File   http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.0/ambari.repo
Tarball md5 | asc   http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.0/ambari-2.4.2.0-centos7.tar.gz

HDP-2.5.3.0
HDP Version Definition File (VDF)   http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.5.3.0/HDP-2.5.3.0-37.xml
Base URL    http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.5.3.0
Repo File   http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.5.3.0/hdp.repo
Tarball md5 | asc   http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.5.3.0/HDP-2.5.3.0-centos7-rpm.tar.gz

HDP-UTILS   1.1.0.21
Base URL    http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7
Tarball md5 | asc   http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7/HDP-UTILS-1.1.0.21-centos7.tar.gz

二、 环境准备

1、主机
172.19.80.2 SY-001.hadoop
172.19.80.3 SY-002.hadoop
172.19.80.4 SY-003.hadoop
172.19.80.5 ambari-mysql

2、安装基础环境 (在主机SY-001.hadoop，SY-002.hadoop，SY-003.hadoop部署)
（1）安装基本包
$ yum install openssh wget vim openssh-clients openssl gcc openssh-server mysql-connector-odbc -y
$ systemctl enable sshd && systemctl start sshd

（2）安装mysql jdbc 驱动
$ yum install mysql-connector-java -y

（3）安装JDK
版本要求：
Oracle JDK 1.8 64-bit (minimum JDK 1.8.0_77) (default)
Oracle JDK 1.7 64-bit (minimum JDK 1.7_67)
OpenJDK 8 64-bit (not supported on SLES)
OpenJDK 7 64-bit (not supported on SLES)

$ mkdir /usr/local/src/Ambari/
$ wget http://download.oracle.com/otn-pub/java/jdk/8u112-b15/jdk-8u112-linux-x64.tar.gz?AuthParam=1480914946_a78f7ba2e70d2f0abed0b0af73c74214
$ mkdir -p /opt/java/ && tar xf jdk-8u112-linux-x64.tar.gz -C /opt/java
$ vim /etc/profile
添加下面内容：
export JAVA_HOME=/opt/java/jdk1.8.0_112
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin
$ source /etc/profile

（4）配置SSH免密码登录
主节点里root用户登录执行如下步骤
$ ssh-keygen
$ cd ~/.ssh/
$ cat id_rsa.pub >> authorized_keys
$ chmod 600 ~/.ssh/authorized_keys
把各个主机上公钥id_rsa.pub文件内容放到各个主机authorized_keys中

（5）开启NTP服务
$ yum -y install ntp
$ systemctl is-enabled ntpd
$ systemctl enable ntpd
$ systemctl start ntpd

（6）检查DNS和NSCD
ambari在安装时需要配置全域名，所以需要检查DNS。为了减轻DNS的负担, 建议在节点里用 Name Service Caching Daemon (NSCD)
$ cat /etc/hosts
172.19.80.2 SY-001.hadoop
172.19.80.3 SY-002.hadoop
172.19.80.4 SY-003.hadoop
172.19.80.5 ambari-mysql

（7）每台节点里配置FQDN，如下以主节点为例
$ cat /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=SY-001.hadoop

（8）关闭防火墙和SELinux
$ systemctl disable firewalld
$ systemctl stop firewalld
临时关闭，不用重启机器：
$ setenforce 0
修改配置文件让机器重启也生效：
$ cat /etc/sysconfig/selinux
SELINUX=disabled