参考
https://zhuanlan.zhihu.com/p/460867086
https://blog.51cto.com/yangxingzhen/5980590
下载相关软件包：
链接：https://pan.baidu.com/s/1U04kjTUPtT58FNh2FwPqGQ
提取码：5ja5
上传至服务器：
| [hadoop@node-01 data]$ pwd /data [hadoop@node-01 data]$ ll total 11839096 -rw-r--r--. 1 hadoop hadoop 2030700094 Jan 23 06:40 ambari-2.7.4.0-centos7.tar.gz -rw-r--r--. 1 hadoop hadoop 255543307 Jan 23 06:15 flink-1.9.1-bin-scala_2.11.tgz -rw-r--r--. 1 hadoop hadoop 43467085 Jan 23 06:12 flink-shaded-hadoop-2-uber-2.8.3-7.0.jar -rw-r--r--. 1 hadoop hadoop 9506255805 Jan 23 07:30 HDP-3.1.4.0-centos7-rpm.tar.gz -rw-r--r--. 1 hadoop hadoop 162038 Jan 23 06:10 HDP-GPL-3.1.4.0-centos7-gpl.tar.gz -rw-r--r--. 1 hadoop hadoop 90606616 Jan 23 06:10 HDP-UTILS-1.1.0.22-centos7.tar.gz -rw-r--r--. 1 hadoop hadoop 194151339 Jan 23 06:11 jdk-8u231-linux-x64.tar.gz -rw-r--r--. 1 hadoop hadoop 2330539 Jan 23 06:09 mysql-connector-java-8.0.18.jar |
|---|
在各个节点上创建hadoop用户
| [root@node-01 ~]# groupadd hadoop [root@node-01 ~]# useradd -g hadoop hadoop [root@node-01 ~]# passwd hadoop ( 密码为hadoop) |
|---|
在工作中一般不会让我们通过root用户来操作，因此我们给hadoop用于赋予sudo权限
| [root@node-01 ~]# visudo 注释该行： # %wheel ALL=(ALL) NOPASSWD: ALL 新增： hadoop ALL=(ALL) NOPASSWD: ALL |
|---|
生产环境一般由专业运维人员来限制相关端口，而不是完全禁用防火墙。在学习中，通过如下命令禁用防火墙，避免多个节点之间通信受到影响:
| [hadoop@node-01 ~]$ sudo systemctl disable firewalld [hadoop@node-01 ~]$ sudo systemctl stop firewalld [hadoop@node-01 ~]$ sudo systemctl status firewalld |
|---|
| [root@node-01 ~]# su - hadoop [hadoop@node-01 ~]$ sudo cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime |
|---|
SELinux是美国国家宇航局开发的Linux的安全子系统，可以避免出现各种莫名其妙的问题。
| #检查SELinux状态 [hadoop@node-01 ~]$ getenforce Enforcing #临时禁用 [hadoop@node-01 ~]$ sudo setenforce 0 #永久禁用 [hadoop@node-01 ~]$ sudo vi /etc/selinux/config #将SELINUX=enforcing改为 SELINUX=disabled |
|---|
umask用于设置在Linux上创建新文件或文件夹时授予的默认权限或基本权限。 大多数Linux发行版将022设置为默认umask值。 umask值022授予新文件或文件夹的755权限。 umask值027授予新文件或文件夹的750权限。
Ambari，HDP和HDF支持的umask值为022（ 等价于0022），027（等价于0027）。 这些值必须在所有主机上设置。
| #查看当前状态 [hadoop@node-01 ~]$ umask 0002 [hadoop@node-01 ~]$ sudo vi /etc/profile #新增 umask配置 umask 0022 [hadoop@node-01 ~]$ source /etc/profile [hadoop@node-01 ~]$ umask 0022 |
|---|
| [hadoop@node-01 ~]$ sudo vi /etc/security/limits.conf # 表示任何一个用户可以打开的最大的文件描述符数量 * soft nofile 655350 * hard nofile 655350 |
|---|
| [hadoop@node-01 ~]$ ssh-keygen [hadoop@node-01 ~]$ cd .ssh/ [hadoop@node-01 .ssh]$ cat id_rsa.pub >> authorized_keys [hadoop@node-01 .ssh]$ chmod 700 ../.ssh [hadoop@node-01 .ssh]$ chmod 600 ../.ssh/authorized_keys |
|---|
| [hadoop@node-01 ~]$ sudo tar -zxvf /data/jdk-8u231-linux-x64.tar.gz -C /usr/local/ [hadoop@node-01 ~]$ sudo ln -s /usr/local/jdk1.8.0_231/ /usr/local/jdk [hadoop@node-01 ~]$ sudo vi /etc/profile # jdk 1.8 export JAVA_HOME=/usr/local/jdk export JRE_HOME=${JAVA_HOME}/jre export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib export PATH=${JAVA_HOME}/bin:$PATH [hadoop@node-01 ~]$ source /etc/profile [hadoop@node-01 ~]$ java -version java version "1.8.0_231" Java(TM) SE Runtime Environment (build 1.8.0_231-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.231-b11, mixed mode) |
|---|
| #安装Apache [hadoop@node-01 ~]$ sudo yum -y install httpd #设置开机启动 [hadoop@node-01 ~]$ sudo systemctl enable httpd #启动服务 [hadoop@node-01 ~]$ sudo systemctl start httpd |
|---|
访问 http://192.169.1.101 或者 http://node-01
Apache Web界面
| #移动HDP软件包到html目录下，并解压： [hadoop@node-01 ~]$ cd /var/www/html/ [hadoop@node-01 html]$ sudo mv /data/ambari-2.7.4.0-centos7.tar.gz ./ [hadoop@node-01 html]$ sudo mv /data/HDP-* ./ #解压安装包 [hadoop@node-01 html]$ sudo tar -zxvf ambari-2.7.4.0-centos7.tar.gz [hadoop@node-01 html]$ sudo tar -zxvf HDP-3.1.4.0-centos7-rpm.tar.gz [hadoop@node-01 html]$ sudo tar -zxvf HDP-GPL-3.1.4.0-centos7-gpl.tar.gz [hadoop@node-01 html]$ sudo tar -zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz #更改目录权限 [hadoop@node-01 html]$ sudo chmod -R ugo+rX /var/www/html/ |
|---|
| [hadoop@node-01 yum]$ cd /etc/yum.repos.d/ [hadoop@node-01 yum.repos.d]$ sudo vi ambari.repo [ambari-repo] name=ambari baseurl=http://node-01/ambari/centos7/2.7.4.0-118/ gpgcheck=0 enabled=1 |
|---|
| [hadoop@node-01 yum.repos.d]$ yum repolist |
|---|
ambari仓库验证截图
验证配置仓库成功。
| #安装MySQL yum仓库 [hadoop@node-01 ~]$ sudo yum -y localinstall https://dev.mysql.com/get/mysql56-community-release-el7-8.noarch.rpm #安装MySQL [hadoop@node-01 ~]$ sudo yum install mysql-community-server #设置开机启动 [hadoop@node-01 ~]$ sudo systemctl enable mysqld [hadoop@node-01 ~]$ sudo systemctl start mysqld #设置密码 [hadoop@node-01 ~]$ sudo mysqladmin -u root password '密码' #进入mysql数据库 [hadoop@node-01 ~]$ sudo mysql -u root -p #配置远程连接 mysql> grant all on *.* to 'root'@'%' identified by '密码'; Query OK, 0 rows affected (0.00 sec) #配置远程连接 mysql> create database ambari; Query OK, 1 row affected (0.00 sec) mysql> grant all on ambari.* to 'ambari'@'%' identified by '密码'; mysql> grant all on ambari.* to 'ambari'@'node-01' identified by '密码'; Query OK, 0 rows affected (0.00 sec) |
|---|
| [hadoop@node-01 ~]$ sudo mkdir -p /usr/share/java [hadoop@node-01 ~]$ sudo mv /data/mysql-connector-java-8.0.18.jar /usr/share/java/ |
|---|
| [hadoop@node-01 ~]$ sudo yum -y install ambari-server |
|---|
进入MySQL数据库，执行初始化Ambari数据表
| [hadoop@node-01 ~]$ sudo mysql -u root -p mysql> use ambari; Database changed mysql> source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql; |
|---|
配置Ambari-server
| [hadoop@node-01 ~]$ sudo ambari-server setup Using python /usr/bin/python Setup ambari-server Checking SELinux... SELinux status is 'disabled' Customize user account for ambari-server daemon [y/n] (n)? --输入n Adjusting ambari-server permissions and ownership... Checking firewall status... Checking JDK... [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8 [2] Custom JDK =====================================================================Enter choice (1): 2 WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts. WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts. Path to JAVA_HOME: /usr/local/jdk Validating JDK on Ambari Server...done. Check JDK version for Ambari Server... JDK version found: 8 Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server. Checking GPL software agreement... GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? y Completing setup... Configuring database... Enter advanced database configuration [y/n] (n)? y Configuring database... ===================================================================== Choose one of the following options: [1] - PostgreSQL (Embedded) [2] - Oracle [3] - MySQL / MariaDB [4] - PostgreSQL [5] - Microsoft SQL Server (Tech Preview) [6] - SQL Anywhere [7] - BDB ===================================================================== Enter choice (1): 3 Hostname (localhost): Port (3306): Database name (ambari): Username (ambari): Enter Database Password (bigdata): Re-enter password: Configuring ambari database... Enter full path to custom jdbc driver: /usr/share/java/mysql-connector-java-8.0.18.jar Configuring remote database connection properties... WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql Proceed with configuring remote database connection properties [y/n] (y)? y Extracting system views... ambari-admin-2.7.4.0.118.jar .... Ambari repo file doesn't contain latest json url, skipping repoinfos modification Adjusting ambari-server permissions and ownership... Ambari Server 'setup' completed successfully. |
|---|
启动Ambari-sever
| [hadoop@node-01 ~]$ sudo ambari-server start Using python /usr/bin/python Starting ambari-server Ambari Server running with administrator privileges. Organizing resource files at /var/lib/ambari-server/resources... Ambari database consistency check started... Server PID at: /var/run/ambari-server/ambari-server.pid Server out at: /var/log/ambari-server/ambari-server.out Server log at: /var/log/ambari-server/ambari-server.log Waiting for server start................................................................................... Server started listening on 8080 DB configs consistency check: no errors and warnings were found. Ambari Server 'start' completed successfully. |
|---|
访问Ambari web界面，http://node-01:8080/#/login
Ambari Web登录界面
启动安装向导，点击“Launch Install Wizard”。
集群命名
版本选择,选择HDP-3.1版本
发布于 2022-01-23 16:49
#########################
@@@@@@@@@@@@@@@@
$$$$$$$$$$$
56的mysql安装不成功，用57的替代
https://blog.csdn.net/weixin_42133329/article/details/116805467
安装server的时候报错
https://blog.51cto.com/u_13045706/5078336
修改密码报错 sudo mysqladmin -u root password 'jwwng'
通过如下方式找到密码【;ojVka0kpMdI】，再用mysql登录
https://blog.csdn.net/Rong_Toa/article/details/88410154
利用如下方法修改root密码
https://blog.csdn.net/wingrez/article/details/99825144
set password='2021-com-CN';
修改hostname
sudo hostnamectl set-hostname node-01
启动失败
.ERROR: Exiting with exit code -1.
链接数据库异常
ERROR 1045 (28000): Access denied for user 'root'@'node-01' (using password: YES)
https://cloud.tencent.com/developer/article/1993884
mysql修改某个用户的密码 https://blog.csdn.net/qq_63844528/article/details/127815952
update user set authentication_string=password('2021-com-CN') where user='ambari' and host='node-01';
我想为Heroku构建一个Rails3应用程序。他们使用Postgres作为他们的数据库，所以我通过MacPorts安装了postgres9.0。现在我需要一个postgresgem并且共识是出于性能原因你想要pggem。但是我对我得到的错误感到非常困惑当我尝试在rvm下通过geminstall安装pg时。我已经非常明确地指定了所有postgres目录的位置可以找到但仍然无法完成安装:$envARCHFLAGS='-archx86_64'geminstallpg--\--with-pg-config=/opt/local/var/db/postgresql90/defaultdb/po
我打算为ruby脚本创建一个安装程序，但我希望能够确保机器安装了RVM。有没有一种方法可以完全离线安装RVM并且不引人注目(通过不引人注目，就像创建一个可以做所有事情的脚本而不是要求用户向他们的bash_profile或bashrc添加一些东西)我不是要脚本本身，只是一个关于如何走这条路的快速指针(如果可能的话)。我们还研究了这个很有帮助的问题:RVM-isthereawayforsimpleofflineinstall?但有点误导，因为答案只向我们展示了如何离线在RVM中安装ruby。我们需要能够离线安装RVM本身，并查看脚本https://raw.github.com/wayn
我有一个奇怪的问题:我在rvm上安装了rubyonrails。一切正常，我可以创建项目。但是在我输入“railsnew”时重新启动后，我有“程序'rails'当前未安装。”。SystemUbuntu12.04ruby-v"1.9.3p194"gemlistactionmailer(3.2.5)actionpack(3.2.5)activemodel(3.2.5)activerecord(3.2.5)activeresource(3.2.5)activesupport(3.2.5)arel(3.0.2)builder(3.0.0)bundler(1.1.4)coffee-rails(
我刚刚为fedora安装了emacs。我想用emacs编写ruby。为ruby提供代码提示、代码完成类型功能所需的工具、扩展是什么？ 最佳答案 ruby-mode已经包含在Emacs23之后的版本中。不过，它也可以通过ELPA获得。您可能感兴趣的其他一些事情是集成RVM、feature-mode(Cucumber)、rspec-mode、ruby-electric、inf-ruby、rinari(用于Rails)等。这是我当前用于Ruby开发的Emacs配置:https://github.com/citizen428/emacs
我正在尝试在我的centos服务器上安装therubyracer，但遇到了麻烦。$geminstalltherubyracerBuildingnativeextensions.Thiscouldtakeawhile...ERROR:Errorinstallingtherubyracer:ERROR:Failedtobuildgemnativeextension./usr/local/rvm/rubies/ruby-1.9.3-p125/bin/rubyextconf.rbcheckingformain()in-lpthread...yescheckingforv8.h...no***e
我的最终目标是安装当前版本的RubyonRails。我在OSXMountainLion上运行。到目前为止，这是我的过程:已安装的RVM$\curl-Lhttps://get.rvm.io|bash-sstable检查已知(我假设已批准)安装$rvmlistknown我看到当前的稳定版本可用[ruby-]2.0.0[-p247]输入命令安装$rvminstall2.0.0-p247注意:我也试过这些安装命令$rvminstallruby-2.0.0-p247$rvminstallruby=2.0.0-p247我很快就无处可去了。结果:$rvminstall2.0.0-p247Search
我实际上是在尝试使用RVM在我的OSX10.7.5上更新ruby，并在输入以下命令后:rvminstallruby我得到了以下回复:Searchingforbinaryrubies,thismighttakesometime.Checkingrequirementsforosx.Installingrequirementsforosx.Updatingsystem.......Errorrunning'requirements_osx_brew_update_systemruby-2.0.0-p247',pleaseread/Users/username/.rvm/log/138121
由于fast-stemmer的问题，我很难安装我想要的任何rubygem。我把我得到的错误放在下面。Buildingnativeextensions.Thiscouldtakeawhile...ERROR:Errorinstallingfast-stemmer:ERROR:Failedtobuildgemnativeextension./System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/rubyextconf.rbcreatingMakefilemake"DESTDIR="cleanmake"DESTDIR=
当我尝试安装Ruby时遇到此错误。我试过查看this和this但无济于事➜~brewinstallrubyWarning:YouareusingOSX10.12.Wedonotprovidesupportforthispre-releaseversion.Youmayencounterbuildfailuresorotherbreakages.Pleasecreatepull-requestsinsteadoffilingissues.==>Installingdependenciesforruby:readline,libyaml,makedepend==>Installingrub
当我执行>rvminstall1.9.2时一切顺利。然后我做>rvmuse1.9.2也很顺利。但是当涉及到ruby-v时..sam@sjones:~$rvminstall1.9.2/home/sam/.rvm/rubies/ruby-1.9.2-p136,thismaytakeawhiledependingonyourcpu(s)...ruby-1.9.2-p136-#fetchingruby-1.9.2-p136-#downloadingruby-1.9.2-p136,thismaytakeawhiledependingonyourconnection...%Total%Rece
