我已在 Amazon EC2 主机中安装了 Apache Hadoop MapReduce 2.6.1，并使用私有(private) IP 地址配置了运行时。你可以在下面[2-5]查看我的配置。

为了尝试轻松调试我的问题，我已允许安全组 [6] 中的所有入站流量。

问题是，当我启动 MapReduce 时，我在 SSH key 中得到 Permission denied。

1. 我注意到我无法从同一主机ssh EC 主机。我该如何解决这个错误？ ubuntu@ip-XXX-XX-XX-XX: ssh ubuntu@ip-XXX-XX-XX
2. 我想在 EC2 的不同站点上运行的 HDFS 实例之间复制数据。使用主机的私有(private) IP 地址是否正确？

[1] 我遇到的错误

ubuntu@ip-XXX-XX-XX-XX:~/Programs/medusa-2.0$ start-all.sh 
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
15/12/16 10:52:18 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [ip-XXX-XX-XX-XX]
ip-XXX-XX-XX-XX: Permission denied (publickey).
ip-XXX-XX-XX-XX: Permission denied (publickey).
Starting secondary namenodes [0.0.0.0]
0.0.0.0: Permission denied (publickey).
15/12/16 10:52:23 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
starting yarn daemons
starting resourcemanager, logging to /home/ubuntu/Programs/hadoop-2.6.2/logs/yarn-ubuntu-resourcemanager-ip-XXX-XX-XX-XX.out
ip-XXX-XX-XX-XX: Permission denied (publickey).

[2] Yarn-site.xml

ubuntu@ip-XXX-XX-XX-XX:~/Programs$ cat ./hadoop/etc/hadoop/yarn-site.xml 
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->
<configuration>
 <property> <name>yarn.log-aggregation-enable</name> <value>true</value> </property>
 <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property>  
 <property> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name> <value>org.apache.hadoop.mapred.ShuffleHandler</value> </property>
 <property> <name>yarn.resourcemanager.resource-tracker.address</name> <value>ip-XXX-XX-XX-XX:8025</value> </property>
 <property> <name>yarn.resourcemanager.scheduler.address</name> <value>ip-XXX-XX-XX-XX:8030</value> </property>
 <property> <name>yarn.resourcemanager.address</name> <value>ip-XXX-XX-XX-XX:8040</value> </property>
</configuration>

[3] 核心站点.xml

ubuntu@ip-XXX-XX-XX-XX:~/Programs$ cat ./hadoop/etc/hadoop/core-site.xml 
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->
<configuration>
  <property> <name>fs.default.name</name> <value>hdfs://ip-XXX-XX-XX-XX:9000</value> </property>
  <property> <name>hadoop.tmp.dir</name> <value>/tmp/hadoop-temp</value> </property>
</configuration>

[4] 奴隶

ubuntu@ip-XXX-XX-XX-XX:~/Programs$ cat ./hadoop/etc/hadoop/slaves 
ip-XXX-XX-XX-XX

[5] SSH 文件

ubuntu@ip-XXX-XX-XX-XX:~/Programs$ ls -alrt ~/.ssh/
total 24
-rw------- 1 ubuntu ubuntu  392 Dec 16 09:49 authorized_keys
-rw-r--r-- 1 ubuntu ubuntu  666 Dec 16 10:33 known_hosts
-rw-r--r-- 1 ubuntu ubuntu  404 Dec 16 10:33 id_rsa.pub
-rw------- 1 ubuntu ubuntu 1675 Dec 16 10:33 id_rsa
drwx------ 2 ubuntu ubuntu 4096 Dec 16 10:33 .
drwxr-xr-x 7 ubuntu ubuntu 4096 Dec 16 10:34 ..

[6] 安全组

All traffic   All   All   0.0.0.0/0
SSH           TCP   22   0.0.0.0/0

最佳答案

与SSH问题有关。您没有正确配置 SSH。

你可以在运行start-all.sh程序之前测试ssh ubuntu@ip****。如果它不起作用，那么您需要解决 ssh 问题并重试。

关于hadoop - EC2 主机中 MapReduce 中的 SSH 错误，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/34310154/