我正在尝试使用以下代码连接到 Kerberized hdfs 集群，使用以下相同的代码我能够使用 HBaseConfiguration 访问 hbase ofcourse，
Configuration config = new Configuration();
config.set("hadoop.security.authentication", "Kerberos");
UserGroupInformation.setConfiguration(config);
UserGroupInformation ugi = null;
ugi = UserGroupInformation.loginUserFromKeytabAndReturnUGI("me@EXAMPLE>COM","me.keytab");
model = ugi.doAs((PrivilegedExceptionAction<Map<String,Object>>) () -> {
testHadoop(hcb.gethDFSConfigBean());
return null;
});
我已经能够使用相同的 key 表和主体成功访问 Solr、Impala，但我遇到了这个奇怪的问题，无法找到 hdfs 的服务名称。
请看下面的堆栈跟踪
java.io.IOException: Failed on local exception: java.io.IOException: java.lang.IllegalArgumentException: Failed to specify server's Kerberos principal name; Host Details : local host is: "Securonix-int3.local/10.0.4.36"; destination host is: "sobd189.securonix.com":8020;
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
at org.apache.hadoop.ipc.Client.call(Client.java:1472)
at org.apache.hadoop.ipc.Client.call(Client.java:1399)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:752)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy10.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1988)
at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1118)
at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1114)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1114)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400)
at com.securonix.application.ui.uiUtil.SnyperUIUtil.lambda$main$4(SnyperUIUtil.java:1226)
at com.securonix.application.ui.uiUtil.SnyperUIUtil$$Lambda$6/1620890840.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at com.securonix.application.ui.uiUtil.SnyperUIUtil.main(SnyperUIUtil.java:1216)
Caused by: java.io.IOException: java.lang.IllegalArgumentException: Failed to specify server's Kerberos principal name
at org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:680)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:643)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:730)
at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:368)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1521)
at org.apache.hadoop.ipc.Client.call(Client.java:1438)
... 23 more
Caused by: java.lang.IllegalArgumentException: Failed to specify server's Kerberos principal name
at org.apache.hadoop.security.SaslRpcClient.getServerPrincipal(SaslRpcClient.java:322)
at org.apache.hadoop.security.SaslRpcClient.createSaslClient(SaslRpcClient.java:231)
at org.apache.hadoop.security.SaslRpcClient.selectSaslClient(SaslRpcClient.java:159)
at org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:396)
at org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:553)
at org.apache.hadoop.ipc.Client$Connection.access$1800(Client.java:368)
at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:722)
at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:718)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:717)
在我为 Kerberos 启用调试代码后，我在调用 FileSystem.get() 时得到了以下调试日志； Kerberor 调试日志:
Java 配置名称:空 Java 配置名称:空
native 配置名称:/etc/krb5.conf native 配置名称:/etc/krb5.conf
从 native 配置加载 从 native 配置加载
22 年 2 月 16 日 15:53:14 警告 util.NativeCodeLoader:无法为您的平台加载 native hadoop 库...在适用的情况下使用内置 java 类
Java 配置名称:空 Java 配置名称:空
native 配置名称:/etc/krb5.conf native 配置名称:/etc/krb5.conf
从 native 配置加载 从 native 配置加载
KdcAccessibility: reset >>> KdcAccessibility: reset KdcAccessibility: reset >>> KdcAccessibility: reset KeyTabInputStream, readName(): EXAMPLE.COM >>> KeyTabInputStream, readName(): EXAMPLE.COM KeyTabInputStream, readName(): securonix >>> KeyTabInputStream, readName(): securonix KeyTab: load() entry length: 55; type: 23 >>> KeyTab: load() entry length: 55; type: 23 KeyTabInputStream, readName(): EXAMPLE.COM >>> KeyTabInputStream, readName(): EXAMPLE.COM KeyTabInputStream, readName(): securonix >>> KeyTabInputStream, readName(): securonix KeyTab: load() entry length: 71; type: 18 >>> KeyTab: load() entry length: 71; type: 18 Looking for keys for: securonix@EXAMPLE.COM Looking for keys for: securonix@EXAMPLE.COM Added key: 18version: 1 Added key: 18version: 1 Added key: 23version: 1 Added key: 23version: 1 Looking for keys for: securonix@EXAMPLE.COM Looking for keys for: securonix@EXAMPLE.COM Added key: 18version: 1 Added key: 18version: 1 Added key: 23version: 1 Added key: 23version: 1 default etypes for default_tkt_enctypes: 18 18 16. default etypes for default_tkt_enctypes: 18 18 16. KrbAsReq creating message >>> KrbAsReq creating message KrbKdcReq send: kdc=sobd189.securonix.com TCP:88, timeout=30000, number of retries =3, #bytes=139 >>> KrbKdcReq send: kdc=sobd189.securonix.com TCP:88, timeout=30000, number of retries =3, #bytes=139 KDCCommunication: kdc=sobd189.securonix.com TCP:88, timeout=30000,Attempt =1, #bytes=139 >>> KDCCommunication: kdc=sobd189.securonix.com TCP:88, timeout=30000,Attempt =1, #bytes=139 DEBUG: TCPClient reading 639 bytes >>>DEBUG: TCPClient reading 639 bytes KrbKdcReq send: #bytes read=639 >>> KrbKdcReq send: #bytes read=639 KdcAccessibility: remove sobd189.securonix.com >>> KdcAccessibility: remove sobd189.securonix.com Looking for keys for: securonix@EXAMPLE.COM Looking for keys for: securonix@EXAMPLE.COM Added key: 18version: 1 Added key: 18version: 1 Added key: 23version: 1 Added key: 23version: 1 EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType >>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType KrbAsRep cons in KrbAsReq.getReply securonix
有趣的是，当我使用像 hdfs.exists() 这样的文件系统的 api 时
>>>KinitOptions cache name is /tmp/krb5cc_501
>> Acquire default native Credentials
default etypes for default_tkt_enctypes: 18 18 16.
>>> Found no TGT's in LSA
最佳答案
我认为问题在于 HDFS 期望配置具有 dfs.datanode.kerberos.principal 的值，它是数据节点的主体，而在这种情况下它丢失了。
当我仅从 core-site.xml 创建配置实例并忘记添加 hdfs-site.xml 时，我遇到了同样的问题。一旦我添加了 hdfs-site.xml 它就开始工作了，并且 hdfs-site.xml 有:
<property>
<name>dfs.datanode.kerberos.principal</name>
<value>....</value>
</property>
希望这对您有所帮助。
关于java - 连接到 Kerberrized HDFS，java.lang.IllegalArgumentException : Failed to specify server's Kerberos principal name;，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/35325720/
我正在尝试测试是否存在表单。我是Rails新手。我的new.html.erb_spec.rb文件的内容是:require'spec_helper'describe"messages/new.html.erb"doit"shouldrendertheform"dorender'/messages/new.html.erb'reponse.shouldhave_form_putting_to(@message)with_submit_buttonendendView本身，new.html.erb，有代码:当我运行rspec时，它失败了:1)messages/new.html.erbshou
我在我的项目目录中完成了compasscreate.和compassinitrails。几个问题:我已将我的.sass文件放在public/stylesheets中。这是放置它们的正确位置吗？当我运行compasswatch时，它不会自动编译这些.sass文件。我必须手动指定文件:compasswatchpublic/stylesheets/myfile.sass等。如何让它自动运行？文件ie.css、print.css和screen.css已放在stylesheets/compiled。如何在编译后不让它们重新出现的情况下删除它们？我自己编译的.sass文件编译成compiled/t
我在我的项目中添加了一个系统来重置用户密码并通过电子邮件将密码发送给他，以防他忘记密码。昨天它运行良好(当我实现它时)。当我今天尝试启动服务器时，出现以下错误。=>BootingWEBrick=>Rails3.2.1applicationstartingindevelopmentonhttp://0.0.0.0:3000=>Callwith-dtodetach=>Ctrl-CtoshutdownserverExiting/Users/vinayshenoy/.rvm/gems/ruby-1.9.3-p0/gems/actionmailer-3.2.1/lib/action_mailer
在MRIRuby中我可以这样做:deftransferinternal_server=self.init_serverpid=forkdointernal_server.runend#Maketheserverprocessrunindependently.Process.detach(pid)internal_client=self.init_client#Dootherstuffwithconnectingtointernal_server...internal_client.post('somedata')ensure#KillserverProcess.kill('KILL',
我已经从我的命令行中获得了一切，所以我可以运行rubymyfile并且它可以正常工作。但是当我尝试从sublime中运行它时，我得到了undefinedmethod`require_relative'formain:Object有人知道我的sublime设置中缺少什么吗？我正在使用OSX并安装了rvm。 最佳答案 或者，您可以只使用“require”，它应该可以正常工作。我认为“require_relative”仅适用于ruby1.9+ 关于ruby-主要:Objectwhenrun
我正在使用Sequel构建一个愿望list系统。我有一个wishlists和itemstable和一个items_wishlists连接表(该名称是续集选择的名称)。items_wishlists表还有一个用于facebookid的额外列(因此我可以存储opengraph操作)，这是一个NOTNULL列。我还有Wishlist和Item具有续集many_to_many关联的模型已建立。Wishlist类也有:selectmany_to_many关联的选项设置为select:[:items.*,:items_wishlists__facebook_action_id].有没有一种方法可以
我花了三天的时间用头撞墙，试图弄清楚为什么简单的“rake”不能通过我的规范文件。如果您遇到这种情况:任何文件夹路径中都不要有空格!。严重地。事实上，从现在开始，您命名的任何内容都没有空格。这是我的控制台输出:(在/Users/*****/Desktop/LearningRuby/learn_ruby)$rake/Users/*******/Desktop/LearningRuby/learn_ruby/00_hello/hello_spec.rb:116:in`require':cannotloadsuchfile--hello(LoadError) 最佳
我已经像这样安装了一个新的Rails项目:$railsnewsite它执行并到达:bundleinstall但是当它似乎尝试安装依赖项时我得到了这个错误Gem::Ext::BuildError:ERROR:Failedtobuildgemnativeextension./System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/rubyextconf.rbcheckingforlibkern/OSAtomic.h...yescreatingMakefilemake"DESTDIR="cleanmake"DESTDIR="
我真的很习惯使用Ruby编写以下代码:my_hash={}my_hash['test']=1Java中对应的数据结构是什么？ 最佳答案 HashMapmap=newHashMap();map.put("test",1);我假设？ 关于java-等价于Java中的RubyHash，我们在StackOverflow上找到一个类似的问题： https://stackoverflow.com/questions/22737685/
关闭。这个问题需要detailsorclarity.它目前不接受答案。想改进这个问题吗？通过editingthispost添加细节并澄清问题.关闭8年前。Improvethisquestion在首页我有:汽车:VolvoSaabMercedesAudistatic_pages_spec.rb中的测试代码:it"shouldhavetherightselect"dovisithome_pathit{shouldhave_select('cars',:options=>['volvo','saab','mercedes','audi'])}end响应是rspec./spec/request