草庐IT

rsparkling

全部标签

运行 h2o、rsparkling、sparklyr

我一直在尝试使用h2o(rsparkling)运行Spark2.2,master=yarn但是当我运行h2o_context(sc)我获取异常:Error:java.lang.NoSuchMethodError:org.apache.spark.util.Utils$.getUserJars(Lorg/apache/spark/SparkConf;Z)Lscala/collection/Seq;atorg.apache.spark.repl.h2o.H2OInterpreter.createSettings(H2OInterpreter.scala:66)atorg.apache.sp

r - sparklyr - rsparkling as_h2o_frame() 错误 java.lang.IllegalArgumentException : Unsupported argument: (spark. dynamicAllocation.enabled,true)

我正在尝试在sparklyrsession期间通过H2o(使用库rsparkling)使用一些机器学习功能。我正在运行hadoop集群。考虑以下示例:library(dplyr)library(sparklyr)library(rsparkling)library(h2o)#configurethesparksessionandconnectsc=spark_connect(master='yarn-client',spark_home='/usr/hdp/current/spark-client',app_name='sparklyr',config=list("sparklyr.s