RubyPDF Blog bigdata the Easiest Way to Install SparkR

the Easiest Way to Install SparkR

  • Install the last version R
  • Dowload and install Spark(you can choose 1.6.2 or 2.0.0, it is also ok to keep both)
  • Install RStudio(optional)
  • Tell R where spark is installed
Sys.setenv(SPARK_HOME="/Users/steven/Applications/spark2")
.libPaths(c(file.path(Sys.getenv("SPARK_HOME"), "R", "lib"), .libPaths()))
library(SparkR)
  • Enjoy it( Spark 2.0.0 example)
sc <- sparkR.session(master="local[2]")
df <- createDataFrame(iris)   #R data.frame to Spark DataFrame
createOrReplaceTempView(df,"iris") #register Spark DataFrame as temp view
x<-sql("select * from iris")# test Spark SQL
nrow(x)#
summary(x)

collect(df)#Spark DataFrame to R data.frame
sparkR.session.stop()

 

reference,

最简便的SparkR安装方法

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.