Installing RHadoop

Based on this, to get RHadoop installed on our system we need Hadoop with either a single node or multimode installation as per the size of our data.

• Installing the R packages:

We can install them by calling the execution of the following R command in

the R console:

install.packages(c('rJava','RJSONIO','itertools','digest','Rcpp','httr','functional','devtools', 'plyr','reshape2'))

• Setting Environment variables

We can set this via the R console using the

following code:

## Setting HADOOP_CMD

Sys.setenv(HADOOP_CMD="/usr/local/hadoop/bin/hadoop")

## Setting up HADOOP_STREAMING

Sys.setenv(HADOOP_STREAMING="/usr/local/hadoop/contrib/streaming/

hadoop-streaming-1.0.3.jar")

or, we can also set the R console via the command line as follows:

export HADOOP_CMD=/usr/local/Hadoop

export HADOOP_STREAMING=/usr/lib/hadoop-0.20-mapreduce/contrib/

streaming/hadoop-streaming-2.0.0-mr1-cdh4.1.1.jar

• Installing RHadoop [rhdfs, rmr, rhbase]

1. Download RHadoop packages from GitHub repository of Revolution

Analytics: 

https://github.com/RevolutionAnalytics/RHadoop.

°° rmr: [rmr-2.2.2.tar.gz]

°° rhdfs: [rhdfs-1.6.0.tar.gz]

°° rhbase: [rhbase-1.2.0.tar.gz]

2. Install this packages in R by

packages -> install packages from local zip files

3. Check the Installation

Once we complete the installation of RHadoop, we can test the setup by running the MapReduce job with the rmr2 and rhdfs libraries in the RHadoop sample program as follows:

## loading the libraries

library(rhdfs)

library(rmr2)

## initializing the RHadoop

hdfs.init()