SPARK installation steps in ubuntu 18.04

1. Download Spark:
http://spark.apache.org/downloads.html

(Steps 1, 2, 3, 4)



2. Unzip and move spark to /usr/lib/

machineName:~$tar xzvf spark-2.3.1-bin-hadoop2.7.tgz

machineName:~$mv spark-2.3.1-bin-hadoop2.7 spark

machineName:~$sudo mv spark /usr/lib/



3. Install Java 8 following steps given in the web page below:

http://tipsonubuntu.com/2016/07/31/install-oracle-java-8-9-ubuntu-16-04-linux-mint-18/

sudo add-apt-repository ppa:webupd8team/java
sudo apt update; sudo apt install oracle-java8-installer
Click OK and YES when the Oracle User Agreement screens appear.
javac -version
sudo apt install oracle-java8-set-default

4. Add Path:
sudo nano ~/.bashrc


5. At the end of .bashrc, add the following and save:

export JAVA_HOME=/usr/lib/jvm/java-8-oracle

export SPARK_HOME=/usr/lib/spark/

export PATH=$PATH:$SPARK_HOME/bin



6. Open a new terminal and verify installation by running the spark shell:

machineName:~ spark-shell




Comments

Post a Comment

Popular posts from this blog

Installation of JDK 8, R,R Studio,sbt, scala, spark

Installation of java 8 sudo su sudo apt install openjdk-8-jdk Steps to install R & R Studio sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E298A3A825C0D65DFD57CBB651716619E084DAB9   sudo add-apt-repository 'deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/'   sudo apt update sudo apt install libclang-dev sudo apt install r-base Test Install Start R’s interactive shell as root. sudo -i R  Install Rstudio a. download latest Rstudio version   https://download1.rstudio.org/desktop/bionic/amd64/rstudio-1.2.1335-amd64.deb cd Downloads sudo dpkg -i rstudio-1.2.1335-amd64.deb COMMANDS TO INSTALL SBT echo "deb https://dl.bintray.com/sbt/debian /" | sudo tee -a /etc/apt/sources.list.d/sbt.list sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2EE0EA64E40A89B84B2DF73499E82A75642AC823 sudo apt-get update sudo apt-get install sbt if anything gives error while sbt in...
Read more

INSTALLATION R and Rstudio IN UBUNTU 18.04

Introduction R is an open-source programming language that specializes in statistical computing and graphics. In this tutorial, we will install R on an Ubuntu 18.04 server. For a more detailed version of this tutorial, with better explanations of each step, please refer to How To Install R on Ubuntu 18.04. Step 1 — Add GPG Key Logged into your Ubuntu 18.04 server as a sudo non-root user, add the relevant GPG key. sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E298A3A825C0D65DFD57CBB651716619E084DAB9 Step 2 — Add the R Repository sudo add-apt-repository 'deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/' If you’re not using 18.04, find the relevant repository from the R Project Ubuntu list, named for each release. Step 3 — Update Package Lists sudo apt update Step 4 — Install R sudo apt install r-base If prompted to confirm installation, press y to continue. Step 5 — Test Install Start R’s interactive shell as root. sudo -i R...
Read more

mongodb

Step 2 — Checking the Service and Database sudo systemctl status mongodb mongo --eval 'db.runCommand({ connectionStatus: 1 })' Step 3 — Managing the MongoDB Service sudo systemctl status mongodb You can stop the server anytime by typing: sudo systemctl stop mongodb To start the server when it is stopped, type: sudo systemctl start mongodb You can also restart the server with a single command: sudo systemctl restart mongodb By default, MongoDB is configured to start automatically with the server. If you wish to disable the automatic startup, type: sudo systemctl disable mongodb It’s just as easy to enable it again. To do this, use: sudo systemctl enable mongodb
Read more