Install Apache Spark on Ubuntu (step by step guide)

Video is ready, Click Here to View ×

This is a step by step video covering how you can install Spark standalone mode on Ubuntu. Covers installation of Java, Scala, Git and finally building Spark using sbt tool.

Install Java
sudo apt-add-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java7-installer

Install Scala
sudo mkdir /usr/local/src/scala
sudo tar -xvf scala-2.11.7.tgz -C /usr/local/src/scala/
nano .bashrc


  1. Hi Bro, Can u please share me the location of theses parameters. I got only one.

    Path= spark-2.2.0-bin-hadoop2.7/conf/spark-defaults.conf


  2. Hi, I followed the video step by step but at the end I couldnt built spark. It gives me this : sbt/sbt assembly no such file or directory….So I read the comments below and I tried every commande : ./build/sbt package , ./build/sbt package …. but it gives me ERROR : [error] (compile:compileIncremental) java.lang.NullPointerException
    Please help. I'm on it from yesterday morning

  3. sir , I follow the same as you said in this tutorial but when i run this command ( sbt/sbt assembly ) , I got the result : no such file or directory …. I stuck to this point becoz after that when i am running any command as you suggested in you tutorial i am getting error.
    scala> var f = sc.textFile("README")
    <console>:17: error: not found: value sc
    var f = sc.textFile("README")
    scala> f.collect()
    <console>:18: error: not found: value f

  4. i have install and follow ur all steps but it give me error on the last command that "f.collect() "
    org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: file:/root/spark-2.1.0/bin/

  5. Hello, I am very thanked for your video. I have followed all the steps but when I enter the command to check if Scala is correctly installed I have the following output:

    diego@DESKTOP-M93VP0O:~$ scala -version
    cat: /release: No such file or directory
    Exception in thread "main" java.lang.UnsupportedClassVersionError: scala/tools/nsc/MainGenericRunner : Unsupported major.minor version 52.0
    at java.lang.ClassLoader.findBootstrapClass(Native Method)
    at java.lang.ClassLoader.findBootstrapClassOrNull(
    at java.lang.ClassLoader.loadClass(
    at java.lang.ClassLoader.loadClass(
    at sun.misc.Launcher$AppClassLoader.loadClass(
    at java.lang.ClassLoader.loadClass(
    at sun.launcher.LauncherHelper.checkAndLoadMain(

    I have checked that JAva is correctly installed and that I have correctly written the code to set the path in the .bashrc.
    I don't know what to do.

  6. Thanks for the installation steps!

    I'm getting the below error when doing build for spark.

    tried increasing the heap size using below:

    export JAVA_OPTS="-Xms512m -Xmx1024m -XX:PermSize=64m -XX:MaxPermSize=128m -Xss512k"

    But not able to resolve yet.

    Any help will be much appreciated.

    Below is the error:

    NOTE: The sbt/sbt script has been relocated to build/sbt.
    Please update references to point to the new location.

    Invoking 'build/sbt assembly' now …

    Error occurred during initialization of VM
    Could not reserve enough space for object heap
    Error: Could not create the Java Virtual Machine.
    Error: A fatal exception has occurred. Program will exit.

  7. Hello Sir,

    Thanks for the video and its awesome.

    While trying to build the SPARK as described here, I'm getting the following error message upon running the below command… Can u help me here ??

    hp@hp-Not-Specified:~/spark-1.6.1$ sbt/sbt assembly

    NOTE: The sbt/sbt script has been relocated to build/sbt.
    Please update references to point to the new location.

    Invoking 'build/sbt assembly' now …

    Using /usr/lib/jvm/java-7-oracle/ as default JAVA_HOME.
    Note, this will be overridden by -java-home if it is set.
    Picked up _JAVA_OPTIONS: -Xmx256M -Xms128M
    Getting org.scala-sbt sbt 0.13.7 …
    [SUCCESSFUL ] org.scala-sbt#precompiled-2_8_2;0.13.7!compiler-interface-bin.jar (81203ms)
    [SUCCESSFUL ] org.scala-sbt#interface;0.13.7!interface.jar (85368ms)

    :: problems summary ::
    :::: WARNINGS
    module not found: org.scala-sbt#main;0.13.7

    ==== local: tried


    — artifact org.scala-sbt#main;0.13.7!main.jar:


    ==== typesafe-ivy-releases: tried

    ==== Maven Central: tried

    — artifact org.scala-sbt#main;0.13.7!main.jar:




    :: org.scala-sbt#main;0.13.7: not found


    :::: ERRORS
    Server access Error: Connection timed out url=

    unresolved dependency: org.scala-sbt#main;0.13.7: not found
    Error during sbt execution: Error retrieving required libraries
    (see /home/hp/.sbt/boot/update.log for complete log)
    Error: Could not retrieve sbt 0.13.7

  8. Getting error in ide installation
     "More than one scala library found in the build path

    is not an optimal configuration, try to limit to one Scala library in
    the build path. examples Unknown Scala Classpath Problem "

  9. I have error "Error during sbt execution: java.lang.AssertionError: Platform not recongnized" after running the command "sbt/sbt assembly"

    I do not know how to fix it. please help me.
    P.S. I'm not good english. I apologize If anything goes wrong.

Leave a Reply

Your email address will not be published.