error initialising SparkContext, running Spark on bash on ubuntu on windows - apache-spark

Question

I am attempting to install and configure spark 2.0.1 on bash on ubuntu on Windows. I followed the instructions at Apache Spark - Installation and everything seemed to get installed OK however when I run spark-shell this happens:
16/11/06 11:25:47 ERROR SparkContext: Error initializing SparkContext.
java.net.SocketException: Invalid argument
at sun.nio.ch.Net.listen(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:224)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:125)
at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:485)
at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1089)
at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:430)
at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:415)
at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:903)
at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:198)
at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:348)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at java.lang.Thread.run(Thread.java:745)
Immediately prior to that error I see a warning which may or may not be related:
16/11/06 11:25:47 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
16/11/06 11:25:47 WARN Utils: Your hostname, DESKTOP-IKGIG97 resolves to a loopback address: 127.0.0.1; using 151.127.0.0 instead (on interface wifi0)
16/11/06 11:25:47 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
I'm a bit of a noob with linux I must admit so am a bit clueless as to what to do next. In case it matters here is the contents of /etc/hosts
127.0.0.1 localhost
127.0.0.1 DESKTOP-IKGIG97
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
Hoping someone here can identify my issue. What do I need to do to investigate and fix this error?

Answer 1

As the error suggests, add the SPARK_LOCAL_IP in the conf/spark-env.sh script in the directory where Spark is installed
http://spark.apache.org/docs/2.0.1/configuration.html
Environment Variables
Certain Spark settings can be configured through environment variables, which are read from the conf/spark-env.sh script in the directory where Spark is installed (or conf/spark-env.cmd on Windows). In Standalone and Mesos modes, this file can give machine specific information such as hostnames. It is also sourced when running local Spark applications or submission scripts.
Note that conf/spark-env.sh does not exist by default when Spark is installed. However, you can copy conf/spark-env.sh.template to create it. Make sure you make the copy executable.
The following variables can be set in spark-env.sh:
SPARK_LOCAL_IP IP address of the machine to bind to.
If it doesn't solve your problem - please share the output executing the following unix command:
ifconfig