Apache spark job failing execution with: ArrayIndexOutOfBoundsException - apache-spark

Question

Below is the code
val path = "C:\\Users\\John\\Downloads\\crimes.csv"
val crimeFile = sc.textFile(path)
val crimerows = crimeFile.map(l=>l.split(",").map(e=>e.trim))
//taking the first row as header
val header = crimerows.first
//filter out the header
val crimes = crimerows.filter(_(0)!=header(0))
//mapping the field to be reduced
val crimetype = crimes.map(l=>(l(5),1))
val stats = crimetype.reduceByKey(_+_)
stats.count
Below is the error I get.
I am using Spark 1.2.0, Using Scala version 2.10.4 (Java HotSpot(TM) Client VM, Java 1.8.0_45). The file is about 1gb in size and the JVM is set at default heap of 256MB.
Appreciate any help, below is the error:
15/04/28 21:17:47 ERROR Executor: Exception in task 32.0 in stage 11.0 (TID 169)
java.lang.ArrayIndexOutOfBoundsException: 5
at $line13.$read$$iwC$$iwC$$iwC$$iwC$$anonfun$1.apply(<console>:22)
at $line13.$read$$iwC$$iwC$$iwC$$iwC$$anonfun$1.apply(<console>:22)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at org.apache.spark.util.Utils$.getIteratorSize(Utils.scala:1311)
at org.apache.spark.rdd.RDD$$anonfun$count$1.apply(RDD.scala:910)
at org.apache.spark.rdd.RDD$$anonfun$count$1.apply(RDD.scala:910)
at org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:1314)
at org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:1314)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61)
at org.apache.spark.scheduler.Task.run(Task.scala:56)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
15/04/28 21:17:47 WARN TaskSetManager: Lost task 32.0 in stage 11.0 (TID 169, localhost): java.lang.ArrayIndexOutOfBoun
sException: 5
at $line13.$read$$iwC$$iwC$$iwC$$iwC$$anonfun$1.apply(<console>:22)
at $line13.$read$$iwC$$iwC$$iwC$$iwC$$anonfun$1.apply(<console>:22)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at org.apache.spark.util.Utils$.getIteratorSize(Utils.scala:1311)
at org.apache.spark.rdd.RDD$$anonfun$count$1.apply(RDD.scala:910)
at org.apache.spark.rdd.RDD$$anonfun$count$1.apply(RDD.scala:910)
at org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:1314)
at org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:1314)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61)
at org.apache.spark.scheduler.Task.run(Task.scala:56)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
15/04/28 21:17:47 ERROR TaskSetManager: Task 32 in stage 11.0 failed 1 times; aborting job
15/04/28 21:17:47 INFO TaskSchedulerImpl: Removed TaskSet 11.0, whose tasks have all completed, from pool
15/04/28 21:17:47 INFO TaskSchedulerImpl: Cancelling stage 11
15/04/28 21:17:47 INFO DAGScheduler: Job 9 failed: count at <console>:25, took 17.366680 s
org.apache.spark.SparkException: Job aborted due to stage failure: Task 32 in stage 11.0 failed 1 times, most recent fa
lure: Lost task 32.0 in stage 11.0 (TID 169, localhost): java.lang.ArrayIndexOutOfBoundsException: 5
at $iwC$$iwC$$iwC$$iwC$$anonfun$1.apply(<console>:22)
at $iwC$$iwC$$iwC$$iwC$$anonfun$1.apply(<console>:22)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at org.apache.spark.util.Utils$.getIteratorSize(Utils.scala:1311)
at org.apache.spark.rdd.RDD$$anonfun$count$1.apply(RDD.scala:910)
at org.apache.spark.rdd.RDD$$anonfun$count$1.apply(RDD.scala:910)
at org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:1314)
at org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:1314)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61)
at org.apache.spark.scheduler.Task.run(Task.scala:56)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Driver stacktrace:
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages
DAGScheduler.scala:1214)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1203)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1202)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1202)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:696)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:696)
at scala.Option.foreach(Option.scala:236)
at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:696)
at org.apache.spark.scheduler.DAGSchedulerEventProcessActor$$anonfun$receive$2.applyOrElse(DAGScheduler.scala:1
20)
at akka.actor.Actor$class.aroundReceive(Actor.scala:465)
at org.apache.spark.scheduler.DAGSchedulerEventProcessActor.aroundReceive(DAGScheduler.scala:1375)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
at akka.actor.ActorCell.invoke(ActorCell.scala:487)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)
at akka.dispatch.Mailbox.run(Mailbox.scala:220)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

Answer 1

Here val crimetype = crimes.map(l=>(l(5),1)) you are expecting the crime array to have at least 6 elements. Some entry in the file does not comply to that condition and you get
java.lang.ArrayIndexOutOfBoundsException: 5
To deal with potentially non-compliant data, one should do more defensive coding, in this case, if ignoring a missing value is allowed (and not an error), we could:
val crimetype = crimes.flatMap(l=> l.lift()(5).map(value => (value,1)))
As an alternative: filter the RDD for the correct values:
val crimetype = crimes.filter(l => l.length > 5).map(l=>(l(5),1))
And not to say that there's only one way to do simple things:
val crimetype = crimes.collect{ case l if (l.length > 5) => (l(5),1)}

Answer 2

Java's String.split is designed to cause bugs. It discards trailing empty strings: "aa,a,".split(",") is ["aa", "a"]. To get the expected ["aa", "a", ""] you need to use "aa,a,".split(",", -1).

Answer 3

It was a data error as suggested #maasg, so the below code worked:
val crimetype = crimes.filter(l => l.length > 5).map(l=>(l(5),1))