Trying to attempt genomic analysis using ADAM and Zeppelin. I'm not sure if I'm doing this right but running into below issue.
%dep
z.reset()
z.addRepo("Spark Packages Repo").url("http://dl.bintray.com/spark-packages/maven")
z.load("com.databricks:spark-csv_2.10:1.2.0")
z.load("mysql:mysql-connector-java:5.1.35")
z.load("org.bdgenomics.adam:adam-core_2.10:0.20.0")
z.load("org.bdgenomics.adam:adam-cli_2.10:0.20.0")
z.load("org.bdgenomics.adam:adam-apis_2.10:0.20.0")
%spark
import org.bdgenomics.adam.rdd.ADAMContext._
import org.bdgenomics.adam.rdd.ADAMContext
import org.bdgenomics.adam.projections.{ AlignmentRecordField, Projection }
import org.apache.spark.SparkContext
import org.apache.spark.SparkConf
import org.bdgenomics.adam.rdd.ADAMContext
import org.bdgenomics.adam.rdd.ADAMContext._
import org.bdgenomics.adam.projections.Projection
import org.bdgenomics.adam.projections.AlignmentRecordField
import scala.io.Source
import org.apache.spark.rdd.RDD
import org.bdgenomics.formats.avro.Genotype
import scala.collection.JavaConverters._
import org.bdgenomics.formats.avro._
import org.apache.spark.SparkContext._
import org.apache.spark.mllib.linalg.{ Vector => MLVector, Vectors }
import org.apache.spark.mllib.clustering.{ KMeans, KMeansModel }
val ac = new ADAMContext(sc)
and I get the following output with a error
import org.bdgenomics.adam.rdd.ADAMContext._
import org.bdgenomics.adam.rdd.ADAMContext
import org.bdgenomics.adam.projections.{AlignmentRecordField, Projection}
import org.apache.spark.SparkContext
import org.apache.spark.SparkConf
import org.bdgenomics.adam.rdd.ADAMContext
import org.bdgenomics.adam.rdd.ADAMContext._
import org.bdgenomics.adam.projections.Projection
import org.bdgenomics.adam.projections.AlignmentRecordField
import scala.io.Source
import org.apache.spark.rdd.RDD
import org.bdgenomics.formats.avro.Genotype
import scala.collection.JavaConverters._
import org.bdgenomics.formats.avro._
import org.apache.spark.SparkContext._
import org.apache.spark.mllib.linalg.{Vector=>MLVector, Vectors}
import org.apache.spark.mllib.clustering.{KMeans, KMeansModel}
res7: org.apache.spark.SparkContext = org.apache.spark.SparkContext#62ec8142
<console>:188: error: constructor ADAMContext in class ADAMContext cannot be accessed in class $iwC
new ADAMContext(sc)
Any idea where to look? Am I missing any dependencies?
^
According file ADAMContext.scala in the version you use. The constructor is private.
class ADAMContext private (#transient val sc: SparkContext)
extends Serializable with Logging {
...
}
You can instead use like this.
import org.bdgenomics.adam.rdd.ADAMContext._
val adamContext: ADAMContext = z.sc
It will use the implicit conversion in object ADAMContext
object ADAMContext {
implicit def sparkContextToADAMContext(sc: SparkContext): ADAMContext =
new ADAMContext(sc)
}
It did work without using Z reference!!
val ac:ADAMContext = sc
val genotypes: RDD[Genotype] = ac.loadGenotypes("/tmp/ADAM2").rdd
Output
ac: org.bdgenomics.adam.rdd.ADAMContext = org.bdgenomics.adam.rdd.ADAMContext#2c60ef7e
genotypes:
org.apache.spark.rdd.RDD[org.bdgenomics.formats.avro.Genotype] = MapPartitionsRDD[3] at map at ADAMContext.scala:207
I had tried doing this at the adam-shell prompt and I don't recall having to use implicit conversion. It was using the 0.19 version of ADAM though.
Related
import android.app.Activity
import android.content.ClipData.newIntent
import android.content.Intent
import android.os.Bundle
import android.util.Log
import android.view.View
import android.widget.Button
import android.widget.TextView
import android.widget.Toast
import androidx.appcompat.app.AppCompatActivity
import androidx.lifecycle.ViewModelProvider
cheatButton.setOnClickListener {
// Start CheatActivity
val answerIsTrue = quizViewModel.currentQuestionAnswer
val intent = CheatActivity.newIntent(this#MainActivity, answerIsTrue)
startActivityForResult(intent, REQUEST_CODE_CHEAT)
}
I tried hovering over the red to import but android studios is not recognizing the import. Also, if you are able to assist with the statement that is crossed out.
I want to save data frame into cassandra table using sparkJava API
I want to add the part of saving in the following code
I want to save people dataframe into cassandra table and make queries on that cassandra table
import org.apache.spark.api.java.*;
import org.apache.spark.SparkConf;
import org.apache.spark.api.java.function.Function;
import org.apache.spark.sql.types.DataTypes;
import org.apache.spark.sql.types.StructType;
import org.apache.spark.sql.types.StructField;
import org.apache.spark.sql.Row;
import org.apache.spark.sql.RowFactory;
import org.apache.spark.sql.SQLContext;
import com.datastax.spark.connector.cql.CassandraConnector;
import com.datastax.spark.connector.japi.CassandraRow;
import org.apache.spark.sql.SQLContext;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spark.sql.DataFrame;
public class SimpleApp {
public static void main(String[] args) {
SparkConf conf = new SparkConf().setAppName("Simple Application");
conf.setMaster("local");
conf.set("spark.cassandra.connection.host", "localhost");
JavaSparkContext sc = new JavaSparkContext(conf);
SQLContext sqlContext = new org.apache.spark.sql.SQLContext(sc);
DataFrame people = sqlContext.read().json("/root/people.json");
people.printSchema();
people.registerTempTable("people");
**//I want to save this TempTable or people dataframe into cassandra table and make teenagers SQL query on that cassandra table**
DataFrame teenagers = sqlContext.sql("SELECT name FROM people WHERE age >= 13 AND age <= 19");
teenagers.show();
}
}
Actually I am using the caldroid library for android but in it I could not find any proper documentation for setBackgroundResourceForDate function. The files I have imported are:-
import android.annotation.SuppressLint;
import android.os.Bundle;
import android.support.v4.app.FragmentTransaction;
import android.support.v7.app.AppCompatActivity;
import android.view.View;
import android.view.View.OnClickListener;
import android.widget.Button;
import android.widget.TextView;
import android.widget.Toast;
import com.roomorama.caldroid.CaldroidFragment;
import com.roomorama.caldroid.CaldroidListener;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Calendar;
import java.util.Date;
and here the setBackgroundResourceForDate is being showed in red:-
if (caldroidFragment != null) {
caldroidFragment.setBackgroundResourceForDate(R.color.blue,
blueDate);
caldroidFragment.setBackgroundResourceForDate(R.color.green,
greenDate);
caldroidFragment.setTextColorForDate(R.color.white, blueDate);
caldroidFragment.setTextColorForDate(R.color.white, greenDate);
}
As far as i can see there is no method defined with the name 'setBackgroundResourceForDate'in class :import com.roomorama.caldroid.CaldroidFragment;
instead you could use setBackgroundDrawableForDate(Drawable drawable, Date date)
You can implement it like this:
if (caldroidFragment != null) {
caldroidFragment.setBackgroundDrawableForDate(R.color.blue,
blueDate);
caldroidFragment.setBackgroundDrawableForDate(R.color.green,
greenDate);
caldroidFragment.setTextColorForDate(R.color.white, blueDate);
caldroidFragment.setTextColorForDate(R.color.white, greenDate);
}
I executed this command to compile my program
java -Xms16m -Xmx64m -cp ".:boilerpipe-1.2.0.jar:lib/nekohtml-1.9.13.jar:lib/xerces-2.9.1.jar:lib/langdetect.jar:lib/jsonic-1.2.8.jar" ExampleProgram.java
It reports this error:
Error: Could not find or load main class ExampleProgram.java
Here is ExampleProgram.java:
import java.io.InputStream;
import java.io.FileReader;
import java.io.FileInputStream;
import java.io.InputStreamReader;
import java.io.IOException;
import org.xml.sax.InputSource;
import de.l3s.boilerpipe.document.TextDocument;
import de.l3s.boilerpipe.extractors.ArticleExtractor;
import de.l3s.boilerpipe.sax.BoilerpipeSAXInput;
// Language detect librarys
import com.cybozu.labs.langdetect.*;
import net.arnx.jsonic.JSON;
import net.arnx.jsonic.JSONException;
import java.io.*;
import java.net.*;
import java.util.concurrent.Executors;
import com.sun.net.httpserver.Headers;
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;
import com.sun.net.httpserver.HttpServer;
public class ExampleProgram {
public static void main(String[] args) throws Exception {
EveryDetector evr = new EveryDetector();
InetSocketAddress addr = new InetSocketAddress("127.0.0.1",8080);
HttpServer server = HttpServer.create(addr, 0);
MyHandler hndl = new MyHandler();
hndl.setDetector(evr);
MyHandlerExtractContent hnd2 = new MyHandlerExtractContent();
hnd2.setDetector(evr);
MyHandlerDetectLanguage hnd3 = new MyHandlerDetectLanguage();
hnd3.setDetector(evr);
server.createContext("/",hndl);
server.createContext("/extractcontent",hnd2);
server.createContext("/detectlanguage",hnd3);
server.setExecutor(Executors.newCachedThreadPool());
server.start();
System.out.println("Server is listening on port 8080" );
}
}
Source: https://github.com/remdex/boilerpipe-and-language-detect-api-server
How can I solve my problem?
For me, in Windows environment this worked (My class location : C:/MyFolder/MyClass.java) :
cd C:/MyFolder/MyClass.java
Compiling:
C:/MyFolder/MyClass.java> javac MyClass.java
Executing:
C:/MyFolder/MyClass.java> java -classpath C:/external.jar;. MyClass
This is a simple program that's supposed to change the background color of a pane from a list of colors. However,
import java.awt.Color;
import java.awt.FlowLayout;
import javax.swing.JFrame;
import javax.swing.JTextField;
import java.awt.Font;
import javax.swing.ButtonGroup;
import javax.swing.JFrame;
import javax.swing.JButton;
import javax.swing.Icon;
import javax.swing.ImageIcon;
import javax.swing.JList;
import javax.swing.JOptionPane;
import javax.swing.JCheckBox;
import javax.swing.JScrollPane;
import javax.swing.ListSelectionModel;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.awt.event.ItemEvent;
import java.awt.event.ItemListener;
import javax.swing.JRadioButton;
import javax.swing.event.ListSelectionListener;
public class JL1st extends JFrame{
private JList list;
private static String[] colornames = {"black","blue","red","white"};
private static Color[] colors = {Color.BLACK, Color.BLUE, Color.RED, Color.WHITE};
public JL1st(){
super("title");
setLayout(new FlowLayout());
list = new JList(colornames);
list.setVisibleRowCount(4);
list.setSelectionMode(ListSelectionModel.SINGLE_SELECTION);
add(new JScrollPane(list));
list.ListSelectionListener(
new ListSelectionListener(){
public void valueChanged(ListSelectionEvent event){
getContentPane().setBackground(colors[list.getSeletedIndex]);
}
}
);
}
}
I keep getting an error on "ListSelectionListener" The method ListSelectionListener(new ListSelectionListener(){}) is undefined for the type JList
Could it be that I have a missing import or is it just messed up syntax?
Thanks
Problem solved,
ListSelectionListener wasn't imported.