I would like to ask how to fetch all doc from solr collection using solrJ.
I have written one code but getting error
Exception in thread "main" org.apache.solr.client.solrj.SolrServerException: No collection param specified on request and no default collection has been set.
String zkHostString = "linux152:2181,linuxUL:2181,linux170:2181/solr";
CloudSolrClient server = new CloudSolrClient(zkHostString);
SolrQuery parameters = new SolrQuery();
public void cursorMark() throws IOException, SolrServerException {
SolrQuery parameters = new SolrQuery();
QueryResponse response = new QueryResponse();
response = server.query(parameters);
parameters.set("q",":");
parameters.set("qt","/select");
parameters.setParam("wt","json");
parameters.set("collection", "RetailDev_Protocol");
int fetchSize = 2;
parameters.setRows(fetchSize);
String cursorMark = CursorMarkParams.CURSOR_MARK_START;
boolean done = false;
while (! done) {
parameters.set(CursorMarkParams.CURSOR_MARK_PARAM, cursorMark);
long offset = 0;
long totalResults = response.getResults().getNumFound();
while (offset < totalResults)
{
parameters.setStart((int) offset);
try {
for (SolrDocument doc : server.query(parameters).getResults())
{
log.info((String) doc.getFieldValue("title"));
}
} catch (SolrServerException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
offset += fetchSize;
}
String nextCursorMark = (response).getNextCursorMark();
}
SolrDocumentList list = response.getResults();
System.out.println(list.toString());
}
You need to set your collection in the following way:
server.setDefaultCollection("<MY_COLLECTION");
otherwise you get the error that you specified in your question.
Related
I want to remove session in case i get certain data from file i have code like this,but i got errors "flowfile has already marked for removal", what should i change to get rid of extra errors?
In case of session rollback flowfile will dissapear in queues also?
2.should i use rollback instead of remove()?
NodeList childNodes = nodeGettingChanged.getChildNodes();
for (int i = 0; i != childNodes.getLength(); ++i) {
Node child = childNodes.item(i);
if (!(child instanceof Element))
continue;
if (child.getNodeName().equals("runAs")) {
if(child.getFirstChild().getTextContent()=="false"){
session.remove(flowFile1);
File deleteExtraFile =new File("C://Users//s.tkhilaishvili//Desktop//try2//nifi-1.3.0//1//conf.xml");
boolean delete=deleteExtraFile.delete();
}
else {
child.getFirstChild().setNodeValue("false");
}
}
}
Document finalXmlDocument = xmlDocument;
session.write(flowFile1, new StreamCallback() {
public void process(InputStream inputStream, OutputStream outputStream) throws IOException {
TransformerFactory transformerFactory = TransformerFactory.newInstance();
Transformer transformer = null;
try {
transformer = transformerFactory.newTransformer();
} catch (TransformerConfigurationException e) {
e.printStackTrace();
}
DOMSource source = new DOMSource(finalXmlDocument);
ffStream.close();
ByteArrayOutputStream bos = new ByteArrayOutputStream();
StreamResult result = new StreamResult(bos);
try {
transformer.transform(source, result);
} catch (TransformerException e) {
e.printStackTrace();
}
byte[] array = bos.toByteArray();
outputStream.write(array);
}
});
session.remove(flowFile);
session.transfer(flowFile1, REL_SUCCESS);
}
If you are executing session.remove(flowFile1) then later trying to transfer it to REL_SUCCESS, you will get that error. It looks like you already have an if-clause checking the firstChild for "false", perhaps you could put the transfer in an else-clause, such that it will only be transferred if it wasn't removed.
Note: This question may look like a repetition of several question posted on the forum, but I am really stuck on this problem from quite some time and I am not able to solve this issue using the solutions posted for similar questions. I have posted my code here and need help to proceed further
So, here is my issue:
I am writing a Java GUI application which loads a file before performing any processing. There is a waiting time on an average of about 10-15 seconds during which the file is parsed. After this waiting time, what I get see on the GUI is,
The parsed file in the form of individual leaves in the JTree in a Jpanel
Some header information (example: data range) in two individual JTextField
A heat map generated after parsing the data in a different JPanel on the GUI.
The program connects to R to parse the file and read the header information.
Now, I want to use swing worker to put the file reading process on a different thread so that it does not block the EDT. I am not sure how I can build my SwingWorker class so that the process is done in the background and the results for the 3 components are displayed when the process is complete. And, during this file reading process I want to display a JProgressBar.
Here is the code which does the whole process, starting from selection of the file selection menu item. This is in the main GUI method.
JScrollPane spectralFilesScrollPane;
if ((e.getSource() == OpenImagingFileButton) || (e.getSource() == loadRawSpectraMenuItem)) {
int returnVal = fcImg.showOpenDialog(GUIMain.this);
// File chooser
if (returnVal == JFileChooser.APPROVE_OPTION) {
file = fcImg.getSelectedFile();
//JTree and treenode creation
DefaultMutableTreeNode root = new DefaultMutableTreeNode(file);
rawSpectraTree = new JTree(root);
DefaultTreeModel model = (DefaultTreeModel) rawSpectraTree.getModel();
try {
// R connection
rc = new RConnection();
final String inputFileDirectory = file.getParent();
System.out.println("Current path: " + currentPath);
rc.assign("importImagingFile", currentPath.concat("/importImagingFile.R"));
rc.eval("source(importImagingFile)");
rc.assign("currentWorkingDirectory", currentPath);
rc.assign("inputFileDirectory", inputFileDirectory);
rawSpectrumObjects = rc.eval("importImagingFile(inputFileDirectory,currentWorkingDirectory)");
rc.assign("plotAverageSpectra", currentPath.concat("/plotAverageSpectra.R"));
rc.eval("source(plotAverageSpectra)");
rc.assign("rawSpectrumObjects", rawSpectrumObjects);
REXP averageSpectraObject = rc.eval("plotAverageSpectra(rawSpectrumObjects)");
rc.assign("AverageMassSpecObjectToSpectra", currentPath.concat("/AverageMassSpecObjectToSpectra.R"));
rc.eval("source(AverageMassSpecObjectToSpectra)");
rc.assign("averageSpectraObject", averageSpectraObject);
REXP averageSpectra = rc.eval("AverageMassSpecObjectToSpectra(averageSpectraObject)");
averageSpectraMatrix = averageSpectra.asDoubleMatrix();
String[] spectrumName = new String[rawSpectrumObjects.asList().size()];
for (int i = 0; i < rawSpectrumObjects.asList().size(); i++) {
DefaultMutableTreeNode node = new DefaultMutableTreeNode("Spectrum_" + (i + 1));
model.insertNodeInto(node, root, i);
}
// Expand all the nodes of the JTree
for(int i=0;i< model.getChildCount(root);++i){
rawSpectraTree.expandRow(i);
}
DefaultMutableTreeNode firstLeaf = ((DefaultMutableTreeNode)rawSpectraTree.getModel().getRoot()).getFirstLeaf();
rawSpectraTree.setSelectionPath(new TreePath(firstLeaf.getPath()));
updateSpectralTableandChartRAW(firstLeaf);
// List the min and the max m/z of in the respective data fields
rc.assign("dataMassRange", currentPath.concat("/dataMassRange.R"));
rc.eval("source(dataMassRange)");
rc.assign("rawSpectrumObjects", rawSpectrumObjects);
REXP massRange = rc.eval("dataMassRange(rawSpectrumObjects)");
double[] massRangeValues = massRange.asDoubles();
minMzValue = (float)massRangeValues[0];
maxMzValue = (float)massRangeValues[1];
GlobalMinMz = minMzValue;
GlobalMaxMz = maxMzValue;
// Adds the range values to the jTextField
minMz.setText(Float.toString(minMzValue));
minMz.validate();
minMz.repaint();
maxMz.setText(Float.toString(maxMzValue));
maxMz.validate();
maxMz.repaint();
// Update status bar with the uploaded data details
statusLabel.setText("File name: " + file.getName() + " | " + "Total spectra: " + rawSpectrumObjects.asList().size() + " | " + "Mass range: " + GlobalMinMz + "-" + GlobalMaxMz);
// Generates a heatmap
rawIntensityMap = gim.generateIntensityMap(rawSpectrumObjects, currentPath, minMzValue, maxMzValue, Gradient.GRADIENT_Rainbow, "RAW");
rawIntensityMap.addMouseListener(this);
rawIntensityMap.addMouseMotionListener(this);
imagePanel.add(rawIntensityMap, BorderLayout.CENTER);
coordinates = new JLabel();
coordinates.setBounds(31, 31, rawIntensityMap.getWidth() - 31, rawIntensityMap.getHeight() - 31);
panelRefresh(imagePanel);
tabbedSpectralFiles.setEnabledAt(1, false);
rawSpectraTree.addTreeSelectionListener(new TreeSelectionListener() {
#Override
public void valueChanged(TreeSelectionEvent e) {
try {
DefaultMutableTreeNode selectedNode =
(DefaultMutableTreeNode) rawSpectraTree.getLastSelectedPathComponent();
int rowCount = listTableModel.getRowCount();
for (int l = 0; l < rowCount; l++) {
listTableModel.removeRow(0);
}
updateSpectralTableandChartRAW(selectedNode);
} catch (RserveException e2) {
e2.printStackTrace();
} catch (REXPMismatchException e1) {
e1.printStackTrace();
}
}
});
spectralFilesScrollPane = new JScrollPane();
spectralFilesScrollPane.setViewportView(rawSpectraTree);
spectralFilesScrollPane.setPreferredSize(rawFilesPanel.getSize());
rawFilesPanel.add(spectralFilesScrollPane);
tabbedSpectralFiles.validate();
tabbedSpectralFiles.repaint();
rawImage.setEnabled(true);
peakPickedImage.setEnabled(false);
loadPeakListMenuItem.setEnabled(true); //active now
loadPeaklistsButton.setEnabled(true); //active now
propertiesMenuItem.setEnabled(true); // active now
propertiesButton.setEnabled(true); //active now
} catch (RserveException e1) {
JOptionPane.showMessageDialog(this,
"There was an error in the R connection. Please try again!", "Error",
JOptionPane.ERROR_MESSAGE);
} catch (REXPMismatchException e1) {
JOptionPane.showMessageDialog(this,
"Operation requested is not supported by the given R object type. Please try again!", "Error",
JOptionPane.ERROR_MESSAGE);
}
// hideProgress();
}
}
I tried creating a SwingWorker class, but I am totally confused how I can get all the three outputs on the GUI, plus have a progress bar. It is not complete, but I don't know how to proceed further.
public class FileReadWorker extends SwingWorker<REXP, String>{
private static void failIfInterrupted() throws InterruptedException {
if (Thread.currentThread().isInterrupted()) {
throw new InterruptedException("Interrupted while loading imaging file!");
}
}
// The file that is being read
private final File fileName;
private JTree rawSpectraTree;
private RConnection rc;
private REXP rawSpectrumObjects;
private double[][] averageSpectraMatrix;
private Path currentRelativePath = Paths.get("");
private final String currentPath = currentRelativePath.toAbsolutePath().toString();
final JProgressBar progressBar = new JProgressBar();
// public FileReadWorker(File fileName)
// {
// this.fileName = fileName;
// System.out.println("I am here");
// }
public FileReadWorker(final JProgressBar progressBar, File fileName) {
this.fileName = fileName;
addPropertyChangeListener(new PropertyChangeListener() {
public void propertyChange(PropertyChangeEvent evt) {
if ("progress".equals(evt.getPropertyName())) {
progressBar.setValue((Integer) evt.getNewValue());
}
}
});
progressBar.setVisible(true);
progressBar.setStringPainted(true);
progressBar.setValue(0);
setProgress(0);
}
#Override
protected REXP doInBackground() throws Exception {
System.out.println("I am here... in background");
DefaultMutableTreeNode root = new DefaultMutableTreeNode(fileName);
rawSpectraTree = new JTree(root);
DefaultTreeModel model = (DefaultTreeModel) rawSpectraTree.getModel();
rc = new RConnection();
final String inputFileDirectory = fileName.getParent();
rc.assign("importImagingFile", currentPath.concat("/importImagingFile.R"));
rc.eval("source(importImagingFile)");
rc.assign("currentWorkingDirectory", currentPath);
rc.assign("inputFileDirectory", inputFileDirectory);
rawSpectrumObjects = rc.eval("importImagingFile(inputFileDirectory,currentWorkingDirectory)");
rc.assign("plotAverageSpectra", currentPath.concat("/plotAverageSpectra.R"));
rc.eval("source(plotAverageSpectra)");
rc.assign("rawSpectrumObjects", rawSpectrumObjects);
REXP averageSpectraObject = rc.eval("plotAverageSpectra(rawSpectrumObjects)");
rc.assign("AverageMassSpecObjectToSpectra", currentPath.concat("/AverageMassSpecObjectToSpectra.R"));
rc.eval("source(AverageMassSpecObjectToSpectra)");
rc.assign("averageSpectraObject", averageSpectraObject);
REXP averageSpectra = rc.eval("AverageMassSpecObjectToSpectra(averageSpectraObject)");
averageSpectraMatrix = averageSpectra.asDoubleMatrix();
for (int i = 0; i < rawSpectrumObjects.asList().size(); i++) {
DefaultMutableTreeNode node = new DefaultMutableTreeNode("Spectrum_" + (i + 1));
model.insertNodeInto(node, root, i);
}
// Expand all the nodes of the JTree
for(int i=0;i< model.getChildCount(root);++i){
rawSpectraTree.expandRow(i);
}
return averageSpectra;
}
#Override
public void done() {
setProgress(100);
progressBar.setValue(100);
progressBar.setStringPainted(false);
progressBar.setVisible(false);
}
}
Any help would be very much appreciated.
I am executing large query,so my app throwing time out error. Some of the thread suggested to added command time out but after adding those lines it take longer to get response back, any idea why or what am i missing in my code?
public int CreateRecord(string theCommand, DataSet theInputData)
{
int functionReturnValue = 0;
int retVal = 0;
SqlParameter objSqlParameter = default(SqlParameter);
DataSet dsParameter = new DataSet();
int i = 0;
try
{
//Set the command text (stored procedure name or SQL statement).
mobj_SqlCommand.CommandTimeout = 120;
mobj_SqlCommand.CommandText = theCommand;
mobj_SqlCommand.CommandType = CommandType.StoredProcedure;
for (i = 0; i <= (theInputData.Tables.Count - 1); i++)
{
if (theInputData.Tables[i].Rows.Count > 0)
{
dsParameter.Tables.Add(theInputData.Tables[i].Copy());
}
}
objSqlParameter = new SqlParameter("#theXmlData", SqlDbType.Text);
objSqlParameter.Direction = ParameterDirection.Input;
objSqlParameter.Value = "<?xml version=\"1.0\" encoding=\"iso-8859-1\"?>" + dsParameter.GetXml();
//Attach to the parameter to mobj_SqlCommand.
mobj_SqlCommand.Parameters.Add(objSqlParameter);
//Finally, execute the command.
retVal = (int)mobj_SqlCommand.ExecuteScalar();
//Detach the parameters from mobj_SqlCommand, so it can be used again.
mobj_SqlCommand.Parameters.Clear();
functionReturnValue = retVal;
}
catch (Exception ex)
{
throw new System.Exception(ex.Message);
}
finally
{
//Clean up the objects created in this object.
if (mobj_SqlConnection.State == ConnectionState.Open)
{
mobj_SqlConnection.Close();
mobj_SqlConnection.Dispose();
mobj_SqlConnection = null;
}
if ((mobj_SqlCommand != null))
{
mobj_SqlCommand.Dispose();
mobj_SqlCommand = null;
}
if ((mobj_SqlDataAdapter != null))
{
mobj_SqlDataAdapter.Dispose();
mobj_SqlDataAdapter = null;
}
if ((dsParameter != null))
{
dsParameter.Dispose();
dsParameter = null;
}
objSqlParameter = null;
}
return functionReturnValue;
}
I am developing an j2me application which is communicating with the database using servlet.
I want to store the data received from servlet into record store and display it.how this can be achieved?Please provide code examples.
Thank you in advance
public void viewcon()
{
StringBuffer sb = new StringBuffer();
try {
HttpConnection c = (HttpConnection) Connector.open(url);
c.setRequestProperty("User-Agent","Profile/MIDP-1.0, Configuration/CLDC-1.0");
c.setRequestProperty("Content-Language","en-US");
c.setRequestMethod(HttpConnection.POST);
DataOutputStream os = (DataOutputStream)c.openDataOutputStream();
os.flush();
os.close();
// Get the response from the servlet page.
DataInputStream is =(DataInputStream)c.openDataInputStream();
//is = c.openInputStream();
int ch;
sb = new StringBuffer();
while ((ch = is.read()) != -1) {
sb.append((char)ch);
}
// return sb;
showAlert(sb.toString());//display data received from servlet
is.close();
c.close();
} catch (Exception e) {
showAlert(e.getMessage());
}
}
Put this function call where you show the response data alert
writeToRecordStore(sb.toString().getBytes());
The function definition is as below:
private static String RMS_NAME = "NETWORK-DATA-STORAGE";
private boolean writeToRecordStore(byte[] inStream) {
RecordStore rs = null;
try {
rs = RecordStore.openRecordStore(RMS_NAME, true);
if (null != rs) {
//Based on your logic either ADD or SET the record
rs.addRecord(inStream, 0, inStream.length);
return true;
} else {
return false;
}
} catch (RecordStoreException ex) {
ex.printStackTrace();
return false;
} finally {
try {
if (null != rs) {
rs.closeRecordStore();
}
} catch (RecordStoreException recordStoreException) {
} finally {
rs = null;
}
}
}
After you have saved the data, read the records store RMS-NAME and check the added index to get the response data.
.
NOTE: The assumption is the network response data is to be appended to the record store. If you want to set it to a particular record modify the method writeToRecordStore(...) accordingly.
//this code throws exception in display class.
try
{
byte[] byteInputData = new byte[5];
int length = 0;
// access all records present in record store
for (int x = 1; x <= rs.getNumRecords(); x++)
{
if (rs.getRecordSize(x) > byteInputData.length)
{
byteInputData = new byte[rs.getRecordSize(x)];
}
length = rs.getRecord(x, byteInputData, 0);
}
alert =new Alert("reading",new String(byteInputData,0,length),null,AlertType.WARNING);
alert.setTimeout(Alert.FOREVER);
display.setCurrent(alert);
}
I'm struggling with this a whole day now and I can't seem to figure it out.
I have a fucntion that gives me a list of all links on a specific url. That works fine.
However I want to make this function recursive so that it searches for the links found with the first search and adds them to the list and continue so that it goes through all my pages on the website.
How can I make this recursive?
My code:
class Program
{
public static List<LinkItem> urls;
private static List<LinkItem> newUrls = new List<LinkItem>();
static void Main(string[] args)
{
WebClient w = new WebClient();
int count = 0;
urls = new List<LinkItem>();
newUrls = new List<LinkItem>();
urls.Add(new LinkItem{Href = "http://www.smartphoto.be", Text = ""});
while (urls.Count > 0)
{
foreach (var url in urls)
{
if (RemoteFileExists(url.Href))
{
string s = w.DownloadString(url.Href);
newUrls.AddRange(LinkFinder.Find(s));
}
}
urls = newUrls.Select(x => new LinkItem{Href = x.Href, Text=""}).ToList();
count += newUrls.Count;
newUrls.Clear();
ReturnLinks();
}
Console.WriteLine();
Console.Write("Found: " + count + " links.");
Console.ReadLine();
}
private static void ReturnLinks()
{
foreach (LinkItem i in urls)
{
Console.WriteLine(i.Href);
//ReturnLinks();
}
}
private static bool RemoteFileExists(string url)
{
try
{
HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
request.Method = "HEAD";
//Getting the Web Response.
HttpWebResponse response = request.GetResponse() as HttpWebResponse;
//Returns TURE if the Status code == 200
return (response.StatusCode == HttpStatusCode.OK);
}
catch
{
return false;
}
}
}
The code behind LinkFinder.Find can be found here: http://www.dotnetperls.com/scraping-html
Anyone knows how I can either make that function recursive or can I make the ReturnLinks function recursive? I prefer to not touch the LinkFinder.Find method as this works perfect for one link, I just should be able to call it as many times as needed to expand my final url list.
I assume you want to load each link and find the link within, and continue until you run out of links?
Since it is likely that the recursion depth could get very large, i would avoid recursion, this should work i think.
WebClient w = new WebClient();
int count = 0;
urls = new List<string>();
newUrls = new List<LinkItem>();
urls.Add("http://www.google.be");
while (urls.Count > 0)
{
foreach(var url in urls)
{
string s = w.DownloadString(url);
newUrls.AddRange(LinkFinder.Find(s));
}
urls = newUrls.Select(x=>x.Href).ToList();
count += newUrls.Count;
newUrls.Clear();
ReturnLinks();
}
Console.WriteLine();
Console.Write("Found: " + count + " links.");
Console.ReadLine();
static void Main()
{
WebClient w = new WebClient();
List<ListItem> allUrls = FindAll(w.DownloadString("http://www.google.be"));
}
private static List<ListItem> FindAll(string address)
{
List<ListItem> list = new List<ListItem>();
foreach (url in LinkFinder.Find(address))
{
list.AddRange(FindAll(url.Address)));//or url.ToString() or what ever the string that represents the address
}
return list;
}