I have code logic inside nifi processor (executeScript processor) which will reduce log files(in this case in my log files i have same text so i want to remove duplicates and i try to choose them by name and file size),but i sometimes ( not always) got negative index error, what should i change in my code to make it work?
import org.apache.nifi.processor.FlowFileFilter;
import groovy.json.JsonSlurper
import groovy.json.JsonBuilder
import java.nio.charset.StandardCharsets
import org.apache.commons.io.IOUtils
def flowFile = session.get();
def n=0;
if(!flowFile)return
def size = flowFile.getAttribute('Size');
int value = size as Integer;
if((value/338)>=1){
def ffList = session.get(new FlowFileFilter(){
public FlowFileFilterResult filter(FlowFile ff) {
if( size == ff.getAttribute('Size') ){ n++; return FlowFileFilterResult.ACCEPT_AND_CONTINUE;}
else{ return FlowFileFilterResult.REJECT_AND_CONTINUE; }
}
});
session.transfer(ffList[n-1],REL_SUCCESS);
session.remove( ffList[0..-2] )
session.remove(flowFile);
}
else{
session.transfer(flowFile,REL_SUCCESS);
}
I think you should check the size of list before accessing elements.
def ffSize = ffList.size()
if (n <= ffSize && ffSize >= 2) {
session.transfer(ffList[n - 1], REL_SUCCESS)
session.remove(ffList[0..-2])
session.remove(flowFile)
}
Related
I am working API which give the list of Ride booking and from the Json response i need to check the DestinationEstimatedTravelTime should not be 0 or Null
Expected via script Assertion: it should scan all the available Array Response and check condition that the "DestinationETA" should be greater then 0.
Below is the image of my Response
Below is the code that i have used I have used For loop.
import groovy.json.JsonSlurper
//grab response
def response = messageExchange.response.responseContent
def jsosl = new JsonSlurper().parseText(response)
for(int i =0 ; i < jsos1.size(); i++)
{
if(Results[i].DestinationETA == 0 | Results[i].DestinationETA != "Null" )
{
log.info("Values are greater than 0")
}
else
{
log.info("test case Fail ")
}
}
Your error is because you call it jsosl in one place, then jsos1 in the next
(last character is a lowercase L then a 1)
You could change your code to:
assert json.Results.every { it.DestinationETA }
As in groovy, 0 or null is false
I was working on this and I my self found the solution
//Import file
import groovy.json.JsonSlurper
import com.eviware.soapui.model.testsuite.TestRunner.*
import com.eviware.soapui.model.testsuite.*
import com.eviware.soapui.model.*
//Grab Response
def response = messageExchange.response.responseContent
def json = new groovy.json.JsonSlurper().parseText(response)
//take count
def count = json.Results.size()
for(int i=0; i<count; i++) //loop to traverse to each object
{
log.info(json.Results[i].DestinationETA) // print result
assert json.Results[i].DestinationETA != null
assert json.Results[i].DestinationETA != 0
}
I have many internal log , which i write myself inside nifi environment, i want to input all this data inside one log flowFile, but this code trows Null pointer exception what should i change?
import java.nio.charset.StandardCharsets;
import org.apache.commons.io.IOUtils;
import org.apache.nifi.processor.FlowFileFilter;
import groovy.json.JsonSlurper;
import groovy.json.JsonBuilder;
def flowFile = session.get();
def n=0;
if(!flowFile)return
def size = flowFile.getAttribute('fileSize');
int value = size as Integer;
log.error("g");
if((value/338)>1){
def ffList = session.get(new FlowFileFilter(){
public FlowFileFilterResult filter(FlowFile ff) {
if( size == ff.getAttribute('fileSize') ){ n++; return FlowFileFilterResult.ACCEPT_AND_CONTINUE;
}
else{
return FlowFileFilterResult.REJECT_AND_CONTINUE
}
}
})
session.transfer(ffList[n-1],REL_SUCCESS);
session.remove( ffList[0..-2] )
session.remove(flowFile);
}
else{
session.transfer(flowFile,REL_SUCCESS);
}
I want to get all flowfile from the queu which fileSize is greater than 831 and then put them into the list after that catch last flowfile from the list and transfer to success relationship and finally remove all other flowfiles , here is my code which throws exception that transfer relationship not specified , what should i change in this case?
import org.apache.nifi.processor.FlowFileFilter;
import groovy.json.JsonSlurper
import groovy.json.JsonBuilder
import java.nio.charset.StandardCharsets
import org.apache.commons.io.IOUtils
def flowFile = session.get()
def n=0;
if(!flowFile)return
def size = flowFile.getAttribute('fileSize');
log.error(size.toString())
int value = size as Integer;
if((value/831)>1){
def ffList = session.get(new FlowFileFilter(){
public FlowFileFilterResult filter(FlowFile ff) {
if( size == ff.getAttribute('fileSize') ) n++; return FlowFileFilterResult.ACCEPT_AND_CONTINUE
return FlowFileFilterResult.REJECT_AND_CONTINUE
}
})
session.transfer(ffList.get(n-1),REL_SUCCESS)
//session.remove(ffList);
}
session.remove(flowFile);
if you get flow file from queue you have to do something with it.
this code returns you a flowFile list:
def ffList = session.get(new FlowFileFilter(){...})
if you wand just remove all of them except the last one just put this code after transferring the last one:
session.remove( ffList[0..-2] )
and i guess there is a mistake in this line:
if( size == ff.getAttribute('fileSize') ) n++; return FlowFileFilterResult.ACCEPT_AND_CONTINUE
the command return FlowFileFilterResult.ACCEPT_AND_CONTINUE executed in any case because it's not under if.
i think should be like this:
if( size == ff.getAttribute('fileSize') ){
n++;
return FlowFileFilterResult.ACCEPT_AND_CONTINUE
}
I'm working on a project with Soapui when I send the request I have this error unexpected end of file here is my code:
import java.io.FileNotFoundException
import java.io.FileOutputStream
import java.io.IOException
import java.util.Date
import java.io.*
import org.apache.poi.hssf.usermodel.HSSFCell
import org.apache.poi.hssf.usermodel.HSSFCellStyle
import org.apache.poi.hssf.usermodel.HSSFDataFormat
import org.apache.poi.hssf.usermodel.HSSFRow
import org.apache.poi.hssf.usermodel.HSSFSheet
import org.apache.poi.hssf.usermodel.HSSFWorkbook
import org.apache.poi.hssf.util.HSSFColor
import org.apache.poi.xssf.usermodel.*
import com.opencsv.CSVReader
import com.opencsv.CSVWriter
CSVReader reader = new CSVReader(new FileReader("C:\\Users\\******\\Desktop\\Groovy Script test\\data.csv"))
CSVWriter writer = new CSVWriter(new FileWriter("C:\\Users\\******\\Desktop\\Groovy Script test\\Output.csv"))
//get property
propTestStep = context.testCase.getTestStepByName("PRO-Number-property") // get the Property step (ours is named "property-loop")
//loop
String [] nextLine
while ((nextLine = reader.readNext()) != null ) {
writer.writeNext(nextLine)
log.info ("The value on the excel sheet is data " + nextLine[5])
//assign value of pro
propTestStep.setPropertyValue("PRO Number", (nextLine[2])) //set the value of Pro" property equal to Excel's column B ( getCell(1) )
//Get node of the response
testRunner.runTestStepByName("GetByProNumber") //we're going to run the Soap Request after each iteration of the Excel's rows.
def groovyUtils = new com.eviware.soapui.support.GroovyUtils( context )
if( "GetByProNumber#Response" == null){
log.info("Null try again")
} else {
def holder = groovyUtils.getXmlHolder("GetByProNumber#Response")
log.info (holder)
holder.namespaces["ns"] = "http://www.SaiaSecure.com/WebService/Shipment"
if (holder != null){
writer.writeNext("${holder.getNodeValue('//ns:Activity')}", nextLine[0])
log.info("Node Response:${holder.getNodeValue('//ns:Activity')}")
} else {
log.info("Null try again")
}
}
}// end of while loop
//writer.close()
writer.close()
log.info ("Done" )
Please I want to know what I'm doing wrong Thanks for any help I'm working on a project with Soapui when I send the request I have this error unexpected end of file here is my code:
Oh ok, your comment is very useful to identify the issue.
That appears to be a trivial error.
Change your code snippet from:
def holder = groovyUtils.getXmlHolder("GetByProNumber#Response")
To:
//Get the specified test step response and assign to variable
def stepResponse = context.expand('${GetByProNumber#Response}')
//Now pass the above variable to xmlholder to create the object
def holder = groovyUtils.getXmlHolder(stepResponse)
I wrote logic for testing urls using threads.
This works good for less number of urls and failing with more than 400 urls to check .
class URL extends Thread{
def valid
def url
URL( url ) {
this.url = url
}
void run() {
try {
def connection = url.toURL().openConnection()
connection.setConnectTimeout(10000)
if(connection.responseCode == 200 ){
valid = Boolean.TRUE
}else{
valid = Boolean.FALSE
}
} catch ( Exception e ) {
valid = Boolean.FALSE
}
}
}
def threads = [];
urls.each { ur ->
def reader = new URL(ur)
reader.start()
threads.add(reader);
}
while (threads.size() > 0) {
for(int i =0; i < threads.size();i++) {
def tr = threads.get(i);
if (!tr.isAlive()) {
if(tr.valid == true){
threads.remove(i);
i--;
}else{
threads.remove(i);
i--;
}
}
}
Could any one please tell me how to optimize the logic and where i was going wrong .
thanks in advance.
Have you considered using the java.util.concurrent helpers? It allows multithreaded programming at a higher level of abstraction. There's a simple interface to run parallel tasks in a thread pool, which is easier to manage and tune than just creating n threads for n tasks and hoping for the best.
Your code then ends up looking something like this, where you can tune nThreads until you get the best performance:
import java.util.concurrent.*
def nThreads = 1000
def pool = Executors.newFixedThreadPool(nThreads)
urls.each { url ->
pool.submit(url)
}
def timeout = 60
pool.awaitTermination(timeout, TimeUnit.SECONDS)
Using ataylor's suggestion, and your code, I got to this:
import java.util.concurrent.Executors
import java.util.concurrent.TimeUnit
class MyURL implements Runnable {
def valid
def url
void run() {
try {
url.toURL().openConnection().with {
connectTimeout = 10000
if( responseCode == 200 ) {
valid = true
}
else {
valid = false
}
disconnect()
}
}
catch( e ) {
valid = false
}
}
}
// A list of URLs to check
def urls = [ 'http://www.google.com',
'http://stackoverflow.com/questions/2720325/groovy-thread-for-urls',
'http://www.nonexistanturlfortesting.co.ch/whatever' ]
// How many threads to kick off
def nThreads = 3
def pool = Executors.newFixedThreadPool( nThreads )
// Construct a list of the URL objects we're running, submitted to the pool
def results = urls.inject( [] ) { list, url ->
def u = new MyURL( url:url )
pool.submit u
list << u
}
// Wait for the poolclose when all threads are completed
def timeout = 10
pool.shutdown()
pool.awaitTermination( timeout, TimeUnit.SECONDS )
// Print our results
results.each {
println "$it.url : $it.valid"
}
Which prints out this:
http://www.google.com : true
http://stackoverflow.com/questions/2720325/groovy-thread-for-urls : true
http://www.nonexistanturlfortesting.co.ch/whatever : false
I changed the classname to MyURL rather than URL as you had it, as it will more likely avoid problems when you start using the java.net.URL class