Groovy include new Groovy script - groovy

I'm pretty new to Groovy an I want to import a class from another groovy script in the main groovy script. I saw already this post. However due to environment restrictions I can't us functions like GroovyClassLoader or or import any libs/frameworks. The only possibility I have is to use the evaluate method.
I tried this:
main script (The evaluate does not throw any error)
File csvFile = new File("./Csvreader.groovy");
evaluate(csvFile);
Csvreader script
class CSVReader{
def csvFile = new File('./test.csv');
def separatorChar = ";"
def values = [];
def headers = [];
void setSeperator(String s){
if(s != ""){
separatorChar = s;
}
}
void readCsv(){
csvFile.eachLine{ line, number ->
if(number == 1){
def head = line.split(separatorChar);
for (entry in head) {
headers.add(entry);
}
} else{
def value = line.split(separatorChar);
def map =[:];
def i = 0;
for (entry in value) {
map.put(headers[i], entry);
i++;
}
values.add(map);
}
}
}
def getValues(){
return values;
}
def getHeaders(){
return headers;
}
def getSize(){
return values.size();
}
def getLine(def keyParam, def value){
for(int i = 0; i < values.size(); i++){
def line = values[i];
if(values[i][keyParam] == value){
return values[i];
}
}
}
}
However I would need to pass parameters with the evaluate call (and I would move the methods outside the class) or I need to create an instance of the class.
Unfortunately I don't really know how I can do that, does anyone have a solution for that?

you could try to put at the end of your Csvreader.groovy file the expression return CSVReader.class that will return you a compiled class.
Then in your caller script you could create an instance of this class and use it as any other class instance.
Csvreader.groovy
class CSVReader{
def separatorChar = ";"
void setSeparator(String s){
if(s != ""){
separatorChar = s;
}
}
}
return CSVReader.class
caller script
File csvFile = new File("./Csvreader.groovy")
def CSVReaderClass = evaluate(csvFile)
def cvsReader = CSVReaderClass.newInstance()
cvsReader.setSeparator("Z")
println cvsReader.separatorChar

Related

Nifi - Groovy Json Script define null attributes

I have a Nifi Groovy Script. Assigns fields to nifi attributes but it define null if json values ​​are empty. if json value is null I want to define attribute empty.
null attributes look like this ;
I want to like this ;
this is the script i use ;
import org.apache.commons.io.IOUtils
import java.nio.charset.*
def flowFile = session.get();
if (flowFile == null) {
return;
}
def slurper = new groovy.json.JsonSlurper()
def attrs = [:] as Map<String,String>
session.read(flowFile,
{ inputStream ->
def text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)
def obj = slurper.parseText(text)
obj.each {k,v ->
attrs[k] = v.toString()
}
} as InputStreamCallback)
flowFile = session.putAllAttributes(flowFile, attrs)
session.transfer(flowFile, REL_SUCCESS)
flowFile = session.putAllAttributes(flowFile, attrs.collectEntries{k,v->[k,v?:'']})
This is i solved it ;
import org.apache.commons.io.IOUtils
import java.nio.charset.*
def flowFile = session.get();
if (flowFile == null) {
return;
}
def slurper = new groovy.json.JsonSlurper()
def attrs = [:] as Map<String,String>
session.read(flowFile,
{ inputStream ->
def text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)
def obj = slurper.parseText(text)
obj.each {k,v ->
if (v.toString()=="null")
{
attrs[k] = ''
} else {
attrs[k] = v.toString()
}
}
} as InputStreamCallback)
flowFile = session.putAllAttributes(flowFile, attrs)
session.transfer(flowFile, REL_SUCCESS)

Can you help me with a solution of hackerrank active traders problem?

How can I rewrite this piece of code so that it works faster? Currently the execution time is more than 10 seconds.
def mostActive(customers):
initial = sorted(list(set(customers)))
filter_object = filter(lambda s: customers.count(s)/len(customers) >= 0.05, initial)
return filter_object
if __name__ == '__main__':
fptr = open(os.environ['OUTPUT_PATH'], 'w')
customers_count = int(input().strip())
customers = []
for _ in range(customers_count):
customers_item = input()
customers.append(customers_item)
result = mostActive(customers)
fptr.write('\n'.join(result))
fptr.write('\n')
fptr.close()
def mostActive(customers):
store=dict()
res=[]
for i in customers:
for i in store:
store[i]+=1
else:
store[i]=1
for i in store.items():
if(i[1]/len(customers))*100>5 or (len(customers)*5)/100>=5:
if(1[1]/len(customers))*100 >=(len(customers)*5)/100>=5 or (1[1]/len(customers))*100>=5:
res.append(i[0])
return sorted (res)
public static List<string> mostActive(List<string> customers)
{
List<string> res = new List<string>();
foreach (var c in customers)
{
double count = customers.Count(a => a.Contains(c));
double per = (count / customers.Count()) * 100;
if (!res.Contains(c) && per>=5)
{
res.Add(c);
}
}
return res.OrderBy(r=>r).ToList();
}
A Java 7 solution (without lambdas)
public static List<String> mostActive(List<String> customers) {
int size = customers.size();
Map<String, Integer> map = new HashMap<>();
for (String customer : customers) {
if (map.containsKey(customer)) {
map.put(customer, map.get(customer) + 1);
} else {
map.put(customer, 1);
}
}
List<String> result = new ArrayList<>();
for (String key : map.keySet()) {
double currentCustomerPercent = (double) (map.get(key)) / (double) size;
if (currentCustomerPercent * 100 >= 5.0) {
result.add(key);
}
}
Collections.sort(result);
return result;
}
def mostActive(customers):
n=len(customers)
res=dict()
for i in customers:
if i not in res:
res[i]=1
else:
res[i]+=1
perc=dict()
for i in res:
perc[i]= (res[i]/n)*100
b=[]
for i in perc:
if perc[i]>=5:
b.append(i)
return(sorted(b))

Nexus3 only downloads 50 in list

So I have a code that downloads the version nr of each nuget package but it all stops after 50 in list.
I use jenkins with groovy code and get out a list of versions.
import groovy.json.JsonSlurperClassic
import groovy.json.JsonBuilder
import wslite.rest.*
def data = new URL("http://nexus.xx.xx.se:8081/service/rest/v1/search?repository=xx-sx-nuget&name=XXXFrontend").getText()
println data
/**
* 'jsonString' is the input json you have shown
* parse it and store it in collection
*/
Map convertedJSONMap = new JsonSlurperClassic().parseText(data)
//If you have the nodes then fetch the first one only
if(convertedJSONMap."items"){
println "Version : " + convertedJSONMap."items"[0]."version"
}
def list = convertedJSONMap.items.version
Collections.sort(list)
list
So the problem is that it only get 50 of the versions. How can I get more than 50? I have read about a continuetoken but I dont understand how to use that?
UPDATE
I have added this but still dont work
while(convertedJSONMap."continuesToken" != null){
def token = convertedJSONMap."continuationToken"
def data2 = new URL("http://nexus.xxx.xxx.se:8081/service/rest/v1/search?repository=xxx-xx-nuget&name=xxxxxx&continuationToken=" +token).getText()
convertedJSONMap = JsonSlurperClassic().parseText(data2)
}
This is how I solved it for me. It is just a snippet of the code that I use
def json = sendRequest(url)
addResultToMap(map2, json, release) //I do something here with the received result
def continuationToken = json.continuationToken
if (continuationToken != null) {
while (continuationToken != null) {
json = sendRequest(url + "&continuationToken=" + continuationToken)
addResultToMap(map2, json, release) //I do something here with the received result as above
continuationToken = json.continuationToken
}
}
And my sendRequest method looks like this
def sendRequest(def url, String method = "GET") {
String userPass = "${nexus.username}:${nexus.password}"
String basicAuth = "Basic " + "${printBase64Binary(userPass.getBytes())}"
def connection = new URL( url ).openConnection() as HttpURLConnection
connection.setRequestProperty('Accept', 'application/json' )
connection.setRequestProperty('Authorization', basicAuth)
connection.setRequestMethod(method)
try {
if ( connection.responseCode <= 299 ) {
if (connection.responseCode == 200) {
return connection.inputStream.withCloseable { inStream -> new JsonSlurper().parse( inStream as InputStream ) }
}
} else {
displayAndLogError(connection.responseCode + ": " + connection.inputStream.text, loglevel.DEBUG)
}
} catch(Exception exc) {
displayAndLogError(exc.getMessage())
}
}
Here is an alternative :
import groovy.json.JsonSlurper
try {
N_PAGES_MAX = 10
List<String> versions = new ArrayList<String>()
continuationToken = "123"
artifactsUrl = "http://nexus.zzz.local/service/rest/v1/components?repository=releases-super-project"
currentPage = 1
while (true) {
artifactsObjectRaw = ["curl", "-s", "-H", "accept: application/json", "-k", "--url", "${artifactsUrl}"].execute().text
artifactsJsonObject = (new JsonSlurper()).parseText(artifactsObjectRaw)
continuationToken = artifactsJsonObject.continuationToken
if (continuationToken!=null && continuationToken!='123') {
artifactsUrl = artifactsUrl + "&continuationToken=$continuationToken"
}
def items = artifactsJsonObject.items
for(item in items){
versions.add(item.name)
}
currentPage += 1
if (continuationToken==null || currentPage>N_PAGES_MAX) break
}
return versions.sort().reverse()
}
catch (Exception e) {
print "There was a problem fetching the versions"
}

Use Groovy To Search A File And Grab Contents Below A String

I'm trying to use Groovy to open a file to search for a specific substring and then grab a different substring that occurs below the first one.
For example the substring I'm searching for is "Charger is enabled. Checking charge parameters..."
and if it is found I want to get a specific string that occurs after this.
Is the best way to do this read the file into memory and search for the index of the first string?
// With Java
import java.io.File;
import java.io.BufferedReader;
import java.io.FileReader;
def localDirectory = "";
def fileName = "";
def searchKey = "Charger is enabled. Checking charge parameters";
def searchKey2 = "";
def errorMessage = "";
FileReader fr;
BufferedReader br;
try
{
File f1 = new File(localDirectory+"/"+fileName);
fr = new FileReader(f1);
br = new BufferedReader(fr);
def keyFound = false;
// Go through line by line.
def line;
while ((line = br.readLine()) != null)
{
// If first string is found, process the second string.
if(line.contains(searchKey))
{
while ((line = br.readLine()) != null)
{
// Do something with the second string.
if(line.contains(searchKey2))
{
keyFound=true;
break;
}
}
}
if(keyFound)
{
break;
}
}
}
catch (Exception e)
{
errorMessage += "\nUnexpected Exception: " + e.getMessage();
for (trace in e.getStackTrace())
{
errorMessage += "\n\t" + trace;
}
}
finally
{
fr?.close();
br?.close();
}
// With Groovy
def localDirectory = "";
def fileName = "";
def searchKey = "Charger is enabled. Checking charge parameters";
def searchKey2 = "";
def errorMessage = "";
try
{
File f1 = new File(localDirectory+"/"+fileName);
def keyFound = false;
// Go through line by line.
def line;
f1.withReader
{
reader ->
while((line = reader.readLine()) != null)
{
// If first string is found, process the second string.
if(line.contains(searchKey))
{
while((line = reader.readLine()) != null)
{
// Do something with the second string.
if(line.contains(searchKey2))
{
keyFound=true;
break;
}
}
}
if(keyFound)
{
break;
}
}
}
}
catch (Exception e)
{
errorMessage += "\nUnexpected Exception: " + e.getMessage();
for (trace in e.getStackTrace())
{
errorMessage += "\n\t" + trace;
}
}

Groovy and POI: Can I read/write at the same time?

I'm new to groovy, and I've used the ExcelBuilder code referenced below to iterate through an excel spreadsheet to grab data. Is there an easy to write data as I iterate?
For example, row 1 might have data like this (CSV):
value1,value2
And after I iterate, I want it to look like this:
value1,value2,value3
http://www.technipelago.se/content/technipelago/blog/44
Yes, this can be done! As I got into the guts of it, I realized that the problem I was trying to solve wasn't the same as trying to read and write from the same file at the same time, but rather the excel data was stored in an object that I could freely manipulate whenever I wanted. So I added methods specific to my needs - which may or many not meet the needs of anyone else - and I post them here for smarter people to pick apart. At the end of it all, it is now doing what I want it to do.
I added a cell method that takes an index (number or label) and a value which will update a cell for the current row in context (specifically while using .eachLine()), and a .putRow() method that adds a whole row to the spreadsheet specified. It also handles Excel 2003, 2007, and 2010 formats. When files, sheets, or cells referenced don't exist, they get created. Since my source spreadsheets often have formulas and charts ready to display the data I'm entering, the .save() and .saveAs() methods call .evaluateAllFormulaCells() before saving.
To see the code I started with and examples of how it works, check out this blog entry
Note that both the .save() and .saveAs() methods reload the workbook from the saved file immediately after saving. This is a workaround to a bug in Apache POI that doesn't seem to be fixed yet (see Exception when writing to the xlsx document several times using apache poi).
import groovy.lang.Closure;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.Map;
import org.apache.poi.ss.usermodel.WorkbookFactory;
import org.apache.poi.ss.usermodel.DataFormatter;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import org.apache.poi.xssf.usermodel.XSSFSheet;
import org.apache.poi.xssf.usermodel.XSSFRow;
import org.apache.poi.xssf.usermodel.XSSFCell;
import org.apache.poi.ss.usermodel.DateUtil;
import org.apache.poi.xssf.usermodel.XSSFFormulaEvaluator;
import org.apache.poi.hssf.usermodel.HSSFWorkbook;
import org.apache.poi.hssf.usermodel.HSSFSheet;
import org.apache.poi.hssf.usermodel.HSSFRow;
import org.apache.poi.hssf.usermodel.HSSFCell;
import org.apache.poi.hssf.usermodel.HSSFDateUtil;
import org.apache.poi.hssf.usermodel.HSSFFormulaEvaluator;
class Excel
{
def workbook;
def sheet;
def labels;
def row;
def infilename;
def outfilename;
Excel(String fileName)
{
HSSFRow.metaClass.getAt = {int index ->
def cell = delegate.getCell(index);
if(! cell)
{
return null;
}
def value;
switch (cell.cellType)
{
case HSSFCell.CELL_TYPE_NUMERIC:
if(HSSFDateUtil.isCellDateFormatted(cell))
{
value = cell.dateCellValue;
}
else
{
value = new DataFormatter().formatCellValue(cell);
}
break;
case HSSFCell.CELL_TYPE_BOOLEAN:
value = cell.booleanCellValue
break;
default:
value = new DataFormatter().formatCellValue(cell);
break;
}
return value
}
XSSFRow.metaClass.getAt = {int index ->
def cell = delegate.getCell(index);
if(! cell)
{
return null;
}
def value = new DataFormatter().formatCellValue(cell);
switch (cell.cellType)
{
case XSSFCell.CELL_TYPE_NUMERIC:
if (DateUtil.isCellDateFormatted(cell))
{
value = cell.dateCellValue;
}
else
{
value = new DataFormatter().formatCellValue(cell);
}
break;
case XSSFCell.CELL_TYPE_BOOLEAN:
value = cell.booleanCellValue
break;
default:
value = new DataFormatter().formatCellValue(cell);
break;
}
return value;
}
infilename = fileName;
outfilename = fileName;
try
{
workbook = WorkbookFactory.create(new FileInputStream(infilename));
}
catch (FileNotFoundException e)
{
workbook = (infilename =~ /(?is:\.xlsx)$/) ? new XSSFWorkbook() : new HSSFWorkbook();
}
catch (Exception e)
{
e.printStackTrace();
}
}
def getSheet(index)
{
def requested_sheet;
if(!index) index = 0;
if(index instanceof Number)
{
requested_sheet = (workbook.getNumberOfSheets >= index) ? workbook.getSheetAt(index) : workbook.createSheet();
}
else if (index ==~ /^\d+$/)
{
requested_sheet = (workbook.getNumberOfSheets >= Integer.valueOf(index)) ? workbook.getSheetAt(Integer.valueOf(index)) : workbook.createSheet();
}
else
{
requested_sheet = (workbook.getSheetIndex(index) > -1) ? workbook.getSheet(index) : workbook.createSheet(index);
}
return requested_sheet;
}
def cell(index)
{
if (labels && (index instanceof String))
{
index = labels.indexOf(index.toLowerCase());
}
if (row[index] == null)
{
row.createCell(index);
}
return row[index];
}
def cell(index, value)
{
if (labels.indexOf(index.toLowerCase()) == -1)
{
labels.push(index.toLowerCase());
def frow = sheet.getRow(0);
def ncell = frow.createCell(labels.indexOf(index.toLowerCase()));
ncell.setCellValue(index.toString());
}
def cell = (labels && (index instanceof String)) ? row.getCell(labels.indexOf(index.toLowerCase())) : row.getCell(index);
if (cell == null)
{
cell = (index instanceof String) ? row.createCell(labels.indexOf(index.toLowerCase())) : row.createCell(index);
}
cell.setCellValue(value);
}
def putRow (sheetName, Map values = [:])
{
def requested_sheet = getSheet(sheetName);
if (requested_sheet)
{
def lrow;
if (requested_sheet.getPhysicalNumberOfRows() == 0)
{
lrow = requested_sheet.createRow(0);
def lcounter = 0;
values.each {entry->
def lcell = lrow.createCell(lcounter);
lcell.setCellValue(entry.key);
lcounter++;
}
}
else
{
lrow = requested_sheet.getRow(0);
}
def sheetLabels = lrow.collect{it.toString().toLowerCase()}
def vrow = requested_sheet.createRow(requested_sheet.getLastRowNum() + 1);
values.each {entry->
def vcell = vrow.createCell(sheetLabels.indexOf(entry.key.toLowerCase()));
vcell.setCellValue(entry.value);
}
}
}
def propertyMissing(String name)
{
cell(name);
}
def propertyMissing(String name, value)
{
cell(name, value);
}
def eachLine (Map params = [:], Closure closure)
{
/*
* Parameters:
* skiprows : The number of rows to skip before the first line of data and/or labels
* offset : The number of rows to skip (after labels) before returning rows
* max : The maximum number of rows to iterate
* sheet : The name (string) or index (integer) of the worksheet to use
* labels : A boolean to treat the first row as a header row (data can be reference by label)
*
*/
def skiprows = params.skiprows ?: 0;
def offset = params.offset ?: 0;
def max = params.max ?: 9999999;
sheet = getSheet(params.sheet);
def rowIterator = sheet.rowIterator();
def linesRead = 0;
skiprows.times{ rowIterator.next() }
if(params.labels)
{
labels = rowIterator.next().collect{it.toString().toLowerCase()}
}
offset.times{ rowIterator.next() }
closure.setDelegate(this);
while(rowIterator.hasNext() && linesRead++ < max)
{
row = rowIterator.next();
closure.call(row);
}
}
def save ()
{
if (workbook.getClass().toString().indexOf("XSSF") > -1)
{
XSSFFormulaEvaluator.evaluateAllFormulaCells((XSSFWorkbook) workbook);
}
else
{
HSSFFormulaEvaluator.evaluateAllFormulaCells((HSSFWorkbook) workbook);
}
if (outfilename != null)
{
try
{
FileOutputStream output = new FileOutputStream(outfilename);
workbook.write(output);
output.close();
workbook = null;
workbook = WorkbookFactory.create(new FileInputStream(outfilename));
}
catch (IOException e)
{
e.printStackTrace();
}
}
}
def saveAs (String fileName)
{
if (workbook.getClass().toString().indexOf("XSSF") > -1)
{
XSSFFormulaEvaluator.evaluateAllFormulaCells((XSSFWorkbook) workbook);
}
else
{
HSSFFormulaEvaluator.evaluateAllFormulaCells((HSSFWorkbook) workbook);
}
try
{
FileOutputStream output = new FileOutputStream(fileName);
workbook.write(output);
output.close();
outfilename = fileName;
workbook = null;
workbook = WorkbookFactory.create(new FileInputStream(outfilename));
}
catch (IOException e)
{
e.printStackTrace();
}
}
}
If you see any glaring errors or ways to improve (other than style), I'd love to hear them. Again, Groovy is not a language I have much experience with, and I haven't done anything with Java in several years, so I'm sure there might be some better ways to do things.

Resources