Groovy code to read rabbitMQ working on Windows, not working on Linux - groovy

Need: Read from rabbitMQ with AMQPS
Problem: ConsumeAMQP is not working so I'm using groovy script that's working on windows and not working on linux. Error message is:
groovy.lang.MissingMethodException: No signature of method: com.rabbitmq.client.ConnectionFactory.setUri() is applicable for argument types: (String) values: [amqps://user:xxxxxxxXXXxxxx#c-s565c7-ag77-etc-etc-etc.mq.us-east-1.amazonaws.com:5671/virtualhost]
Possible solutions: getAt(java.lang.String), every(), every(groovy.lang.Closure)
Troubleshooting:
Developed code on python to test from my machine using pika lib and it's working with URL amqps. It reads from rabbitMQ. no connection issues.
put the python code on the nifi server (1.15.3) machine, installed python and pika lib, execute on the command line, it's working on the server and reads from rabbitMQ.
Develop groovy code to test from my windows apache nifi (1.15.3)` and it's working, it's reading from rabbitMQ client system.
Copy the code (copy past) to the nifi server, uploaded the .jar lib also. Not working with this error message. create a groovy file and execute the code. not working.
Can anyone help me?
NOTE: I want to use groovy code to output the results to the flowfile.
#Grab('com.rabbitmq:amqp-client:5.14.2')
import com.rabbitmq.client.*
import org.apache.commons.io.IOUtils
import java.nio.charset.*
// -- Define connection
def ConnectionFactory factory = new ConnectionFactory();
factory.setUri('amqps://user:password#a-r5t60-etc-etc-etc.mq.us-east-1.amazonaws.com:5671/virtualhost');
factory.useSslProtocol();
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
// -- Waiting for messages.");
boolean noAck = false;
int count = 0;
while(count<10) {
GetResponse response = channel.basicGet("db-user-q" , noAck)
if (response != null) {
byte[] body = response.getBody()
long deliveryTag = response.getEnvelope().getDeliveryTag()
def msg = new String(body, "UTF-8")
channel.basicAck(response.envelope.deliveryTag, false)
def flowFile = session.create()
flowFile = session.putAttribute(flowFile, 'myAttr', msg)
session.transfer(flowFile, REL_SUCCESS);
}
count++;
}
channel.close();
connection.close();

The following code is suspect:
def ConnectionFactory factory = new ConnectionFactory();
You don't need both def and a type ConnectionFactory. Just change it to this:
ConnectionFactory factory = new ConnectionFactory()
You don't need the semi-colon either. The keyword def is used for dynamic typing situations (or laziness), and specifying the type (ie ConnectionFactory) is for static typing situations. You can't have both. It's either dynamic or static typing. I suspect Groovy VM is confused by what type the object is hence why it can't figure out if setUri exists or not.

Related

Soap UI: Groovy Test Step : How to call a Specific Method in a groovy script from another Groovy Script

In my project, I want to keep all groovy utilities test step under one test case and to call them again and again where ever is needed. Like reading the test data file etc. I would be able to achieve that if the below problem is resolved. I tried a lot of ways but couldn't make it. Any help is welcome!!
For Example
script 1: test1Script
def sayHellow(){
log.info "Hello!!"
}
Script 2 : test2Script
import groovy.lang.Binding
import groovy.util.GroovyScriptEngine
def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context)
def projectPath = groovyUtils.projectPath
def scriptPath = projectPath + "\\GroovyScripts\\"
//GroovyShell shell = new GroovyShell()
//Util = shell.parse(new File(directoryName + "groovyUtilities.groovy"))
//groovyUtilities gu = new groovyUtilities(Util)
// Create Groovy Script Engine to run the script.
GroovyScriptEngine gse = new GroovyScriptEngine(scriptPath)
// Load the Groovy Script file
externalScript = gse.loadScriptByName("sayHello.groovy")
// Create a runtime instance of script
instance = externalScript.newInstance()
// Sanity check
assert instance!= null
// run the foo method in the external script
instance.sayhellowTest()
When I'm calling that method from another script, I'm getting below exception
groovy.lang.MissingPropertyException: No such log for class test1Script
The error shows that groovy runtime calls your method but it can't find the log property. I assume that this log variable is declared in the test1Script body, e.g. def log = ... In this case the variable becomes local to its declaration scope and it's not visible to the script functions. To make it visible, it can be annotated by #Field or it should be undeclared (doesn’t have type declaration, just log = ...). The latter, however, requires you to pass the variable value via so-called bindings when running the script as you run it. So given my assumptions above, you can annotate your log variable as a field and it should work:
//just for the sake of example it prints to stdout whatever it receives
#groovy.transform.Field
def log = [info: {
println it
}]
def sayHellow() {
log.info "Hello!!"
}
Now calling sayHellow from another script prints "Hello" to stdout:
...
instance.sayHellow()
It is very important to declare, context, testRunner, and Log variables in the called script.
script 1: sayHello.groovy
public class groovyUtilities {
def context
def testRunner
def log
def sayhellowTest() {
log.info "Hi i'm arpan"
}
}
script 2: RunAnotherGroovyScript.groovy
def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context)
def projectPath = groovyUtils.projectPath
def scriptPath = projectPath + "\\GroovyScripts\\"
// Create Groovy Script Engine to run the script and pass the location of the directory of your script.
GroovyScriptEngine gse = new GroovyScriptEngine(scriptPath)
// Load the Groovy Script file
externalScript = gse.loadScriptByName("sayHello.groovy")
// Create a runtime instance of script
instance = externalScript.newInstance(context: context, log: log, testRunner: testRunner)
// Sanity check if the instance is null or not
assert instance != null
// run the foo method in the external script
instance.sayhellowTest()
Standoutput:
Hi i'm arpan
"I want to keep all groovy utilities test step under one test case and
to call them again and again where ever is needed. Like reading the
test data file etc."
OK, so to me this simply sounds like you have a library of reusable functions and want to be able to call them from any test you might be running.
I suppose you could store them with another test and then call them from the test you're currently running, but SoapUI comes with the neat feature in that you can store your common functions/libraries 'outside' of the SoapUI project.
I have lots of such Groovy libraries and I store mine under the bin/scripts folder of SoapUI. I typically call common functions from a Script assertion test step in the test I'm running. For example, I have a getUserDetails type test step. I can do all the usual assertions against the step like valid response code, SLA etc. I can then use a Script assertion test step. This type of step allows you to run a chunk of Groovy script. This is OK for specific cases, but you wouldn't want to paste in something common as you need to update every Script assertion if something changes. But you can call an 'external' groovy script. Also, the Script Assertion step is just a method that has log, context and message exchange passed into it, so no need to instantiate your own. Just pass them into you external groovy script...
So, as an illustration...
ValidateUser.groovy (stored in bin/scripts/groovyScripts/yourOrg/common)
package groovyScripts.yourOrg.common; // Package aligns with folder it's stored in.
Class ValidateUser {
def log = null;
def context = null;
def messageExchange = null;
// Constructor for the class
ValidateUser(logFromTestStep, contextFromTestStep, messageExchangeFromTestStep) {
// Assign log, context and messageExchange to the groovy class.
this.log = logFromTestStep;
this.context = contextFromTestStep;
this.messageExhange = messageExchangeFromTestStep;
}
def runNameCheck() {
// Performs some validation. You have access to the API response via
// this.messageExchange
log.info("Running the Name Check");
}
}
In the test step of interest, go to the assertions and create a 'Script Assertion' From here you can instantiate your external class and call some method. E.g.
def validateUserObject = new groovyScripts.yourOrg.common.ValidateUser(log, context, messageExchange);
validateUserObject.runNameCheck();
What I like about these external type scripts is that I can use any text editor I like. Also, when I make a change and press Save, SoapUI is monitoring the scripts folder for changes and reloads the script so no need to restart SoapUI.

groovy script not working in nifi executescript processor

I'm trying to execute something via executescript processor; a groovy code inside. In the code I'm trying to create a scala script which is to be executed on spark in a further processor.
// Get flow file
def flowFile = session.get()
if (!flowFile) return
// Create output directory
def userInputDir = flowFile.getAttribute("user.input.path")
def finalFolder = new File(userInputDir + "/" + innerDir)
try
{
if (!finalFolder.exists()) finalFolder.mkdirs()
// Write script
file = "spark.sqlContext.setConf(\"hive.exec.dynamic.partition\", \"true\")\n"
file = file + "spark.sqlContext.setConf(\"hive.exec.dynamic.partition.mode\", \"nonstrict\")\n"
file = file + "import org.apache.spark.sql._"
file = file + "\n"
file = file + "import java.io._"
file = file + "\n"
}
.
.
.
The rest other steps are adding some other spark specific commands to the script variable. Script is huge so skipping the full code paste.
Finally, closing with a catch
// Output file path
flowFile = session.putAttribute(flowFile, "generatedScript", scalaFile.getCanonicalPath())
session.transfer(flowFile, REL_SUCCESS)
}
catch(Exception e)
{
log.info("File: {}\n", finalFolder.file)
session.transfer(flowFile, REL_FAILURE)
}
The processor is not even beginning to start the groovy script execution and it fails with the error:
groovy.lang.MissingPropertyException: No such property: script for calss: javal.io.File
By the statement 'not even beginning to start' means that the previous queue is not empty and the processor throws the error. I'm guessing it's a syntax issue but I don't find any syntactical problems related in the script. I also tried running the script in local machine's groovy shell and have the same error there as well, but no syntax issue.
Googling the error got me the suggestion to include the imports in the script but even after including the relevant imports the error is the same.
Any clues?
You're referring to a variable "innerDir" which is not defined anywhere. Are you expecting the user to add a user-defined property to ExecuteScript called innerDir? If so, the innerDir variable in the script is a PropertyValue object, so you'd need to call getValue() on it to get the actual value of the property:
innerDir.value
Also you refer to scalaFile.getCanonicalPath() but scalaFile is not defined above, and getCanonicalPath() won't give you the contents of the script, is that what you meant?
I reworked the partial script above to assume that innerDir is a user-defined property and you write the contents of the file variable to a File pointed to by scalaFile; also I made it more Groovy by using heredoc instead of appending to the file variable:
// Get flow file
def flowFile = session.get()
if (!flowFile) return
// Create output directory
def userInputDir = flowFile.getAttribute("user.input.path")
def finalFolder = new File(userInputDir + "/" + innerDir?.value ?: '')
try
{
if (!finalFolder.exists()) finalFolder.mkdirs()
// Write script
file =
"""
spark.sqlContext.setConf("hive.exec.dynamic.partition", "true")
spark.sqlContext.setConf("hive.exec.dynamic.partition.mode", "nonstrict")
import org.apache.spark.sql._
import java.io._
"""
scalaFile = new File(finalFolder, 'script.scala')
scalaFile.withWriter {w -> w.write(file)}
// Output file path
flowFile = session.putAttribute(flowFile, "generatedScript", scalaFile.canonicalPath)
session.transfer(flowFile, REL_SUCCESS)
}
catch(Exception e) {
log.info("File: {}\n", finalFolder.file)
session.transfer(flowFile, REL_FAILURE)
}

How to use createTempFile in groovy/Jenkins to create a file in non-default directory?

What I am trying to achieve is to create a temporary file in groovy in workspace directory, but as an example /tmp/foo will be good enough.
So, here is perfectly working java code:
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.Files;
class foo {
public static void main(String[] args) {
try {
String s="/tmp/foo";
Path p=Paths.get(s);
Path tmp=Files.createTempFile(p,"pref",".suf");
System.out.println(tmp.toString());
} catch (Exception e) {
e.printStackTrace();
}
}
}
however, when used in context of Jenkins pipeline it simply does not work:
def mktemp() {
//String s=pwd(tmp:true)
String s="/tmp/foo"
Path p=Paths.get(s)
Path tmp=Files.createTempFile(p,"pref",".suf")
return tmp;
}
The result is array element type mismatch message with nothing helpful in pipeline log:
java.lang.IllegalArgumentException: array element type mismatch
at java.lang.reflect.Array.set(Native Method)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovyCallSiteSelector.parametersForVarargs(GroovyCallSiteSelector.java:104)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovyCallSiteSelector.matches(GroovyCallSiteSelector.java:51)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovyCallSiteSelector.findMatchingMethod(GroovyCallSiteSelector.java:197)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovyCallSiteSelector.staticMethod(GroovyCallSiteSelector.java:191)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onStaticCall(SandboxInterceptor.java:153)
at org.kohsuke.groovy.sandbox.impl.Checker$2.call(Checker.java:184)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedStaticCall(Checker.java:188)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:95)
at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
at WorkflowScript.mktemp(WorkflowScript:16)
The java.io.File.createTempFile() is not any better. In plain java code it works perfectly. In groovy it throws java.io.IOException: No such file or directory.
BTW, /tmp/foo directory exists, methods are added on script approval screen.
From the IOException I suspect you're calling mktemp from within a node {} block and expecting to create the temporary file on that node. Pipeline scripts are run entirely on the Jenkins master. Pipeline steps that interact with the filesystem (e.g. writeFile) are aware of node {} blocks and will be sent over to the node to be executed there, but any pure-Java methods know nothing about remote nodes and are going to interact with the master's filesystem.

Running SoapUI test cases using testRunner

I am working on a SoapUI project where I need to run my test suite using test runner. I am using external groovy scripting for environment variable. The problem I am facing here is whenever I am running test case from test runner its return Workspace as null, which is used in External groovy. So in external groovy I am getting workspace as null causing error [getProjectByname() cannot be invoked on null]. Below is the
constructor of global script where workspace is used
AvengerAPITestManager(String TestProject, String TestSuite, String TestCase,String TestStep)
{
TestName = "AvengerAPITests";
testProject = SoapUI.getWorkspace().getProjectByName(TestProject);
tSuite = testProject.getTestSuiteByName(TestSuite);
tCase = testProject.getTestSuiteByName(TestSuite).getTestCaseByName(TestCase);
tStepName = TestStep.toString();
tStep=testProject.getTestSuiteByName(TestSuite).getTestCaseByName(TestCase).getTestStepByName (TestStep);
}
Above we have user SoapUI.getWorkspace() which is working fine when trying to run from soapUI but whever I m trying to run from testrunner SoapUI.getWorkspace comes out to be null. I even tried passing workspace like I am passing testproject name still it didnt worked.
I tried something like this also
AvengerAPITestManager(Object workspace,String TestProject, String TestSuite, String TestCase, String TestStep)
{
TestName = "AvengerAPITests";
testProject = workspace.getProjectByName(TestProject);
tSuite = testProject.getTestSuiteByName(TestSuite);
tCase = testProject.getTestSuiteByName(TestSuite).getTestCaseByName(TestCase);
tStepName = TestStep.toString();
tStep = testProject.getTestSuiteByName(TestSuite).getTestCaseByName(TestCase).getTestStepByName(TestStep);
}
In the above code I tries passing Workspace object from the test case as I passed Testcase name and all but still I m getting null for workspace. Please tell me how do I deal with the problem.
Here is usefull working example https://github.com/stokito/soapui-junit
You should place your sample-soapui-project.xml to /src/test/resources folder that will expose it to classpath
If you want to use soap ui in external code, try to directly create new test runner with specific project file:
SoapUITestCaseRunner runner = new SoapUITestCaseRunner();
runner.setProjectFile( "src/dist/sample-soapui-project.xml" );
runner.run();
Or if you want to define test execution more precisely, you can use something like this:
WsdlProject project = new WsdlProject( "src/dist/sample-soapui-project.xml" );
TestSuite testSuite = project.getTestSuiteByName( "Test Suite" );
TestCase testCase = testSuite.getTestCaseByName( "Test Conversions" );
// create empty properties and run synchronously
TestRunner runner = testCase.run( new PropertiesMap(), false );
PS: don't forget to import soap ui classes, that you use in your code and put them to classpath.
PPS: If you need just run test cases outside the soap ui and/or automate this process, why not just use testrunner.sh/.bat for the same thing? (here is description of this way: http://www.soapui.org/Test-Automation/functional-tests.html)
I am not sure if this is going to help anyone out there but here is what I did to fix the problem I was having with workspace as null causing error[getProjectByname() cannot be invoked on null] When i run from cmd
try this:
import com.eviware.soapui.model.project.ProjectFactoryRegistry
import com.eviware.soapui.impl.wsdl.WsdlProjectFactory
import com.eviware.soapui.impl.wsdl.WsdlProject
//get the Util project
def project = null
def workspace = testRunner.testCase.testSuite.project.getWorkspace();
//if running Soapui
if (workspace != null) {
project = workspace.getProjectByName("Your Project")
}
//if running in Jenkins/Hudson
else{
project = new WsdlProject("C:\\...\\....\\....\\-soapui-project.xml");
}
if (project.open && project.name == "Your Project") {
def properties = new com.eviware.soapui.support.types.StringToObjectMap()
def testCase = project.getTestSuiteByName("TestSuite 1").getTestCaseByName("TestCase");
if(testCase == null)
{
throw new RuntimeException("Could not locate testcase 'TestCase'! ");
} else {
// This will run everything in the selected project
runner = testCase.run(new com.eviware.soapui.support.types.StringToObjectMap(), false)
}
}
else {
throw new RuntimeException("Could not find project ' Order Id....' !")
}
The above code will run everything in the selected project.

Write to file via jenkins post-groovy script on slave

I'd like to do something very simple: Create/write to a file located in the remote workspace of a slave via the jenkins groovy post-build script plug-in
def props_file = new File(manager.build.workspace.getRemote() + "/temp/module.properties")
def build_num = manager.build.buildVariables.get("MODULE_BUILD_NUMBER").toInteger()
def build_props = new Properties()
build_props["build.number"] = build_num
props_file.withOutputStream { p ->
build_props.store(p, null)
}
The last line fails, as the file doesn't exist. I'm thinking it has something to do with the output stream pointing to the master executor, rather than the remote workspace, but I'm not sure:
Groovy script failed:
java.io.FileNotFoundException: /views/build_view/temp/module.properties (No such file or directory)
Am I not writing to the file correctly?
While writing onto slave you need to check the channel first and then you can successfully create a file handle and start reading or writing to that file:
if(manager.build.workspace.isRemote())
{
channel = manager.build.workspace.channel;
}
fp = new hudson.FilePath(channel, manager.build.workspace.toString() + "\\test.properties")
if(fp != null)
{
String str = "test";
fp.write(str, null); //writing to file
versionString = fp.readToString(); //reading from file
}
hope this helps!
Search for words The post build plugin runs on the manager and doing it as you say will fail if you are working with slaves! on the plugin page (the link to which you've provided) and see if the workaround there helps.
Does the folder /views/build_view/temp exist?
If not, you will need to do new File( "${manager.build.workspace.remote}/temp" ).mkdirs()

Resources