I am trying to write a Jenkins job (say CopyJob) that copies another job (in this job using the Multijob plugin) and also copies all its downstream jobs to new jobs. The idea is to have a Multijob that serves as a template so it can be copied to new Multijobs (e.g. for a specific branch or feature).
See:
MultiJob_Template
|
+-- Sub1_Template
+-- Sub2_Template
+-- Sub3_Template
CopyJob (Parameters: NewSuffix)
When manually triggering the "CopyJob" it shall create a new MultiJob with new SubJobs:
MultiJob_BranchXYZ
|
+-- Sub1_BranchXYZ
+-- Sub2_BranchXYZ
+-- Sub3_BranchXYZ
So far I was successful with copiing the Multijob and copiing the Subjobs, but I couldn't find a way to make the new Multijob actually depend on the new Subjobs.
My code (for the CopyJob groovy script) so far is:
import jenkins.model.*
import com.tikal.jenkins.plugins.multijob.*
def templateJobName = "MultiJob_Template"
// Retrieve parameters
def newSfx = build.buildVariableResolver.resolve("NewSuffix")
def templateJob = Jenkins.instance.getJob(templateJobName)
// copy Multijob
def newJob = Jenkins.instance.copy(templateJob, 'Multijob_' + newSfx)
newJob.save()
// copy all downstreamjobs
def subs = newJob.getDownstreamProjects()
for (s in subs) {
def oldSubJob = Jenkins.instance.getJob(s.getDisplayName())
def newSubJob = Jenkins.instance.copy(oldSubJob, s.getDisplayName().replaceFirst(/Template/, newSfx))
newSubJob.save()
// how to update the MultiJob_newSfx DownstreamJoblist to use the newSubJob?
// ????
}
I actually managed to solve it myself. Maybe there are other ways too, but it seems best to step through the list of builders and then the list of PhaseJobs of the MultiJob template.
The code of the MultiJob plugin itself helped for this solution.
It is also worth having a look at this question if you are looking for similar things.
import jenkins.model.*
import com.tikal.jenkins.plugins.multijob.*
def jenkinsInstance = jenkins.model.Jenkins.instance
def templateJobName = "Multijob_Template"
// Retrieve parameters
def newSfx = build.buildVariableResolver.resolve("NewSuffix")
// create new MultiJob
def templateJob = Jenkins.instance.getJob(templateJobName)
def newJob = Jenkins.instance.copy(templateJob, 'Multijob_' + newSfx)
newJob.save()
// get MultiJob BuildPhases and clone each PhaseJob
def builders = newJob.getBuilders()
builders.each { builder ->
builder.getPhaseJobs().each() { pj ->
println "cloning phasejob: " + pj.getJobName()
def subTemplate = Jenkins.instance.getJob(pj.getJobName())
def newSubJob = Jenkins.instance.copy(subTemplate, pj.getJobName().replaceFirst(/Template/, newSfx))
newSubJob.save()
pj.setJobName(newSubJob.getDisplayName())
}
}
// update dependencies
jenkinsInstance.rebuildDependencyGraph()
I am using the Scriptler plugin in Jenkins with the parameters:
NewSuffix, TemplateStr, and templateJobName. I tweaked the script from pitseeker to use these and work around a runtime issue in Jenkins v1.580.3:
import jenkins.model.*
import com.tikal.jenkins.plugins.multijob.*
def jenkinsInstance = jenkins.model.Jenkins.instance
// Retrieve parameters
def newSfx = NewSuffix
println "using new suffix " + newSfx
// create new MultiJob
def templateJob = Jenkins.instance.getJob(templateJobName)
println "Found templateJob " + templateJobName
def newJobName = templateJobName.replaceFirst(/$TemplateStr/, newSfx)
def newJob = Jenkins.instance.copy(templateJob, templateJobName + newSfx)
newJob.save()
println "Copied template job to " + newJob.getName()
// get MultiJob BuildPhases and clone each PhaseJob
def builders = newJob.getBuilders()
builders.each { builder ->
builder.getPhaseJobs().each() { pj ->
def pjNameOrig = pj.getJobName()
def pjNameNew = pjNameOrig.replaceFirst(/$TemplateStr/, newSfx)
println "pjNameOrig = $pjNameOrig, pjNameNew=$pjNameNew"
if (pjNameNew != pjNameOrig)
{
println "cloning phasejob: " + pjNameOrig
def subTemplate = Jenkins.instance.getJob(pjNameOrig)
def newSubJob = Jenkins.instance.copy(subTemplate, pjNameNew)
newSubJob.save()
pj.setJobName(newSubJob.getDisplayName())
}
else
{
println "Not cloning phasejob, keeping original job: " + pjNameOrig
}
}
}
// update dependencies
jenkinsInstance.rebuildDependencyGraph()
Related
I have a contants.groovy file as below
def testFilesList = 'test-package.xml'
def testdataFilesList = 'TestData.xml'
def gitId = '9ddfsfc4-fdfdf-sdsd-bd18-fgdgdgdf'
I have another groovy file that will be called in Jenkins pipeline job
def constants
node ('auto111') {
stage("First Stage") {
container('alpine') {
script {
constants = evaluate readTrusted('jenkins_pipeline/constants.groovy')
def gitCredentialsId = constants."${gitId}"
}
}
}
}
But constants."${gitId}" is says "cannot get gitID from null object". How do I get it?
It's because they are local variables and cannot be referenced from outside. Use #Field to turn them into fields.
import groovy.transform.Field
#Field
def testFilesList = 'test-package.xml'
#Field
def testdataFilesList = 'TestData.xml'
#Field
def gitId = '9ddfsfc4-fdfdf-sdsd-bd18-fgdgdgdf'
return this;
Then in the main script you should load it using load step.
script {
//make sure that file exists on this node
checkout scm
def constants = load 'jenkins_pipeline/constants.groovy'
def gitCredentialsId = constants.gitId
}
You can find more details about variable scope in this answer
Trying to create new jobs whenever there is a new branch entry in my SVN repo and below is the script.
svnCommand = "svn list --xml http://myrepo/svn/repo_name/branches"
def proc = svnCommand.execute()
proc.waitFor()
def xmlOutput = proc.in.text
def lists = new XmlSlurper().parseText(xmlOutput)
def listOfBranches = lists.list.entry.name
listOfBranches.each(){
def branchName = it.text()
println "found branch: '${branchName}'"
}
mavenJob('${branchName}'){
mavenInstallation('M3.3.9')
logRotator(365, 25, -1, -1)
scm {
svn {
location('http://myrepo/svn/repo_name/branches/${branchName}') {
credentials('4t4d8ef-p67a-5298-a011-580ghe898a65')
}
}
}
}
Script is able to iterate thru branches and print the branch names,
found branch: 'feature_01'
but I'm facing the issue , while variable substitution while creating the job name and with the svn branch name.
hudson.model.Failure: ‘$’ is an unsafe character
Jenkins - V.2.32
Job DSL - V.1.57
Any suggestions please.Thanks.
#Rao is right: first - you have to change:
mavenJob('${branchName}')
to:
mavenJob(branchName)
and:
location('http://myrepo/svn/repo_name/branches/${branchName}')
to:
location("http://myrepo/svn/repo_name/branches/${branchName}")
Moreover def branchName = it.text() inside iteration limits scope of variable to this iteration only. Try:
listOfBranches.each() {
branchName = it.text()
println "found branch: '${branchName}'"
}
I am new to SoapUI and Groovy so please forgive this post as it has been posted a number of times in Stackoverflow however I cannot find a fix.
SoapUI version: 4.5.2
I have 2 questions if you guys don't mind:
I have an executable jar file that I've put in the the \bin\ext directory along with another jar that is considered a dependency jar within the code in the jar so I hope it will reference there. The groovy code I found in Stackoverflow that should execute this jar is as follows but does not work as I don't see any output anywhere in the SoapUI directory.
Here is the code:
def command = "java -jar UpdateAppIdXMLRequest.jar file.xml"
def process = command.execute()
process.waitFor()
def output = process.in.text
log.info output
This jar creates 25 xml files that should be able to be picked up by the SoapUI and put them in as TestSteps in the same project. In my java code in what path do I put these files?
Here is the code in my jar:
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import org.jdom2.Document;
import org.jdom2.Element;
import org.jdom2.JDOMException;
import org.jdom2.input.SAXBuilder;
import org.jdom2.output.Format;
import org.jdom2.output.XMLOutputter;
class UpdateAppIdXMLRequest {
static main(args) {
try {
SAXBuilder builder = new SAXBuilder();
File xmlFile = new File("c:\\file.xml");
Document doc = (Document) builder.build(xmlFile);
Element rootNode = doc.getRootElement();
// Create loop to create 25 testStepApps
for (int i = 1; i < 26; i++) {
// Get current AppID, incrementAppID and update the ApplicationNumber attribute value for next test script.
int appID = Integer.parseInt(rootNode.getAttributeValue("ApplicationNumber"));
appID++;
String appIDValue = Integer.toString(appID);
rootNode.getAttribute("ApplicationNumber").setValue(appIDValue);
XMLOutputter xmlOutput = new XMLOutputter();
// Create new XML file with next AppID
xmlOutput.setFormat(Format.getPrettyFormat());
xmlOutput.output(doc, new FileWriter("c:\\testStepApp" + i + ".xml"));
// xmlOutput.output(doc, System.out);
// System.out.println("File updated!");
}
} catch (IOException io) {
io.printStackTrace();
} catch (JDOMException e) {
e.printStackTrace();
}
}
}
Any help/direction would be appreciated.
Thanks.
In order to do so, I recommend that you use directly groovy test step instead of a jar, this way you have more flexibility an you have not recompile the jar each time you must need to change something.
So, in order to achieve your goal, at first you need to create a TestCase inside your project, create a SOAP Test Step and Groovy Test Step inside like this:
I will use SOAP Test Step to create the other test steps (to create test steps it needs the wsdl:operation and so on, and it's more easy to copy the test step that create directly).
In the Groovy Test Step I will put the necessary code to do the work which is showed below:
import com.eviware.soapui.support.XmlHolder
import com.eviware.soapui.impl.wsdl.teststeps.registry.WsdlTestRequestStepFactory
// read your request template
def requestFile = new File("C:/file.xml");
// parse it as xml
def requestXml = new XmlHolder(requestFile.text)
// get the current testCase to add testSteps later
def tc = testRunner.testCase;
// get the testStep as template to create the other requests
def tsTemplate = tc.getTestStepByName("Template");
// loop to create 25 testStep
for(int i = 1; i < 26; i++){
// xpath expression to get applicationNumber attribute in root node
def xpathNodeAttr = "/*/#ApplicationNumber";
// get the root node attribute applicationNumber throught an XPATH
int appId = Integer.parseInt(requestXml.getNodeValue(xpathNodeAttr));
// add 1 to appId
appId++;
// set the value again in the attribute
requestXml.setNodeValue(xpathNodeAttr,appId);
def testStepName = "TestStep_ApplicationNumber_"+ String.valueOf(appId)
log.info testStepName;
log.info testStepName.getClass().getName()
log.info tc.getClass().getName()
// create a new testStepConfig
def testStepFactory = new WsdlTestRequestStepFactory();
def testStepConfig = testStepFactory.createConfig( tsTemplate.getOperation(), testStepName )
// add the new testStep to TestCase
def newTestStep = tc.insertTestStep( testStepConfig, -1 )
// set the request which just create
newTestStep.getTestRequest().setRequestContent(requestXml.getXml())
}
This code it's basically your java code "translated" to groovy and added the necessary code to create the test steps. In a few words this code reads a request from a file, and create 25 test steps in the current test case using the request, in each request it only changes the ApplicationNumber attribute of the root node adding it +1.
EDIT BASED ON COMMENT:
If you use a REST Request step instead of SOAP Request Step you have to change a bit your groovy code to use com.eviware.soapui.impl.wsdl.teststeps.registry.RestRequestStepFactory and getTestRequest() method on it. So if you have a REST Service with a POST method create a Test Case with a REST Request test step and Groovy Test Step like this:
And use this groovy code instead, basically this code it's the same and works like the code above and makes the same thing with REST Request instead of SOAP Request:
import com.eviware.soapui.support.XmlHolder
import com.eviware.soapui.impl.wsdl.teststeps.registry.RestRequestStepFactory
// read your request template
def requestFile = new File("C:/file.xml");
// parse it as xml
def requestXml = new XmlHolder(requestFile.text)
// get the current testCase to add testSteps later
def tc = testRunner.testCase;
// get the testStep as template to create the other requests
def tsTemplate = tc.getTestStepByName("Template");
// loop to create 25 testStep
for(int i = 1; i < 26; i++){
// xpath expression to get applicationNumber attribute in root node
def xpathNodeAttr = "/*/#ApplicationNumber";
// get the root node attribute applicationNumber throught an XPATH
int appId = Integer.parseInt(requestXml.getNodeValue(xpathNodeAttr));
// add 1 to appId
appId++;
// set the value again in the attribute
requestXml.setNodeValue(xpathNodeAttr,appId);
def testStepName = "TestStep_ApplicationNumber_"+ String.valueOf(appId)
log.info testStepName;
log.info testStepName.getClass().getName()
log.info tc.getClass().getName()
// create a new testStepConfig
def testStepFactory = new RestRequestStepFactory();
def testStepConfig = testStepFactory.createConfig( tsTemplate.getTestRequest(), testStepName )
// add the new testStep to TestCase
def newTestStep = tc.insertTestStep( testStepConfig, -1 )
// set the request which just create
newTestStep.getTestRequest().setRequestContent(requestXml.getXml())
}
Hope this helps.
I use to import data from excel ,but i use the bootstrap.groovy to write the code and my import script method is called when the application starts.
Here the scenarios is i m having 8000 related data once to import if they are not on my database.And,also when i deploy it to tomcat6 it is blocking other apps from deployment ,until it finish the import.So,i want to use separate thread for to run the script in anyway without affecting performance AND BLOCKING OTHER FROM DEPLOYMENT.
code excerpt ...
class BootStrap {
def grailsApplication
def sessionFactory
def excelService
def importStateLgaArea(){
String fileName = grailsApplication.mainContext.servletContext.getRealPath('filename.xlsx')
ExcelImporter importer = new ExcelImporter(fileName)
def listState = importer.getStateLgaTerritoryList() //get the map,form excel
log.info "List form excel:${listState}"
def checkPreviousImport = Area.findByName('Osusu')
if(!checkPreviousImport) {
int i = 0
int j = 0 // up
date cases
def beforeTime = System.currentTimeMillis()
for(row in listState){
def state = State.findByName(row['state'])
if(!state) {
// log.info "Saving State:${row['state']}"
row['state'] = row['state'].toString().toLowerCase().capitalize()
// log.info "after capitalized" + row['state']
state = new State(name:row['state'])
if(!state.save(flash:true)){
log.info "${state.errors}"
break;
}
}
}
}
For import of large data I suggest to take in consideration the use of Spring Batch. Is easy to integrate it in grails. You can try with this plugin or integrate it manually.
I have prepared a test case in SoapUI Open Source which loops over values in csv file and sends request for each set of values (taken care of by groovy script). I want to modify it so each thread for each new iteration uses value from next row of csv file.
import com.eviware.soapui.impl.wsdl.teststeps.*
def testDataSet = []
def fileName = "C:\\sSHhrTqA5OH55qy.csv"
new File(fileName).eachLine { line -> testDataSet.add( line.split(",") ) }
def myProps = new java.util.Properties();
myProps = testRunner.testCase.getTestStepByName("Properties");
def groovyUtils = new com.eviware.soapui.support.GroovyUtils( context );
def testCase = testRunner.testCase;
def testStep = testCase.getTestStepByName("TestRequest");
testRunner = new com.eviware.soapui.impl.wsdl.testcase.WsdlTestCaseRunner(testCase, null);
testStepContext = new com.eviware.soapui.impl.wsdl.testcase.WsdlTestRunContext(testStep);
while (true) {
for ( i in testDataSet ) {
myProps.setPropertyValue("parameter0",i[0]);
myProps.setPropertyValue("username",i[1]);
myProps.setPropertyValue("parameter1",i[2]);
myProps.setPropertyValue("password",i[3]);
testStep.getTestRequest().setUsername(myProps.getPropertyValue("username"))
testStep.getTestRequest().setPassword(myProps.getPropertyValue("password"))
testStep.run(testRunner, testStepContext);
}
}
I want to modify this script so each thread from the pool gets unique (next) unused value from data source
I tried to use newFixedThreadPool from java.util.concurrent as suggested here (Concurrency with Groovy), however I can't get it to work - either requests are duplicated or SoapUI crashes (I am new to concurrency).
Can you please help me to get it right?
I think this would work for you:
while (true) {
for ( i in testDataSet ) {
def th = Thread.start(){
myProps.setPropertyValue("parameter0",i[0]);
myProps.setPropertyValue("username",i[1]);
myProps.setPropertyValue("parameter1",i[2]);
myProps.setPropertyValue("password",i[3]);
testStep.getTestRequest().setUsername(myProps.getPropertyValue("username"))
testStep.getTestRequest().setPassword(myProps.getPropertyValue("password"))
testStep.run(testRunner, testStepContext);
}
th.join()
}
So, new threads would be created on each loop.
If you wanted to test out if its working you could place loginfo(s) in the code...
log.info("Thread Id: " + Thread.currentThread().getId() as String)
I don't see your point. SoapUi already gives you a datasource test step that accepts a csv file as input.
So once you have all these values you can transfer the properties and run the test.