I am currently working on Scriptrunner for bitbucket. I made a pre-receive hook that check the push, if there is a file named "conanfile.py" it checks the inside of the file and compare it with a regex I made. My problem is that whenever I try to push to test this, my terminal tells me there is a mistake :
Total 13 (delta 8), reused 0 (delta 0), pack-reused 0
remote: Push blocked
remote: Conanfile.py lines starting with "self.requires" should match regex
To https://bitbucket.ecatools.io/scm/ttdo/test_branch.git
! [remote rejected] testscript -> testscript (pre-receive hook declined)
While Scriptrunner tells me that everything is fine :
I am quite lost at what to do and I am here asking for help please.
I tried several ways of doing this while learning Groovy, here is the code of the pre-receive hook (i deleted the import to not take too much space, tell me if you need them) :
#ShortTextInput(label = 'regex', description = 'Regex expression used to verify file content')
String regexValue
def pattern = Pattern.compile(".*requires\\(\"[^\\d|_|\\-](\\S)+/\\[(\\d)(\\.\\d)*\\]#(xxx|xxxxx)/stable\"")
def commitService = ComponentLocator.getComponent(CommitService)
def contentService = ComponentLocator.getComponent(ContentService)
def commits = refChanges.getCommits(repository) as List<Commit>
// filePaths to commit Id, if file was modified in multiple commits, take the last commit id
def filePaths = [:] as Map<String, String>
commits.reverseEach { commit ->
def changesRequest = new ChangesRequest.Builder(repository, commit.id).build()
commitService.streamChanges(changesRequest) { Change change ->
if (change.path.toString().endsWith("conanfile.py")) {
filePaths[change.path.toString()] = commit.id
}
true
}
}
def result = filePaths.keySet().collect { filePath ->
def fileStream = new ByteArrayOutputStream()
contentService.streamFile(repository, filePaths[filePath], filePath) {
fileStream
}
fileStream.toString()
}.every {
it.split("\n").any { line ->
line.startsWith("self.requires") && pattern.matcher(line).find()
}
}
result ? RepositoryHookResult.accepted('push approved') : RepositoryHookResult.rejected('Push blocked', 'Conanfile.py lines starting with "self.requires" should match regex')
I have forked an android library on github, applied some fixes to it and now would like to get a snapshot version as a dependency to my own project, but the build task in jitpack.io fails
* What went wrong:
Execution failed for task ':project_name:signReleasePublication'.
> path may not be null or empty string. path=''
The project has publish-mavencentral.gradle script setup for publishing task with signing properties read from a file or envirnment variables which I guess is causing the issue
ext["signing.keyId"] = ''
ext["signing.password"] = ''
ext["signing.secretKeyRingFile"] = ''
ext["ossrhUsername"] = ''
ext["ossrhPassword"] = ''
ext["sonatypeStagingProfileId"] = ''
File secretPropsFile = project.rootProject.file('local.properties')
if (secretPropsFile.exists()) {
Properties p = new Properties()
p.load(new FileInputStream(secretPropsFile))
p.each { name, value ->
ext[name] = value
}
} else {
ext["signing.keyId"] = System.getenv('SIGNING_KEY_ID')
ext["signing.password"] = System.getenv('SIGNING_PASSWORD')
ext["signing.secretKeyRingFile"] = System.getenv('SIGNING_SECRET_KEY_RING_FILE')
ext["ossrhUsername"] = System.getenv('OSSRH_USERNAME')
ext["ossrhPassword"] = System.getenv('OSSRH_PASSWORD')
ext["sonatypeStagingProfileId"] = System.getenv('SONATYPE_STAGING_PROFILE_ID')
}
signing {
sign publishing.publications
}
Tried adding the required() directive, but it did not work
signing {
required { hasProperty("signing.keyId") }
sign publishing.publications
}
How to skip signing if the properties are empty?
The problem was in the key composite name. Renamed signing.keyId to signingKeyId and required() started to work as expected.
My code delete just files which parse to Jenkins name in the file. I would like to delete file based on the author (Jenkins) in the last commit. What is the best solution for that?
def changelogPath = "C:\\test"
def PackID = "test"
def delete(String changelogPath, String PackID) {
String folderPath = "$changelogPath"+ "\\" + "$PackID"
new File(folderPath).eachFile(FileType.FILES) { file ->
if (file.name.contains('Jenkins')) file.delete()
}
delete(changelogPath, PackID)
In order to find all files that have been changed with a certain commit, you need a diff of that commit with its predecessor.
You can let JGit compute a list of DiffEntries like this:
ObjectReader reader = git.getRepository().newObjectReader();
CanonicalTreeParser oldTreeIter = new CanonicalTreeParser();
ObjectId oldTree = git.getRepository().resolve( "HEAD^{tree}" );
oldTreeIter.reset( reader, oldTree );
CanonicalTreeParser newTreeIter = new CanonicalTreeParser();
ObjectId newTree = git.getRepository().resolve( "HEAD~1^{tree}" );
newTreeIter.reset( reader, newTree );
DiffFormatter df = new DiffFormatter( new ByteArrayOutputStream() );
df.setRepository( git.getRepository() );
List<DiffEntry> entries = df.scan( oldTreeIter, newTreeIter );
Each DiffEntry has a path that denotes the file which was added, changed, or deleted. The path is relative to the root of the working directory of the repository. Actually, there is an oldPath and newPath, see the JavaDoc which one to use when.
See also here for a general overview of JGit's diff API: http://www.codeaffine.com/2016/06/16/jgit-diff/
I would like to write a system groovy script which inspects the queued jobs in Jenkins, and extracts the build parameters (and build cause as a bonus) supplied as the job was scheduled. Ideas?
Specifically:
def q = Jenkins.instance.queue
q.items.each { println it.task.name }
retrieves the queued items. I can't for the life of me figure out where the build parameters live.
The closest I am getting is this:
def q = Jenkins.instance.queue
q.items.each {
println("${it.task.name}:")
it.task.properties.each { key, val ->
println(" ${key}=${val}")
}
}
This gets me this:
4.1.next-build-launcher:
com.sonyericsson.jenkins.plugins.bfa.model.ScannerJobProperty$ScannerJobPropertyDescriptor#b299407=com.sonyericsson.jenkins.plugins.bfa.model.ScannerJobProperty#5e04bfd7
com.chikli.hudson.plugin.naginator.NaginatorOptOutProperty$DescriptorImpl#40d04eaa=com.chikli.hudson.plugin.naginator.NaginatorOptOutProperty#16b308db
hudson.model.ParametersDefinitionProperty$DescriptorImpl#b744c43=hudson.mod el.ParametersDefinitionProperty#440a6d81
...
The params property of the queue element itself contains a string with the parameters in a property file format -- key=value with multiple parameters separated by newlines.
def q = Jenkins.instance.queue
q.items.each {
println("${it.task.name}:")
println("Parameters: ${it.params}")
}
yields:
dbacher params:
Parameters:
MyParameter=Hello world
BoolParameter=true
I'm no Groovy expert, but when exploring the Jenkins scripting interface, I've found the following functions to be very helpful:
def showProps(inst, prefix="Properties:") {
println prefix
for (prop in inst.properties) {
def pc = ""
if (prop.value != null) {
pc = prop.value.class
}
println(" $prop.key : $prop.value ($pc)")
}
}
def showMethods(inst, prefix="Methods:") {
println prefix
inst.metaClass.methods.name.unique().each {
println " $it"
}
}
The showProps function reveals that the queue element has another property named causes that you'll need to do some more decoding on:
causes : [hudson.model.Cause$UserIdCause#56af8f1c] (class java.util.Collections$UnmodifiableRandomAccessList)
I have the following Groovy script which deletes all builds of a given Jenkins job except one build number that user provides (i.e. wants to retain).
/*** BEGIN META {
"name" : "Bulk Delete Builds except the given build number",
"comment" : "For a given job and a given build number, delete all build except the user provided one.",
"parameters" : [ 'jobName', 'buildNumber' ],
"core": "1.409",
"authors" : [
{ name : "Arun Sangal" }
]
} END META **/
// NOTE: Uncomment parameters below if not using Scriptler >= 2.0, or if you're just pasting the script in manually.
// ----- Logic in this script takes 5000 as the infinite number, decrease / increase this value from your own experience.
// The name of the job.
//def jobName = "some-job"
// The range of build numbers to delete.
//def buildNumber = "5"
def lastBuildNumber = buildNumber.toInteger() - 1;
def nextBuildNumber = buildNumber.toInteger() + 1;
import jenkins.model.*;
import hudson.model.Fingerprint.RangeSet;
def jij = jenkins.model.Jenkins.instance.getItem(jobName);
println("Keeping Job_Name: ${jobName} and build Number: ${buildNumber}");
println ""
def setBuildRange = "1-${lastBuildNumber}"
def range = RangeSet.fromString(setBuildRange, true);
jij.getBuilds(range).each { it.delete() }
println("Builds have been deleted - Range: " + setBuildRange)
setBuildRange = "${nextBuildNumber}-5000"
range = RangeSet.fromString(setBuildRange, true);
jij.getBuilds(range).each { it.delete() }
println("Builds have been deleted - Range: " + setBuildRange)
This works well for any Jenkins job. For ex: If your Jenkins job name is "TestJob" and you have 15 builds i.e. build# 1 to build 15 in Jenkins, and you want to delete all but retain build# 13, then this script will delete the builds (build# 1-12 and 14-15 - even if you mark any build as "Keep this build forever") and only keep build#13.
Now, what I want is:
what should I change in this script to not delete a build - if a build is marked in Jenkins as "Keep this build forever". I tried the script and it deleted that keep forever build too.
Lets say, if I'm using "Build name setter plugin" in Jenkins, which can give me build names as what name I want i.e. instead of getting just build as build#1 or #2, or #15, I will get build as build# 2.75.0.1, 2.75.0.2, 2.75.0.3, ..... , 2.75.0.15 (as I would have set the build name/description as use some variable which contains 2.75.0 (as a release version value) and suffixed it with the actual Jenkins job's build number i.e. the last 4th digit - ex: set the name as:
${ENV,var="somepropertyvariable"}.${BUILD_NUMBER}
In this case, I'll start getting Jenkins builds as 2.75.0.1 to 2.75.0.x (where x is the last build# of that release (2.75.0)). Similarly, when I'll change the property release version to next i.e. 2.75.1 or 2.76.0, then the same Jenkins job will start giving me builds as 2.75.1.0, 2.75.1.1, ...., 2.75.1.x or 2.76.0.1, 2.76.0.2, ...., 2.76.0.x and so on. During the release version change, let say, our build will start from 1 again (as I mentioned above for 2.75.1 and 2.76.0 release versions).
In this case, if my Jenkins job's build history (shows all builds for 2.75.0.x, 2.75.1.x and 2.76.0.x), then what change should I make in this script to include a 3rd parameter/argument. This 3rd argument will take release /version value i.e. either 2.75.0 or 2.75.1 or 2.76.0 and then this script should delete build numbers on that release only (and should NOT delete other release's builds).
If you want to test whether a build has been marked permanent, use
if (!build.isKeepLog()) {
// Build can be deleted
} else {
// Build is marked permanent
}
I think you should be able to use the getName() method on each build to check whether you should delete a given build. The API JavaDoc can be fairly obscure, so I often go on GitHub and look through the code for a Jenkins plugin that's doing something similar to what I need. The public Scriptler repository can be useful, too.
Final Answer: This includes deleting the build artifacts from Artifactory as well using Artifactor's REST API call. This script will delete Jenkins/Artifactory builds/artifacts of a given Release/Version (as sometimes over the time - a given Jenkins job can create multiple release / version builds for ex: 2.75.0.1, 2.75.0.2, 2.75.0.3,....,2.75.0.54, 2.76.0.1, 2.76.0.2, ..., 2.76.0.16, 2.76.1.1, 2.76.1.2, ...., 2.76.1.5). In this case, for every new release of that job, we start the build# from 1 fresh. If you have to delete the all builds except one / even all (change the script a little bit for your own needs) and don't change older/other release builds, then use the following script.
Scriptler Catalog link: http://scriptlerweb.appspot.com/script/show/103001
Enjoy!
/*** BEGIN META {
"name" : "Bulk Delete Builds except the given build number",
"comment" : "For a given job and a given build numnber, delete all builds of a given release version (M.m.interim) only and except the user provided one. Sometimes a Jenkins job use Build Name setter plugin and same job generates 2.75.0.1 and 2.76.0.43",
"parameters" : [ 'jobName', 'releaseVersion', 'buildNumber' ],
"core": "1.409",
"authors" : [
{ name : "Arun Sangal - Maddys Version" }
]
} END META **/
import groovy.json.*
import jenkins.model.*;
import hudson.model.Fingerprint.RangeSet;
import hudson.model.Job;
import hudson.model.Fingerprint;
//these should be passed in as arguments to the script
if(!artifactoryURL) throw new Exception("artifactoryURL not provided")
if(!artifactoryUser) throw new Exception("artifactoryUser not provided")
if(!artifactoryPassword) throw new Exception("artifactoryPassword not provided")
def authString = "${artifactoryUser}:${artifactoryPassword}".getBytes().encodeBase64().toString()
def artifactorySettings = [artifactoryURL: artifactoryURL, authString: authString]
if(!jobName) throw new Exception("jobName not provided")
if(!buildNumber) throw new Exception("buildNumber not provided")
def lastBuildNumber = buildNumber.toInteger() - 1;
def nextBuildNumber = buildNumber.toInteger() + 1;
def jij = jenkins.model.Jenkins.instance.getItem(jobName);
def promotedBuildRange = new Fingerprint.RangeSet()
promotedBuildRange.add(buildNumber.toInteger())
def promoteBuildsList = jij.getBuilds(promotedBuildRange)
assert promoteBuildsList.size() == 1
def promotedBuild = promoteBuildsList[0]
// The release / version of a Jenkins job - i.e. in case you use "Build name" setter plugin in Jenkins for getting builds like 2.75.0.1, 2.75.0.2, .. , 2.75.0.15 etc.
// and over the time, change the release/version value (2.75.0) to a newer value i.e. 2.75.1 or 2.76.0 and start builds of this new release/version from #1 onwards.
def releaseVersion = promotedBuild.getDisplayName().split("\\.")[0..2].join(".")
println ""
println("- Jenkins Job_Name: ${jobName} -- Version: ${releaseVersion} -- Keep Build Number: ${buildNumber}");
println ""
/** delete the indicated build and its artifacts from artifactory */
def deleteBuildFromArtifactory(String jobName, int deleteBuildNumber, Map<String, String> artifactorySettings){
println " ## Deleting >>>>>>>>>: - ${jobName}:${deleteBuildNumber} from artifactory"
def artifactSearchUri = "api/build/${jobName}?buildNumbers=${deleteBuildNumber}&artifacts=1"
def conn = "${artifactorySettings['artifactoryURL']}/${artifactSearchUri}".toURL().openConnection()
conn.setRequestProperty("Authorization", "Basic " + artifactorySettings['authString']);
conn.setRequestMethod("DELETE")
if( conn.responseCode != 200 ) {
println "Failed to delete the build artifacts from artifactory for ${jobName}/${deleteBuildNumber}: ${conn.responseCode} - ${conn.responseMessage}"
}
}
/** delete all builds in the indicated range that match the releaseVersion */
def deleteBuildsInRange(String buildRange, String releaseVersion, Job theJob, Map<String, String> artifactorySettings){
def range = RangeSet.fromString(buildRange, true);
theJob.getBuilds(range).each {
if ( it.getDisplayName().find(/${releaseVersion}.*/)) {
println " ## Deleting >>>>>>>>>: " + it.getDisplayName();
deleteBuildFromArtifactory(theJob.name, it.number, artifactorySettings)
it.delete();
}
}
}
//delete all the matching builds before the promoted build number
deleteBuildsInRange("1-${lastBuildNumber}", releaseVersion, jij, artifactorySettings)
//delete all the matching builds after the promoted build number
deleteBuildsInRange("${nextBuildNumber}-${jij.nextBuildNumber}", releaseVersion, jij, artifactorySettings)
println ""
println("- Builds have been successfully deleted for the above mentioned release: ${releaseVersion}")
println ""
OK - Solution for my question 2 is here: I'm still working on fixing question 1.
http://scriptlerweb.appspot.com/script/show/102001
bulkDeleteJenkinsBuildsExceptOne_OfAGivenRelease.groovy
/*** BEGIN META {
"name" : "Bulk Delete Builds except the given build number",
"comment" : "For a given job and a given build numnber, delete all builds of a given release version (M.m.interim) only and except the user provided one. Sometimes a Jenkins job use Build Name setter plugin and same job generates 2.75.0.1 and 2.76.0.43",
"parameters" : [ 'jobName', 'releaseVersion', 'buildNumber' ],
"core": "1.409",
"authors" : [
{ name : "Arun Sangal" }
]
} END META **/
// NOTE: Uncomment parameters below if not using Scriptler >= 2.0, or if you're just pasting the script in manually.
// ----- Logic in this script takes 5000 as the infinite number, decrease / increase this value from your own experience.
// The name of the job.
//def jobName = "some-job"
// The release / version of a Jenkins job - i.e. in case you use "Build name" setter plugin in Jenkins for getting builds like 2.75.0.1, 2.75.0.2, .. , 2.75.0.15 etc.
// and over the time, change the release/version value (2.75.0) to a newer value i.e. 2.75.1 or 2.76.0 and start builds of this new release/version from #1 onwards.
//def releaseVersion = "2.75.0"
// The range of build numbers to delete.
//def buildNumber = "5"
def lastBuildNumber = buildNumber.toInteger() - 1;
def nextBuildNumber = buildNumber.toInteger() + 1;
import jenkins.model.*;
import hudson.model.Fingerprint.RangeSet;
def jij = jenkins.model.Jenkins.instance.getItem(jobName);
//def build = jij.getLastBuild();
println ""
println("- Jenkins Job_Name: ${jobName} -- Version: ${releaseVersion} -- Keep Build Number: ${buildNumber}");
println ""
println " -- Range before given build number: ${buildNumber}"
println ""
def setBuildRange = "1-${lastBuildNumber}"
def range = RangeSet.fromString(setBuildRange, true);
jij.getBuilds(range).each {
if ( it.getDisplayName().find(/${releaseVersion}.*/)) {
println " ## Deleting >>>>>>>>>: " + it.getDisplayName();
// Trying to find - how to NOT delete a build in Jenkins if it's marked as "keep this build forever". If someone has an idea, please update this script with a newer version in GitHub.
//if ( !build.isKeepLog()) {
it.delete();
//} else {
// println "build -- can't be deleted as :" + build.getWhyKeepLog();
//}
}
}
println ""
println " -- Range after given build number: ${buildNumber}"
println ""
setBuildRange = "${nextBuildNumber}-5000"
range = RangeSet.fromString(setBuildRange, true);
jij.getBuilds(range).each {
if ( it.getDisplayName().find(/${releaseVersion}.*/)) {
println " ## Deleting >>>>>>>>>: " + it.getDisplayName();
it.delete();
}
}
println ""
println("- Builds have been successfully deleted for the above mentioned release: ${releaseVersion}")
println ""
One can also call this script via a Jenkins job (requires 3 parameters as mentioned in the scriptler script) -OR call it from browser as well: using the following link:
http ://YourJenkinsServerName:PORT/job/Some_Jenkins_Job_That_You_Will_Create/buildWithParameters?jobName=Test_AppSvc&releaseVersion=2.75.0&buildNumber=15