I am experimenting with Gradle to build a few jars, rather than maintain a list of classes that hold EJBs so that I can deploy them separately I thought it might be neat to scan the classes when making the jar.
Rather than load the classes and use reflection to get the annotations I thought it may be simpler to scan the classes with asm, hence the chuncky ClassReader in one of the tasks.
I don't think this is the issue so can be ignored, basically I have 2 tasks that I use to define the contents of the jars, both report that different content is going into them via the eachFile print out, however when I look in the publish repository location both files and associated sha1 are identical.
Either Gradle is broken or, more likely, I've done something crazy but can't see what it is, can anyone help?
By the way if I disable the publish of either of the jar files the one that does get created is correct so I think it's something wrong with the publish rather than the jarring up, but could be wrong.
// ASM is used to interpret the class files, this avoids having to load all classes in the vm and use reflection
import org.objectweb.asm.*
task ejbJar(type: Jar) {
//outputs.upToDateWhen { false }
from "${project.buildDir}/classes/main"
eachFile { println "EJB server: ${name}" }
include getEjbClassFiles(project.buildDir)
}
task clientEjbJar(type: Jar) {
//outputs.upToDateWhen { false }
from "${project.buildDir}/classes/main/com/company/core/versioner"
eachFile { println "Client EJB ${name}" }
include '**/*'
}
artifacts {
archives clientEjbJar
archives ejbJar
}
String[] getEjbClassFiles(base) {
def includedFiles = []
def baseDir = project.file("${base}/classes/main")
def parentPath = baseDir.toPath()
if (baseDir.isDirectory()) {
baseDir.eachFileRecurse(groovy.io.FileType.FILES) { file ->
if(file.name.endsWith('.class')) {
//get hold of annotations in there --- org.objectweb.asm.Opcodes.ASM4
def reader = new ClassReader(file.bytes).accept(
new ClassVisitor(Opcodes.ASM4) {
public AnnotationVisitor visitAnnotation(String desc, boolean visible) {
if(desc.equals("Ljavax/ejb/Stateless;") ||
desc.equals("Ljavax/ejb/Stateful;")) {
includedFiles += parentPath.relativize(file.toPath())
}
return null //no interest in actually visiting the annotation values
}
},
ClassReader.SKIP_DEBUG | ClassReader.EXPAND_FRAMES | ClassReader.SKIP_FRAMES | ClassReader.SKIP_CODE
)
}
}
}
return includedFiles
}
publishing {
publications {
mypub(IvyPublication) {
artifact(ejbJar) {
name 'ejb'
}
artifact(clientEjbJar) {
name 'client-ejb'
}
}
}
repositories {
ivy {
name 'personal'
url "${ant['developer.repository']}/"
layout 'pattern', {
artifact "[organisation]/[module]/[artifact]/[revision]/[type]/[artifact]-[revision].[ext]"
ivy "[organisation]/[module]/[type]/[revision]/[type]/[type]-[revision].[ext]"
}
}
}
}
I did break the thing down into a simpler form as I thought it may be a Gradle bug.
The simplified form was:
apply plugin: 'java'
apply plugin: 'ivy-publish'
task bigJar(type: Jar) {
from "${rootDir}/src/main/resources"
include '**/*'
}
task smallJar(type: Jar) {
from "${rootDir}/src/main/resources/A/B"
include '**/*'
}
group 'ICantBeEmpty'
artifacts {
archives bigJar
archives smallJar
}
publishing {
publications {
mypub(IvyPublication) {
artifact(bigJar) { name 'biggie' }
artifact(smallJar) { name 'smallie' }
}
repositories {
ivy {
name 'personal'
url "c:/temp/gradletest"
layout 'pattern', {
artifact "[organisation]/[module]/[artifact]/[revision]/[type]/[artifact]-[revision].[ext]"
ivy "[organisation]/[module]/[type]/[revision]/[type]/[type]-[revision].[ext]"
}
}
}
}
}
This results in 2 files in c:/temp/gradletest/ICantBeEmpty/report-bug/biggie/unspecified/biggie-unspecified.jar and c:/temp/gradletest/ICantBeEmpty/report-bug/smallie/unspecified/smallie-unspecified.jar
Both of these files are identical, however I think I know why see my later answer.
Whilst looking at some configurations I noticed some odd behaviour that led me to a resolution of this issue, and it is a Gradle bug.
In my build I had a scratch task doing
configurations.archives.artifacts.each { println it }
This gave me 5 different lines output, however changing it to this
configurations.archives.artifacts.each { println it.file }
produced the same filename 5 times.
It turns out this is related to my issue, although the artifacts are there as separate entities the name used to uniquely identify them was the same so the same file was always chosen during a publish. The name of the artifacts is given by ${baseName}-${appendix}-${version}-${classifier}.${extension} by default in the java plugin. This means that if neither appendix or classifier is specified then the artifact will have the same name.
I tested this using the above sample code by adding an appendix name
task bigJar(type: Jar) {
appendix = 'big'
from "${rootDir}/src/main/resources"
include '**/*'
}
task smallJar(type: Jar) {
appendix = 'small'
from "${rootDir}/src/main/resources/A/B"
include '**/*'
}
Using this rather than the code from the question produces 2 different jars.
It's not a complete answer but is a good enough work around, if I add a new publication definition I can publish the artifacts that I want to to the location that I want, the only downside is that it will create another gradle task which isn't ideal.
publications {
mypub(IvyPublication) {
artifact(ejbJar) {
name 'ejb'
}
}
newpub(IvyPublication) {
artifact(clientEjbJar) {
name 'client-ejb'
}
}
}
The above answer works in the short term, however does reveal yet another short coming in the Gradle world enter link description here
Not sure Gradle is all it could be at the moment, and so far no one has answered my questions so maybe it's not that actively developed!!
I'm no expert in this part of Gradle, but the functionality you are using is marked as "incubating"; you are using the new publishing feature which might or might not be complete. Perhaps you should use the old way of doing things. You also seem to be mixing both ways by using the artifacts closure.
Related
I have a generic Artifatory repo that contains some zip files that I need to download and extract in an Android Studio project. I am new to Gradle but found that using an Ivy repository object maybe the best way to do this instead of just downloading the files using a Gradle task which calls curl.
I have managed to get a single zip file downloaded and extracted but soon as I try to add another dependency it overwrites my configuration. See the dependencies block in the code below.
Why is it not adding another dependency and is there a better way to do this?
app:build.gradle:
repositories {
ivy {
url property("artifactoryUrl")
credentials {
username property("artifactoryUser")
password property("artifactoryPassword")
}
authentication {
basic(BasicAuthentication)
}
patternLayout {
artifact '/[organisation]/[module]/third-party/[revision](.[ext])'
}
metadataSources {
artifact()
}
}
}
configurations {
thirdPartyDependencies
}
dependencies {
thirdPartyDependencies "artifactory:test-generic:FreeImage#zip"
thirdPartyDependencies "artifactory:test-generic:GLEW#zip" // thirdPartyDependencies has been overwritten
}
task CopyThirdPartyDependencies(type: Copy) {
def thirdPartyDirectory = getProject().getRootDir().toString() + "/third-party/"
println(configurations.thirdPartyDependencies.resolvedConfiguration.resolvedArtifacts)
configurations.thirdPartyDependencies.resolvedConfiguration.resolvedArtifacts.each { artifact ->
copy {
from zipTree(artifact.getFile())
into thirdPartyDirectory
}
}
}
preBuild.dependsOn(CopyThirdPartyDependencies)
I'm currently struggling with a build process in gradle. My goal is to not have a specific java class in the final .apk for a specific flavor.
The logic goes like this:
1.) before compiling my project delete MyClass.java and copy it to a temp folder
2.) after assembling the apk copy back MyClass.java to my original folder
3.) delete the temp folder
This happens only if I build a specific flavor, so it doesn't happen for all build variants. My code works perfectly when I build only one flavor and one build variant e.g. assembleFlavorRelease, but if I wan't to make my code work for multiple build types; if I run assembleFlavor it should build flavorDebug the same way it does flavorRelease.
However my logic goes trough only the first time and after that it stops, so flavorDebug is build with MyClass and it shouldn't be, while flavorRelease doesn't include MyClass in the .apk because my code runs the first time.
Here is what the code looks like:
task copyResource << {
copy {
from 'src/main/java/MyClass.java'
into 'src/temp'
}
}
task deleteResource << {
delete {
'src/main/java/MyClass.java'
}
}
task deleteTemp << {
delete {
'src/temp'
}
}
task copyBackToSource << {
copy {
from 'src/temp/MyClass.java'
into 'src/main/java'
}
deleteTemp.execute()
}
android.applicationVariants.all { variant ->
if (variant.name.contains('flavor')) {
deleteResource.dependsOn copyResource
variant.javaCompile.dependsOn deleteResource
variant.assemble.doLast {
copyBackToSource.execute()
}
}
}
I think that the directories which I use in my code are somehow locked when trying to execute the whole process the second time?
I feel that you are approaching the problem in the wrong way. Instead of thinking
Put the java file in src/main/java and remove it from flavorX
You should instead be approaching it as
Add an extra source directory to the source sets for flavorY and flavorZ
Once you approach the problem like this it becomes much easier
If you only want the file in one flavor, you can put the file in the flavor specific source folder using the built in conventions (ie src/flavorX/java)
If you want the file in more than one flavor you could put MyClass.java in src/common/java and do something like
android {
productFlavors {
flavor1 {
}
flavor2 {
}
flavor3 {
}
}
}
sourceSets {
flavor1.java = ['src/flavor1/java','src/common/java']
flavor2.java = ['src/flavor2/java','src/common/java']
}
Ok so I got the right solution from here: https://discuss.gradle.org/t/android-gradle-assemble-tasks/10711
If you want to avoid compiling a specific class then use this:
variant.javaCompile.exclude '**/SourceFileToExclude.java'
I would like feedback on the best practices for defining plugin tasks that depend on external state (i.e. defined in the build.gradle that referenced the plugin). I'm using extension objects and closures to defer accessing those settings until they're needed and available. I'm also interested in sharing state between tasks, e.g. configuring the outputs of one task to be the inputs of another.
The code uses "project.afterEvaluate" to define the tasks when the required settings have been configured through the extension object. This seems more complex than should be needed. If I move the code out of the "afterEvaluate", it gets compileFlag == null which isn't the external setting. If the code is changed again to use the << or doLast syntax, then it will get the external flag... but then it fails to work with type:Exec and other similarly helpful types.
I feel that I'm fighting Gradle in some ways, which means I don't understand better how to work well with it. The following is a simplified pseudo-code of what I'm using. This works but I'm looking to see if this can be simplified, or indeed what the best practices are. Also, the exception shouldn't be thrown unless the tasks are being executed.
apply plugin: MyPlugin
class MyPluginExtension {
String compileFlag = null
}
class MyPlugin implements Plugin<Project> {
void apply(Project project) {
project.extensions.create("myPluginConfig", MyPluginExtension)
project.afterEvaluate {
// Closure delays getting and checking flag until strictly needed
def compileFlag = {
if (project.myPluginConfig.compileFlag == null) {
throw new InvalidUserDataException(
"Must set compileFlag: myPluginConfig { compileFlag = '-flag' }")
}
return project.myPluginConfig.compileFlag
}
// Inputs for translateTask
def javaInputs = {
project.files(project.fileTree(
dir: project.projectDir, includes: ['**/*.java']))
}
// This is the output of the first task and input to the second
def translatedOutputs = {
project.files(javaInputs().collect { file ->
return file.path.replace('src/', 'build/dir/')
})
}
// Translates all java files into 'translatedOutputs'
project.tasks.create(name: 'translateTask', type:Exec) {
inputs.files javaInputs()
outputs.files translatedOutputs()
executable '/bin/echo'
inputs.files.each { file ->
args file.path
}
}
// Compiles 'translatedOutputs' to binary
project.tasks.create(name: 'compileTask', type:Exec, dependsOn: 'translateTask') {
inputs.files translatedOutputs()
outputs.file project.file(project.buildDir.path + '/compiledBinary')
executable '/bin/echo'
args compileFlag()
translatedOutputs().each { file ->
args file.path
}
}
}
}
}
I'd look at this problem another way. It seems like what you want to put in your extension is really owned by each of your tasks. If you had something that was a "global" plugin configuration option, would it be treated as an input necessarily?
Another way of doing this would have been to use your own SourceSets and wire those into your custom tasks. That's not quite easy enough yet, IMO. We're still pulling together the JVM and native representations of sources.
I'd recommend extracting your Exec tasks as custom tasks with a #TaskAction that does the heavy lifting (even if it just calls project.exec {}). You can then annotate your inputs with #Input, #InputFiles, etc and your outputs with #OutputFiles, #OutputDirectory, etc. Those annotations will help auto-wire your dependencies and inputs/outputs (I think that's where some of the fighting is coming from).
Another thing that you're missing is if the compileFlag effects the final output, you'd want to detect changes to it and force a rebuild (but not a re-translate).
I simplified the body of the plugin class by using the Groovy .with method.
I'm not completely happy with this (I think the translatedFiles could be done differently), but I hope it shows you some of the best practices. I made this a working example (as long as you have a src/something.java) by implementing the translate as a copy/rename and the compile as something that just creates an 'executable' file (contents is just the list of the inputs). I've also left your extension class in place to demonstrate the "global" plug-in config. Also take a look at what happens with compileFlag is not set (I wish the error was a little better).
The translateTask isn't going to be incremental (although, I think you could probably figure out a way to do that). So you'd probably need to delete the output directory each time. I wouldn't mix other output into that directory if you want to keep that simple.
HTH
apply plugin: 'base'
apply plugin: MyPlugin
class MyTranslateTask extends DefaultTask {
#InputFiles FileCollection srcFiles
#OutputDirectory File translatedDir
#TaskAction
public void translate() {
// println "toolhome is ${project.myPluginConfig.toolHome}"
// translate java files by renaming them
project.copy {
includeEmptyDirs = false
from(srcFiles)
into(translatedDir)
rename '(.+).java', '$1.m'
}
}
}
class MyCompileTask extends DefaultTask {
#Input String compileFlag
#InputFiles FileCollection translatedFiles
#OutputDirectory File outputDir
#TaskAction
public void compile() {
// write inputs to the executable file
project.file("$outputDir/executable") << "${project.myPluginConfig.toolHome} $compileFlag ${translatedFiles.collect { it.path }}"
}
}
class MyPluginExtension {
File toolHome = new File("/some/sane/default")
}
class MyPlugin implements Plugin<Project> {
void apply(Project project) {
project.with {
extensions.create("myPluginConfig", MyPluginExtension)
tasks.create(name: 'translateTask', type: MyTranslateTask) {
description = "Translates all java files into translatedDir"
srcFiles = fileTree(dir: projectDir, includes: [ '**/*.java' ])
translatedDir = file("${buildDir}/dir")
}
tasks.create(name: 'compileTask', type: MyCompileTask) {
description = "Compiles translated files into outputDir"
translatedFiles = fileTree(tasks.translateTask.outputs.files.singleFile) {
includes [ '**/*.m' ]
builtBy tasks.translateTask
}
outputDir = file("${buildDir}/compiledBinary")
}
}
}
}
myPluginConfig {
toolHome = file("/some/custom/path")
}
compileTask {
compileFlag = '-flag'
}
I have a gradle project that has java applications as well as android applications.
root/
build.gradle
settings.gradle
java1/
java2/
android1/
android2/
java3/
etc.
What is the best practice for structuring my build script? I am a total gradle novice and am migrating the project from maven to gradle.
I wanted to do something instead of
configure(subprojects) {}
to apply plugins and other specific things.
such as
configure(java1, java2, java3) { // java specifics }
configure(android1, android2) { // android specifics }
I am probably approaching this from the wrong way.
More explicitly I need to apply the plugin java only for the java projects and the android plugin for the android projects.
configure(subprojects.findAll {it.name == "java1" || it.name == "java2"}) {
Under the filtering section in the guide
Hope this helps someone else out.
There are multiple ways, depending on what you want... Some examples:
// Configures just project java1
project(":java1") { ... }
// Configures projects java1 and java2
["java1","java2"].each { name ->
project(":$name") { ... }
}
You can use normal groovy to find/iterate over all the projects.
Another option:
configure([ project(':sub1'), project(':sub2'), ... ]) {
...
}
The shortest and easiest option:
configure(subprojects.findAll()) {
if (it.name.equals('MY_PROJECT')) {
...
} else {
...
}
}
Another approach...
In the settings.gradle you do define your projects like this:
gradle.ext.javaProjects=[]
gradle.ext.androidProjects=[]
javaProject('java1')
javaProject('java2')
javaProject('java3')
androidProject('android1')
androidProject('android2')
def javaProject(String name) {
gradle.ext.javaProjects.add(name)
include ":$name"
}
def androidProject(String name) {
gradle.ext.androidProjects.add(name)
include ":$name"
}
Now you can reference those in your root build.gradle:
def javaProjects = subprojects.findAll {gradle.ext.javaProjects.contains(it.name)};
def androidProjects = subprojects.findAll {gradle.ext.javaProjects.contains(it.name)};
configure(javaProjects) {
...
}
configure(androidProjects) {
...
}
Maybe thats overkill... but i usually have the project definition method in my settings.gradle anyway. E.g. if you want to put all java projects under a java folder you could define things like this:
def javaProject(String name) {
gradle.ext.javaProjects.add(name)
include ":$name"
project(":$name").projectDir = file('java/'+ name)
}
Is there a way I can easily make the processing of FileTree files in a smart way in gradle tasks? I basically need to wait for the execution of all files, much like what you can do with GPars, but how do I do this gradle with FileTree?
task compressJs(dependsOn: [copyJsToBuild]) << {
println 'Minifying JS'
fileTree {
from 'build/js'
include '**/*.js'
}.visit { element ->
if (element.file.isFile()) {
println "Minifying ${element.relativePath}"
ant.java(jar: "lib/yuicompressor-2.4.6.jar", fork: true) {
arg(value: "build/js/${element.relativePath}")
arg(value: "-o")
arg(value: "build/js/${element.relativePath}")
}
}
}
}
It would be lovely if I could do something like .visit{}.async(wait:true), but my googling turned up nothing. Is there a way I can easily make this multi-threaded? The processing of one element has no effect on the processing of any other element.
Before thinking about going multi-threaded, I'd try the following:
Run everything in the same JVM. Forking a new JVM for each input file is very inefficient.
Make the compressJs task incremental so that it only executes if some input file has changed since the previous run.
Run the minifier directly rather than via Ant (saves creation of a new class loader for each input file; not sure if it matters).
If this still leaves you unhappy with the performance, and you can't use a more performant minifier, you can still try to go multi-threaded. Gradle won't help you there (yet), but libraries like GPars or the Java Fork/Join framework will.
The GPars solution. Note that the compress() function could be modified to properly accept source dir/target dir/etc, but since all my names are consistent, I'm just using the one argument for now. I was able to cut my build time from 7.3s to 5.4s with only 3 files being minified. I've seen build times spiral out of control, so I'm always wary of performance with this kind of behavior.
import groovyx.gpars.GParsPool
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath 'org.codehaus.gpars:gpars:0.12'
}
}
def compress(String type) {
def elementsToMinify = []
fileTree {
from type
include "**/*.$type"
}.visit { element ->
if (element.file.isFile()) {
elementsToMinify << element
}
}
GParsPool.withPool(8) {
elementsToMinify.eachParallel { element ->
println "Minifying ${element.relativePath}"
def outputFileLocation = "build/$type/${element.relativePath}"
new File(outputFileLocation).parentFile.mkdirs()
ant.java(jar: "lib/yuicompressor-2.4.6.jar", fork: true) {
arg(value: "$type/${element.relativePath}")
arg(value: "-o")
arg(value: outputFileLocation)
}
}
}
}
task compressJs {
inputs.dir new File('js')
outputs.dir new File('build/js')
doLast {
compress('js')
}
}
task compressCss {
inputs.dir new File('css')
outputs.dir new File('build/css')
doLast {
compress('css')
}
}