Downloading Dependences From Private Amazon S3 Repository with Gradle - groovy

I am looking to add Groovy support to an existing java project so that I can seemlessly compile mixed Java and Groovy code using invokedynamic so that I can get Java-like execution speed without needing to waste excessive amounts of time with verbose Java syntax
After reading that the gmaven plugin no longer supports compilation -and that the groovy eclipse compiler plugin doesn't yet support invokedynamic, I asked myself, why would I want to continue using Maven if it compiles Groovy code that is needlessly slow?
Consequently, I decided I would try scrapping maven for Gradle so that I could obtain faster code while also porting some python deployment scripts to Gradle tasks so as to only need one codebase.
I have some libraries stored on a simple password protected s3 maven repository (in order to avoid needing enterprise overkill like artifactory). After doing some basic research, I have found that Gradle has no built in support for adding in custom dependency management as determined by this stack overlow question and this support forums post.
I did manage to find a s3 plugin for gradle -but it doesn't deal with management of dependencies.
If the whole point of Gradle is to be more flexible than Maven and if the core purpose of a dependency management/ build system is to effectively manage dependencies from a variety of sources-then lack of support for custom repositories appears to be a fairly significant significant design flaw which makes any issues I have encounted with Maven thus far pale in comparison.
However, it is quite possible that I am missing something, and I have already invested several hours learning Gradle -so I figured I would see if there is some reasonable way to emulate dependency management for these s3 dependencies until Gradle developers fix this critical issue. Otherwise I will have to conclude that I am better off just using Maven and tolerating slower Groovy code until the compiler plugin supports invokedynamic.
Basically I need a solution that does the following:
Downloads dependencies and transitive dependencies to the gradle cache
Doesn't require me to hardcode the path to the gradle cache -so that my build script is platform independent.
Doesn't download the dependencies again if they are already in the cache.
Works with a multi-module project.
However, I cannot find anything in the documentation that would even give me a clue as to where to begin:

Gradle 2.4 has native support for S3 repositories. Both downloading dependencies and publishing artifacts.
To download with IAM credentials (paraphrased from the link above):
repositories {
maven {
url "s3://someS3Bucket/path/to/repo/root"
credentials(AwsCredentials) {
accessKey 'access key'
secretKey 'secret key'
}
}
}
Then specify your dependencies as usual.

You don't need any custom repository support to make this work. Just declare a maven repository with the correct URL. If the repository works when used from Maven, it will also work with Gradle. (Uploading may be a different matter.)

You can use S3 and http
repositories {
mavenCentral()
ivy {
url "https://s3-eu-west-1.amazonaws.com/my-bucket"
layout "pattern", {
artifact "[artifact]-[revision].[ext]"
m2compatible = true
}
}
}
Name the jar in S3 to name-rev.jar (joda-time-3.2.jar) in my-bucket.
Also upload a pom file.
And in S3 give all permission to Download the jar and pom.

Related

composite builds broken on android or did I do this wrong?

In 10 minutes, I created an empty android library and an empty android application in this git repository
https://github.com/deanhiller/compositeAndroid
As seen in my last commit, I quickly convert the android application do depend on the library via gradle's awesome composite build feature (We use this feature a TON in our monorepo so loading a project loads all the libraries source code that it uses as well). Our library is shared amongst a few projects.
I cd into compositeAndroid/MyApplication and run ./gradlew build and it fails with
* What went wrong:
Could not determine the dependencies of task ':app:mergeReleaseAssets'.
> Could not resolve all task dependencies for configuration ':app:releaseRuntimeClasspath'.
> Could not resolve com.tray.android:MyLib.
Required by:
project :app
> No matching configuration of project :MyLib was found. The consumer was configured to find a runtime of a component, preferably optimized for Android, as well as attribute 'com.android.build.api.attributes.BuildTypeAttr' with value 'release', attribute 'com.android.build.api.attributes.AgpVersionAttr' with value '7.1.0' but:
- None of the consumable configurations have attributes.
I am not sure how to work around this. I have a work around to publish/consume but would much prefer composite builds as it brings the source of libraries into intellij cleanly.
Why is composite builds not working? Is there something special I have to do for android projects? The above repo I setup in 10 minutes with those 2 projects(brand new).
You can always clone and play with it yourself as well. (We will actually be releasing our monorepo open-source template however it is not working to well with android just yet).
After looking into the code under MyLib folder in the repository you shared here - it seems you've opened a regular project and intend to use it as a library
Can you please follow the steps required here and test it under a new module?
Hint: your build should result with an aar file

Docker security scan detects vulnerability in gradle 7.4.1

Creating a docker image with gradle 7.4.1 triggers the security scan which shows vulnerability CVE-2020-36518. How can this particular jar file within the gradle package be updated?
I would just reject the security issue, explaining that it is not possible to exploit the vulnerability as the Gradle build runs isolated on controlled input, and is not accessible by any potential attackers.
(Assuming this is the case, of cause, and you don't have a custom Gradle plugin that reads untrusted JSON documents using Jackson from the Gradle classpath. But even then, all you are risking is a denial-of-service on the build.)
Fiddling around with jar files in external tools could easily lead to problems hard to debug later. But if you like, you could create an issue for them, asking if they could bump the Jackson version to avoid unnecessary noise from security scans like this. There is an example of that here.

What is the difference between maven and npm respository?

I want to know the differences between npm and maven respository
Same tool, different language?
Maven is the most popular build and dependency resolution tool for Java, just like NPM is for JS. But it's not just the same tool for a different language. There are obviously huge differences between Java and JS builds, and these differences are directly visible in the way Maven operates. For example, while many JS tools rely on Git to do some heavy-lifting, Maven works with custom filesystem-based Maven repositories, as Maven predates Git and needs to handle binary artifacts, which Git historically didn't handle well. In Maven there's a clear separation between sources and binaries, while they are often the same thing in JS world.
Maven basics
Maven in its purest form follows a declarative model, where pom.xml (similar to package.json) defines different properties of the build, but contains no scripts. The disadvantage is it can be a challenge to fine-tune some aspects of the build without using scripts as you have to rely on plugins. The advantage is it can be easier to understand other builds just by looking at pom.xml, as they usually follow the same approach without too much customization. Gradle is a popular Groovy-based tool built on top of Maven standards and conventions, and is specifically designed to simplify pom.xml and break this "no script" barrier.
Referencing your dependencies
Similarly to package.json, you don't work with pom.xml of your dependency directly, but rather define dependency coordinates and let your build tool handle the rest. In Maven the basic form of these coordinates is GAV (groupId, artifactId, version).
Flat dependency tree?
Based on comments in the other answer, Maven provides "flat dependency tree", not "nested dependency tree" that NPM provides by default. Maven does not allow multiple versions of the same dependency. If it happens that different versions are requested, Maven uses dependency resolution to pick a single version. This means that sometimes your transitive dependencies will get a different version than they require, but there are ways to manage this. However, this limitation comes from Java, not Maven, as (normally) in Java a class loader will only provide access to a single class definition even if multiple definitions are found on the classpath. Since Java is not particularly good at handling this, Maven tries to avoid this scenario in the first place.
Note: since npm v3 the dependencies are flatten. The alternative package manager yarn also does the same.
Maturity
Furthermore, Maven is considerably older than NPM, has a larger user base, huge number of custom plugins, and so far could probably be considered more mature overall. Sometimes Maven is used for non-Java or even polyglot projects, as there are plugins for handling other languages or specific environments, such as Android. There are plugins that bridge Maven and other build tools, such as frontend-maven-plugin that actually handles multiple JS build tools.
NPM is focused on JavaScript while Maven is focused on Java and JVM derived (that is Scala, Kotlin, Groovy).
So comparing them doesn't make sense at all as these are not concurrent.
While comparing Maven to Gradle (Java world concurrent) or NPM to Yarn (JavaScript world concurrent) makes more sense.
Now Maven and NPM have closed features. Which is not surprising : good recipes spread beyond a language.
Here are some common points/features:
- these are package(JavaScript term)/ dependency (Maven term) managers that works with local/remote repositories.
- these are also a way to manage dependencies in your projects and to execute build tasks for them.

CodeNarc Maven Plugin

We have a project that uses Groovy extensively and we use Maven to build our artifacts. (IntelliJ as our IDE)
We wanted to incorporate some automated code-style checking, and thought we might use codecarc-maven-plugin. However, since that was from Codehaus, which gone now, is the plugin actively supported somewhere else?
Any other good options to run a Groovy style checker automatically during a Maven build?
That's a good catch. I'll add a pull request to update the website link. You can find the new plugin information on GitHub: https://github.com/gleclaire/codenarc-maven-plugin

Do I need to be careful which Maven Repositories I hook into?

Generally speaking, should one only add the central Maven Repository to a pom.xml + optionally any local Maven Repositories ? In theory (I think?) anybody can set up a repository - is there a 'Maven Repository<->Maven Repository' circle of trust or something ?
How do I know for instance that I'm really downloading (say) the log4j compiled JARs and not some bastardized / evil version ?
Few things you can do to feel comfortable:
Use a local repository manager like Nexus or JFrog, and proxy any repositories that you want to use. There are few benefits to this:
A local manager can keep track of the SHA hashes to make sure that a jar didn't change under your feet.
You can limit the repositories that your developers can access.
Stick with Maven Central when you can - so many people use it that if someone switched out the log4j version with something untrustworthy everyone would know very quickly (because the hashes wouldn't line up). Generally this argument will also hold true for any other repositories that hold popular libraries (eg sourceforge, google code, codehaus, etc)
Things are only likely to get risky if you're using some dude's repo who wrote some library that's not very popular out in the wild. In practice this rarely happens. In those cases, maybe you can just build the code yourself to be sure.
Best practice is not to add any repository into the pom.xml. The best solution is to configure either into the settings.xml or the best solution is to use a repository manager. Furthermore the best thing is to work with maven central if you don't have a repo-manager, but for that you don't need to configure anything, cause Maven Central is the default within Maven itself. Maven Central is control adminstrative by people of Sonatype and it is not that simple to get something into Maven Central. What you can do to secure the transport a little bit more is to turn on the checksum checking which is controled by a configuration in the settings.xml.

Resources