I have a hundred repos in an organization and I am required to get the latest pom version for each repo. I have tried the current way of sh(returnStdout: true, script: "mvn org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate -Dexpression=project.${repoName} -Drevision=${env.BUILD_NUMBER} -q -DforceStdout) but since I am looping through multiple repos, i am unable to get the build number since the pipeline I'm running is not specific to one repo.
Related
How can I do the following when using GitLab pipeline defined using .gitlab-ci.yml
Get a version number from a file in the Git repo. For example using something like VSN=$(yq '.version' galaxy.yml | sed 's/"//g')
Check if a tag exists in the repository for that version.
Add jobs to the pipeline when there is no tag for the version.
Databricks recently added support for "files in repos" which is a neat feature. It gives a lot more flexibility to the projects, since we can now add .json config files and even write custom python modules that exists solely in our closed environment.
However, I just noticed that the standard way of deploying from an Azure git repo to a workspace does not support arbitrary files. First off, all .py files are converted to notebooks, breaking the custom modules that we wrote for our project. Secondly, it intentionally skips files ending in one of the following: .scala, .py, .sql, .SQL, .r, .R, .ipynb, .html, .dbc, which means our .json config files are missing when the deployment is finished.
Is there any way to get around these issues or will we have to revert everything to use notebooks like we used to?
You need to stop doing deployment the old way as it depends on the Workspace REST API that doesn't support arbitrary files. Instead you need to have a Git checkout in your destination workspace, and update that checkout to a given branch/tag when doing release. This is could be done via Repos API, or databricks cli. Here is an example of how to do that with cli from DevOps pipeline.
- script: |
echo "Checking out the releases branch"
databricks repos update --path $(STAGING_DIRECTORY) --branch "$(Build.SourceBranchName)"
env:
DATABRICKS_HOST: $(DATABRICKS_HOST)
DATABRICKS_TOKEN: $(DATABRICKS_TOKEN)
displayName: 'Update Staging repository'
I want to create a general Jenkins script that changes stuff in the project, which has to be committed. All changed files are pom.xml files. In some cases, we are talking about multi-module projects, but it isn't always the case. Either way, I want to stage every pom.xml-s in the given git repo.
The command I wanted to use is git add '**/pom.xml', but it doesn't stage the pom.xml in the repo root. What's worse is if the project is not a multi-module one, it throws an error, because it can't find any pom.xml-s.
I tried git add pom.xml '**/pom.xml', but it returns with the same error: fatal: pathspec '**/pom.xml' did not match any files
What other options do I have?
The command has become from git add '**/pom.xml' to find . -name 'pom.xml' | xargs git add and it works now
Is it possible to stop verify/download newer repo from internet, such as
test $ repo init -u git#1.1.1.1/test/iot_manifest.git
Get https://gerrit.googlesource.com/git-repo/clone.bundle
Get https://gerrit.googlesource.com/git-repo
remote: Counting objects: 1, done
remote: Finding sources: 100% (36/36)
...
You can't, the repo installed on your computer (the one in your $PATH) is not the full version of repo it is only a launcher.
From the repo documentation on source.android.com:
Repo comes in two parts: One is a launcher script you install, and it communicates with the second part, the full Repo tool included in a source code checkout.
When you run repo init for the first time it gets the full repo and store it in the .repo/repo directory. Every time you'll run repo init again in a brand new repository, the full repo will be downloaded again in .repo/repo.
One thing though you can stop getting the clone.bundle line with repo init --no-clone-bundle
Get repo from your own computer (without internet)
You can use a local version of repo, you need the internet at least to get the git-repo code once. After that you can use this version stored locally in place of the remote ones on Google server.
cd workspace
git clone https://gerrit.googlesource.com/git-repo
mkdir repo_init_no_internet && cd repo_init_no_internet
repo init --repo-url=/home/<user>/workspace/git-repo
I´m using the mvn dependency:get to download some specific *.jar (latest Release Version) files on Linux Ubuntu.
I am using the following command:
mvn dependency:get -Dartifact=org.apache.httpcomponents:httpmime:RELEASE:jar -DrepoUrl=https://repo1.maven.org/maven2 -Ddest=/home/dev/workspace/"
The artifact is downloaded as expected, but only to the local maven repository and not to copied to the specified Location. I have also tried using
-Ddest=home/dev/workspace/
-Ddest=/home/dev/workspace
But there is no difference in the Result.
I know there is the possibility to use dependency:copy, but therefore i would need a pom.xml (which i don´t want to create only for the copy command).
Thanks for your help
I have given an examples. Please check.
mvn dependency:get -Dartifact=org.apache.maven:maven-core:2.2.1:jar:sources -DremoteRepositories=http://repo1.maven.apache.org/maven2 -Ddest=/home/dev/workspace/something.jar
You can also use dependency:copy as
Artifacts can also be resolved by specifying the classifier and optionally type. Type is only used with the classifier and defaults to java-sources. When the classifier is set, the list of dependencies is used as the base to resolve artifacts with the classifier and type.
For example:
mvn dependency:copy-dependencies -Dclassifier=sources
will try to find the sources for all dependencies and copy them.
For more data command line like:
mvn dependency:copy-dependencies [optional params]
Resource Link:
http://maven.apache.org/plugins/maven-dependency-plugin/usage.html
https://maven.apache.org/plugins/maven-dependency-plugin/copy-dependencies-mojo.html
UPDATE:
use "-Ddest=path/to/my.jar"; otherwise, it is just copied to your local ~/.m2/repository (makes it look like nothing happened).
See http://maven.apache.org/plugins/maven-dependency-plugin/get-mojo.html#destination
example=>
mvn org.apache.maven.plugins:maven-dependency-plugin:2.5.1:get
-DremoteRepositories=repo.maven.apache.org
-Dartifact=org.apache.ant:ant:1.8.1
-Ddest=ant-1.8.1.jar
(result: ant-1.8.1.jar in current directory)