I have 2 maven projects hosted on Gitlab. Let's call them A and B. Project A depends on project B.
I want to use gitlab-ci to build A.
Here is the gitlab-ci.yml file for project B:
image: maven:3-jdk-8
build:
script: "mvn install -B"
What should the gitlab-ci in project A look like?
Use GIT SUBMODULES with your project A to refer to project B and then add
GIT_SUBMODULE_STRATEGY: recursive
to the gitlab-ci.yml file in project A. Further project B needs also a CI configuration file in the project root.
https://docs.gitlab.com/ce/ci/git_submodules.html
Related
I have a nested project structure such that there is a submodule within one of my submodules. The project structure is like this:
project A
|_project B
|_project C
|_project D
Where A is a superproject, B and C are submodules of A, and D is a submodule of C. Project's C's file structure does not put project D at the top level of the directory, though this isn't a problem when I clone and update recursively manually. The problem I am encountering occurs only for project D, and only on my gitlab runner. Specifically, when the pipeline runs, it fails to instantiate project D with the following error:
fatal: could not read Username for 'the-gitlab-server': No such device or address
Eventually leading to:
Failed to recurse into submodule path 'Project C'
Projects B and C appear to instantiate correctly according to the job output. Something to do with the nesting appears to be affecting this, and only on the runner, but I cannot figure out what.
I have tried various configurations of .gitlab-ci.yml, both with variables and before_script. For example, in .gitlab-ci.yml:
variables:
GIT_SUBMODULE_STRATEGY: recursive
GIT_STRATEGY: clone
and on the triggering job I have instead tried:
<the-job-that-causes-the-failure>:
stage: smoke
script:
- <job>
before_script:
- git submodule sync --recursive
- git submodule update --init --recursive
Those exact commands correctly instantiate the completed project structure, including D, when I clone the directory personally and then run them, which makes me think the .gitmodules are fine for each project. Both the variables and before_script approach result in the same error.
The url field for each module is using a relative path as suggested in another answer elsewhere that did not resolve my problem.
The versions of git between the runner and my personal VM are the same (2.25).
Appreciate any help you can give; thanks!
I have the same problem, tried the same things and nothing.
I think I will remove recursive submodule strategy an add the inner submodules to the root .gitmodule.
But is a poor solution if works.
What's the easiest way to run a Node.js script as a step in Azure DevOps Pipeline (release step)?
I tried a few approaches and it seems like this is not as straightforward as I'd like it to be. I'm looking for a general approach idea.
My last attempt was adding the script to the source repository under tools/script.js. I need to run it in a release pipeline (after build), from where I can't access the repo directly, so I have added the entire repo as a second build artifact. Now I can reach the script file from the release agent, but I haven't found a way to actually run the script, there seems to be no option to run a Node.js script on an agent in general.
Option 1: Visual Pipeline Editor
1) Create the script file you want to run in your build/release steps
You can add the file to your Azure project repository, e.g. under tools/script.js and add any node modules it needs to run to your package.json, for example:
npm install --save-dev <module>
Commit and push, so your changes are online and Azure can see them.
2) Add your repo as an artifact for your release pipeline
You can skip this for build pipelines as they already have access to the repository.
3) Edit your release pipeline to ensure environment
Add a step to make sure correct Node version is on agent (Node.js Tool Installer):
Add a step to install all required node modules (npm):
4) Add your node script step
Use the Bash step to run your node script, make sure the working directory is set to root of the project (where package.json is located):
Option 2: YAML
you have a script\shell step where you can execute custom commands, just use that to achieve the goal. Agent have node installed on them, one thing you might need to do is use the pick node version step to use the right node version for your script
Example:
trigger:
- master
pool:
vmImage: 'Ubuntu-16.04'
steps:
- checkout: self
persistCredentials: true
clean: true
- bash: |
curl $BEDROCK_BUILD_SCRIPT > build.sh
chmod +x ./build.sh
displayName: My script download
env:
BEDROCK_BUILD_SCRIPT: https://url/yourscript.sh
- task: ShellScript#2
displayName: My script execution
inputs:
scriptPath: build.sh
I'd like Gitlab CI to fetch source code of another project. Is there a better way than adding a read-only deploy key and setting it up in .gitlab-ci.yml?
You can also use GIT SUBMODULES within your project A to refer to project B and then add
GIT_SUBMODULE_STRATEGY: recursive
to the gitlab-ci.yml file in project A.
This also enables you to specifically include a specific branch or commit from your subproject.
https://docs.gitlab.com/ce/ci/git_submodules.html
I have my .gitlab-ci.yml file set up in the typical three stages: test, build, deploy. During the build stage, I run a command that compiles my project and puts it in a tarball. The build stage appears to execute successfully because it moves on to the deploy stage, but the deploy stage then says it can't find the tarball. Is it in another directory? What happened to it? Thanks.
For each test gitlab-ci clean the build folder, therefore the output files of the build stage are not available in the deploy stage.
You need to rebuild your project also in the deploy stage.
The "stages" are only useful to order your tests, i.e. avoid to try to do a deploy test if a build test failed.
EDIT:
Since Gitlab 8.6, it is possible using dependencies feature
I was surprised to see the same behaviour (on GitLab 8.4).
I use cmake to create makefiles, then make to build, and then make test to run the test. I run all these in a build/ directory.
I don't want to repeat myself and identify easily which steps are failing. As such, I've created different gitlab-ci stages: cmake, make, test, etc. I then tell gitlab-ci to keep the build directory using the cache option:
cache:
key: "$CI_BUILD_REF_NAME"
untracked: true
paths:
- build/
I think that the key option will keep the same build directory for all stages acting on the same branch. See the gitlab-ci doc here: http://doc.gitlab.com/ce/ci/yaml/README.html#cache
EDIT: Don't use the cache for this! GitLab implemented reusable artifacts between stages in 8.4: https://gitlab.com/gitlab-org/gitlab-ce/issues/3423
The CI runners will have to be adapted to support this. See: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/336
I have a teamcity (4.something) install that creates .wsp file for deployment to sharepoint. Currently I have to copy the wsp out of the build artifacts directory and into a little deploy folder I have created. In the folder I run a .bat that deploys the new .wsp to our test server.
What steps can I take to automate this?
Either copy the .bat into the artifacts folder and update the paths etc or copy from the artifacts folder into the 'deploy' folder and run the .bat from there.
I am a neophyte when it comes to the intricacies (or basics!) of MSBuild and the like... so hand holding is appreciated!
In more recent versions of TeamCity...
In the build definition you can identify artifacts which can be copied/zipped. Artifacts can then be downloaded manually or referenced from another build (Artifact Dependency).
You can setup a 'build configuration' to do your deployment directly from artifacts produced by your ci build.
Create a build to do your deployment
Build Step
Run: Executable with parameters
Command executable: .bat file (make sure it as part of the ci build artifacts generated)
Command parameters: whatever parameters your patch files needs
Dependencies
Add New Artifact dependency
Depend on: select the ci build you want to deploy
GetArtifacts from: Last successful build
Artifact rules: +:**/*.*
So, given artifacts (like your batch file) are in the CI build... You now have a 'deploy' build. When you run it (manually or setup a Build Trigger) it will copy all the CI build artifacts to it's working directory (Artifact Dependency) and then run your batch file to do the deployment.
Pretty slick.
note: just make sure that the account running the TeamCity BuildAgent has permissions to do all the deployment stuff.
Hope this helps somebody as it took me a while to sort this out ;)
I've done this by creating a nant task, and then having TeamCity execute the nant task. It's more of a pain than it should be. You should be able to do the same as a post-build event with MSBuild.