I'd like Gitlab CI to fetch source code of another project. Is there a better way than adding a read-only deploy key and setting it up in .gitlab-ci.yml?
You can also use GIT SUBMODULES within your project A to refer to project B and then add
GIT_SUBMODULE_STRATEGY: recursive
to the gitlab-ci.yml file in project A.
This also enables you to specifically include a specific branch or commit from your subproject.
https://docs.gitlab.com/ce/ci/git_submodules.html
Related
I have a nested project structure such that there is a submodule within one of my submodules. The project structure is like this:
project A
|_project B
|_project C
|_project D
Where A is a superproject, B and C are submodules of A, and D is a submodule of C. Project's C's file structure does not put project D at the top level of the directory, though this isn't a problem when I clone and update recursively manually. The problem I am encountering occurs only for project D, and only on my gitlab runner. Specifically, when the pipeline runs, it fails to instantiate project D with the following error:
fatal: could not read Username for 'the-gitlab-server': No such device or address
Eventually leading to:
Failed to recurse into submodule path 'Project C'
Projects B and C appear to instantiate correctly according to the job output. Something to do with the nesting appears to be affecting this, and only on the runner, but I cannot figure out what.
I have tried various configurations of .gitlab-ci.yml, both with variables and before_script. For example, in .gitlab-ci.yml:
variables:
GIT_SUBMODULE_STRATEGY: recursive
GIT_STRATEGY: clone
and on the triggering job I have instead tried:
<the-job-that-causes-the-failure>:
stage: smoke
script:
- <job>
before_script:
- git submodule sync --recursive
- git submodule update --init --recursive
Those exact commands correctly instantiate the completed project structure, including D, when I clone the directory personally and then run them, which makes me think the .gitmodules are fine for each project. Both the variables and before_script approach result in the same error.
The url field for each module is using a relative path as suggested in another answer elsewhere that did not resolve my problem.
The versions of git between the runner and my personal VM are the same (2.25).
Appreciate any help you can give; thanks!
I have the same problem, tried the same things and nothing.
I think I will remove recursive submodule strategy an add the inner submodules to the root .gitmodule.
But is a poor solution if works.
I need to push to GitLab a folder (this will be my repo) that contains several modified gihub repos. What is the best way to do it? As far as I understood, what I should do it
Make a GitLab repo for the main folder
Push the submodules
Push the main module
The content of the .gitmodules file is as follows:
[submodule "restyle"]
path = restyle
url = ./restyle
Once I push the main folder, the submodules are visible on GitLab, but I cannot open them. How can I fix this? I would like to see the actual folder and to be able to open them.
The .gitmodules file you presented contains a circular definition; it says that Git should create a submodule for your repository at path restyle, and should populate that path with a clone of the Git repository at URL ./restyle, which is exactly the same directory. The URL needs to point to a Git repository located outside of the current repository. That is why you are unable to open your submodule.
I recommend the Git Tools - Submodules documentation for a more complete description.
If you want a single monolithic repository instead of a main repository which uses submodules, then your "submodules to simple folders" solution should be fine.
However, if you want to retain the submodule aspect of your existing repository, you will need a .gitmodules file that has appropriate URLs for the source of each of the submodules. If you make changes to a submodule, you have a couple of options:
If your changes would help others, make a pull request at the submodule's origin and get your changes moved upstream.
Once done, pull the updates back into your submodule directory and check out the new commit.
In your main repository top directory, commit the new revision pointer for the updated submodule.
If your changes cannot or should not be incorporated in the original upstream location, then it would be best to clone that upstream repository into your GitLab account/instance and change your submodule to point to your GitLab copy.
Push your changes to GitLab.
In your main repository top directory, commit the new revision pointer for the updated submodule.
Has anyone experienced how to add an existing NodeJS API to an existing (Angular) Nx Monrepo?
Unfortunately the manual doesn´t help me so much
https://nx.dev/migration/manual
The process of migrating a repo into your mono repo requires a few manual steps. I think there would not be a simpler way to do it.
Assuming your node project does not share files with your current monorepo, this should be the steps:
on your node repo, create a branch 'to-monorepo' and in it, move all the files to folders that match the nx folder structure and push the commits to it.
remove your package.json file (we will later merge it with the monorepo's one)
once the folders match the nx folder structure, time to merge into the monorepo. From the monorepo add the remote of the other repo
git remote add node-repo <your git repo's node url>
at the monorepo folder, checkout your master
run a pull to make the node repo branches be available in the monorepo
git pull
create a new branch 'merging-node-repo' on your monorepo.
merge the branch node-repo/to-monorepo into your merging-node-repo branch, preserving the history:
git merge node-repo/to-monorepo --allow-unrelated-histories
push your new branch (all the code and its history will now be listed in this new branch)
remove the remote node-repo from your local monorepo configs
git remote rm node-repo
manually merge all the node repo's original package.json file dependencies into the monorepo's one, and run npm install from the monorepo. This way your package-lock.json file is updated. Once you are done, create a commit and push it.
this last step is more tricky. You have now to manually update the monorepo's config files to allow nx to start managing it. This is where the link you had in your question might help. Once you are done, create a commit and push it.
With these steps you can then merge your merging-node-repo branch into master.
I recommend you to create a separated nx workspace with a nodejs project on it. This helps you with having a baseline for all the necessary nx configurations and dependencies.
You might want to make sure your project works via nx commands from this separated workspace; this way you will have a better chance of getting configurations of your monorepo right.
Hopefully this gets you started.
Here is a solution that I wrote and used to import multiple repos into a single monorepo, under whatever subdirectories are wanted, while maintaining commit history:
https://github.com/marcuswestin/monorepo-merge
I've also found two other scripts that look like they might work, but I haven't tried them:
http://choly.ca/post/git-merge-to-monorepo/
https://github.com/ksindi/monoreaper
How to disable any optional feature in jhipster such as "Kafka" rather than regenerating another Jhipster app from scratch?
Edit the .yo-rc.json file in your project, commit it, delete src folder and run jhipster --with-entities.
Of course, if you already have added custom code, you may want to do this in a git branch and use adequate git merging strategy like git merge -s recursive -Xours <branch name> to keep your changes
I installed GithubHQ in one server and GitlabCI in another server. But now I need do integration between GitlabHQ and GitlabCI. When I go to add a new project in GitlabCI he requests a path .git project, but the project is on another server where the GitlabHQ.
I tried use the path remote, like: http://[domain-name]/[user]/[project].git but he not accept.
I researched about how GitlabCI search the path and found that it does not support remote paths. He use "Rugged::Repository.new(path)" just to get the project on the server.
Does anyone know a way to use paths .git remotes in GitlabCI?
As illustrated by Issue 36:
Actually the purpose of gitlab-ci implies that you install it on deployment point. You install it where you deploy your project
So you are supposed to use a local non-bare repo.
You could, in your case, clone your remote repo on the gitlab-ci server, and use that local path.
In order to build an integration between gitlab and gitlab-ci:
add gitlab_ci user to git group for read access
clone your project via git clone /home/git/repositories to somewhere like /home/gitlab_ci/projects/...
add this project to ci.
setup gitlabhq to use ci service
Thats all.
On gitlab push it will trigger gitlab ci to make git fetch origin, so testing repo will be always up to date.