in my project i have 2 submodules (both are maintained by us), for some reason when pushing our code into the CI the runners are all failing because:
Running with gitlab-runner 14.4.0 (4b9e985a)
on bv1 runner docker DcUNN4JN
Resolving secrets
00:00
Preparing the "docker" executor
Using Docker executor with image registry.gitlab.com/visionary.ai/brightervision/gitlab-cuda-trt-v2 ...
Authenticating with credentials from job payload (GitLab Registry)
Pulling docker image registry.gitlab.com/<company-name>/<repository>/gitlab-cuda-trt-v2 ...
Using docker image sha256:ASDFASDFSADFSDAF for registry.gitlab.com/<company-name>/<repository>/gitlab-cuda-trt-v2 with digest registry.gitlab.com/<company-name>/<repository>/gitlab-cuda-trt-v2#sha256:fgsdgsdfgsdfgdsfgsdfg ...
Preparing environment
00:07
Running on runner-dcunn4jn-project-30307801-concurrent-0 via bv1...
Getting source from Git repository
00:16
Fetching changes with git depth set to 1...
Reinitialized existing Git repository in /ci_builds/DcUNN4JN/0/<company-name>/<repository>/.git/
Checking out 09e4b628 as ci-add-bokeh...
Updating/initializing submodules recursively with git depth set to 1...
Synchronizing submodule url for 'jetson/isp-arc-implementations'
Synchronizing submodule url for 'models'
Entering 'jetson/isp-arc-implementations'
Entering 'models'
Entering 'jetson/isp-arc-implementations'
HEAD is now at b9480b7 cleanup
Entering 'models'
HEAD is now at 9c17371 Merge branch 'mar-22' into 'main'
fatal: refusing to merge unrelated histories
Unable to merge 'c90d7c8a3564ff09bb5e02513f28e64a688b325b' in submodule path 'jetson/isp-arc-implementations'
Uploading artifacts for failed job
00:09
Uploading artifacts...
WARNING: untracked: no files
ERROR: No files to upload
Cleaning up project directory and file based variables
00:06
ERROR: Job failed: exit code 1
our .gitmodules looks like this:
[submodule "models"]
path = models
url =../models.git
branch = main
[submodule "jetson/isp-arc-implementations"]
path = jetson/isp-arc-implementations
url = ../isp-arc-implementations.git
branch = main
we are all using linux OS (ubuntu18.04 distribution), couldn't find anything in the gitlab documentations nor anywhere else so i wanted to try my luck here,
if anyone has encountered this and knows how to solve it, it would be much appreciated
a few notes:
for some reason the submodule commit sha's don't match the ones we have on our repositories, but as far as i understand it should fetch the most recent one?
when going to the directory where the runner builds everything, i can't manually pull and update the submodules there (getting authentication errors, i read that the runner can't pull\push in gitlab)
Related
I have a two private repositories: MyProject, MyProjetUtils.
My project uses the MyProjectUtils as a submodule.
My .gitsubmodules looks like this:
[submodule "MyProjetUtils"]
path = MyProjetUtils
url = git#gitlab.com:MyCompany/MyProjetUtils.git
My .gitlab-ci.yml file looks like this:
default:
image: python:latest
variables:
GIT_SUBMODULE_STRATEGY: recursive
all_test:
stage: test
script:
- apt-get update
- pip install -r requirements.txt
- python tests/run_tests.py
The error I'm getting during the job run:
Updating/initializing submodules recursively with git depth set to 50...
Submodule 'MyProjetUtils' (git#gitlab.com:MyCompany/MyProjetUtils.git) registered for path 'MyProjetUtils'
Cloning into '/builds/MyCompany/MyProject/MyProjetUtils'...
error: cannot run ssh: No such file or directory
fatal: unable to fork
fatal: clone of 'git#gitlab.com:MyCompany/MyProjetUtils.git' into submodule path '/builds/MyCompany/MyProject/MyProjetUtils' failed
Failed to clone 'MyProjetUtils'. Retry scheduled
This error occur before the test stage. I've looked for answer here, and here but could not find an answer.
The first link you posted has the solution you are looking for:
When your submodule is on the same GitLab server, you should use relative URLs in your .gitmodules file. Then you can clone with HTTPS in all your CI/CD jobs. You can also use SSH for all your local checkouts.
Assuming that your submodules are in the same group, update your .gitmodules to use a relative URL.
ie:
[submodule "MyProjetUtils"]
path = MyProjetUtils
url = ../../MyCompany/MyProjetUtils.git
May need to update ../../ to work for your groups.
I have a post-receive hook in my server.
I also have a remote repo(origin) (branch:stage) where another developer pushes to.
I pull the commits and push them to production from my local computer.
I recently created a new branch(feature) from the latest head of the existing branch(stage), added some code and pushed the feature branch to remote repo. I merged the new features from feature branch to stage branch.
Then I pulled from the remote repo to local.
Now, I pushed the stage branch to production.
My post-receive file consists:
git --work-tree=/path/to/project --git-dir=/path/to/project.git checkout -
f stage
My head in the remote repo after merge is currently at ae228b9.
And, at the remote production hook's latest logs also points to ae228b9.
The problem is that the code from the merge are not seen in the production server.
Do I have to do something else to get the merged code to production?
After I pulled from origin and pushed to production, I tried changing the new files locally, committed them and push to the server. But, the changes are not reflected in the server.
Edit:
Also, I uninstalled apache php mysql and installed them again. Before the unistallation/installation, the home page was showing in server but the routes were not working(404 error) but the git hook was working. After the unistallation/installation, the git hook stopped working
Update:
I created a new hook with the same code as above. Now during the push I get:
remote: error: unable to unlink old '.gitignore' (Permission denied)
remote: error: unable to unlink old '.htaccess' (Permission denied)
remote: fatal: cannot create directory at '.idea': Permission denied
I tried the following code inside my project folder and project.git(hook) folder:
sudo chmod -R ug+w .;
unable to link
after which the push is a success but the changes are not reflecting.
I solved this by deleting the contents of the project folder and pushing again with some new changes.
I faced the same issue(Azure deployment)
There are two ways to ways to handle this -
Increase timeout setting (but deployment may stuck some times)
Push new changes (redeployment)
I prefer 2nd option.
I have a GitLab project that utilises GitLab CI.
The project also uses submodules, both the project and it's submodules are under the same GitLab account.
Here is my .gitmodules file
[submodule "proto_contracts"]
path = proto_contracts
url = https://gitlab.com/areller/proto_contracts.git
I also have this piece in the .gitlab-ci.yml file
variables:
GIT_SUBMODULE_STRATEGY: recursive
However, when i run the CI I get this error
fatal: could not read Username for 'https://gitlab.com': No such device or address
Both the project and the submodules are in a private repository so you would expect to be prompted for authentication, but as I've mentioned, the project and the submodule are under the same account and one of the runner's jobs is to clone the original repository
So it's odd that it's unable to reach the submodule
Is there a way around it?
You must use relative URLs for submodules. Update your .gitmodules as follow:
[submodule "proto_contracts"]
path = proto_contracts
url = ../../areller/proto_contracts.git
Further reading: Using Git submodules with GitLab CI | GitLab Docs
First of all my gitlab connection was broken by error 500 and 502. The server was unattended for a while. So i decided to reinstall:
#gitlab-ctl cleanse
#apt-get purge gitlab-ce
Installation:
https://about.gitlab.com/downloads/#ubuntu1404
After the gitlab was reachable and i could create a project and clone it.
After i tried to push a new file to my repo and got following message:
git add .
git commit -m "inital"
git push --set-upstream origin master
error: Cannot access URL http://<username>#gitlab.domain.de:<port>/group/project.git/, return code 22
fatal: git-http-push failed
error: failed to push some refs to 'http://<username>#gitlab.domain.de:<port>/group/project.git/'
The gitlab-check gives this information:
#gitlab-rake gitlab:check
everything was green, besides violett ones:
Uploads directory setup correctly? ... skipped (no tmp uploads folder yet)
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Why i can clone but not push? I searched in several log-files, but got nothing useful.
I've got an account with OpenShift which provides a git repository for each project you to work in. I've got one main project and two smaller projects resulting in a total of 3 git repositories.
I have just started to use the two smaller projects in the main project and seen that git can use the submodule command. I've added them both to the custom directory like so:
git submodule add ssh://...#app.rhcloud.com/~/git/app.git/
git submodule add ssh://...#api.rhcloud.com/~/git/api.git/
which has produced a directory structure like so:
node_modules
custom
app ------ full of tasty files
api ------ full of tasty files
index.js
.gitmodules
Within the .gitmodules file I have
[submodule "custom/api"]
path = custom/api
url = ssh://...#api.rhcloud.com/~/git/api.git/
[submodule "custom/app"]
path = custom/app
url = ssh://...#app.rhcloud.com/~/git/app.git/
Which is exactly what I want. It all works locally.
git add --all
git commit -m "new submodules"
git push
The issue is when I run the git push, it comes back with this error:
remote: Host key verification failed.
remote: fatal: Could not read from remote repository.
remote:
remote: Please make sure you have the correct access rights
remote: and the repository exists.
It looks like I need to add the ssh key of the main project to the two smaller projects so that I can ssh. Does anyone know how I can do this or have a better way of including submodules in OpenShift?
Thanks for the help in advance.
It looks like I need to add the ssh key of the main project to the two smaller projects so that I can ssh
At the parent (main) repo level, this shouldn't be the case: all the push does is to push the .gitmodules, and 2 gitlinks (special entries in the index) representing those 2 submodules.
It is more likely that the push doesn't find the proper .ssh/known_hosts, or the public/private ssh keys in order to push back to the app.rhcloud.com upstream repo: See "SSH connection problem with “Host key verification failed…” error" and "The authenticity of host can't be established".