I have a repository repoB on a private server serverB, and get a deploy token (user + password) from it.
This repository is used as a submodule in the project I'm trying to configure.
In this project, I want to be able to init this submodule during the Gitlab-CI.
My .gitmodules is:
[submodule "repoB"]
path = repoB
url = https://serverB/groupB/repoB.git
And I have in my .gitlab-ci.yml:
test_build:
variables:
GIT_SUBMODULE_STRATEGY: recursive
Actually, the error I get is the following (group: japan7, project: nanachan, repoB: karaneko, serverB: git.inpt.fr)
I suppose that I should put the deploy token user and password in the project secrets (in Settings −> CI / CD −> Variables) but I could not find the name of these variables, nor any help to solve this particular case.
How should I proceed?
Thank you in advance for your help!
I solved same issue using deploy token. Just added submodule with deploy token and everything on CI runs ok.
http://<username>:<deploy_token>#gitlab.example.com/tanuki/awesome_project.git
And .gitmodules looks like:
[submodule "tanuki/awesome_project"]
path = tanuki/awesome_project
url = https://<username>:<deploy_token>#gitlab.example.com/tanuki/awesome_project.git
https://docs.gitlab.com/ee/user/project/deploy_tokens/
Related
Scenario: I've created two repositories inside my GitLab account: 'work' & 'project'. There are multiple branches in each of the following repos. I'm currently in the project repo and have created the .gitlab-ci.yml file.
Task: I want to copy two files 'A' & 'B' from the feature branch of 'work' repository to the current location (i.e., in the root of my project repository).
Thanks in advance :)
It's very easy, just add it to your script section
- git clone https://user:password#gitlab.com/group/project.git
Or to download specific files
- curl --header 'Private-Token: <your_access_token>' https://gitlab.com/api/v4/projects/:id/repository/files/:filename\?ref\=:ref
Also it will be a great practice to add Private-Token to project masked varible
You can get access token here
I have a Node.js app. I want to clone a private repository from GitHub using this app. I have access to this repository. In an attempt to clone the repository from my Node.js app, I'm using simple-git, with the following code:
const git = simpleGit();
git.clone('[repository-url]', './repository');
This code runs successfully. A directory named repository with the following structure is created:
/repository
/.git
.gitignore
However, this is not the contents of the repository. I successfully cloned a public repository using this code. This makes me believe that it's an authentication issue, even though no error is being shown. I created a Personal Access Token and stored it in an environment variable. However, I can't figure out how to actually use that Personal Access Token when I clone my private repository.
How do I clone a private repository using a Personal Access Token using simple-git?
You can use the config plugin to add your access token as a custom header:
const git = simpleGit({
config: [
`Authorization: token ${TOKEN}`
]
});
You can also set up a .netrc file with your auth details which simple-git will pick up by default.
I'm working on a Azure DevOps build pipeline for a project. I can't make any changes to the code itself besides the azure-pipeline.yaml file. (And to be honest, I know very little about the project itself)
I'm stuck on the NPM install dependencies step. I'm currently working with the YAML pipeline, but if there's a solution in the classic mode I'll go with that.
The issue is the following:
I've created the pipeline with and I check out a private Bitbucket repository according to the documentation:
resources:
repositories:
- repository: MyBitBucketRepo1
type: bitbucket
endpoint: MyBitBucketServiceConnection
name: MyBitBucketOrgOrUser/MyBitBucketRepo
Next I set the correct version of node, and execute a npm install task
- task: Npm#1
displayName: 'NPM install'
inputs:
command: 'install'
workingDir: 'the working directory'
So far so good. But, there is a dependency to another Bitbucket repository. In the package.json there is a dependecy like this:
another-dependency: git:https://bitbucket.org/organisation/repo.git#v1.1.3
I do have access to this repository, but if I run NPM install it can't re-use the credentials from the first repository.
I've tried adding both repositories to the resources in the hope that would work. But still the same error:
error fatal: Authentication failed for 'https://bitbucket.org/organisation/repo.git/'
I've tried to set up some caching mechanism, run npm install on the 2nd repo, store the dependencies, run npm install on the first one. But that didn't work unfortunately.
Is there a way in Azure Devops pipelines -without making changes to the project set-up- to make this work?
Thanks!
Normally I have the .npmrc on the Repo so I dont have to add any other task. Something like in this guide:
https://learn.microsoft.com/en-us/azure/devops/artifacts/get-started-npm?view=azure-devops&tabs=windows
And I never do something like that, but I think that you can authenticate with the external feed adding this task:
https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/package/npm-authenticate?view=azure-devops
Reading a bit more, I dont know if you can do this without adding a .npmrc on your Repo. You have to create a ServiceConnection to store your login credentials, but on that you will need the .npmrc on your Repo.
Try it and tell my if this help!!
Npm will prompt for passwords when you run npm install command for your package.json locally. Since we can't enter the password during pipeline run in CI/CD pipeline, it causes the Authentication failed error.
An alternative workaround is to add credentials directly in url, like this:
"dependencies": {
"another-dependency": "git+https://<username>:<password>#bitbucket.org/xxx/repo.git"
}
See app-password:
username: your normal Bitbucket username
password: the app password
It has disadvantage since we store the app-password directly as plain-text in package.json file, which lacks security if someone else can access your package.json file. So it depends on you whether to use this workaround.
As a workaround for Azure Devops pipeline:
You can add a File Transform task to replace the old url with new Username+Password url before your npm install steps.
1.I have a package.json in root directory with content like git:https://bitbucket.org/organisation/repo.git#v1.1.3.
2.Define a dependencies.another-dependency variable with value git+https://<username>:<password>#bitbucket.org/..., set it as secret!
3.Then add the File Transform task like this:
4.Finally you'll get a new package.json file with content below:
It won't actually affect your package.json file under version control, it just add credentials temporarily during your pipeline.
Is there a way to get a list of files and directories in the gitlab repo using python? So if I use the gitlab repository url as my source, can I traverse and get a list of all files and directories/ sub-directories within the repo? Like using os.walk but for a web url.
Can you try python-gitlab?
Use repository_tree method can do it for you.
Here is the code
import gitlab
gl = gitlab.Gitlab.from_config()
# Get a project by ID
project_id = 851
project = gl.projects.get(project_id)
branch = {your branch}
directory = {your directory ref to root directory of the repository}
project.repository_tree(direcotry, ref=branch)
For more details, please visit python-gitlab
I have a GitLab project that utilises GitLab CI.
The project also uses submodules, both the project and it's submodules are under the same GitLab account.
Here is my .gitmodules file
[submodule "proto_contracts"]
path = proto_contracts
url = https://gitlab.com/areller/proto_contracts.git
I also have this piece in the .gitlab-ci.yml file
variables:
GIT_SUBMODULE_STRATEGY: recursive
However, when i run the CI I get this error
fatal: could not read Username for 'https://gitlab.com': No such device or address
Both the project and the submodules are in a private repository so you would expect to be prompted for authentication, but as I've mentioned, the project and the submodule are under the same account and one of the runner's jobs is to clone the original repository
So it's odd that it's unable to reach the submodule
Is there a way around it?
You must use relative URLs for submodules. Update your .gitmodules as follow:
[submodule "proto_contracts"]
path = proto_contracts
url = ../../areller/proto_contracts.git
Further reading: Using Git submodules with GitLab CI | GitLab Docs