Github Actions + private repository doensn't work well - rust

I have seen some answers on the internet on how to clone private repositories in CI, but they seem to use Gitlab or Bitbucket. I want to do the same thing with Github Actions.
I have private repo A which depends on private repo B. When I try to execute 'cargo build' in Github Actions for repo A, I get this error:
error: failed to get `b` as a dependency of package `a`
Caused by:
failed to load source for dependency `b`
Caused by:
Unable to update https://github.com/me/b.git
Caused by:
failed to clone into: /home/runner/.cargo/git/db/b-c42bbf84fef750c8
Caused by:
failed to authenticate when downloading repository
* attempted to find username/password via git's `credential.helper` support, but failed
if the git CLI succeeds then `net.git-fetch-with-cli` may help here
https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli
Caused by:
failed to acquire username/password from local configuration
I have seen something with SSH agents, but it looks very complex from my POV.

You need to pass the needed credentials to access the private repo to the failing step. This means git SSH downloader needs a key to authenticate against GitHub service.
If using the git command (what is what cargo build looks like), you need a dedicated key (AKA: deploy key) for this to work.
Alternatively, you can write your own action using GITHUB_TOKEN to perform the download, just how checkout does.

Related

GitLab runner unable to clone private GitLab repositories

I'm using a shared runner that has access to the entire project group. In this one particular project on my GitLab server whose visibility is set to "private", when the runner attempts to clone the repository, it presents an error message noting that it could not clone the repository with a 403 response.
I searched high and low in the documentation but couldn't find the explanation for this nor a solution to this problem. I noticed when I switched the project's visibility to internal, everything started working.
Does anyone know why GitLab runners cannot access private repositories? If I wish to grant the runner access and the ability to execute the repo's CICD pipeline, how can I do so?
Here are the logs:
Fetching changes with git depth set to 50...
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: git branch -m <name>
Initialized empty Git repository in /builds/4q6GE3ka/0/mysecretproject/myproject/.git/
Created fresh repository.
remote: You are not allowed to download code from this project.
fatal: unable to access 'https://example.com/mysecretproject/project.git/': The requested URL returned error: 403

I can not use private repo as npm dependency in circleci deploys

I'm using circle ci to deploy a serverless built in nodejs. And I added as dependency of the main repo,a private github repo. E.g:
// package.json
.....
"dependencies": {
"my-private-github-repo": "git+ssh://git#github.com:company-name/my-private-github-repo.git",
.....
},
.....
The problem is that I need to give access the deploy process to read and clone the private repo when npm install runs
I have configured my ssh user-keys in circle ci,I followed the steps in this documentation: creating-a-bitbucket-user-key, and I m also adding it in my config.yml like this:
// .circleci/config.xml
....
steps:
- add_ssh_keys:
fingerprints:
- "My fingerprint"
....
But during the cicd it throws this message: 'There are no configured ssh keys to install'
There are no configured ssh keys to install
and, of course, npm install fails because can not access to the repo
Any clue? Thanks anyway
This thread mentions:
When this error appears, it typically means that the ssh keys have not been configured in all locations.
SSH keys will need to be set in both the project setting's page and within the config.yml.
Just in case, double-check the URL https://app.circleci.com/settings/project/github/<your organization name>/<project name>/ssh and see if it matches Checkout SSH Keys page mentioned in the official documentation

runing an Azure DevOps pipeline for a Talend ESB project maven error

I'm trying to run an Azure DevOps pipeline for talend ESB project but I got this error message:
[ERROR] Failed to execute goal on project routines: Could not resolve dependencies for project org.example.local_project.code:routines:jar:7.3.1: Could not find artifact org.talend.libraries:crypto-utils:jar:5.2.0 in central (https://repo.maven.apache.org/maven2) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <args> -rf :routines
Any help please.
According to the error message log, this error generally occurs when Maven could not download dependencies. Possible causes for this error are:
The POM misses the declaration of the which hosts the
artifact.
The repository you have configured requires authentication and Maven
failed to provide the correct credentials to the server. In this
case, make sure your ${user.home}/.m2/settings.xml contains a
declaration whose matches the of the remote
repository to use. See the Maven Settings Reference for more
details.
The remote repository in question uses SSL and the JVM running Maven
does not trust the certificate of the server.
There is a general network problem that prevents Maven from
accessing any remote repository, e.g. a missing proxy configuration.
You have configured Maven to perform strict checksum validation and
the files to download got corrupted.
Maven failed to save the files to your local repository, see
LocalRepositoryNotAccessibleException for more details.
You can refer to this document to troubleshoot.

How to add remote azure repo for terraform modules to make terraform code work on Azure pipelines

Source definition given below works for terraform modules BUT it has a PAT TOKEN. Works fine on local VM as well as on Azure Pipelines. This question is about how to define source definition for terraform modules but without hard coding PAT TOKEN
Working copy of code:
source = "git::https://<PAT TOKEN>#<AZURE DEVOPS URL>/DefaultCollection/<Project Name>y/_git/terraform-modules//<sub directory>"
I tried the below:
git::https://<AZURE DEVOPS URL>/DefaultCollection/<Project Name>/_git/terraform-modules.git//<sub directory>
That gave me error like below:
"git::https://<AZURE DEVOPS URL>/DefaultCollection/<Project Name>/_git/terraform-modules":
error downloading
'https://<AZURE DEVOPS URL>/DefaultCollection/<Project Name>/_git/terraform-modules':
/usr/bin/git exited with 128: Cloning into
'.terraform/modules/resource_group'...
fatal: could not read Username for 'https://<AZURE DEVOPS URL>':
terminal prompts disabled
Added my user name without the domain part like below:
source = "git::https://<USERNAMEM#<AZURE DEVOPS URL>/DefaultCollection/<PROJECT NAME>/_git/terraform-modules.git//compute"
Error below:
"git::https://<USERNAME>#<AZURE DEVOPS>/DefaultCollection/<PROJECT>/_git/terraform-modules.git":
error downloading
'https://<USERNAME>#<AZURE DEVOPS>/DefaultCollection/<PROJECT>/_git/terraform-modules.git':
/usr/bin/git exited with 128: Cloning into '.terraform/modules/sql_vms'...
fatal: could not read Password for
'https://<USERNAME>#<AZURE DEVOPS>': terminal prompts disabled
When Build pipeline can do checkout even without specifying username and password why do we have to mention in terraform code.
Azure Pipeline Agent has git credentials. Not sure if this is going to work at all without PAT Token?
Have a look at this - Is it possible to authenticate to a remote Git repository using the default windows credentials non interactively?
So, in our case we discovered that just running git config --global http.emptyAuth true before terraform resolves the problem. The :# business is not needed, unless your terraform module repository is an LFS repo. But this is not our case, so we did not need it.

How to store downloads folder on our private repo in yocto

After a successful "bitbake core-image-sato" build, i moved the downloads folder to my private repository, and then deleted downloads the folder and fetched it from my private repository.
I added BB_NO_NETWORK = "1" in local.conf, and when I tried to do "bitbake core-image-sato" it fails.
NOTE: Executing RunQueue Tasks
ERROR: gnu-config-native-20150728+gitAUTOINC+b576fa87c1-r0 do_fetch: Network access disabled through BB_NO_NETWORK (or set indirectly due to use of BB_FETCH_PREMIRRORONLY) but access requested with command LANG=C git -c core.fsyncobjectfiles=0 fetch -f --prune --progress git://git.savannah.gnu.org/config.git refs/*:refs/* (for url git://git.savannah.gnu.org/config.git)
ERROR: gnu-config-native-20150728+gitAUTOINC+b576fa87c1-r0 do_fetch: Function failed: base_do_fetch
ERROR: Logfile of failure stored in: /home/jamal/test/new_repot/build/tmp/work/x86_64-linux/gnu-config-native/20150728+gitAUTOINC+b576fa87c1-r0/temp/log.do_fetch.29816
ERROR: Task (virtual:native:/home/jamal/test/new_repot/sources/poky/meta/recipes-devtools/gnu-config/gnu-config_git.bb:do_fetch) failed with exit code '1'
It is trying to fetch the source code again, from network, as network access is disabled it fails.
Can you guys please help me in resolving this problem. Thanks for your time and patience.
The problem is missing BB_GENERATE_MIRROR_TARBALLS = "1" in local.conf. Tarballs from git repositories are automatically not created due to performance reasons, see the manual. Setting that variable enables creating of the tarballs, so they can be used later on and git server don't need to be contacted.
(Please see comments for the question for more information, we discussed the solution there. Thanks to #md.jamal for testing it.)

Resources