After a successful "bitbake core-image-sato" build, i moved the downloads folder to my private repository, and then deleted downloads the folder and fetched it from my private repository.
I added BB_NO_NETWORK = "1" in local.conf, and when I tried to do "bitbake core-image-sato" it fails.
NOTE: Executing RunQueue Tasks
ERROR: gnu-config-native-20150728+gitAUTOINC+b576fa87c1-r0 do_fetch: Network access disabled through BB_NO_NETWORK (or set indirectly due to use of BB_FETCH_PREMIRRORONLY) but access requested with command LANG=C git -c core.fsyncobjectfiles=0 fetch -f --prune --progress git://git.savannah.gnu.org/config.git refs/*:refs/* (for url git://git.savannah.gnu.org/config.git)
ERROR: gnu-config-native-20150728+gitAUTOINC+b576fa87c1-r0 do_fetch: Function failed: base_do_fetch
ERROR: Logfile of failure stored in: /home/jamal/test/new_repot/build/tmp/work/x86_64-linux/gnu-config-native/20150728+gitAUTOINC+b576fa87c1-r0/temp/log.do_fetch.29816
ERROR: Task (virtual:native:/home/jamal/test/new_repot/sources/poky/meta/recipes-devtools/gnu-config/gnu-config_git.bb:do_fetch) failed with exit code '1'
It is trying to fetch the source code again, from network, as network access is disabled it fails.
Can you guys please help me in resolving this problem. Thanks for your time and patience.
The problem is missing BB_GENERATE_MIRROR_TARBALLS = "1" in local.conf. Tarballs from git repositories are automatically not created due to performance reasons, see the manual. Setting that variable enables creating of the tarballs, so they can be used later on and git server don't need to be contacted.
(Please see comments for the question for more information, we discussed the solution there. Thanks to #md.jamal for testing it.)
Related
I'm using a shared runner that has access to the entire project group. In this one particular project on my GitLab server whose visibility is set to "private", when the runner attempts to clone the repository, it presents an error message noting that it could not clone the repository with a 403 response.
I searched high and low in the documentation but couldn't find the explanation for this nor a solution to this problem. I noticed when I switched the project's visibility to internal, everything started working.
Does anyone know why GitLab runners cannot access private repositories? If I wish to grant the runner access and the ability to execute the repo's CICD pipeline, how can I do so?
Here are the logs:
Fetching changes with git depth set to 50...
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: git branch -m <name>
Initialized empty Git repository in /builds/4q6GE3ka/0/mysecretproject/myproject/.git/
Created fresh repository.
remote: You are not allowed to download code from this project.
fatal: unable to access 'https://example.com/mysecretproject/project.git/': The requested URL returned error: 403
When I used yocto to build my first linux system and after 'bitbake imx-image-multimedia' was excuted, I faced the odd error:
ERROR: gnu-config-native-20190501+gitAUTOINC+b98424c249-r0 do_unpack: Unpack failure for URL: 'git://git.savannah.gnu.org/config.git'. No up to date source found: clone directory not available or not up to date: /home/admin/Linux/Yocto/fsl/downloads//git2/git.savannah.gnu.org.config.git; shallow clone not enabled
ERROR: Logfile of failure stored in: /home/admin/Linux/Yocto/fsl/build/tmp/work/x86_64-linux/gnu-config-native/20190501+gitAUTOINC+b98424c249-r0/temp/log.do_unpack.73483
ERROR: Task (virtual:native:/home/admin/Linux/Yocto/fsl/sources/poky/meta/recipes-devtools/gnu-config/gnu-config_git.bb:do_unpack) failed with exit code '1'
Curious about the logfile, I opened /home/admin/Linux/Yocto/fsl/build/tmp/work/x86_64-linux/gnu-config-native/20190501+gitAUTOINC+b98424c249-r0/temp/log.do_unpack.73483 and I see:
DEBUG: Executing python function do_unpack
DEBUG: Executing python function base_do_unpack
DEBUG: Running 'export PSEUDO_DISABLED=1; unset _PYTHON_SYSCONFIGDATA_NAME; export SSH_AUTH_SOCK="/run/user/0/vscode-ssh-auth-sock-7925763"; export PATH="/home/admin/Linux/Yocto/fsl/sources/poky/scripts/native-intercept:/home/admin/Linux/Yocto/fsl/sources/poky/scripts:/home/admin/Linux/Yocto/fsl/build/tmp/work/x86_64-linux/gnu-config-native/20190501+gitAUTOINC+b98424c249-r0/recipe-sysroot-native/usr/bin/x86_64-linux:/home/admin/Linux/Yocto/fsl/build/tmp/work/x86_64-linux/gnu-config-native/20190501+gitAUTOINC+b98424c249-r0/recipe-sysroot-native/usr/bin:/home/admin/Linux/Yocto/fsl/build/tmp/work/x86_64-linux/gnu-config-native/20190501+gitAUTOINC+b98424c249-r0/recipe-sysroot-native/usr/sbin:/home/admin/Linux/Yocto/fsl/build/tmp/work/x86_64-linux/gnu-config-native/20190501+gitAUTOINC+b98424c249-r0/recipe-sysroot-native/usr/bin:/home/admin/Linux/Yocto/fsl/build/tmp/work/x86_64-linux/gnu-config-native/20190501+gitAUTOINC+b98424c249-r0/recipe-sysroot-native/sbin:/home/admin/Linux/Yocto/fsl/build/tmp/work/x86_64-linux/gnu-config-native/20190501+gitAUTOINC+b98424c249-r0/recipe-sysroot-native/bin:/home/admin/Linux/Yocto/fsl/sources/poky/bitbake/bin:/home/admin/Linux/Yocto/fsl/build/tmp/hosttools"; export HOME="/root"; git -c core.fsyncobjectfiles=0 branch --contains b98424c249119b79d3f709e26eb86f2fd4d5e5f3 --list master 2> /dev/null | wc -l' in /home/admin/Linux/Yocto/fsl/downloads//git2/git.savannah.gnu.org.config.git
ERROR: Unpack failure for URL: 'git://git.savannah.gnu.org/config.git'. No up to date source found: clone directory not available or not up to date: /home/admin/Linux/Yocto/fsl/downloads//git2/git.savannah.gnu.org.config.git; shallow clone not enabled
DEBUG: Python function base_do_unpack finished
DEBUG: Python function do_unpack finished
What is 'ERROR: Unpack failure for URL: git://git.savannah.gnu.org/config.git'. No up to date source found: clone directory not available or not up to date: /home/admin/Linux/Yocto/fsl/downloads//git2/git.savannah.gnu.org.config.git; shallow clone not enabled means?
What can I do to fix it? Thanks!
See first if this is similar to this thread
Currently, for my Yocto builds (for nxp and other boards) I used to share a same "downloads" DL_DIR to avoid unnecessary fetch operations.
I tried to use an empty DL_DIR...and it worked fine.
After investigating, I found out there is something wrecked in the "git2" sub-directory of DL_DIR.
I don't know what exactly.
So if you have a custom DL_DIR with a lot of stuff, try to rename your "git2" subdir as "git2.bak".
Check also if the // seen in /home/admin/Linux/Yocto/fsl/downloads//git2/git.savannah.gnu.org.config.git means an empty environment variable, meaning check if there should be an intermediate folder between downloads/ and /git2.
I've recently encountered this problem. I had an empty do_fetch function
do_fetch(){
:
}
. Just by removing it, the git repo was cloned properly.
I have seen some answers on the internet on how to clone private repositories in CI, but they seem to use Gitlab or Bitbucket. I want to do the same thing with Github Actions.
I have private repo A which depends on private repo B. When I try to execute 'cargo build' in Github Actions for repo A, I get this error:
error: failed to get `b` as a dependency of package `a`
Caused by:
failed to load source for dependency `b`
Caused by:
Unable to update https://github.com/me/b.git
Caused by:
failed to clone into: /home/runner/.cargo/git/db/b-c42bbf84fef750c8
Caused by:
failed to authenticate when downloading repository
* attempted to find username/password via git's `credential.helper` support, but failed
if the git CLI succeeds then `net.git-fetch-with-cli` may help here
https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli
Caused by:
failed to acquire username/password from local configuration
I have seen something with SSH agents, but it looks very complex from my POV.
You need to pass the needed credentials to access the private repo to the failing step. This means git SSH downloader needs a key to authenticate against GitHub service.
If using the git command (what is what cargo build looks like), you need a dedicated key (AKA: deploy key) for this to work.
Alternatively, you can write your own action using GITHUB_TOKEN to perform the download, just how checkout does.
Occasionally the first upload of artifacts during a GitLab pipeline fail.
I'm getting the following error message in the logs:
2019-08-01 13:43:14,149 [http-nio-8082-exec-187] [ERROR]
(o.j.s.b.p.t.FilePersistenceHelper:87) - Failed moving
'path_to_artifactory\filestore_pre\dbRecord123.bin' to
'path_to_artifactory\filestore\5e\5ecc5f719b4442b9b04f9010646d34917aca8ca2'.
Access to file denied null 2019-08-01 13:43:14,149
[http-nio-8082-exec-187] [ERROR] (o.a.w.s.RepoFilter :251) - Upload
request of products-stage-qa:file_to_upload failed due to {}
java.nio.file.AccessDeniedException: Failed to persist file with sha1:
5ecc5f719b4442b9b04f9010646d34917aca8ca2
This seems to happen only during builds, but not during other uploads directly by a user.
It doesn't happen all the time, and only on first tries. But I haven't found any logic when the first try fails or succeeds. It doesn't seem to have anything to do with file types or the like. I can't really determine if it has anything to do with network speeds though since I only have access to part of the infrastructure.
I found an open ticket with the same error message, but only for Conan and for us it only happens with ivy repositories
We are using Artifactory 6.9.1 and GitLab 12.0.3 starter
This looks to be a permission issue. You are getting an error message that states that the move failed due to "Access to file denied".
You can try to log in to the server using the "artifactory" user and manually move the file called "path_to_artifactory\filestore_pre\dbRecord123.bin" to "path_to_artifactory\filestore\5e\5ecc5f719b4442b9b04f9010646d34917aca8ca2" and see if you have any issues with this. To log in to the server with the "artifactory" user you can use the command "sudo -s -u artifactory".
You will also need to make sure that all filestore and its subdirectories are owned by the "artifactory" user and have the correct permissions.
Hope this helps.
I'm trying to build Yocto for Raspberry Pi3, with console-image, and it gives me some build errors, most I have been able to resolve with
bitbake -c cleansstate libname
bitbake libname
However, now it got to libtalloc and it can't do_fetch the source files.
I went to the URL of the sources, and I was able to download the exact tar.gz archive it was trying to fetch. i.e. http://samba.org/ftp/talloc/talloc-2.1.8.tar.gz
I even put it into /build/downloads folder.
But when I try to bitbake, it keeps giving me the same errors
Is there a way I can configure the build process to always fetch with http or wget, it seems that the these scripts are all broken, because it cant fetch a file that exists.
Thanks,
Here is the full printout:
WARNING: libtalloc-2.1.8-r0 do_fetch: Failed to fetch URL http://samba.org/ftp/talloc/talloc-2.1.8.tar.gz, attempting MIRRORS if available
ERROR: libtalloc-2.1.8-r0 do_fetch: Fetcher failure: Fetch command export DBUS_SESSION_BUS_ADDRESS="unix:abstract=/tmp/dbus-ATqIt180d4"; export SSH_AUTH_SOCK="/run/user/1000/keyring-Ubo22d/ssh"; export PATH="/home/dmitry/rpi/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/dmitry/rpi/build/tmp/sysroots/x86_64-linux/usr/bin/python-native:/home/dmitry/poky-morty/scripts:/home/dmitry/rpi/build/tmp/sysroots/x86_64-linux/usr/bin/arm-poky-linux-gnueabi:/home/dmitry/rpi/build/tmp/sysroots/raspberrypi2/usr/bin/crossscripts:/home/dmitry/rpi/build/tmp/sysroots/x86_64-linux/usr/sbin:/home/dmitry/rpi/build/tmp/sysroots/x86_64-linux/usr/bin:/home/dmitry/rpi/build/tmp/sysroots/x86_64-linux/sbin:/home/dmitry/rpi/build/tmp/sysroots/x86_64-linux/bin:/home/dmitry/poky-morty/scripts:/home/dmitry/poky-morty/bitbake/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"; export HOME="/home/dmitry"; /usr/bin/env wget -t 2 -T 30 -nv --passive-ftp --no-check-certificate -P /home/dmitry/rpi/build/downloads 'http://samba.org/ftp/talloc/talloc-2.1.8.tar.gz' --progress=dot -v failed with exit code 4, output:
--2017-01-24 12:35:19-- http://samba.org/ftp/talloc/talloc-2.1.8.tar.gz
Resolving samba.org (samba.org)... 144.76.82.156, 2a01:4f8:192:486::443:2
Connecting to samba.org (samba.org)|144.76.82.156|:80... connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Retrying.
--2017-01-24 12:35:20-- (try: 2) http://samba.org/ftp/talloc/talloc-2.1.8.tar.gz
Connecting to samba.org (samba.org)|144.76.82.156|:80... connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Giving up.
ERROR: libtalloc-2.1.8-r0 do_fetch: Fetcher failure for URL: 'http://samba.org/ftp/talloc/talloc-2.1.8.tar.gz'. Unable to fetch URL from any source.
ERROR: libtalloc-2.1.8-r0 do_fetch: Function failed: base_do_fetch
ERROR: Logfile of failure stored in: /home/dmitry/rpi/build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/libtalloc/2.1.8-r0/temp/log.do_fetch.80102
ERROR: Task (/home/dmitry/poky-morty/meta-openembedded/meta-networking/recipes-support/libtalloc/libtalloc_2.1.8.bb:do_fetch) failed with exit code '1'
Is there a way I can configure the build process to always fetch with http or wget, it seems that the these scripts are all broken, because it cant fetch a file that exists.
The scripts already use both wget and http. They're also not really broken, the people maintaining the samba download servers just changed several things in the past week: I believe the libtalloc recipes main SRC_URI just needs to be changed to https://download.samba.org/pub/talloc/talloc-${PV}.tar.gz (the current canonical samba download server).
I'm sure meta-oe maintainer would appreciate a patch if this is indeed the case.
I applied the following patch to meta-openembedded and got it built. There are several samba links already broken.
http://pastebin.com/0uTnAY4g
Regards,
M.