I am trying to run my GitLab CI tests locally, to speed up developing CI.
I installed gitlab-runner locally (on my GNU/Linux box) following these instructions.
Now, when I run sudo gitlab-runner exec docker test, I get:
sudo gitlab-runner exec docker test
Runtime platform arch=amd64 os=linux pid=7133 revision=0e5417a3 version=12.0.1
Running with gitlab-runner 12.0.1 (0e5417a3)
Using Docker executor with image joyzoursky/python-chromedriver:2.7 ...
Pulling docker image joyzoursky/python-chromedriver:2.7 ...
Using docker image sha256:f593ebe67d0c75eb8e3d663412c8fc80df70bd08e835d7a64ba6afa21b1c2d8a for joyzoursky/python-chromedriver:2.7 ...
Running on runner--project-0-concurrent-0 via hostname...
Fetching changes...
Initialized empty Git repository in /builds/project-0/.git/
Created fresh repository.
fatal: '/home/USER/path/to/repo' does not appear to be a git repository
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
ERROR: Job failed: exit code 1
FATAL: exit code 1
(/home/USER/path/to/repo is the actual path to my working directory containing .gitlab-ci.yml, and my project code)
So, it looks like gitlab-runner is trying to use the directory path as a git clone URL, which obviously won't work.
I tried specifying --clone-url=, but I can't understand what part of the full project URL (either in the main git repo, or the gitlab mirror) I should be using.
Any clues?
If you are on Fedora and have SELinux enabled; try setting that to permissive:
sudo setenforce 0
SELinux blocks any activity that isn't allowed by some existing policy. The GitLab runner is basically a docker container trying to access files on the host machine, so that gets blocked...which leads the runner to report that you don't have a valid repository at (an error saying it can't read the files would have made more sense but we don't live in a perfect world)
The lazy fix is to make SELinux blanket permit the activity, if you don't do this sort of thing often enough to warrant a policy change for example.
The more security conscious will prefer to edit the policy instead. Figure out some unique identifier in the denial error, then create a new policy to allow it
sudo grep <needle> /var/log/audit/audit.log | audit2allow -M gitlab
sudo semodule -i gitlab.pp
Related
I'm trying to clone an internal gitlab repository, but it keeps giving me this error message
fatal: unable to access 'https://gitlab**************.git/': gnutls_handshake() failed: Error in the pull function.
But somehow, if I'm running as root, the cloning process would run perfectly.
The problem is, I needed this to work because I'm trying to use this repository on Jenkins.
Can someone help me or explain why it only worked if I'm running as root?
root uses the same version of git and curl.
But it does not use the same:
global git config: compare the output of git config --global -l in both instances (root and regular user account).
environment configuration (type env in both cases, and compare the environment variables)
In particular, look for sslcainfo (git config --global -l|grep -i ssl) which could reference the certificate needed to contact through TLS your on-premise HTTPS GitLab URL.
I am working on a project which I want to publish in GitHub pages. To automate the deploying process I'm using Travis CI. For this I have created a deploy.sh file, which has the following code.
# build
npm run docs:build
# navigate into the build output directory
cd docs/.vuepress/dist
# if you are deploying to a custom domain
# echo 'www.example.com' > CNAME
git add -A
git commit -m 'deploy'
# if you are deploying to https://<USERNAME>.github.io
# git push -f git#github.com:<USERNAME>/<USERNAME>.github.io.git master
# if you are deploying to https://<USERNAME>.github.io/<REPO>
git push -f git#github.com:<username>/<repo>.git master:gh-pages
cd -
I have added this in my .travis.yml file, which is below
language: nodejs
node_js:
- "lts/*"
before_script:
- npm install
script:
- bash ./scripts/deploy.sh
Now when I push my code to the master, In travis-ci.org it shows that the build is failed, with the following outputs,
Warning: Permanently added the RSA host key for IP address '192.30.253.113' to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
The command "bash ./scripts/deploy.sh" exited with 128.
I have followed other SO answers of the same type of errors like this link and also followed the way to generate and adding ssh key to my GitHub account but no success. I will be grateful if you could help me out. Thank you.
If you want to push via ssh then travis needs to have access to the private part of the ssh key you generated. What you want to do is use the travis cli gem to encrypt the private key, add it to your repo and during the deploy stage decrypt it again and use it.
Here's a step-by-step
I've created a jenkins server, and I am trying to build a .net core 2.0.0 project on the server. I've been able to successfully pull from source control and store source files in the workspace. However, I'm running into an issue with running the dotnet build command. This is what I'm getting.
/usr/share/dotnet/sdk/2.0.0/Microsoft.Common.CurrentVersion.targets(4116,5):
error MSB3021: Unable to copy file
"obj/Debug/netcoreapp2.0/ubuntu.16.04-x64/Musify.pdb" to
"bin/Debug/netcoreapp2.0/ubuntu.16.04-x64/Musify.pdb". Access to the
path is denied. [/var/lib/jenkins/workspace/Musify/Musify.csproj]
now, I've given read write and execute permissions to every file and directory in /usr/share/dotnet/sdk/2.0.0/, and I've given read write and execute to every file and directory in my workspace (/var/lib/jenkins/workspace/Musify). I also believe my jenkins user is part of the sudo group.
The weird thing I am experiencing, is that I am able to, as root, run dotnet build in my workspace directory (/var/lib/jenkins/workspace/Musify), and the project builds. I cannot however, get the same results under the jenkins user (who should be part of the sudo group). My question is, how can I verify that Jenkins is using the jenkins system user, and that this user has the correct permissions to run this command. I am hosting jenkins on an ubuntu 16.04 x64 server.
UPDATE:
At the command line on your jenkins host run
ps -ef | grep jenkins
the first column will give you the USERID and it should be, as you say, jenkins
Then if you can login as jenkins to the host where the jenkins server is running run the following ....
groups
this will list out the groups that jenkins is a part of
If you want to fix the dotnet build issue take following actions:
Set DOTNET_CLI_HOME environment variable on the docker to a common
path like /tmp on the container. This path is used by the dotnet
to create necessary files to build the project. Check
Dotnet build permission denied in Docker container running Jenkins
Use -o or another accessible path to create the artifacts in the desired directory. e.g. dotnet build -o /tmp/dotnet/build/
microsoftisnotthatbad.sln
Re the jenkins user problem, run whoami in the container. If you get whoami: cannot find name for user ID blahblah it means the user is not found in the passwd file. There are 2 answers under Docker Plugin for Jenkins Pipeline - No user exists for uid 1005, if item 1 did not work, try the second:
Mount the host passwd to the container.
If the jenkins user is logged using an identity provider like LDAP on the Jenkins server or the slave server your job is using, the passwd file of the host will not have the jenkins user. Check the other answer on that post.
Error:
Failed to connect to repository : Command "/usr/bin/git ls-remote -h file:///home/myuser/path/to/project HEAD" returned status code 128:
stdout:
stderr: fatal: 'home/myuser/path/to/project' does not appear to be a git repository
fatal: The remote end hung up unexpectedly
I have tried the following:
chmod 777 to the repo folder(folder containing .git directory)
chowned to jenkins:jenkins on the repo folder
tried to clone into another folder from this local repo folder: this works!
When I run the above command: /usr/bin/git ls-remote -h file:///home/myuser/path/to/project HEAD on cmd I get the branches.
My questions are:
why is git ls-remote -h ... command called when it should be git clone ...?
How to configure jenkins git plugin to fetch code from local repo
My environment:
RHEL 5.9
Jenkins 1.519 installed as a service(no Web container)
Git plugin
When installing Jenkins as a service, by default, Jenkins does not create a user directory as in: /home/jenkins. Jenkins default home directory is set to /var/lib/jenkins. From my work-around, as you would expect, jenkins has trouble accessing local resources from other users directory.
I moved my cloned repo under Jenkins default home directory i.e. under /var/lib/jenkins so my Repository URLin Jenkins Project configuration looks like: file:///${JENKINS_HOME}/repo/<myprojectname>
UPDATE:
The above works fine ...but I found a better way to do it from this blog
The steps are outlined here:
look up /etc/init.d/jenkins script. There are a few $JENKINS variables defined
. This should lead you to the sysconfig for jenkins i.e. /etc/sysconfig/jenkins.
Stop your jenkins instance:
sudo /sbin/service jenkins stop
Take a backup
cp /etc/sysconfig/jenkins /etc/sysconfig/jenkins.bak
In this file, change the following property:
$JENKINS_USER="<your desired user>"
Change ownership of all related Jenkins directories:
chown -R <your desired user>:<your user group> /var/lib/jenkins
chown -R <your desired user>:<your user group> /var/cache/jenkins
chown -R <your desired user>:<your user group> /var/log/jenkins
Restart jenkins and that error should disappear
sudo /sbin/service jenkins start
This error should go away now!
It's been a while since this question was asked, but I had this problem today and there are very few resources. Most probably, because people tend to connect to git repositories remotely.
I checked using strace what exactly jenkins was doing and yes, it was a problem with permissions.
But I solved it in a simpler way than answer #2 - by adding jenkins to the git server group - in my case, git1:
root# gpasswd -a jenkins git1
root# service jenkins restart
I'm running Jenkins on Windows and had the same problem. I was able to solve this by having the Jenkins service log in as my user on my laptop.
(Windows 7)
Open Task Manager (Ctrl + Shift + Escape)
(Windows 10 only) Click on More Details in the lower left corner of the pop up window
Go to the Services tab
Click the Services... button
Find "Jenkins" in the list of services
Right-click "Jenkins" and click on Properties
Click the Log On tab in the Jenkins Properties window
Choose This account: under Log on as:
Enter your username and password
Click OK
Restart the Jenkins service
Then Bob's your uncle.
Jenkins uses git clone command only for the first time when a workspace is configured for a project. Further instances uses the git ls-remote command.
I had the same issue when I configured Jenkins. It was resolved by playing around with the SSH Keys. This looks like a configuration issue as well. Check if SSH Keys are setup for the Jenkins account.
Also, see the step by step procedure of configuration of SSH in the link provided. This might not give you exact solution, but can point you to the solution.
http://oodlestechnologies.com/blogs/How-to-setup-Jenkins-With-Grails-on-Ubuntu
I find that the other solutions are a bit "hacky" for me. What I did was move the Jenkins Home folder from /Users/Shared/ to /Users/[myacccount]/. This way, my Jenkins will have access to my repos and to my Android SDK (because that's where I use Jenkins for). Then change the JENKINS_HOME environment variable. I did this by entering the JENKINS_HOME in my .bash_profile (but there are other ways to do this).
Note: I use OSX
Instead of file:/// you can also use ssh:// as in this answer:
ssh://YOUR_USER#localhost/PATH_TO_YOUR_PROJECT
Note that you need to do the standard ssh setup:
Generate a keypair using ssh-keygen if you don't already have one in ~/.ssh
Paste private key (default ~/.ssh/id_rsa) into Jenkins (project settings, git repo, credentials)
Paste public key into ~/.ssh/authorized_keys
I've installed GitLab on Google Compute Engine using "Click to Deploy" from the project interface. The deployment is successful after a few minutes. I can SSH into the instance, and muck around with it as expected.
I can also log in to GitLab using the web interface, and add SSH keys to my profile. So far, so good. However, when I attempt to push or pull to a new example repository, I receive this message:
Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
I've removed my local SSH config so it doesn't interfere. Do I need to setup an SSH tunnel of some sort? What am I missing?
UPDATE: Wiping out my local ~/.ssh folder, and regenerating an SSH key (which I've added to my profile in GitLab) produces the following error:
Received disconnect from {GITLAB_IP_ADDRESS}: 2: Too many authentication failures for git
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
UPDATE 2: It seems GitLab may already have a solution: run sudo gitlab-ctl reconfigure. See here: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md#git-ssh-access-stops-working-on-selinux-enabled-systems
You need to create an SSH tunnel to communicate with GitLab.
1. Log into your development server as your user, and create a key.
ssh-keygen -t rsa
Follow the steps, and create a passcode (that you can remember) as you'd need this to pull and push code from/to GitLab.
2. Now that you've created your key, we can copy it;
cat id_rsa.pub
Copy the output of that command (including ssh-rsa), and add it to your GitLab profile. (http://my-gitlab-server.com/profile/keys/new).
3. Ensure you have the correct privilege to the project(s)
Ensure you are at role developer at the very least. (Screengrab of roles: http://i.stack.imgur.com/DSSvl.jpg)
4. Now, copy the project link
Go into your project, and find the SSH link in the top right;
5. Now back to your development server
Navigate to your directory where you'd like to work, and run the following;
$ git init
$ git remote add origin <<project_url>>
$ git fetch
Where <<project_url>> is the link we copied in step 4.
You will be prompted your password (this is your ssh key password, not your server password) and to add the host to your known_hosts file. After that, the project will start to download and you can enjoy development.
I did these steps on a CentOS 6.4 machine with Digital Ocean. But they shouldn't differ from using Google CE.
Edit
Quote from Marty Penner answer as per this comment
Solved it! Thanks to #sxleixer and #Alexander Wenzowski for figuring this out.
Apparently, SELinux was interfering with a non-standard location for the .ssh directory. I needed to run the following commands on the Compute Engine instance:
sudo yum -y install policycoreutils-python # Install the `semanage` tool
sudo semanage fcontext -a -t ssh_home_t "/var/opt/gitlab/.ssh/authorized_keys" # Allow the nonstandard ssh_home_t
See the full thread here:
Google Cloud Engine. Permission denied (publickey,gssapi-keyex,gssapi-with-mic)
Solved it! Thanks to #sxleixer and #Alexander Wenzowski for figuring this out.
Apparently, SELinux was interfering with a non-standard location for the .ssh directory. I needed to run the following commands on the Compute Engine instance:
sudo yum -y install policycoreutils-python # Install the `semanage` tool
sudo semanage fcontext -a -t ssh_home_t "/var/opt/gitlab/.ssh/authorized_keys" # Allow the nonstandard ssh_home_t
See the full thread here:
Google Cloud Engine. Permission denied (publickey,gssapi-keyex,gssapi-with-mic)
UPDATE: It seems GitLab may already have a solution: run sudo gitlab-ctl reconfigure. See here: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md#git-ssh-access-stops-working-on-selinux-enabled-systems
In my situation the git user wasn´t set up completely. If you get in your log files messages like "User git not allowed because account is locked" (Under Centos or Redhat it´s /var/log/secure) than you simply need to activate the user via "passwd -d git"