I am following the following doc to deploy to Kubernetes from Jenkins. I have installed jenkins in my own VM. But getting following error when build is run
+ docker build -t myregistry.azurecr.io/my-svc:latest7 ./my-svc
cannot create user data directory: /var/lib/jenkins/snap/docker/321: Read-only file system
Build step 'Execute shell' marked build as failure
Finished: FAILURE
However all the directories have jenkins use as the owner, I am not sure why it is getting into permission issues.
poc#poc-ubuntu:~$ ls -ltr /var/lib/
drwxr-xr-x 18 jenkins jenkins 4096 Feb 18 16:45 jenkins
Was running into the same exact issue. Here's the workaround (from user Gargoyle (g-rgoyle) here) that worked for me:
Stop Jenkins: service jenkins stop
Manually change Jenkins home dir: mv /var/lib/jenkins /home/jenkins
Change jenkins users homedir in (might not be necessary after #2): usermod -m -d /home/jenkins jenkins
Set homedir in Jenkins config: nano /etc/default/jenkins then go down to variable "JENKINS_HOME" and set value to "/home/$NAME"
Start Jenkins: service jenkins start
I believe you must be extending the Jenkins image. /var/lib/jenkins is a mount point. You can't create a directory at build time from dockerfile. If you need to create a folder in that directory then try to create using init script at container runtime
Related
I am trying to run a Docker container with Singularity that executes a Windows program (msconvert from proteowizard) via Wine. This image: https://hub.docker.com/r/chambm/pwiz-skyline-i-agree-to-the-vendor-licenses
The Wine installation inside the Docker container is owned by root. Running with Singularity on a standard user account with
singularity run --env WINEDEBUG=-all pwiz.sif wine msconvert
results in:
wine: '/wineprefix64' is not owned by you
So I must use the --fakeroot option to overcome this.
HOWEVER, on the remote HPC system I wish to run the container I cannot use --fakeroot because it is not allowed by the system admins:
FATAL: while extracting pwiz.sif: root filesystem extraction failed: extract command failed: FATAL: configuration disallows users from running sandbox based containers
My workaround is to add a %post to build the container and change the ownership of the directory to the user of the remote system, then rebuild the Singularity image on a machine I have root access. Simply:
Bootstrap: docker
From: chambm/pwiz-skyline-i-agree-to-the-vendor-licenses
%post
useradd -u 1234 user
chown -Rf --no-preserve-root user /wineprefix64
This works for me. But my question is, is there a better way to generalise this so any unprivileged user can run this without having to manually re-build it for their username?
I have built self-hosted ubuntu agents running in docker.
All works fine, build agent is working etc.
except everything in pipeline steps is being executed as root instead of non-root user.
E.g. When I try to execute "npm ci" and I look at my build-agent logs:
; node bin location = /usr/bin/node
; cwd = /azp/agent/_work/1/s/Core
; HOME = /root
vs logs from Microsoft hosted build-agent:
; node bin location = /usr/local/bin/node
; cwd = /home/vsts/work/1/s/Core
; HOME = /home/vsts
this gives me issues in lot of stages as running some commands as root requires different settings etc.
Does anyone know how to change it from ROOT?
I tried within dockerfile and within start.sh script provided by microsoft but that did not worked. Build agent would not start at all if I would execute start.sh as docker user instead of root.
Any clues? ideas?
You can install your linux agent under the account to change it from root like /home/{accountname}/.
Below is the steps about how to install the linux agent under the non-root user.
Connect to linux agent by non-root user
ssh useraccount#linuxIp
Create a folder and cd the folder.
mkdir myagent && cd myagent
Download the package from the url.
wget https://vstsagentpackage.azureedge.net/agent/2.155.1/vsts-agent-linux-x64-2.155.1.tar.gz
Unzip the file.
tar xzf vsts-agent-linux-x64-2.155.1.gz
Config the linux Agent
I using the following Dockerfile to extend a docker image:
FROM solr:6.6
COPY --chown=solr:solr ./services-core/search/A12Core /A12Core/
Note that solr:6.6 has a USER solr statement.
When running a container built from that Dockerfile I get a permission denied when trying to access a file or directory under /A12Core:
$ docker run -it 2f3c58f093e6 /bin/bash
solr#c091f0cd9127:/opt/solr$ cd /A12Core
solr#c091f0cd9127:/A12Core$ cd conf
bash: cd: conf: Permission denied
solr#c091f0cd9127:/A12Core$ ls -l
total 8
drw-r--r-- 3 solr solr 4096 Aug 31 14:21 conf
-rw-r--r-- 1 solr solr 158 Jun 28 14:25 core.properties
solr#c091f0cd9127:/A12Core$ whoami
solr
solr#c091f0cd9127:/A12Core$
What do I need to do in order to get permission to access the fiels and folders in the /A12Core directory?
Note that I'm running the docker build from windows 7. My docker version is 18.03.0-ce.
Your directory does not have execute permission:
drw-r--r-- 3 solr solr 4096 Aug 31 14:21 conf
Without that, you cannot cd into the directory according to Linux filesystem permissions. You can fix that in your host with a chmod:
chmod +x conf
If you perform this command inside your Dockerfile (with a RUN line), it will result in any modified file being copied to a new layer, so if you run this recursively, it could double the size of your image, hence the suggestion to fix it on your build host if possible.
I had another answer here, which was wrong (but still solved your problem :), but now I see the typo in your Dockerfile. Let's take a look at this line.
COPY --chown=solr:solr ./services-core/search/A12Core /A12Core/
The COPY command checks if the target path in the container exists. If not, it creates it, before copying.
It takes A12Core from ./services-core/search.
Then it checks if path /A12Core exists.
Obviously, it does not. So, the command creates it with permissions root:root.
Lastly, it copies contents of A12Core to newly created A12Core.
In the end your have everything in /A12Core, but it belongs to root and you can't access it.
Since solr docker image already sets USER solr, the way to go would be
RUN mkdir /A12Core
COPY ./services-core/search/A12Core /A12Core
As the docs say
The USER instruction sets the user name ... the user group ... for any RUN, CMD and ENTRYPOINT instructions that follow it in the Dockerfile.
I've created a jenkins server, and I am trying to build a .net core 2.0.0 project on the server. I've been able to successfully pull from source control and store source files in the workspace. However, I'm running into an issue with running the dotnet build command. This is what I'm getting.
/usr/share/dotnet/sdk/2.0.0/Microsoft.Common.CurrentVersion.targets(4116,5):
error MSB3021: Unable to copy file
"obj/Debug/netcoreapp2.0/ubuntu.16.04-x64/Musify.pdb" to
"bin/Debug/netcoreapp2.0/ubuntu.16.04-x64/Musify.pdb". Access to the
path is denied. [/var/lib/jenkins/workspace/Musify/Musify.csproj]
now, I've given read write and execute permissions to every file and directory in /usr/share/dotnet/sdk/2.0.0/, and I've given read write and execute to every file and directory in my workspace (/var/lib/jenkins/workspace/Musify). I also believe my jenkins user is part of the sudo group.
The weird thing I am experiencing, is that I am able to, as root, run dotnet build in my workspace directory (/var/lib/jenkins/workspace/Musify), and the project builds. I cannot however, get the same results under the jenkins user (who should be part of the sudo group). My question is, how can I verify that Jenkins is using the jenkins system user, and that this user has the correct permissions to run this command. I am hosting jenkins on an ubuntu 16.04 x64 server.
UPDATE:
At the command line on your jenkins host run
ps -ef | grep jenkins
the first column will give you the USERID and it should be, as you say, jenkins
Then if you can login as jenkins to the host where the jenkins server is running run the following ....
groups
this will list out the groups that jenkins is a part of
If you want to fix the dotnet build issue take following actions:
Set DOTNET_CLI_HOME environment variable on the docker to a common
path like /tmp on the container. This path is used by the dotnet
to create necessary files to build the project. Check
Dotnet build permission denied in Docker container running Jenkins
Use -o or another accessible path to create the artifacts in the desired directory. e.g. dotnet build -o /tmp/dotnet/build/
microsoftisnotthatbad.sln
Re the jenkins user problem, run whoami in the container. If you get whoami: cannot find name for user ID blahblah it means the user is not found in the passwd file. There are 2 answers under Docker Plugin for Jenkins Pipeline - No user exists for uid 1005, if item 1 did not work, try the second:
Mount the host passwd to the container.
If the jenkins user is logged using an identity provider like LDAP on the Jenkins server or the slave server your job is using, the passwd file of the host will not have the jenkins user. Check the other answer on that post.
Error:
Failed to connect to repository : Command "/usr/bin/git ls-remote -h file:///home/myuser/path/to/project HEAD" returned status code 128:
stdout:
stderr: fatal: 'home/myuser/path/to/project' does not appear to be a git repository
fatal: The remote end hung up unexpectedly
I have tried the following:
chmod 777 to the repo folder(folder containing .git directory)
chowned to jenkins:jenkins on the repo folder
tried to clone into another folder from this local repo folder: this works!
When I run the above command: /usr/bin/git ls-remote -h file:///home/myuser/path/to/project HEAD on cmd I get the branches.
My questions are:
why is git ls-remote -h ... command called when it should be git clone ...?
How to configure jenkins git plugin to fetch code from local repo
My environment:
RHEL 5.9
Jenkins 1.519 installed as a service(no Web container)
Git plugin
When installing Jenkins as a service, by default, Jenkins does not create a user directory as in: /home/jenkins. Jenkins default home directory is set to /var/lib/jenkins. From my work-around, as you would expect, jenkins has trouble accessing local resources from other users directory.
I moved my cloned repo under Jenkins default home directory i.e. under /var/lib/jenkins so my Repository URLin Jenkins Project configuration looks like: file:///${JENKINS_HOME}/repo/<myprojectname>
UPDATE:
The above works fine ...but I found a better way to do it from this blog
The steps are outlined here:
look up /etc/init.d/jenkins script. There are a few $JENKINS variables defined
. This should lead you to the sysconfig for jenkins i.e. /etc/sysconfig/jenkins.
Stop your jenkins instance:
sudo /sbin/service jenkins stop
Take a backup
cp /etc/sysconfig/jenkins /etc/sysconfig/jenkins.bak
In this file, change the following property:
$JENKINS_USER="<your desired user>"
Change ownership of all related Jenkins directories:
chown -R <your desired user>:<your user group> /var/lib/jenkins
chown -R <your desired user>:<your user group> /var/cache/jenkins
chown -R <your desired user>:<your user group> /var/log/jenkins
Restart jenkins and that error should disappear
sudo /sbin/service jenkins start
This error should go away now!
It's been a while since this question was asked, but I had this problem today and there are very few resources. Most probably, because people tend to connect to git repositories remotely.
I checked using strace what exactly jenkins was doing and yes, it was a problem with permissions.
But I solved it in a simpler way than answer #2 - by adding jenkins to the git server group - in my case, git1:
root# gpasswd -a jenkins git1
root# service jenkins restart
I'm running Jenkins on Windows and had the same problem. I was able to solve this by having the Jenkins service log in as my user on my laptop.
(Windows 7)
Open Task Manager (Ctrl + Shift + Escape)
(Windows 10 only) Click on More Details in the lower left corner of the pop up window
Go to the Services tab
Click the Services... button
Find "Jenkins" in the list of services
Right-click "Jenkins" and click on Properties
Click the Log On tab in the Jenkins Properties window
Choose This account: under Log on as:
Enter your username and password
Click OK
Restart the Jenkins service
Then Bob's your uncle.
Jenkins uses git clone command only for the first time when a workspace is configured for a project. Further instances uses the git ls-remote command.
I had the same issue when I configured Jenkins. It was resolved by playing around with the SSH Keys. This looks like a configuration issue as well. Check if SSH Keys are setup for the Jenkins account.
Also, see the step by step procedure of configuration of SSH in the link provided. This might not give you exact solution, but can point you to the solution.
http://oodlestechnologies.com/blogs/How-to-setup-Jenkins-With-Grails-on-Ubuntu
I find that the other solutions are a bit "hacky" for me. What I did was move the Jenkins Home folder from /Users/Shared/ to /Users/[myacccount]/. This way, my Jenkins will have access to my repos and to my Android SDK (because that's where I use Jenkins for). Then change the JENKINS_HOME environment variable. I did this by entering the JENKINS_HOME in my .bash_profile (but there are other ways to do this).
Note: I use OSX
Instead of file:/// you can also use ssh:// as in this answer:
ssh://YOUR_USER#localhost/PATH_TO_YOUR_PROJECT
Note that you need to do the standard ssh setup:
Generate a keypair using ssh-keygen if you don't already have one in ~/.ssh
Paste private key (default ~/.ssh/id_rsa) into Jenkins (project settings, git repo, credentials)
Paste public key into ~/.ssh/authorized_keys