How can I stop Jenkins from resetting user permissions after each build? - node.js

I have a 'hello world' NodeJS project I'm trying to build on a fresh install of Jenkins (running on Ubuntu 18.04.3 LTS on a Digital Ocean server). I have limited experience with Linux so please let me know anything I'm missing here.
Jenkins build "execute shell":
whoami
npm install
./script/test
Jenkins console output:
Started by user twulz
Running as SYSTEM
Building in workspace /var/lib/jenkins/workspace/node-app
No credentials specified
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/Twulz/node-app.git # timeout=10
Fetching upstream changes from https://github.com/Twulz/node-app.git
> git --version # timeout=10
> git fetch --tags --progress -- https://github.com/Twulz/node-app.git +refs/heads/*:refs/remotes/origin/*
> git rev-parse refs/remotes/origin/master^{commit} # timeout=10
> git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 81d9f909cfd34cd5eb65a123dd9f2a1e67686512 (refs/remotes/origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 81d9f909cfd34cd5eb65a123dd9f2a1e67686512
Commit message: "Solve vulnerabilities"
> git rev-list --no-walk 81d9f909cfd34cd5eb65a123dd9f2a1e67686512 # timeout=10
[node-app] $ /bin/sh -xe /tmp/jenkins4626689890347957662.sh
+ whoami
jenkins
+ npm install
audited 12356 packages in 4.203s
found 0 vulnerabilities
+ ./script/test
/tmp/jenkins4626689890347957662.sh: 4: /tmp/jenkins4626689890347957662.sh: ./script/test: Permission denied
Build step 'Execute shell' marked build as failure
Finished: FAILURE
So I think the problem is the jenkins user does not have permission to execute the test script /var/lib/jenkins/workspace/script/test.
On the command line I ran these commands to try to change the permissions:
$cd /var/lib/jenkins/workspace/node-app/script
$ls -l test
-rw-r--r-- 1 jenkins jenkins 51 Sep 22 07:30 test
$sudo chmod -R 757 /var/lib/jenkins/workspace/node-app
$sudo systemctl restart jenkins
$ls -l test
-rwxr-xrwx 1 jenkins jenkins 51 Sep 22 07:35 test
Then selected "Build Now" on my jenkins project, immediately after I ran again:
$ls -l test
-rw-r--r-- 1 jenkins jenkins 51 Sep 22 07:50 test
I also tried giving recursive permissions to the whole jenkins folder, or just to the test file, but both still failed: sudo chmod -R 757 /var/lib/jenkins or sudo chmod -R 757 /var/lib/jenkins/workspace/node-app/script/test
I've re-applied and re-saved my configuration in jenkins, as suggested in another thread but there was no change.
So something in the build process is resetting the permissions - how can I ensure the jenkins user retains the right permissions to run this script?

So I found the answer, I hope this helps others - git saves the executable permissions in GitHub too so in my case each time Jenkins pulled the latest code, it was overwriting the permissions to what was saved in the git repo. I was initially working in Windows so I didn't realise this would be an issue.
To fix the problem I simply had to clone the repo, change the permissions and commit again:
$git clone https://github.com/Twulz/node-app.git
$sudo chmod -R 757 node-app
$cd node-app/
$git add .
$git status
$git commit -m "Update execute permissions"
$git push

Related

gitlab-ci.yml looks for .gitconfig in different path than global user folder

I have my gitlab-ci.yml as follows, with reference to global config:
before script:
- git config --global user.email 'myname#test.com'
- git config --global user.name 'myname'
dev:
git code goes here.
Please note due to NDA agreement, I am unable to share the actual yml code
My gitlab-runner is running on a Linux EC2 instance:
git version: 2.34.1
gitlab-runner version: 14.9.1
When I run the pipeline from my gitlab browser console, it fails with the following error:
error: could not lock config file /home/gitlab-runner/.gitconfig: no such file or directory
When I try to create the directory manually from the command prompt in my EC2 instance, am getting permission denied?
sudo mkdir gitlab-runner
mkdir: cannot create directory 'gitlab-runner': permission denied
Any help please how to resolve this issue. I am curious why the pipeline is looking for /home/gitlab-runner/.gitconfig instead of /home/.gitconfig.
gitlab-runner user is already created, as follows:
sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash
The above command creates an user, but directory is not created.
Thanks

Jenkins2 - Getting error while trying to execute a Git command using 'Execute Shell'

I'm using Jenkins 2 and trying to execute below GIT command using 'Execute Shell' in Build section.
git config --global user.email $GITHUB_LOGIN
git config --global user.name $GITHUB_LOGIN
git tag -a $BUILD_NUMBER -m "Version $BUILD_NUMBER"
git push origin --tags
With this, I'm getting below error when i run the job
git push origin --tags
fatal: could not read Username for 'https://github.com': Device not configured
In the build logs, And I've observed below statement and per my understanding, I believe Jenkins is using GIT_ASKPASS to pull the changes from GitHub.
using GIT_ASKPASS to set credentials GitHubCredentials
With that said, Can someone help me understand if I can use 'GIT_ASKPASS' to fix this issue? If yes, How can i use it in my situation?
I got this resolved with the below code snippet
git config --global user.name <git_username>
git push https://<git_username>:<git_password>#<git_repo_url> --tags

Heroku build failing, 'fatal: Not a git repository'

I'm trying to deploy my app to Heroku but keep getting the error
fatal: Not a git repository (or any parent up to mount point /tmp) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).'
Here under the commands I type
$ git branch
-> master
$ git add -A
$ git commit -m'commit'
...your branch is up to date
$ git push
everything-up-to-date
$ heroku create drumbo
Creating ⬢ drumbo... done
https://drumbo.herokuapp.com/ | https://git.heroku.com/drumbo.git
$ heroku buildpacks:set heroku/nodejs
Buildpack set. Next release on sampleapp will use heroku/nodejs.
Run git push heroku master to create a new release using this buildpack.
$ git init (added this in because of https://stackoverflow.com/questions/16853624/git-discovery-across-filesystem-not-set)
Reinitialized existing Git repository in /Users/Berta/Desktop/Codes-senior/sampleapp/.git/
$ git push heroku master
...(fails with) - fatal: Not a git repository (or any parent up to mount parent /home/kozi) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
$ git status
On branch master
your branch is up to date with 'origin/master'.
nothing to commit, working tree clean
$ ls -A
... .git, .github ...
There's also a decent amount of verbiage about npm-merge-driver in the terminal log. Looks like the build fails right after npm install npm-merge-driver is called -
> drumbo#1.0.0 prepare /tmp/build_77f785ef804afd5c02fbe1c7b6f209fd
remote: > npm-merge-driver install
remote:
remote: fatal: Not a git repository (or any parent up to mount
point /tmp)
remote: Stopping at filesystem boundary
(GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
remote:
/tmp/build_77f785ef804afd5c02fbe1c7b6f209fd/node_modules/npm-merge-
driver/node_modules/yargs/yargs.js:1100
However when I uninstalled the npm-merge-driver, build failed because it wasn't found.
What's happening?
Two things - you need to delete the prepare line from your package.json file (should be the line referring to npm-merge-driver,
and you need to make sure you add, commit, and push your code before you run npm run deploy - should clear anything up

Gitlab-runner fails to init a submodule using ssh-agent

I'm trying to configure gitlab-runner for my project (baseProject) using shell executor. My project has two submodules: submoduleOne and submoduleTwo.
This is how my .gitmodules looks like:
[submodule "lib/submoduleOne"]
path = lib/submoduleOne
url = ../submoduleOne.git
[submodule "third_party/submoduleTwo"]
path = third_party/submoduleTwo
url = ../submoduleTwo.git
And this is how relevant part of my .gitlab-ci.yml looks like:
stages:
- preparation
clean:
stage: preparation
script:
- eval $(ssh-agent -s)
- ssh-add ~/.ssh/submoduleOne
- ssh-add -l
- git submodule update --init -- lib/submoduleOne
- ssh-add -D
- ssh-add ~/.ssh/submoduleTwo
- ssh-add -l
- git submodule update --init -- third_party/submoduleTwo
- ./setup_project.sh
Public keys ~/.ssh/submoduleTwo and ~/.ssh/submoduleOne are added to the respective projects in gitlab as "deploy keys" and the fingerprints match. Both projects seem to be configured in the same way.
Yet, the job fails and the output is:
Running with gitlab-runner 11.0.0 (5396d320)
on wawsrvbuild70 3295e1f6
Using Shell executor...
Running on wawsrvbuild70...
Fetching changes...
HEAD is now at 5e3be99 CI test
From https://gitlab01.mydomain.pl/Group/baseProject
5e3be99..0029318 p.czechows2/configure-gitlab-ci#1038 -> origin/p.czechows2/configure-gitlab-ci#1038
Checking out 00293180 as p.czechows2/configure-gitlab-ci#1038...
Skipping Git submodules setup
$ eval $(ssh-agent -s)
Agent pid 18755
$ ssh-add ~/.ssh/submoduleOne
Identity added: /home/gitlab-runner/.ssh/submoduleOne (/home/gitlab-runner/.ssh/submoduleOne)
$ ssh-add -l
4096 27:f5:e5:86:af:a7:7a:aa:52:03:b1:6d:05:72:56:29 /home/gitlab-runner/.ssh/submoduleOne (RSA)
$ git submodule update --init -- lib/submoduleOne
$ ssh-add -D
All identities removed.
$ ssh-add ~/.ssh/submoduleTwo
Identity added: /home/gitlab-runner/.ssh/submoduleTwo (/home/gitlab-runner/.ssh/submoduleTwo)
$ ssh-add -l
4096 89:aa:68:f9:88:72:4c:cf:8a:a3:c7:7e:34:92:91:6b /home/gitlab-runner/.ssh/submoduleTwo (RSA)
$ git submodule update --init -- third_party/submoduleTwo
Cloning into 'third_party/submoduleTwo'...
fatal: Authentication failed for 'https://gitlab-ci-token:yXkvnfXZadABF4FZWTSb#gitlab01.mydomain.pl/Group/submoduleTwo.git/'
Clone of 'https://gitlab-ci-token:yXkvnfXZadABF4FZWTSb#gitlab01.mydomain.pl/Group/submoduleTwo.git' into submodule path 'third_party/submoduleTwo' failed
ERROR: Job failed: exit status 1
This token yXkvnfXZadABF4FZWTSb is I think the token I gave to gitlab-runner while configuring it for the baseProject. But I'm not sure where to check this.
Also, changing the order of submodules initialization won't help: the problem is still with submoduleTwo.
So my question is: why wouldn't gitlab-runner use the private key stored in ssh-agent for cloning this repository? or maybe I've made a mistake somewhere? Suggestions for how to configure submodules initialization in gitlab-runner differently are also welcome.

git-ftp fatal: Remote host not set

I'm getting this error after pipeline runs:
Preparing to unpack .../git-ftp_1.3.1-1_all.deb ...
Unpacking git-ftp (1.3.1-1) ...
Setting up libcurl3:amd64 (7.52.1-5+deb9u6) ...
Processing triggers for libc-bin (2.24-11+deb9u3) ...
Setting up curl (7.52.1-5+deb9u6) ...
Setting up git-ftp (1.3.1-1) ...
$ git ftp push --user $FTP_USERNAME --passwd $FTP_PASSWORD sftp://$FTP_HOST
fatal: Remote host not set.
ERROR: Job failed: exit code 1
This is my .yml config:
image: samueldebruyn/debian-git
stage_deploy:
only:
- develop
script:
- apt-get update
- apt-get -qq install git-ftp
- git ftp push --user $FTP_USERNAME --passwd $FTP_PASSWORD sftp://$FTP_HOST
A month ago it worked fine. The values of the variables are correct..
Any ideas?
Before you make git ftp push you have to initialize git ftp, so add the line:
- git ftp init --user $FTP_USERNAME --passwd $FTP_PASSWORD ftp://$FTP_HOST
note: this only work for the first time
or if you want to execute this every commit add the line before git ftp push :
- git reset --hard
I finally solved it pressing the "Clear Runner Caches" button and running it again.

Resources