How do I use SSH in a Jenkins pipeline? - node.js

I have some Jenkins jobs defined using a Jenkins Pipeline Model Definition, which builds NPM projects. I use Docker containers to build these projects (using a common image with
just Node.js + npm + yarn).
The results of the builds are contained in the dist/ folder that I zipped using a zip pipeline command.
I want to copy this ZIP file to another server using SSH/SCP (with private key authentication). My private key is added to the Jenkins environment (credentials manager), but when I use Docker containers, an SSH connection cannot be established.
I tried to add agent { label 'master' } to use the master Jenkins node for file transfer, but it seems to create a clean workspace with new Git fetch, and without my built files.
After I tried the SSH Agent Plugin, I have this output:
Identity added: /srv/jenkins3/workspace/myjob-TFD#tmp/private_key_370451445598243031.key (rsa w/o comment)
[ssh-agent] Started.
[myjob-TFD] Running shell script
+ scp -r dist test#myremotehost:/var/www/xxx
$ docker exec bfda17664965b14281eef8670b34f83e0ff60218b04cfa56ba3c0ab23d94d035 env SSH_AGENT_PID=1424 SSH_AUTH_SOCK=/tmp/ssh-k658r0O76Yqb/agent.1419 ssh-agent -k
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 1424 killed;
[ssh-agent] Stopped.
Host key verification failed.
lost connection
How do I add a remote host as authorized?

I had a similar issue. I did not use the label 'master', and I identified that the file transfer works across slaves when I do it like this:
Step 1 - create SSH keys in a remote host server, include the key to authorized_keys
Step 2 - Create credential using SSH keys in Jenkins, use the private key from the remote host
Use the SSH agent plugin:
stage ('Deploy') {
steps{
sshagent(credentials : ['use-the-id-from-credential-generated-by-jenkins']) {
sh 'ssh -o StrictHostKeyChecking=no user#hostname.com uptime'
sh 'ssh -v user#hostname.com'
sh 'scp ./source/filename user#hostname.com:/remotehost/target'
}
}
}

Use the SSH agent plugin:
SSH Agent Plugin
SSH Agent Plugin
When using this plugin you can use the global credentials.

To add a remote host to known hosts and hopefully cope with your error try to manually ssh from the Jenkins host to the target host as the Jenkins user.
Get on the host where Jenkins is installed. Type
sudo su jenkins
Now use ssh or scp like
ssh username#server
You should be prompted like this:
The authenticity of host 'server (ip)' can't be established.
ECDSA key fingerprint is SHA256:some-weird-string.
Are you sure you want to continue connecting (yes/no)?
Type yes. The server will be permanently added as a known host. Don't even bother passing a password, just Ctrl + C and try running a Jenkins job.

Like #haschibaschi recommends, I also use the ssh-agent plugin. I have a need to use my personal UID credentials on the remote machine, because it doesn't have any UID Jenkins account. The code looks like this (using, for example, my personal UID="myuid" and remote server hostname="non_jenkins_svr":
sshagent(['e4fbd939-914a-41ed-92d9-8eededfb9243']) {
// 'myuid#' required for scp (this is from UID jenkins to UID myuid)
sh "scp $WORKSPACE/example.txt myuid#non_jenkins_svr:${dest_dir}"
}
The ID e4fbd939-914a-41ed-92d9-8eededfb9243 was generated by the Jenkins credentials manager after I created a global domain credentials entry.
After creating the credentials entry, the ID is found under the "ID" column on the credentials page. When creating the entry, I selected type 'SSH Username with private key' ('Kind' field), and copied the RSA private key I had created for this purpose under the myuid account on host non_jenkins_svr without a passphrase.

Related

Gitalb SSH into ubuntu server

in my local dev windows machine I generated shh key using PuttyGen. I also pasted public key into gitlab ssh keys section so now are linked.
I can correcty use ssh now from my windows manchine but I want to use it also in my production server which uses ubuntu.
For example I wan to ssh clone a repository into my ubuntu machine, where and how should I add the ssh keys to my ubuntu server so I can link it with gitlab.
I used this tutorial to generate ssh keys in windows with Putty.
https://ourcodeworld.com/articles/read/1421/how-to-create-a-ssh-key-to-work-with-github-and-gitlab-using-puttygen-in-windows-10
How should I add the ssh keys to my Ubuntu server so I can link it with GitLab?
Ideally, you would create a dedicated key pair on your Ubuntu server, in order to be able to clone GitLab repositories.
On that Ubuntu server, go to your $HOME folder of your account 'user' (replace user by the actual user name you are login with on that server).
cd
# assuming you do not have a default key yet:
ssh-keygen -t rsa -P "" -f ~/.ssh/id_rsa
# copy ~/.ssh/id_rsa.pub to your GitLab account
# Check the key is working
ssh -Tv git#gitlab.com
# Use your key to clone repositories
git clone git#gitlab.com:me/myRepository

How can I `git push` to GitHub on Ubuntu using a single command, without having to type in my email each time?

I want to git push on Ubuntu via a single command, such as:
echo -e "email\ntoken" | git push origin branchName
git push origin branchName && email && token
But after the command I have to put in my email:
How to use ssh keys to easily push to / pull from GitHub
You need to:
Configure your remote to use the ssh version of the GitHub repo address instead of the http version.
Generate a public/private ssh key pair, and add the public key to your GitHub account manually via your web browser.
Details
Configure your remote to use the ssh version of the GitHub repo address instead of the http version. Ex:
For this repo of mine: https://github.com/ElectricRCAircraftGuy/eRCaGuy_hello_world, use this ssh URL: git#github.com:ElectricRCAircraftGuy/eRCaGuy_hello_world.git instead of this HTTPS one: https://github.com/ElectricRCAircraftGuy/eRCaGuy_hello_world.git:
# View your current remote servers and their URLs
git remote -v
# Set your `origin` remote server to use the ssh URL instead
# of the HTTPS one
git remote set-url origin https://github.com/ElectricRCAircraftGuy/eRCaGuy_hello_world.git
Generate a public/private ssh key pair, and add the public key to your GitHub account manually via your web browser.
See my full notes on ssh stuff here: https://github.com/ElectricRCAircraftGuy/eRCaGuy_dotfiles/tree/master/home/.ssh
# generate a public/private ssh key pair
ssh-keygen -t ed25519 -C "your_email#example.com"
# Ensure the ssh-agent is running (this starts the `ssh-agent`)
eval "$(ssh-agent -s)"
# Add your private key to it; update the path to your private key below, as
# required, based on what path you interactively selected above when
# generating the key
ssh-add ~/.ssh/id_ed25519
# Verify what keys have been added to the ssh-agent by listing
# (`-l`) currently-added keys.
# A. If you see "Could not open a connection to your authentication agent.",
# it means the `ssh-agent` has not been started yet, so you must start it
# with `eval "$(ssh-agent -s)"`.
# B. If you see "The agent has no identities.", it means the ssh-agent is
# running but you haven't added any ssh keys to it, so run `ssh-add
# path/to/private_key` to add a key to the agent.
ssh-add -l
Now log into github in a web browser and click on your profile image --> Settings --> SSH and GPG keys (on left) --> New SSH key --> copy and paste the contents of your .pub key file (ex: run cat ~/.ssh/id_ed25519.pub on your Ubuntu machine to read the public key--adjust that path as necessary if you used a different file name) into GitHub here --> click "Add SSH key".
Now, whenever you type git push it automatically works, using your ssh key.
References
My full ssh notes: https://github.com/ElectricRCAircraftGuy/eRCaGuy_dotfiles/tree/master/home/.ssh
You can provide the username once as part of the https git remote address.
First run git remote -vv to get the full, current remote URL.
Then to change your existing remote, you can do a command like:
git remote set-url origin https://yourname#github.com/yourname/yourrepo.git
where the new part is yourname# (substitute your github user name) and the rest of the URL should be the same as shown in git remote -vv

How do I access a remote (local gitlab instance on remote server) repository over SSH?

The setup is as follows:
remote private server far far away
remote private server has private gitlab instance on port XXXX
remote private server is configured to allow SSH sign-on via SSH key
gitlab instance on port XXXX of remote private server requires SSH key authentication using different SSH key
How can I clone that repository onto my local machine, and push/pull data remotely given that setup?
This is how I access it locally when I am not far, far away from remote private server:
git clone git#XXX.XXX.XX.X:REPODIR/repo_name.git
In this case, XXX.XXX.XX.X is the IP of the local git-lab instance on the remote network.
Is there anyway to tunnel into the remote network and access the gitlab instance by proxy (forgive me for using the word wrong likely).
Thank you.
Ok, mostly thanks to #o11c for this, although here are my findings that led me to be able to clone my repo remotely.
Disclaimer: ProxyJump (-J see ssh manpage) is the shorthand, more modern, version of this but I couldn't get it working -- if anyone wants to update with their implementation of ProxyJump that would be useful!
SSH into your remote account to the main server with port to your gitlab or other application instance, using your main identity (this can be in ~/.ssh or you can manually reference it with -i)
ssh -ND 3131 nkunes#XXX.XXX.1.146 -i ../../keys/XXX-ssh &
I then source this bash script in the shell I intend to run git commands (notice the ProxyCommand usage instead of ProxyJump, this is the old method of doing this yet it works well for me. also notice the 127.0.0.1:PORT should be swapped with your application's port)
alias ssh="ssh -o ProxyCommand='/usr/bin/nc -X 4 -x 127.0.0.1:3131 %h %p'"
export GIT_SSH=~/Desktop/XXX-eng/ssh-access/ssh-proxy.sh
export PRE_SSH_ALIAS_PROMPT="$PS1"
export PS1="<< SSH ALIAS >>$PS1"
Where ssh-proxy.sh is defined as follows: (again, swap the port out for your application, and possibly use ProxyJump if want modern implementation)
ssh -o ProxyCommand='/usr/bin/nc -X 4 -x 127.0.0.1:3131 %h %p' "$#"
Then, you can clone normally using:
git clone git#XXX.XXX.XX.X:REPODIR/repo_name.git

EC2 ssh-add identity doesn't "stick"

I'm trying to connect my Atlassian BitBucket with an AWS EC2.
I followed all the right steps and it's working. The one thing that got me into trouble was launching the ssh-agent with eval ssh-agent -s and then ssh-add mybitbucket.pub to add the identity.
However, the issue is that the identity does not persist. Meaning that if log back in, in order to do any git operations, I have to do eval ssh-agent -s and ssh-add mybitbucket.pub again.
[root#ip-10-0-1-112 themes]# ssh-add -l
The agent has no identities.
Any recommended workarounds?
Steps taken so far:
Login EC2
Sudo su -
ssh-keygen -t rsa
eval ssh-agent -s
ssh-add mybitbucket.pub
copy the key in BitBucket's web interface.
Thanks!
In the case where you only need the key when you are ssh'd to the instance, you can set up ssh-agent forwarding. This means that when you connect to a specified host, the remote server is allowed to use the keys from your local ssh-agent in order to connect to things, such as the bitbucket account.
So, what you could do is add your public key to the BitBucket account, which would then allow you to access BitBucket via ssh because your local machine has your private key. Then, by enabling ssh-agent forwarding, when you ssh to the EC2 instance, you allow that instance to use your private key access BitBucket without ever storing your private key on the instance.
Here's an article on how to set this up:
https://developer.github.com/guides/using-ssh-agent-forwarding/
In short, add the following to your ~/.ssh/config:
Host example.com
ForwardAgent yes
Where example.com is the public IP of your AWS instance, or the EIP assigned to it, etc.

How to allow jenkins from local machine to run remote python test scripts

I have a jenkins running on my local centos machine.
I have configured my local jenkins and was able to run a successful local build .
Now, i want to run remote tests which are python scripts on a remote centos machine which is not having jenkins installed. also, i dont want to install any jenkins process on the remote linux system as it is "like a" production server and am advised not to install any apps on it.
How do i use my local jenkins to run a build to execute those remote tests and report/output on my local jenkins console.
Do i need to use jenkins master-slave architecture ? if yes, how do i configure that given my above requirement.
You might want to have a look at this:
https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds
for you req, precisely this part:
https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds#Distributedbuilds-Launchslaveagentheadlessly
However, i believe you still have to have java on your slave unix node to run the slave.jar on it
This answer is assuming the scripts are in GitHub. May it helps to think in your case.
So.. First you need to install Git in you server machine by:
$ sudo apt-get update
$ sudo apt-get install git
Now you need to get the path of Git by $ which git
it will give like "/usr/local/bin/git"
copy that path into ManageJenkins->Global Tool Configuration-> in the git section, paste into "Path to Git executable".
it will allows you to access git sources.
Now you need to provive SSH keys.
Type sudo su- jenkins in you remote machine.You have to generate ssh key for "jenkins" user.
Now add public key to GitHub account(You can see https://www.youtube.com/watch?v=Vi-WqFKYpnw).
and add the private key to Jenkins by
Go to Credentials
Click in Global in Stores scoped
Add Credentials
Kind: SSH Username with private key
Username: your server username
Private Key: give the private key of user "Jenkins"
Specify ID as "jenkins-private-key" or anything else to identify
Now
Go to job configuration->select credentials that you have created and
Copy the ssh url of repository(Where you scripts are stored) Now you can run the scripts which are stored in Git.

Resources