I am working on a Jenkins job that will download a specific file from a GIT repo and then deploy it on a server to a specified location.
Jenkins job will ask below variables from the user:
Commit ID and name of the file in repo that needs to be copied.
Path on the target server where the file needs to be copied.
Here's what I have done so far:
Installed and configured the GIT so it will pull the repo to the Jenkins server
Configured the target servers so Jenkins can copy the file to one of the 3 target servers using 'execute shell option'.
In the build action, wrote an ssh command that will checkout a specific commit of a file from the git repo.
Wrote a shell command that will scp the checked out file and to the one of the 3 target servers. Of course the target server will be provided by the user.
Here's what the code looks like so far:
git checkout $commit $filename
scp $filename username#$servername:/path/to/deploy/
The challenge is how to feed the variables to the shell script.
Is it even possible to do it this way? Any help would be appreciated.
Below solution worked.
Added the GIT repo to the Jenkins freestyle job
After checking the option, 'this project is parameterized', I added 3 variables as shown.
Then modified the shell script as below
#!/bin/bash
pwd
ssh ansible#$server_name "cp "$File_path"/"$file_name" "$File_path"/"$file_name"_"$(date +"%Y-%m-%d")";"
scp $file_name ansible#$server_name:$File_path
Related
So I have inherited an application, and am not too happy with the current deployment workflow. There is a docker instance running on an ec2 server, and developers have been merging their local work like this:
rsync -arhvz --progress ./ ubuntu#52.12.345.67:/home/ubuntu/app --exclude node_modules
And then going into the server itself and running a deploy.sh script. I would simply like a push event to the master branch to trigger this sync and deploy. How is it possible? As far as I know we don't pay for a Docker hub account - we don’t have private docker registry.
Some solutions :
use a git pre-push hook script which will execute the script before each local push to remote
create a bare git repo on your server, push your change to this git remote and checkout the last change using post-receive hook
track push events on Github with Github webhook, you will need to have your server listen to a port in order to receive the webhook events
pre-push hook
In your local repository, edit/create .git/hooks/pre-push file :
#!/bin/bash
rsync -arhvz --progress ./ ubuntu#52.12.345.67:/home/ubuntu/app --exclude node_modules
ssh ubuntu#52.12.345.67 "/home/ubuntu/deploy.sh"
This way, your modification will sync before each local push to remote. The deploy.sh is also triggered
setup a git repo on your server and deploy to it using git
You can create a bare git repo on your server and create a post-receive hook script that will use git to checkout your server repo. From this guide :
#!/bin/bash
while read oldrev newrev ref
do
# only checking out the master (or whatever branch you would like to deploy)
if [[ $ref =~ .*/master$ ]];
then
echo "Master ref received. Deploying master branch to production..."
git --work-tree=/home/ubuntu/app/deploy-folder/ --git-dir=/home/ubuntu/app/project.git/ checkout -f
else
echo "Ref $ref successfully received. Doing nothing: only the master branch may be deployed on this server."
fi
done
Note that this solution only uses git
Github Webhook
You can use this docker-hook project which is a Python server listening for a POST on your webhook uri. It was originally used for Docker Hub webhook but also works for Github webhook (though not parsing the events).
Webhook
On your server :
download docker-hook python script
curl https://raw.githubusercontent.com/schickling/docker-hook/master/docker-hook > /usr/local/bin/docker-hook; chmod +x /usr/local/bin/docker-hook
start it with a token (generated with uuidgen for example) :
docker-hook -t 3ea4e9d8-8fff-47e5-a704-65ab21de6963 -c /path/to/deploy.sh
In Github, go to your repo settings, create the webhook with the token as path :
Check that the webhook is working (in Recent Deliveries tab)
Deploy key
Now, as you want to trigger a git pull on your server, you will need to generate a Deploy Key which is a ssh key used to deploy (read-only), you will need, on server side :
install git if not already installed
generate a key
copy this key in your repo/settings/Deploy key tab
See this post for a complete tutorial
Now you need to edit your deploy.sh script to perform a git pull or a fresh git clone to the location of your choice
We are using Gitlab (7.5.3).
I created a Repository, but I want to upload my project to it.
I've looked on the repository page for an upload button of some kind but I haven't seen anything of the sort.
I've looked at the links provided so far but I'm still getting nowhere. They mention command line, is that Windows command line ?
I am able to upload through Git GUI, how to upload without Git GUI?
And when I login to user account through putty ,it shows
login as: root
root#192.168.1.5's password:
Last login: Tue May 26 12:20:10 2015 from 192.168.1.12
![grep: write error]
Does anyone knows how we can solve these issue?
If you have Git GUI, you probably also have Git Bash which provide command line support for git inside Windows.
You can use this Git Bash to upload your project. For this, you just need some git command, and you don't have to log in the Gitlab's server (I supposed it's why you tried). This command are provided by Gitlab when you create a new project and want to upload it.
First open Git Bash. Then :
cd c:\Users\Me\..\my_project\ # Go to your project directory in Windows
git init # Initialize this directory as a git repo
git remote add origin git#your_gitlab_server_ip:your_username/your_project_name.git
git push -u origin master` # Send your code to your Gitlab instance
It's also a good idead to provide to git some info about you. This infos will be displayed by Gitlab :
git config --global user.name "Your Name"
git config --global user.email "your_name#your_mail.com"
Remember that with Gitlab the only interaction between your git's repo and Gitlab should happend through the git command, and not with login to this server.
Try this 4 step push project to gitlab --> open command line.
cd to/path/local
git init
git remote add anything_name name http://scmgit.ktb/username/projectname.git
git push -u test anything_name master
Hope is save you day.
SOLUTION BELOW - How to use git to push to cpanel server
I finally got somewhere with setting up Git between my localhost (WAMP setup on Windows 8.1) and my Linux server (CentOS 6.6 x64 with cPanel 11.46.2).
Locally I created a bare clone: git clone --bare my_project my_project.git
NOTE: my_project is an example name, not the real name, and from this doc here: http://git-scm.com/book/en/v2/Git-on-the-Server-Getting-Git-on-a-Server
I copied the my_project.git folder to my server's root directory /home/myuser/public_html/
so now in the root directory I have:
cgi-bin
my_project.git
This is one area I am unsure of. Do I have to do an init (using putty) on my server in the public_html directory? I read something about a bare init? I just want to push (from my PC) the website I already have under Git control, to the server. When I make a change to 1 file, push that change to the server so it's updated live with a push. The website is DONE and ready to be live. I have already manually moved back and forth for live testing on the server. My last step is to get the Git setup correctly, so any further changes I can just push them to the server without the need of FTP.
I added a remote origin: git remote add origin ssh://myuser#thedomain.com/home/myuser/public_html/my_project.git
I tried to push to it, and got "Permission denied (publickey)". I already had an id_rsa and id_rsa.pub key locally on my PC, so I copied them and renamed them to id_rsa.myname id_rsa.myname.pub (where myname is my first name). I then copied them to the .ssh folder through FTP (FTP as cpanel user, and it's the directory your dumped into, above public_html), same as /home/myuser/.ssh/ directory.
Once they where there, I added them to 'authorized_keys' using Putty logged in as the cpanel user (my private ppk) by doing:
cd .ssh
cat id_rsa.myname >> ~/.ssh/authorized_keys
cat id_rsa.myname.pub >> ~/.ssh/authorized_keys
After doing that, a push appeared to work. Because I was having a key/auth issue, I used Git Gui version, which was setup and worked fine locally. I added the origins through Git Bash though. When I did "Remote > Push" in the Gui version, I got:
Pushing to ssh://theregistrybank#theregistrybank.com/home/theregistrybank/public_html/yiire gistrybank.git
stdin: is not a tty
To ssh://myuser#thedomain.com/home/myuser/public_html/my_project.git
44ae034..0388a05 master -> master
updating local tracking ref 'refs/remotes/origin/master'
Before doing the push, the only file modified (diff from the bare clone I transferred to the server) was my .gitignore file. I added 2 more exclusions to it, and committed it locally. So I was trying to push the change in that file. After I did the push, it said "success" in green and appeared to work. However, when I check the file in FileZilla, the .gitignore file is not the updated one that I just committed locally.
I think I am close, but missed a step somewhere. I tried to be as descriptive as possible.
And putting the source on GitHub is not an option as the client does not want the source public, and does not want to pay for the private repos. I should be able to push from my local setup to the cPanel server so I don't have to transfer thousands of files every time. I actually transfer a zip file, and unzip on the server lol.
Server Info
cPanel 11.46.2 build 0
CentOS 6.6 x86_64 kvm build01
Yes, Git is setup on the server, and working, and git --version reports:git version 1.7.1
Git on my PC: git version 1.9.4.msysgit.2
Thank you in advance.
SOLUTION
Thanks to #VonC I was able to get this to work :)
You need somewhere for your git repo to sit. I created a 'git-repos' folder in '/home/cpaneluser/git-repos' to house my repos for this cpanel user.
First step is to create a bare repo: http://git-scm.com/book/en/v2/Git-on-the-Server-Getting-Git-on-a-Server - I only followed the first step, basically created the bare repo 'my-project.git'
Before putting it on your server, rename 'my-project.git/hooks/post-receive.sample' to just 'post-receive' so it will be ran. Edit it with your editor, and add the line that #VonC gave us in his chosen answer:
#!/bin/sh
umask 0022
GIT_WORK_TREE=/home/cpaneluser/public_html GIT_DIR=/home/cpaneluser/git-repos/my-project.git git checkout -f
Note: I am using cpanel, so your path's may be different, and your umask could be different. 0022 is for 0644 file permissions. Without the umask, I was getting 500 Internal Server Errors, because the files were created with 0664 permissions instead.
Using FTP or whatever you like, copy the 'my-project.git' bare repo to your server to '/home/cpaneluser/git-repos'. Then go into 'my-project.git/hooks' and change the permissions of post-receive to have execute permissions. For me, 0744 worked fine. This was the magic sauce :)
Locally, add your remote (must be in your git project): git remote set-url origin ssh://cpaneluser#yourdomain.com/home/cpaneluser/git-repos/my-project.git
Now if you try to push now, it won't put the files in 'public_html' because the git tree (terminology?) matches and is up to date. If they are up to date, it seems to skip over executing your 'post-receive' hook. That means your bash script never ran, and it never checked out the files to your working tree.
We need to manually run the 'post-receive' bash script to create all the files of our project in the 'public_html' directory.
cd to '/home/cpaneluser/git-repos/my-project.git/hooks'
Run: ./post-receive
Boom, all your files are in 'public_html'. Now you can work locally, then push to your cpanel server as expected :)
I had a similar problem with my cPanel account, git was set up but for some reason I couldn't push to it. After a lot of head-banging I realized that the root of the problem is like you pointed out that although git is set up, the repo is empty so there's nothing for it to track.
This happens when you set up an empty repo in cPanel and then try pushing a local repo that you've already created, as opposed to cloning it first from github or your local repo (which is what happened to me because for some reason I couldn't access my github from the cPanel git interface even though I had set up a SSH key)
The simple solution that I found is to manually clone the repo using the terminal in your cPanel account
On your cPanel dashboard, under the "advanced" section, you'll find the terminal. You'll get a warning saying that you could mess up your server if you don't know what you're doing, click ok and you're in.
Now you just have to clone your repo the same way you would if you're cloning onto a local machine.
Navigate to where git was set up in your cPanel
cd repositories/<nameOfYourRepo>
And run the clone command
git clone <URLofYourGithubReop>
You'll be asked for your github username and password if it's a private repository
And that's it, you're good to go
What you have copied (my_project.git) is a bare repo, meaning one without a working tree (the actual checked out files).
Read for instance "Git workflow - Setting up a build process".
That means pushing to if won't change anything in /home/myuser/public_html/
The missing piece is a post-receive hook (in /home/myuser/public_html/my_project.git/hooks/post-receive, make sure it is executable: chmod +x), in order to checkout the repo in /home/myuser/public_html/.
#!/bin/sh
GIT_WORK_TREE=/home/myuser/public_html GIT_DIR=/home/myuser/public_html/my_project.git git checkout -f
I would like to pull the changes as user 'ubuntu' during startup.
The upstart file is:
description "Custom startup script"
start on filesystem
script
cd /var/www/gitstuff
git checkout master
git pull
end script
When I rebooted I got this log message:
Already on 'master'
Host key verification failed.^M
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
I have a feeling when the upstart script runs it is being executed as root user.
Used this for the pull statement:
git pull ubuntu#example.com:/etc/drupal/7/gitstuff
Also had to add keys to the root user
try:
script
cd /var/www/gitstuff
su - ubuntu git checkout master
su - ubuntu git pull
end script
I have written scripts for Windows and Linux to essentially set up a new users workspace with all the git repositories from our server.
I would like the user to enter the password for our server once, store it in a local variable, pass that variable to each git pull command, then erase the password variable and exit.
How can I input the password when the git pull command requests it? Both for Windows batch file and a Linux shell script.
Here is code from the Linux script:
#!/bin/bash
echo "Enter password: "
read pswd
clear #No screen peaking
#This is repeated for each repo
location=folderName
mkdir $location
cd $location
git init
git remote add origin git#<server>:$location.git
git pull origin master
#Above prompts for password & is where I want to automatically input $pswd
I've tried various things recommended on SO and elsewhere, such as piping, reading from .txt file, etc. I would prefer to not need anything more than plain old windows cmd and Linux terminal commands. And as this script is just for set up purposes, I do not need to securely store the password permanently with something like ssh agent.
I'm running Windows 7 and Ubuntu 12.10, but this script is meant for setting up new users, so it should ideally work on most distributions.
Synopsis:
git pull "https://<username>:<password>#github.com/<github_account>/<repository_name>.git" <branch_name>
Example:
git pull "https://admin:12345#github.com/Jet/myProject.git" master
Note: This works for me on a bash script
I would really recommend to not try and manage that password step, and delegate that (both on Linux and Windows) to git credential helper.
See:
"Git http - securely remember credentials"
"How to use git with gnome-keyring integration"
The user will enter the password only once per session.
Read the remote url from git and then insert the ID and password (PW) to the url might work.
For example try the following:
cd ${REPOSITORY_DIR}
origin=$(git remote get-url origin)
origin_with_pass=${origin/"//"/"//${USER_ID}:${USER_PW}#"}
git pull ${origin_with_pass} master