I have several servers that can't be reach via SSH or HTTP directly but they are able to connect to internet.
I would like to find a smart way to deploy automatically updates via git. I could do a script that periodically do a git pull but it seems not the best way.
Which is the best way to manage git deploy in this scenario?
You might want to look into "git bundle"
I have used this in "air gapped" systems before.
git bundle create <file> <git-rev-list-args>
git bundle verify <file>
Once you create the bundle, you can use "git clone" or "git pull" against the bundle.
This is from the man page for "bundle":
Some workflows require that one or more branches of development on one machine be replicated on another
machine, but the two machines cannot be directly connected, and therefore the interactive Git protocols (git,
ssh, rsync, http) cannot be used. This command provides support for git fetch and git pull to operate by
packaging objects and references in an archive at the originating machine, then importing those into another
repository using git fetch and git pull after moving the archive by some means (e.g., by sneakernet). As no
direct connection between the repositories exists, the user must specify a basis for the bundle that is held by
the destination repository: the bundle assumes that all objects in the basis are already in the destination
repository.
you just need to copy the bundle to a machine that you can reach from the other system and then copy the file over and just do a git pull from the bundle
Git 2.25 (Q1 2020) explains why.
See commit 0e40a73 (20 Oct 2019) by Philip Oakley (PhilipOakley).
(Merged by Junio C Hamano -- gitster -- in commit 7c88714, 10 Dec 2019)
Doc: Bundle file usage
Signed-off-by: Philip Oakley
Improve the command description, including paragraph spacing.
Git URLs can accept bundle files for fetch, pull and clone, include in that section. Include git clone in the bundle usage description. Correct the quoting of <git-rev-list-args>.
Detail the <git-rev-list-args> options for cloning a complete repo.
So the updated git bundle documentation now includes:
The 'git bundle' command packages objects and references in an archive at the originating machine, which can then be imported into another repository using 'git fetch', 'git pull', or 'git clone', after moving the archive by some means (e.g., by sneakernet).
Also:
git clone can use any bundle created without negative refspecs (e.g., new, but not old..new).
If you want to match git clone --mirror, which would include your refs such as refs/remotes/*, use --all.
If you want to provide the same set of refs that a clone directly from the source repository would get, use --branches --tags for the <git-rev-list-args>.
Related
I have a VPS running a Node.js / React app.
I want to update the files in the vps each time I push data to the git(hub).
I found out, using this answer, that I can add some hooks in git, executing commands on "post-receive".
What I didn't quite understand :
Why did he init another git ? Couldn't he have done this in the .git directory and create the hooks/post-receive file?
Why git checkout -f ? If the goal is to update local files, so nodemon / create-react-app restarts the server / app, why not execute a git pull instead ?
Is there a better way of doing this ?
In the recommended answer there, nobody is using GitHub and there is no other Git repository yet. So the answer to your question:
Couldn't he have done this in the .git directory and create the hooks/post-receive file?
is: No, there was no .git directory in the first place. The target machine had nothing at all, no Git repository, no working tree, etc. The git init --bare created the Git repository (the ".git directory").
The git checkout -f is a poor-man's / low-quality implementation of push to deploy. A receiving repository is normally "bare", and this one is no exception.
why not execute a git pull instead ?
That would require creating a third Git repository. That would have been an option.
"Better" is in the eye of the beholder. There are many ways of doing this, each with its own pluses and minuses. See also Deploy a project using Git push, which notes that since Git 2.3, receive.denyCurrentBranch = updateInstead is available; it was not available prior to 2015 (and in 2015, many people had older versions of Git installed).
Note further that if you're using GitHub as a hosting system, this changes a number of variables. The questions and answers you and I have linked here are aimed at those not using GitHub.
This question already has answers here:
How do I move my local Git repository to a remote Git repository
(9 answers)
Closed 6 years ago.
I apologize if this has been asked somewhere else and I missed it. I've found many questions and answers that are close to what I'm trying to do, but none that fit exactly.
I have local files and projects in a local git repository. I created the repository a few months back as a way to safeguard and track my own work. The rest of our company uses SVN so I am the first to use git here. I set it up in my home directory on a Linux server. This repo contains many commits, branches, and tags that span my work over the past few months.
I want to move that repo to a more accessible location within our company's Linux server (not in my home directory) so others can clone and work with the entire repo (including all the tags, branches, history, etc.) as they see fit.
I've thought about copying or moving the entire file structure, but I want it to be a single file - i.e. myRepo.git.
How do I do this so that myRepo.git contains all the information I've committed such that when someone clones this repo and runs 'gitk --all', they will see all the branches, tags, etc.?
Thanks for your help.
You can git clone --bare your repo from somewhere other people can access it.
The most basic remote Git repo is nothing but a bare clone. A bare clone means it doesn't have a checkout, it's just the contents of the .git directory. Having a checkout would mean people might try to use that checkout as their own git working directory and make changes and that would get confusing with other people pushing changes to the repository.
For example, I'll make a new repository in foo/ and commit a file.
$ git init foo
Reinitialized existing Git repository in /Users/schwern/tmp/foo/.git/
$ rm -rf foo
$ git init foo
Initialized empty Git repository in /Users/schwern/tmp/foo/.git/
$ cd foo
$ touch this
$ git add this
$ git ci -m 'First commit'
[master (root-commit) 1a0bddf] First commit
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 this
Then I'll make a bare clone of it into bar/ (cloning doesn't have to happen over a network).
$ git clone --bare foo bar
$ ls
HEAD config description hooks info objects packed-refs refs
Instead of the checkout, it has the contents of the .git directory.
Now I can clone from that.
$ git clone bar baz
Cloning into 'baz'...
done.
$ cd baz/
$ git log
commit 1a0bddf415f48d71fa3aacc79c07c233d91ed4a9 (HEAD -> master, origin/master, origin/HEAD)
Author: Michael G. Schwern <schwern#pobox.com>
Date: Tue Feb 14 12:01:13 2017 -0800
First commit
That's cloning on a local filesystem. Git will, of course, clone over a network. Typically it uses ssh or https. git clone user#host/path is really sshing into host as user and copying what's in path.
So you can put your repo anywhere the other folks have ssh access, like a dev server, or the same host that has the SVN server.
That's the most bare bones way to share your Git repository. There are plenty of more sophisticated Git servers out there that provide web access and more protocols. Everything from the very sparse GitWeb to Gitlab, a Github clone you can host locally for free.
You can read more about this in Pro Git's chapter on Git Servers.
A fully functional git repo is not a single file, and there is no way to make it so. That said, copying your working repo isn't the best thing to do either.
If having a single file is more important that having the repo be fully functional - i.e. you don't care if your coworkers can push to it - then you can create a bundle file.
git bundle create /path/to/my/repo.bundle --all
Your coworkers can clone from this, add it as a remote, pull or fetch from it... but they can't push to it (and neither can you; to add to it you'd have to recreate it).
A shared repo to be cloned from should normally be a bare repo. This is roughly equivalent to a copy of the .git directory from your working repo. If your existing repo is at /path/to/my/repo, you can create a suitable bare repo at /path/to/shared/repo by saying
cd /path/to/shared
git clone --mirror /path/to/my/repo
Note that I used --mirror instead of --bare; this has to do with how refs are copied and will work better if you intend this to be treated as the origin henceforth.
And yet at this point /path/to/my/repo is still considered the upstream. To fix that
cd /path/to/shared/repo
git remote remove origin
cd /path/to/my/repo
git remote add origin /path/to/shared/repo
I know there is git branch -a that lists the branches in the repository but how do I see the files inside this branch when the repository is remote (can't access it from the github website).
You can also use git ls-files.
Short of using the GitHub API or another hack, it's not really possible to do this without pulling down the repository first. However, you can if you do pull from the remote.
In order to fetch all branches from remote, do git fetch --all. Then you can view all your branches via git branch and checkout to them via git checkout -b [branch]. Then you can simply view the files as you please from your remote using standard command line using ls, cd, cat, etc. If you're really rebelious you can try Midnight Commander.
I recognise it might be slow pulling down large repositories over limited bandwidth (try pulling it through a VPS you SSH into?), but unfortunately there isn't much choice.
Just installed a fresh new 6.8 Gitlab on a brand new high performance server.
Before considering to forget my repositories history (comments, issues, etc...), do one know of a way to export a repository data from a Gitlab server to another Gitlab Server ?
I just failed to found anything on the documentation for exporting/migrating the whole project data (not just the git repository and its wiki).
For GitLab versions >= 8.9 (released in June 2016) you can use the built-in export and import project feature.
Please not that for the existing installations of GitLab, this option has to be enabled in the application settings (URL: /admin/application_settings) under 'Import sources'. You have to be a GitLab admin user to enable and use the import functionality.
Here is the feature complete documentation: https://gitlab.com/help/user/project/settings/import_export.md
I have actually done this recently, we were upgrading our instance of gitlab and needed to save and import repositories to the new installation.
First, create a bundle of the checked-out repository. For example, say you checked out a repository we will call myrepository
To check out the repository use git clone (let's assume your repository is under the root account and the ipaddress is 192.168.1.1)
git clone http://192.168.1.1/root/myrepository.git (or match your environment)
Now this step is somewhat important; you need to change into the working directory that has the .git folder of your checked out repository.
cd myrepository
Next, you create a bundle file:
git bundle create myrepository.bundle --all
Import the bundle file into the new instance of gitlab.
Create a new 'myrepository' on the gitlab gui interface
clone the empty repository; let's say this new gitlab has the ipaddress 192.168.1.2:
git clone http:\\192.168.1.2\root\myrepository.git (or match your environment)
You will get warnings that you cloned an empty repository. This is normal.
Change into the working directory of your checked out repository and do a git pull:
cd myrepository
git pull file/path/to/myrepository.bundle
this will pull the repository into your clone. Next you can do a git add, git commit and git push
This should work assuming you have the gitlab server settings set up correctly; you may have issues such as needing to add a client_max_body_size parameter in your nginx.conf file and also a 'git config --global http.postBuffer' to push large files.
Another way to do this is to make patch files of each commit and then deploy them:
This involves doing a 'git format-patch -C 0badil..68elid -o patch_directory_path' and reference the range of all your commits and have them pushed to an output directory; this should give you one patch file per commit. Next would involve git cloning the new empty repository, changing into the working directory of the clone and applying the patches to the new repository using 'git am patch_directory_path'
For GitLab versions < 8.9, without built-in export/import feature, I recommend a great tool from Marcus Chmelar, gitlab-migrator. I used it successfully many times with older GitLab versions so you should too. Just be aware of its limitations.
For the repos themselves, you can use git bundle: that will generate one file, that it is easy to copy around.
(as I described in "Backup a Local Git Repository")
But another way is simply to git clone --mirror your repos from the first server on a local workstation, and git push --mirror to the new server.
This is what GitHub details in its help page "Duplicating a repository".
In both cases, you need first to declare those repos on the new GitLab server, in order for them to be initialized, and ready to receive commits.
But for the rest... not easily. There is a feature request pending:
(Update August 2016, 2 years later: GitLab 8.9 has that feature implemented)
(for GitLab version older than 8.9, see and upvote Greg Dubicki's answer)
I agree that issues are the main thing to make exportable first.
They are stored in the database. Storing them in git is not an option. Maybe export them as a formatted file (SQL, YAML or something else).
This blog post illustrates the export of a mysql database.
Use mysqldump to create dump of old database, then create a new database on the new server and import this.
On old:
mysqldump gitlab | gzip > gitlab.sql.gz
On new:
gunzip < gitlab.sql.gz | mysql gitlab
Run the db migrate command to make sure the schema is updated to the latest version.
sudo -u gitlab -H bundle exec rake db:migrate RAILS_ENV=production
First off, forgive me if this is a duplicate question. I don't know anything but the basic terminology, and it's difficult to find an answer just using laymen's terms.
I made a project, and I made a repository on Github. I've been able to work with that and upload stuff to it for some time, on Windows. The Github Windows application is nice, but I wish there was a GUI for the Linux git.
I want to be able to download the source for this project, and be able to edit it on my Linux machine, and be able to do git commit -m 'durrhurr' and have it upload it to the master repository.
Forgive me if you've already done most of this:
The first step is to set up your ssh keys if you are trying to go through ssh, if you are going through https you can skip this step. Detailed instructions are provided at https://help.github.com/articles/generating-ssh-keys
The next step is to make a local clone of the repository. Using the command line it will be git clone <url> The url you should be able to find on your github page.
After that you should be able to commit and push over the command line using git commit -am "commit message" and git push
You can use SmartGit for a GUI for git on Linux: http://www.syntevo.com/smartgit/index.html
But learning git first on the command line is generally a good idea:
Below are some basic examples assuming you are only working from the master branch:
Example for starting a local repo based on what you have from github:
git clone https://github.com/sampson-chen/sack.git
To see the status of the repo, do:
git status
Example for syncing your local repo to more recent changes on github:
git pull
Example for adding new or modified files to a "stage" for commit
git add /path/file1 /path/file2
Think of the stage as the files that you explicitly tell git to keep track of for revision control. git will see the all the files in the repo (and changes to tracked files), but it will only do work on the files that you add to a stage to be committed.
Example for committing the files in your "stage"
git commit
Example for pushing your local repo (whatever you have committed to your local repo) to github
git push
What you need to do is clone your git repository. From terminal cd to the directory you want the project in and do
git clone https://github.com/[username]/[repository].git
Remember not to use sudo as you will mess up the remote permissions.
You then need to commit any changes locally, i.e your git commit -m and then you can do.
git push
This will update the remote repository.
Lastly if you need to update your local project cd to the required directory and then:
git pull
To start working on the project in linux, clone the repo to linux machine. Add the ssh public key to github. Add your username and email to git-config.
For GUI you can use gitg.
PS : Get used to git cli, It is worth to spend time on it.