We are migrating SVN to Git. On a project.
Everything is well but I have a problem with preserving the permissions (file mode) of some untracked (ignored) dirs and files.
We the following process:
CleanUp the SVM repo (tidy up, clean uncommited stuff and so forth)
svn2git on local environment (migrating from networked SVN - so latest rev, no further commits, this does include a proper authors.txt mapping and proper tracking of branches, tags, etc.)
Create ignored files & dirs li (.gitignore)st - partially automated from svn propget, manual finish (row by row)
Add the remote and push to it (We're using github but it should hardly matter)
At this stage the remote is ready, all the history is converted. Now comes the hard part I want to hot-migrate the server deployments without changing the dirs, moving files or creating symlinks.
Let's say there are two mirrored environments are on two different servers - one for beta and one for prod. Let us say the dir is /var/www/depl/ for example/. The thing is that as every Web project, there are dirs and files that we don't need to track.
I will be writing my steps wit the commands since I think it could also be a nice guide for others.
So the strategy for on the servers is:
Go there cd /var/www/depl/
BackUp to another dir via rsync and preserve the permissions as they are!!
Delete all .svn directories recursively find -type d -name .svn -exec rm -rf {} \;
Init an empty git repo git init (notice we're not using a 'bare' repo)
Add the GitHub remote and call it "origin", also download all branches git remote add -f https://github.com/ORG/REPO
Check status (the local copy should be clean)
Pull (so fetch + merge but it is actually only a merge) git pull origin master
This last step is what breaks my permissions. Remember those files that I don't want/need to track? It now seems that those have their permissions modified.
They are all properly ignored but when I apply the pull/merge it breaks.
OK, for sure the issue is coming from the remote repository and via the merge. And it is not an issue, but rather how the files were commited (with what file modes).
The question is:
At the last step, when pulling the updates in, can I instruct git to preserve the current file permissions for all files? As in the current dir and recursively down?
So: do not change any local permissions?
Yes, maybe I will have a diff and stuff to commit afterwards, but that is not such a big issue as are the broken permissions.
So? Is there a way? The servers are running Linux of course.
Thanks in advance.
Cheers!
Well, I managed to find a way. The 'secret', as I thought, was in using rsync.
I actually solved my issue, while I was writing the question. :)
Anyway, after some research it turns out that rsync --archive will update the target's permissions provided that the timestamps didn't change. The SAMBA mailing list helped a lot!
A prerequisite is that you ensure that NOTHING WILL CHANGE in those dirs - so put the website in maintenance mode, stop all crons. Make it inaccessible so you don't have headaches later.
The migration (after you have migrated all you history into git) steps are as follows:
Take your production env to the SVN stage that you desire - update, checkout etc. Make sure that this is the same as the incommig git copy and/or keep your changes in patches.
BackUp the deployment directory with rsync rsync -av --progress /var/www/depl/ /var/www/deplBackUp1/ - notice that this includes the .svn dirs
Delete the .svn dirs: cd /var/www/delp/ find -type d -name .svn -exec rm -rf {} \;
Now backUp again in a different location rsync -av --progress /var/www/depl/ /var/www/deplBackUp2/ - the goal is to have two copies
Init the new git repo git init
Add a remote git remote add -f REMOTE_NAME https://github.com/ORG/REPO
Pull everything in from the branch you desire git pull origin master
Now rsync back from your (no-.svn dir backUp) rsync -av --progress /var/www/deplBackUp2/ /var/www/depl
You may also want to look into using bare git repositories for your server deployments.
This last step will fix any file mode issues you may have created with your new VCS.
If you ever need to do this and run into trouble, tag me in a comment - I will try to help.
Cheers!
Related
I completed MyProject1 and have uploaded it in git in fine way with commits after adding each new features. And now I'm starting MyProject2 and was trying to add the URL for the remote repository. But then I found out that I had mistakenly added the URL in Documents instead of MyProject1 folder because of which MyProject2 folder is also being tracked in MyProject1. And I'm not being able to add URL to MyProject2 but instead facing merge issues.
Is there any way to detach the track from my Document folder without deleting any of my files from Documents.
Structure is this way(I'm using Linux):
Documents
(And inside Documents there is:)
MyProject1
MyProject2
And other folders which are also being tracked.
We can manually do it with below steps
create a file .gitignore in base directory.
If MyProject1 and MyProject2 are already part of git tracking, Please run commands
git rm -r --cached MyProject1/
git rm -r --cached MyProject2/
Open the file in text editor and add below lines in file
MyProject1/
MyProject2/
Best way:
Create a .gitignore file following #Vinayagam R
Ignore file locally
Those methods won't affect other contributors working on the same remote repository:
Use update-index:
If you want to stop tracking a file at certian point.
git update-index --assume-unchanged yourDirectoryName
--assume-unchanged is the flag which means the files should not change locally. In other words, it is used when ignore files that you do not need to change locally (or should not change).
To revert it use update-index --no-assume-unchanged yourDirectoryName
Using .exclude
In your working directory edit .git/info/exclude
I am new to git and I keep all my git repos in one folder inside an NTFS disk so that both Windows and Linux partitions can see it. The problem is that executable files in that folder can't be given execution permissions in Linux.
For the repos that contain executable files, I've been working in a directory of the Linux partition, so that I can execute them, and then once I'm done editing them, I copy them back to the relevant repo folder to keep version-controlling them.
Side note (for GolezTrol):
When I say executables, I mean, for example, *.cpp or *.java files. If I want to compile those on an NTFS partition, I will get an error if I try to run the executable. So I can't work in the NTFS folder, I have to work in the Linux partition. Think for example, the workspace folder in Eclipse. I can't make any app work if I place it in the NTFS folder. So I have to move it to the Linux partition, edit my files, and then copy the relevant files back to the repo folder in the NTFS partition. I'm not tracking the executables, I'm tracking the files that produce those executables. I hope this clears it up.
I was wondering if there is a way to keep the version-control of those files in their repo folder, while the files live in the Linux partition, so that I don't have to copy them back when I'm finished editing, and I can still see them from Windows. Something like a link to the executables folder that lives in the repos folder, maybe (it would probably be nice to be able to edit them from Windows too, as if they really lived in the repo folder, but I guess that won't be possible).
I searched SO and found a similar question. I tried the first answer, although it's not exactly what I'm aiming for, but it didn't work for me. I placed the .git file in my executables folder, containing:
gitdir: path/to/repo/in/NTFS/disk/.git
and then initialized the repo, but I got an error:
$ git init
fatal: Not a git repository: path/to/repo/in/NTFS/disk/.git
So I went back to the repo folder (in the NTFS disk), did git init and came back to the executables folder. Now I get a different error:
$ git init
fatal: Invalid gitfile format: path/to/executables/folder
I think I didn't understand the answer.
Regarding the second answer, I couldn't understand how to implement it either.
So, wrapping up, my question is... how can I have the executable files in a folder in the Linux partition but version control them from the repo folder in the NTFS disk? Would it be possible to make Windows see those files?
P.S. I would like to avoid (if possible) getting into mounting or editing fstab files.
P.S2. This question is different to the one I linked. I'm thinking more of a sort of link to the executables folder from the repo folder. It would be ideal to see the files from Windows too. If that is not possible, I'm open to other answers, since the answers in the linked question didn't work for me. Had those answers solved my problem, I wouldn't be asking this question.
Update:
My executables directory:
$ tree -L 1 -aF --dirsfirst
.
├── file.txt
└── .git
0 directories, 2 files
Contents of file.txt: file
Contents of .git: gitdir: /media/admin/DATA/github/af-62/.git
My NTFS directory (that is, /media/admin/DATA/github/af-62):
$ tree -L 1 -aF --dirsfirst
.
├── .git/
├── .gitignore
└── LICENSE.md
1 directory, 2 files
(Please don't pay too much attention to the files, they're placeholder files for now.)
The -a option shows hidden files, as those whose name starts with a dot, and the --dirsfirst shows directories first, so in the first case, .git is at the end while in the second .git is at the beginning.
In the NTFS directory:
$ git log
commit f8f7e3bac01f51e5d819a31e28c1a42c181b0407
Author: private <user#example.com>
Date: Wed Jul 15 11:50:44 2015 +0200
First commit.
$ git checkout /path/to/executables/folder/file.txt
fatal: /path/to/executables/folder/file.txt: '/path/to/executables/folder/file.txt' is outside repository
In the executables directory:
$ git init
fatal: Invalid gitfile format: /path/to/executables/folder/.git
$ git checkout file.txt
fatal: Invalid gitfile format: .git
$ git checkout /media/admin/DATA/github/af-62/LICENSE.md
fatal: Invalid gitfile format: .git
According to jvdm's linked resource, "[...the .git file...] It must point to a valid Git bare repository", but /media/admin/DATA/github/af-62/ is a valid repository, isn't it? What am I doing wrong?
You need to tell git that /path/to/executables/folder is the working
dir of the git repository /path/to/repo/in/NTFS/disk/.git, one way
to do it:
cd /path/to/executables/folder
echo gitdir: /path/to/repo/in/NTFS/disk/.git > .git
Now notice that a git status (if /path/to/executables/folder was
empty) will show all your files as deleted. Then you can:
git checkout .
To checkout them into your working dir.
I've already read several posts about this here (like Git ignore & changing the past, How to remove files that are listed in the .gitignore but still on the repository?, and Applying .gitignore to committed files), but they have several problems:
Commands that only work on Linux.
Incomplete commands (like the first post I've linked to).
Only for one file.
I have pretty much no experience with Git so I was hoping for some help here.
What I'm basically trying to do is rescue one of my projects history. It's currently Hg and I converted it to Git with Hg-Git (all very easy) and it includes the history (great!). However, I also added a .gitignore file and added several new files & folders that I want completely gone from the history (like the bin and obj folders, but also files from ReSharper). So I'm looking for a way to apply the .gitignore file to all of my history. The commands should work on Windows as I have no intention of installing Linux for this.
No need to add the .gitignore in the history (there is no added value to do it), just add it for your future commits.
For the remove of files and directories in your history, use bfg-repo-cleaner which is fast, easy and works very well on Windows (done in scala).
It will do the job for you!
This is working for me:
Install hg-git.
cd HgFolder
hg bookmark -r default master
mkdir ../GitFolder
cd ../GitFolder
git init --bare
cd ../HgFolder
hg push ../GitFolder
Move all files from GitFolder to a '.git' folder (in this GitFolder) and set this folder to hidden (not the subfolders and files).
cd ../GitFolder
git init
git remote add origin https://url.git
Copy all current content (including .gitignore) to GitFolder.
git add .
git commit -m "Added existing content and .gitignore".
git filter-branch --index-filter "git rm --cache d -r --ignore-unmatch 'LINES' 'FROM' 'GITIGNORE'" --prune-empty --tag-name-filter cat -- --all
git rm -r --cached .
git add .
git gc --prune=now --aggressive
git push origin master --force
There is probably an easier way to do this and it might not be perfect but this had the result I wanted.
When I do php programming, I tend to edit in-place on the (development) server (cpanel VPS, not local) via a text editor that has built in sFTP. I like this because there isn't 100 thinkgs you have to do to "publish" the changes, you save (via FTP) and reload the page.
I want to get the source into SVN mostly so I can track other people's edits.
I currently use tortoiseSVN on my windows machine to access hosted SVN repos (csvdude/codison) for non web projects so I'm familiar with that. Linux SVN, not so much.
My thought here (open to other ideas) is that I want the public_html folder on the server to be the working copy (Is there a problem with this?). People can make edits as they would via sFTP and then the working copy would be committed to the external SVN repo, possibly on a late night cron..hoping I can set it up to auto-add and commit.
So aside from thoughts on the idea, I'm curious how to set up the linux svn client to commit to external repos a la tortoiseSVN. It seems all the tutorials I see set up a local repo.
Additionally I'm worried when I check out with public_html dir as the working copy that it will overwrite everything. (yes I have backups...)
Any input or suggestions would be awesome.. Thanks.
I created a new repo on my hosted SVN service.
I manually created a public_html folder using the repo browser.
On the cPanel server:
cd /home/[account]/
svn checkout https://svnservice.com/my_repo/public_html
I set up an ignore list for directories in public_html I didn't want in the repo.
Then you can add files and directories to the repo
cd public_html
svn add [directory] // adds directory for commit
svn add --force . // adds all new files without ignoring the ignore list
In hindsight, I think the import command would have done something similar, I just didn't quite understand it enough.
This is a useful command as well:
svn status | grep '^\!' | sed 's/! *//' | xargs -I% svn rm % // removes files from repo that no longer exist in the working copy
EDIT: One thing I should mention is to make sure your .htaccess blocks access to .svn directories or it could expose source code.
So, my situation is the following: I want to maintain my website using emacs-muse mode. For transferring the website (and version control), I want to use git. So I would create a hook in the remote (bare) repository that automatically copies the HTML subdir to the web directory. How can I do that from a hook? Also note that the whole directory contains a lot of files, so I can't export the whole directory.
Any help welcome.
You can use git archive, as it takes a path command. So, in your post-update hook you can do something like
git archive $SHA HTML | (cd dir/where/html/should/go && tar x)
this first creates a tarball of the subdir, and pipes that tarball to a tar command to untar it in the specified directory
(just a suggestion, as it may not be applicable in your exact configuration)
You might consider adopting this Web-Focused Git Workflow which, instead of copying, does push your repo directly on the web directory.
(From Joe MALLER)
Less complicated: Using Git to maintain your website (Daniel MIESSLER):
Daniel MIESSLER has an updated version of that same process:
Other suggestions are available at this SO question, like this post-update script mentioned in the Git FAQ.