On my project, there are two branches I'm working on (Develop and Release), each of which makes generous use of submodules. The Develop branch uses about twice as many submodules as Release, because it is where we test ideas.
When I switch branches from Develop to Release, the directories of Develop-specific submodules stay where they are, and so they become untracked. This makes things a bit confusing for me, because I do occasionally need to add or remove submodules from Release as well, and the git status message becomes a long list of untracked modules, some of which I want to use and some I don't.
What I would like to do is remove all untracked submodules from my project as soon as I switch from Develop to Release, so that I'm working with a "clean slate", (IE no untracked submodules sitting in my working directory).
I have found several solutions to removing individual submodules one at a time, such as here: How do I remove a submodule?
However solutions such as this assume that the git submodules are in use and being tracked (which they are not), and it is also a pain in the neck to remove them one at a time when I'm working with something like 15-20 submodules.
I have also tried piping linux commands like so:
git ls-files --others --exclude-standard | rm -rf
But the command does not appear to do anything. I have also tried using the same with git rm -rf, to no avail.
Does anyone know if there is an easy to remove all untracked git submodules from a working directory? Any advice anyone can share on this matter would be greatly appreciated. Thank you!
With some advice from helpful folks in the comments section, I determined that there is no obvious, non-tedious solution to this problem. Instead I created a bash script that does the job for me. Here it is, in case anyone else has the same issue:
#!/bin/bash
clear
git ls-files --others --directory --exclude-standard
echo
read -r -p "Are you sure you want to remove these untracked submodules? [y/N] " response
if [[ $response =~ ^([yY][eE][sS]|[yY])$ ]]
then
git ls-files --others --directory --exclude-standard | while read line; do
rm $line/.git &> /dev/null;
done
git clean -d -f
fi
Easy steps for people not super comfortable with Bash scripts:
Step 1: Copy the above script into a file and name it 'cleanUSM'. Save the file to /usr/bin. If you are having trouble saving it or finding /usr/bin, just save it to your current directory, and then use 'sudo mv cleanUSM /usr/bin/cleanUSM' to get it where it needs to go.
Step 2: From within your root directory, run the command 'cleanUSM'
Thanks to everyone who contributed!
Related
I have a git repo that has a folder with about 70,000 files in it. This has been a pain so instead of using the 70,000 files I have compressed them into one tar.gz.
I'm now trying to remove the directory from the git repo, but it seems to be taking a LOT longer than I would think. I'm wondering if git is just hanging every time I try to remove the folder.
I do see git running when I do an htop:
This is the command I am using to remove the folder:
git rm -r <folder_name>
Is there a better way to do this?
**Update: It eventually did finish, took about 20 mins.
A faster way would be to separate the index update (recording the deletion) from the filesystem actual deletion:
git rm -r --cached <folder_name>
rm -Rf <folder_name>
git commit -m "delete folder"
I am using git in my project at Linux platform. I have plenty of files in a particular directory. I modified some 50 above files in that directory and didn't stage and commit it. I wish to delete all other unmodified files from that directory? Is there a way to do this, using git and Linux Commands?
Not sure why you would want to do this.... but you can:
# Save changes to stash
git stash save
# Remove everything left
rm -rf ./*
# Checkout (restore) all of the changed files
git stash show --stat | grep -v changed | sed -e 's/|.*$//;' | xargs git checkout
# Restore the changes to those files
git stash pop
git reset --hard [HEAD] should work for you repeated
Repeated question How can I discard modified files?
You can also use more simple commands for this purpose:
git clean -Xfd // capital X
git clean -xfd // lower x
It will clean your working directory from the desired files.
Using git clean is what you want. To remove (-x) those files and directories (-d), run:
$ git clean -fdx
If you use the -X option instead of -x, then the files you have told git to ignore will still be kept (e.g., build artifacts). Recent versions of git require either "-f" (force) or "-n" (dry-run) to be specified.
You should run a dry-run first, to show what will happen, but not actually do anything:
$ git clean -ndx
I use this so often, that I have an alias for this (added to your .gitconfig) to check for files that would be deleted when you run git clean. It's also useful to remind me if I've forgotten to "git add" a file that I want to keep.
[alias]
# list files that would be removed via 'clean' (non-destructive)
ifc = clean -ndx
Then, running git ifc (i.e,. "ifc" = "if clean") shows everything that isn't tracked and could be removed, or isn't tracked and should be added.
https://git-scm.com/docs/git-clean
I've got a git workflow set up similar to this http://joemaller.com/990/a-web-focused-git-workflow/. Essentially i have local repositories that report to a remote repository that is bare. Then I have my deployment directory accessible via web also set up as a repository that also reports to the same bare repository.
It's set up with git hooks so that when local developer pushes changes to remote repository, hook goes into web folder and pulls from the repository so it always has the latest and greatest. All works pretty good.
My crux is that I'm looking to appease to the people who don't want to you GIT and just want to upload files to the web folder via FTP. I've kinda got this working by setting up an inotifywait monitor on the web folder for whenever files are written, modified, moved, deleted, created, etc... my bash script for this is as follows.
!/bin/sh
inotifywait #*.swp -rm -e modify,move,create,delete,delete_self,unmount /var/www/html/mysite | while read
do
now=$(date +"%m_%d_%Y:%T")
echo $now >> temp.txt
cd /var/www/html/mysite || exit
git add --all
git commit -m "ftp update $now" -a
done
This too actually works, but what I'm observing is that I'm stuck in the while loop once I trigger the inotifywait by modifying a file in my web folder. Anyone have any ideas on this? Ideally would love it to do it's thing and not be stuck in the while loop continuously running unnecessary git commands.
Thanks!
The man page for inotifywait suggests that you do a different loop style:
while inotifywait -e modify /var/log/messages; do
…
done
have you tried that?
I am experimenting some linux configuration and I want to track my changes? Of course I don't want to to put my whole OS under version control?
Is there a way (with git, mercurial or any VCS) to track the change without storing the whole OS?
This is what I imagine:
I do a kind of git init -> all hashes of all files are stored, but not the content of the files
I make some changes to my file system -> git detect that the hash of this file has changed
I commit -> the content of the file is stored (or even better the original file and the diff are stored! I know, that is impossible... )
Possible? Impossible? Work-arounds?
EDIT: What I care about is just to minimize the size of the repository and to have a repository containing only my changes. Having all files in my repository is not relevant for me. For example if i push to github I just want it to contain only the files that has changed.
Take a look at etckeeper, it will probably do the job.
What you want is git update-index --info-only or ... --index-info, from the man page: " --info-only is used to register files without placing them in the object database. This is useful for status-only repositories.". --index-info is its industrial-scale cousin.
Do that with the files you want to track, write-tree to write the index structure into the object db, commit-tree that, and update-ref to update a branch.
To get the object name use git hash-objectfilename.
Here is what we do...
su -
cd /etc
echo "*.cache" > .gitignore
git init
chmod 700 .git
cd /etc; git add . && git add -u && git commit -m "Daily Commit"
Then setup crontab:
su -
crontab -e
# Put the following in:
0 3 * * * cd /etc; git add . && git add -u && git commit -m "Daily Commit"
Now you will have a nightly commit of all changes in /etc
If you want to track more than /etc in one repo, then you could simply do it at the root of your filesystem, except add the proper ignore paths to your /.gitignore. I am unclear on the effects of having git within git, so you might want to be extra careful in that case.
I know this question is old, but I thought this might help someone. Inspired by #Jonathon's comment on the How to record concrete modification of specific files question, I have created a shell script that enables you to monitors all the changes done on a specific file, while keeping all the changes history. the script depends on the inotifywait and git packages being installed.
You can find the script here
https://github.com/hisham-hassan/linux-file-monitor
Usage: file-monitor.sh [-f|--file] <absolute-file-path> [-m|--monitor|-h|--history]
file-monitor.sh --help
-f,--file <absolute-file-path> Adding a file to the monitored files List. The <absolute-file-path>
is the absolute file path of the file we need to action.
PLEASE NOTE: Relative file path could cause issues in the script,
please make sure to use the abolute path of the file. also try to
avoid sym links, as it has not been tested.
example: file-monitor.sh -f /absolute/path/to/file/test.txt -m
-m, --monitor Monitoring all the changes on the file. the monitoring will keep
happening as long as the script is running; you may need to run it
in the background.
example: file-monitor.sh -f /absolute/path/to/file/test.txt -m
-h, --history showing the full history of the file.
To exit, press "q"
example: file-monitor.sh -f /absolute/path/to/file/test.txt -h
--uninstall uninstalls the script from the bin direcotry,
and removes the monitoring history.
--install Adds the script to the bin directory, and creates
the directories and files needed for monitoring.
--help Prints this help message.
I have a git clone/repo on a development server, but I am now moving to another one. I don't want to commit all my local branches and changes to the main repository, so how can I make an exact copy of everything on oldserver to newserver?
I tried oldserver:~$ scp -rp project newserver:~/project
but then I just get loads and loads of "typechange" errors when trying to do anything on newserver.
Someone said something about x-modes, but how can I preserve that when moving files between servers?
If you want a git solution, you could try
git clone --mirror <oldurl> <newurl>
though this is only for bare repositories.
If this is a non-bare repo, you could also do the normal clone, followed by something like this:
git fetch origin
git branch -r | grep '^ *origin/[^ ]*$' |
while read rb; do git branch --no-track ${rb#*/} $rb; done
git remote rm origin
The middle step can of course be done in 5000 different ways, but that's one! (note that the continuation line \ isn't necessary after the pipe in bash - it knows it needs more input)
Finally, I'd suggest using rsync instead of scp (probably with -avz options?) if you want to directly copy. (What exactly are these typechange errors?)
I've actually done this, and all I did was tar the repo up first and scp it over. I would think that scp -rp would work as well.
"Typechange" would normally refer to things like a symlink becoming a file or vice-versa. Are the two servers running the same OS?
You may also want to try the simple dumb solution -- don't worry about how the typechanges got there, but let git fix them with a reset command:
git reset --hard HEAD
That only makes sense if (1) the problems all pertain to the checked-out files (and not the repository structure itself) and (2) you haven't made any changes on newserver which you need to preserve.
Given those caveats, it worked for me when I found myself with the same problem, and it doesn't require you to think about git's internals or how well your file-transfer process is preserving attributes.