On my local machine I have several plugins, and on the server I have a few.
Local Machine:
~/wp-content/plugins/plugin1
~/wp-content/plugins/plugin1/includes
~/wp-content/plugins/plugin2
~/wp-content/plugins/plugin3
~/wp-content/plugins/plugin4
~/wp-content/plugins/plugin5
Remote Machine:
~/wp-content/plugins/plugin1
~/wp-content/plugins/plugin1/includes
~/wp-content/plugins/plugin3
~/wp-content/plugins/plugin5
What rsync command can I use to update all the files in the remote directories, but only if they exist? For this I would like to have plugin1, plugin1/includes, plugin3, and plugin5 synced - with all files and directories inside - but not plugin2 or plugin4. Basically, update all of the plugins if they exist with a single rsync command.
This is not completely possible with only one rsync command. If you only want to update the remote files, but do not want to add new files, you can use this command:
rsync -rP --existing source/ user#remote:target/
This will not create new files or directories at all, but update differing ones.
Edit: Maybe you could do something like this (assuming GNU find, if BSD/OS X: replace maxdepth with depth):
#!/bin/bash
REMOTE_TARGET_DIR=/some/path
REMOTE_DIRS=$(ssh user#remote "find $REMOTE_TARGET_DIR -maxdepth 1 -type d -printf '%P\n' | sed 1d"
for DIR in $REMOTE_DIRS
do
rsync -rP "$DIR" "user#remote:$REMOTE_TARGET_DIR/"
done
Warning: I have not tested this, you might want to add a "-n" to rsync (dry run) to see what happens.
Related
I try to move DB2 Archive Log files to a /backup filesystem before I do additional actions on them. Important here is that we preserv the full path.
tar -czvf $OUTFILE db2/???/log_archive/db2???/???/NODE*/LOGSTREAM*/C*/* --remove-files
At the moment I use tar from the root,but it has some downsides; if prefer that the files are simply moved.
So therefore I am playing with rsync, like for instance:
rsync -nrv --include='/db2/???/log_archive/*' --include '*.LOG' --exclude '*' --prune-empty-dirs / /backup
But what ever I try.... or I have just (nearly) all files and folders from root, or nothing at all.
Does anyone have a good idea?
When I noticed that rsync can work with "files-from", I managed it in a totally other way.....
So to move the DB2 Log files to a different /backup filesystem and preserving full path, I can use find and then rsync
find /db2/???/log_archive/* -name *.LOG -type f -mmin +1 | rsync -rv --files-from=- --remove-source-files / /backup/
sending incremental file list
db2/
db2/BWP/
db2/BWP/log_archive/
db2/BWP/log_archive/db2bwp/
db2/BWP/log_archive/db2bwp/BWP/
db2/BWP/log_archive/db2bwp/BWP/NODE0000/
db2/BWP/log_archive/db2bwp/BWP/NODE0000/LOGSTREAM0000/
db2/BWP/log_archive/db2bwp/BWP/NODE0000/LOGSTREAM0000/C0000000/
db2/BWP/log_archive/db2bwp/BWP/NODE0000/LOGSTREAM0000/C0000000/S0006869.LOG
db2/BWP/log_archive/db2bwp/BWP/NODE0000/LOGSTREAM0000/C0000000/S0006870.LOG
db2/BWP/log_archive/db2bwp/BWP/NODE0000/LOGSTREAM0000/C0000000/S0006871.LOG
db2/BWP/log_archive/db2bwp/BWP/NODE0000/LOGSTREAM0000/C0000000/S0006872.LOG
db2/BWP/log_archive/db2bwp/BWP/NODE0000/LOGSTREAM0000/C0000000/S0006873.LOG
etc etc etc
Here I use 'find' to select the files (older then 1 minute to ensure DB2 is finished with archiving this file) and then transport this list of files to 'rsync' who moves them to the new location.
I have a folder called documentaries on my Linux computer.
I have SSH access to seedbox (also Linux).
How do I find out which documentaries I have in both computers?
On seedbox it's a flat file structure. Some documentaries are files, some are folders which contain many files, but all in same folder
For example:
data/lions_botswana.mp4
data/lions serengeti/S01E01.mkv
data/lions serengeti/S01E02.mkv
data/strosek_on_capitalism.mp4
data/something_random.mp4
Locally structure is more organized
documentaries/animals/lions_botswana.mp4
documentaries/animals/lions serengeti/S01E01.mkv
documentaries/animals/lions serengeti/S01E02.mkv
documentaries/economy/strosek_on_capitalism.mp4
documentaries/something_random.mp4
I am not looking for command like diff, I am looking for command like same (opposite of diff) if such command exists.
Based on the answer from Zumo de Vidrio, and my comment:
on one computer
cd directory1/; find | sort > filelist1
on the other
cd directory2/; find | sort > filelist2
copy them in one place an run:
comm -12 filelist1 filelist2
or as a one liner:
ssh user#host 'cd remotedir/; find|sort' | comm -12 - <(cd localdir/; find|sort)
Edit: With multiple folders this would look as follows
on one computer
cd remotedir/; find | sort > remotelist
on the other
cd localdir/subdir1/; find > locallist1
cd -;
cd localdir/subdir2/; find > locallist2
cd -;
#... and so on
sort locallist1 locallist2 > locallistall
copy them in one place an run:
comm -12 remotelist locallistall
or as a (now very long) one liner:
ssh user#host 'cd remotedir/; find|sort' | comm -12 - <({cd localdir/subdir1/; find; cd -; cd localdir/subdir2/; find; cd -; cd localdir/subdir3/; find}|sort)
Export list of remote files to local file by:
ssh user#seedbox 'find /path/to/data -type f -execdir echo {} ";"' > remote.txt
Note: On Linux you've to use absolute path to avoid leading ./ or use with "$PWD"/data.
Then grep the result of find command:
find documentaries/ -type f | grep -wFf remote.txt
This will display only these local files which also exist on remote.
If you would like to generate similar list on local and compare two files, try:
find "$PWD"/documentaries/ -type f -execdir echo {} ';' > local.txt
grep -wFf remote.txt local.txt
However above methods aren't reliable, since one file could have a different size. If files would have the same structure, you could use rsync to keep your files up-to-date.
For more reliable solution, you can use fdupes which can find all files which exist in both directories by comparing file sizes and MD5 signatures.
Sample syntax:
fdupes -r documentaries/ data/
However both directories needs to be accessible locally, so you can always use sshfs tool to mount the remote directory locally. Then you can use fdupes to find all duplicate files. It has also option to remove the other duplicates (-d).
Copy the ls output of each Computer to a same folder and then apply diff over them:
In your computer:
ls -R documentaries/ > documentaries_computer.txt
In seedbox:
ls -R documentaries/ > documentaries_seedbox.txt
Copy both files to a same location and execute:
diff documentaries_computer.txt documentaries_seedbox.txt
You can mount remote folder using sshfs, then you can use diff -r to find the differences between them.
E.g.
sshfs user#seedbox-host:/path/to/documentaries documentaries/
diff -rs /local/path/documentaries/animals documentaries/ | grep identical
diff -rs /local/path/documentaries/economy documentaries/ | grep identical
Let's assume I have following source directory
source/
subdir1/file1
subdir1/file2
subdir2/file3
subdir3/file4
and target directory
target
subdir1/file5
subdir2/file6
subdir4/file7
I would like to move content of source subdirectories to right target subdirectories so result look like this
target
subdir1/file1
subdir1/file2
subdir1/file5
subdir2/file6
subdir2/file3
subdir3/file4
subdir4/file7
Is there some Linux command to do this or must I write a script myself?
To suimmarize, it is important to move, not copy. That rules out cp and rsync but allows mv. mv, however, has the issue that it is not good at merging the old directory into the new.
In the examples that you gave, the target directory had the complete directory tree but lacked files. If that is the case, try:
cd /source ; find . -type f -exec sh -c 'mv "$1" "/target/$1"' _ {} \;
The above starts by selecting the source as the current directory with cd /source. Next, we use find which is the usual *nix utility for finding files/directories. In this case, give find the -type f option to tell it to look only for files. With the -exec option, we tell it to move any such files found to the target directory.
You have choices for how to deal with conflicts between the two directory trees. You can give mv the -f option and it will overwrite files in the target without asking, or you can give it the -n option and it will never overwrite a target file, or your can give it the -i option and it will ask you each time.
In case the target directory tree is incomplete
If the target directory tree is missing some directories that are in the source, the we have to create them on the fly. This adds just minor complication:
cd /source ; find . -type f -exec sh -c 'mkdir -p "/target/${1%/*}"; mv "$1" "/target/$1"' _ {} \;
The mkdir -p command assures that the directory we want exists before we try to move the file there.
Additional notes
The form ${1%/*} is an example of one of the shells powerful features called "parameter expansion". This particular feature is suffix removal. In general, it looks like ${parameter%word} which tells bash to expand word and remove it from the end of parameter. In our case, the name of the parameter is 1, meaning the first argument to the script. We want to remove the file name and just leave behind the directory that the file is in. So, the word /* tells the shell to remove the last slash and any characters which follow.
The commands above use both single and double quotes. They have to be copied exactly for the command to work properly.
To sync dorectory maybe used rsync
Example:
rsync -avzh source/ target/
More info man rsync
Move (no copy)
rsync --remove-source-files -avzh source/ target/
We are running subversion on a Linux server. Someone in our organization overwrote about 3k files with mixed case when they need to all be lowercase.
This works in the CLI
rename 'y/A-Z/a-z/' *
But obviously will screw up subversion.
svn rename 'y/A-Z/a-z/' *
Doesn't work because subversion deals with renames differently I guess. So How can I do this as a batch job? I suck with CLI, so please explain it like I'm your parents. hand renaming all 3k files is not a task I wish to undertake.
Thanks
Create a little script svn-lowername.sh:
#!/bin/bash
# svn rename each supplied file with its lowercase
for src; do
dst="$(dirname "$src")/$(basename "$src" | tr '[A-Z]' '[a-z]')"
[ "$src" = "$dst" ] || svn rename "$src" "$dst"
done
Make sure to chmod +x svn-lowername.sh. Then execute:
How it works:
Loop over each supplied filename.
Basename part of filename is lowercased, directory part is left alone, using
tr to do character by character translations. (If you need to support
non-ASCII letters, you'll need to make a more elaborate mapping.)
If source and destination filenames are the same we're okay, otherwise, svn rename. (You could use an if statement instead, but for one-liner
conditionals using || is pretty idiomatic.)
You can use this for all files in a directory:
./svn-lowername.sh *
...or if you want to do it recursively:
find -name .svn -prune -o -type f -print0 | xargs -0 ./svn-lowername.sh
If you think this problem might happen again, stash svn-lowername.sh in your
path somewhere (~/bin/ might be a good spot) and you can then lose the ./
prefix when running it.
This is a bit hacky, but should work on any number of nested directories:
find -mindepth 1 -name .svn -prune -o -print | (
while read file; do
svn mv "$file" "`perl -e 'print lc(shift)' "$file"`"
done
)
Also, you could just revert their commit. Administering the clue-by-four may be prudent.
Renames in Subversion require two steps, an svn copy followed by an svn delete if you want to retain history. If you use svn move you will loose the history of the file.
While this answer doesn't help with your question, it will help keep this from happening again. Implement the case-insenstive.py script in your pre-commit hook on your Subversion repository. The next time someone tries to add a file with a different case, the pre-commit hook won't let them.
I need to recursively copy a directory tree, ignoring any subdirectories named 'CVS'. Is there a simple way to do this?
rsync -av --exclude=CVS <src> <dst>
tar -cpf - --exclude=CVS directory | sh -c 'cd /wherever/it/goes && tar -xpf -'
Modify the right-hand tar's options to -xvpf if you'd like to see what's going on.
Why not approach it from a slightly different angle and check out the files from CVS using the export command.
This'll give you the directories without any of the CVS artifacts.