I'd like to copy a file to a directory without changing the modification timestamp of the directory on an ext4 filesytem. Well, many files and many directories in a script.
I've looked at rsync and cp options.
So to frame the question, how do I copy a file on an ext4 filesystem and preserve the timestamp on the destination directory?
There are many ways to copy files and preserve their attributes but they modify the parent directory timestamp. What's needed is the two step process of recording that timestamp and applying it after the copy. That was not addressed in the question referenced. Giving a file/directory the same modification date as another
One option is to save and restore the timestamp:
# Save current modification time
timestamp=$(stat -c #%Y mydir)
[.. copy/sync files ..]
# Restore that timestamp
touch -d "$timestamp" mydir
See this guid. If you want to preserve original timestamp use
$ touch -r <original_file> <new_file>.
This copy the attributes from another file. If you need to change specific attributes use following.
For Access time
$ touch -a --date="1988-02-15" file.txt
For Modify time:
$ touch -m --date="2020-01-20" file.txt
To see stat of a file or folder, Use stat <file>
$ cp --preserve=timestamps oldfile newfile
or
$ cp -p oldfile newfile would do that. Man pages clearly state that.
Related
I need to copy a file from multiple locations to the BACK UP directory by retaining its directory structure. For example, I have a file "a.txt" at the following locations /a/b/a.txt /a/c/a.txt a/d/a.txt a/e/a.txt, I now need to copy this file from multiple locations to the backup directory /tmp/backup. The end result should be:
when i list /tmp/backup/a --> it should contain /b/a.txt /c/a.txt /d/a.txt & /e/a.txt.
For this, I had used the command: echo /a/*/a.txt | xargs -I {} -n 1 sudo cp --parent -vp {} /tmp/backup. This is throwing the error "cp: cannot stat '/a/b/a.txt /a/c/a.txt a/d/a.txt a/e/a.txt': No such file or directory"
-I option is taking the complete input from echo instead of individual values (like -n 1 does). If someone can help debug this issue that would be very helpful instead of providing an alternative command.
Use rsync with the --relative (-R) option to keep (parts of) the source paths.
I've used a wildcard for the source to match your example command rather than the explicit list of directories mentioned in your question.
rsync -avR /a/*/a.txt /tmp/backup/
Do the backups need to be exactly the same as the originals? In most cases, I'd prefer a little compression. [tar](https://man7.org/linux/man-pages/man1/tar.1.html) does a great job of bundling things including the directory structure.
tar cvzf /path/to/backup/tarball.tgz /source/path/
tar can't update compressed archives, so you can skip the compression
tar uf /path/to/backup/tarball.tar /source/path/
This gives you versioning of a sort, as if only updates changed files, but keeps the before and after versions, both.
If you have time and cycles and still want the compression, you can decompress before and recompress after.
I am doing a clean up activity on files based on the time with which they create , however I also have to take a backup with scp without modifying the ctime or mtime of files
Later I will use find command to pick the qualified files with mtime in my shells script
find $V_Filepath -mmin +7200 -name "*.ack"
To keep the mtime unchanged scp -p option would help in terms of doing scp between two servers.
eg:
scp -pr myfile.* username#server://path/
Comparing the mtime (using stat command) before and after remains same. scp -p option preserves the mtime of the files
eg:
stat -c '%y' myfile.*
Modification time is the mtime, not ctime. scp -p already preserves mtime.
ctime is the inode change time, updated every time the file itself is touched in any way – renamed, moved, chmodded, etc.
Generally, there is no way to preserve it, as the OS does not provide any function for that, and even if it did, the very act of setting the ctime would be a change that would cause the ctime to be updated again.
Hello I am trying to copy all files from Documents directory to the backup directory that has a timestamp. So I have created a folder called bk$( the time stamp of the folder) and I am trying to copy files from the Documents directory to the new created folder that is unique. This will be in a crontab backing up files from documents and when the backup will kick in, it will create new directory for each backup that is uniquely identified by the folder timestamp. For some reason I cannot get the cp or cpio -mdp. Now someone had mentioned I could use $PATH variable which seems promising, if that is the solution, if someone could help me out on making it work.
bkdest=home/user/backup/
bksource="/home/user/Documents/"
export PATH=/$bkdest:$PATH
mkdir /"$bkdest"bk.$(date +%Y_%m_%d_%H_%M_%S)
cp /"$bksource"* $PATH
My other approach which I have tried to use to make it work:
cp $bksource * ls | tail -l | $PATH
I could have gone with the ctime but unfortunately it does not work with the folder creation date.
This was my approach but with the latest created folder and not file
find $HOME -type d -daystart ctime 0
If someone could please help me out to copy to that new folder, I would really appreciate it. Thank you!
Store the target name in a variable:
bkdest=/home/user/backup
bksource=/home/user/Documents
target=${bkdest}/bk.$(date +%Y_%m_%d_%H_%M_%S)
mkdir -p $target
cp ${bksource}/* ${target}/
Note I tidied up your use of variables a little.
Also, this won't copy subdirectories. For that you need to use cp -R. When I do backups I prefer to use rsync.
I did not fully understand your approach or what exactly you want to do but here it goes.
CP Approach
You should not use cp for backups, rsync is far more suitable for this. But if for some reason you really need to use cp, you can use the following script.
#!/bin/bash
BKP_DIR=/tmp/bkp
BKP_SRC=/tmp/foo
SNAPSHOT=${BKP_DIR}/$(date +%F.%H-%M-%S.%N)
mkdir -p ${SNAPSHOT}
cp -r ${BKP_SRC}/* ${SNAPSHOT}
Rsync Approach
No big change here.
#!/bin/bash
BKP_DIR=/tmp/bkp
BKP_SRC=/tmp/foo
SNAPSHOT=${BKP_DIR}/$(date +%F.%H-%M-%S.%N)
rsync -a ${BKP_SRC}/ ${SNAPSHOT}/
Improved Rsync Approach (RECOMMENDED)
#!/bin/bash
BKP_DIR=/tmp/bkp
BKP_SRC=/tmp/foo
SNAPSHOT=${BKP_DIR}/$(date +%F.%H-%M-%S.%N)
LATEST=${BKP_DIR}/latest
rsync \
--archive \
--delete \
--backup \
--backup-dir=${SNAPSHOT} \
--log-file=${BKP_DIR}/rsync.log \
${BKP_SRC}/ ${LATEST}/
EXPLAINING: --archive plus --delete will make sure that $LATEST is a perfect copy of $BKP_SRC, it means that files that no longer exist in $BKP_SRC will be deleted from $LATEST. The --archive option also ensure that permissions and owners will be maintained, symlinks will be copied as symlinks, and more (look at man rsync for more information).
The --backup plus --backup-dir options will create a backup directory to put differential files. In other words, all files that were deleted or modified since last backup will be put in there, so you do not lost them as they are deleted from $LATEST.
--log-file is optional, but it is aways good to keep logs for debug purposes.
At the end you have an incremental backup.
I have a directory containing a set of subdirectories and files. I need to recursively copy all the content of this directory to all the subdirectories of another directory, also recursively.
How do I achieve this, preferably without using a script and only with the cp command?
You can write this in a script but you don't have to. Just write it line by line in the terminal:
# $TARGET is the directory containing subdirectories where you want to STORE the copies
# $SOURCE is the directory containing the subdirectories you want to COPY
for dir in $(ls $TARGET); do
cp -r $SOURCE/* $TARGET/$dir
done
Only uses cp and runs on both bash and zsh.
You can't. cp can copy multiple sources but will only copy to a single destination. You need to arrange to invoke cp multiple times - once per destination - for what you want to do; using, as you say, a loop or some other tool.
The first part of the command before the pipe instruct tar to create an archive of everything in the current directory and write it to standard output (the – in place of a file-name frequently indicates stdout).
tar cf - * | ( cd /target; tar xfp -)
The commands within parentheses cause the shell to change directory to the target directory and untar data from standard input. Since the cd and tar commands are contained within parentheses, their actions are performed together.
The -p option in the tar extraction command directs tar to preserve permission and ownership information, if possible given the user executing the command. If you are running the command as superuser, this option is turned on by default and can be omitted.
Also you can use the following command, but it seems to be quite slower than tar;
cp -a * /target
Can Rsync be configured to verify the contents of the file before they are being synced. I have heard about checksum, but I came to know that checksum only does a sampling. I want to transfer a file only if it is contents are changed and not timestamp, is there a way to do it with any of the rsync modes.
In my scenario, say file sample.text will be created every week and I want to sync it with a remote server only if the contents of sample.text are changed, since it is created every week, the time stamp would obviously change. But I want the transfer only on a content change.
Yes:
$ man rsync | grep "\--checksum"
-c, --checksum skip based on checksum, not mod-time & size
Rsync is pretty complicated. I recommend cuddle time with the man page and experimentation with test data before using it for anything remotely important.
Most of the time, people use rsync -ac source dest.
$ man rsync | grep "\--archive"
-a, --archive archive mode; same as -rlptgoD (no -H)
And that -rlptgoD garbage means: recursive (r), copy symlinks as symlinks (l), preserve permissions (p), preserve times (t), preserve group (g), preserve owner (o), preserve device files (D, super-user only), preserve special files (also part of D).
The -c or --checksum is really what you are looking for (skip based on checksum, not mod-time & size). Your supposition that rsync only samples mtime and size is wrong.
See the --checksum option on the rsync man page.
Also, the --size-only option will be a faster choice if you know for sure that a change of contents also means a change of size.
Example:
#!/bin/bash
echo > diff.txt
rsync -rvcn --delete /source/ /destination/ > diff.txt
second_row=$(sed -n '2p' diff.txt)
if [ "$second_row" = "" ]; then
echo there was no change
else
echo there was change
fi