I'm passing a tar command to shell executor in an application. But it seems that my tar syntax is incorrect. (This is Windows (bsdtar command) but works the same as Linux as far as I know; I can also test on Linux if need be.)
I'm trying to tar gz everything up all files ending in ext without storing the full path in my tar file.
tar -cvzf test.tar.gz -C C:/mydir/toTar/ *.ext
I get an error:
tar: *.ext: Cannot stat: No such file or directory
I can give the whole path but then my tar will contain C->mydir->toTar->. I just want the files, not mydir and toTar in the result.
So far only thing that is close to what I want is . instead of *.ext, but that tars other things too, which I obviously don't want.
The problem is that * is a wildcard character that is expanded by the shell, but you are bypassing the shell and calling tar directly. The tar command is looking for one file which is named literally *.ext and it does not exist.
Your options are:
Expand the list of files in your own code and pass that list to tar.
Call the shell from your code by calling something like /bin/sh -c tar ...
With option 2 there may be security implications -- if the shell sees something it thinks is a command, it will run it. Option 1 is therefore safer, but it's up to you which makes more sense.
I am befuddled by how you're using dos-style paths in an apparently linux-like context. But this is how I'd do it. Hopefully the concept is clear if the details may be incorrect.
cd C:/mydir/toTar/
mkdir ~/tmpwork
find . -name '*.ext' > ~/tmpwork/extfiles
tar czvfT ~/tmpwork/test.tar.gz ~/tmpwork/extfiles
rm ~/tmpfiles/extfiles
There is no way around the shell expansion without using pipes, etc.
Related
I need to copy a file from multiple locations to the BACK UP directory by retaining its directory structure. For example, I have a file "a.txt" at the following locations /a/b/a.txt /a/c/a.txt a/d/a.txt a/e/a.txt, I now need to copy this file from multiple locations to the backup directory /tmp/backup. The end result should be:
when i list /tmp/backup/a --> it should contain /b/a.txt /c/a.txt /d/a.txt & /e/a.txt.
For this, I had used the command: echo /a/*/a.txt | xargs -I {} -n 1 sudo cp --parent -vp {} /tmp/backup. This is throwing the error "cp: cannot stat '/a/b/a.txt /a/c/a.txt a/d/a.txt a/e/a.txt': No such file or directory"
-I option is taking the complete input from echo instead of individual values (like -n 1 does). If someone can help debug this issue that would be very helpful instead of providing an alternative command.
Use rsync with the --relative (-R) option to keep (parts of) the source paths.
I've used a wildcard for the source to match your example command rather than the explicit list of directories mentioned in your question.
rsync -avR /a/*/a.txt /tmp/backup/
Do the backups need to be exactly the same as the originals? In most cases, I'd prefer a little compression. [tar](https://man7.org/linux/man-pages/man1/tar.1.html) does a great job of bundling things including the directory structure.
tar cvzf /path/to/backup/tarball.tgz /source/path/
tar can't update compressed archives, so you can skip the compression
tar uf /path/to/backup/tarball.tar /source/path/
This gives you versioning of a sort, as if only updates changed files, but keeps the before and after versions, both.
If you have time and cycles and still want the compression, you can decompress before and recompress after.
I am attempting to make a script that will make a backup file of a video file in the same directory with the time stamp at the end of the tar file. The script is for demonstration purposes only, that is why I do not intend on sending the file to a different directory. Below is how far I have come with it.
#!/bin/bash
cd Compression_Play/
echo Me.$(date +%d_%b_%Y-%k:%M:%S).tar.xz
tar -I "pixz -9t" -cvf Me.$(date +%d_%b_%Y-%k:%M:%S).tar.xz 2017-03-23_01-13-02.avi
My problem is whenever I try to execute the script it gives me this:
Me.29_Mar_2017-22:03:49.tar.xz
tar: -9: (PROGRAM ERROR) Option should have been recognized!?
Try 'tar --help' or 'tar --usage' for more information.
As best as I can tell the problem is with the quotes in my tar command. Is there a way to make the script so I can keep the quotes or substitute them?
Given your goal to make an archive, presumably preserving owner, file modes, file flags and ACLs, if available, then this should do what you need:
#!/bin/bash
archive_name="Me.$(date +%d_%b_%Y-%k:%M:%S).tar.xz"
$(cd Compression_Play/
tar -cvp 2017-03-23_01-13-02.avi |
pixz -9t > $archive_name
)
Based on the GitHub page for tarball compress/decompress for pixz, you will need to reverse the pipe to get your data out:
pixz -x9T < $archive_name | tar xvpf -
Greg gave a good answer and it works very well. I also figured out another way after nocking my head on it for several hours and taking the things from the help I got.
#!/bin/bash
cd /home/apowell/Compression_Play/Test_Backup
tar -I 'pixz -9t' -cvf Me.$(date +%d_%b_%Y-%H_%M_%S).tar.xz /home/apowell/Compression_Play/Linux_Programs
and this one compresses a little more by replacing pixz with just plain xz.
#!/bin/bash
cd /home/apowell/Compression_Play/Test_Backup
tar -I 'xz -9ve -T8 -k' -cvf Me.$(date +%d_%b_%Y-%H_%M_%S).tar.xz /home/apowell/Compression_Play/Linux_Programs
Have got a problem executing the command as below:
tar -xvf arch.tar.gz -s '/^bundle//'
Could be the
-s '/^bundle//'
is a problem as I've got errors like:
$ tar -xvf arch.tar.gz -s '/^bundle//'
tar: /^bundle: Not found in archive
tar: Exiting with failure status due to previous errors
The tried to run the command under Cygwin/Win10.
It's part of the longer script but I'm not sure what was the idea of original author. Archive does include the 'bundle' folder inside... and it's the only first level file there.
Thank you in advance :)
-s does not mean to do a substitution, which seems to be how you're trying to use it. You probably want --xform='s/^bundle//'
-s has the following entry in the help listing:
-s, --preserve-order, --same-order
member arguments are listed in the same order as
the files in the archive
With your code it's actually trying to find a file with the name /^bundle// which does not exist, even if bundle does. Also, the --xform option I gave will rewrite the names of files to strip the string bundle from the front. If you are just trying to not extract the file bundle you would want the flag --exclude='bundle'
In this case, if bundle is a top level directory in the archive, and it's the only one, you could also use the flag --strip-components=1, though this would get rid of all the top level directories, so might not be exactly what you want depending on your archive
Thanks all,
Problem solved other, then 'tar', way but for those who may be interested here is the answer I have found on the web:
If you are developing on Linux, or using GNU tar, this command should work:
tar -xvf arch.tar.gz --transform 's|^bundle/||'
For Mac or BSD-based operating systems:
tar -xvf arch.tar.gz -s '/^bundle//'
Yes, the idea was to remove the /bundle/ folder from files paths.
Let's say I'm trying to tar.gz all the files and folders in /usr/local/bin/data/*
The file name would be data-2015-10-01.tar.gz. When I untar it, is it possible that the root directory would be data-2015-10-01 followed by the contents of whatever is inside of data/* ?
If not, how can I tar /usr/local/bin/data/* but start at the /data/ folder level?
I can't do this unfortunately since the program spits out /usr/local/bin/data/ and I'm unable to change it.
cd /usr/local/bin
tar ... /data/*
There are a couple of ways to do what I think you're trying to accomplish. First, you can use the -C option to tar when creating the archive. That changes tar's current working directory to that directory before creating the archive. Not strictly required in your case, but probably helpful.
# tar -C /usr/local/bin -czf data-2015-10-01.tar.gz data/*
That at least gets you to a single directory named data. If you have control of the extraction (manually or via a script you provide to whomever is unpacking this), then you can do something like this on the extraction:
# mkdir -f data-2015-10-01 && tar -C data-2015-10-01 --strip-components=1 -xzf data-2015-10-01.tar.gz
This will remove the first path, which is "data" and extract everything from there into the directory which is your current working directory, data-2015-10-01. So, it isn't specifically tar that's doing the renaming, but you will effectively end up with the same result.
I've accomplished something similar with a symlink. This is not a great solution if you have (or might have) symlinks in the directory structure you're trying to archive. I have to say that I prefer #geis' solution to strip out the top-level directory on extract, but this gives you another option.
ln -s /usr/local/bin/data data-2015-10-01
tar -cvhf data-2015-10-01.tar.gz data-2015-10-01/
rm data-2015-10-01
(Note the additional -h option in the tar invocation.)
I have a directory containing a set of subdirectories and files. I need to recursively copy all the content of this directory to all the subdirectories of another directory, also recursively.
How do I achieve this, preferably without using a script and only with the cp command?
You can write this in a script but you don't have to. Just write it line by line in the terminal:
# $TARGET is the directory containing subdirectories where you want to STORE the copies
# $SOURCE is the directory containing the subdirectories you want to COPY
for dir in $(ls $TARGET); do
cp -r $SOURCE/* $TARGET/$dir
done
Only uses cp and runs on both bash and zsh.
You can't. cp can copy multiple sources but will only copy to a single destination. You need to arrange to invoke cp multiple times - once per destination - for what you want to do; using, as you say, a loop or some other tool.
The first part of the command before the pipe instruct tar to create an archive of everything in the current directory and write it to standard output (the – in place of a file-name frequently indicates stdout).
tar cf - * | ( cd /target; tar xfp -)
The commands within parentheses cause the shell to change directory to the target directory and untar data from standard input. Since the cd and tar commands are contained within parentheses, their actions are performed together.
The -p option in the tar extraction command directs tar to preserve permission and ownership information, if possible given the user executing the command. If you are running the command as superuser, this option is turned on by default and can be omitted.
Also you can use the following command, but it seems to be quite slower than tar;
cp -a * /target