I want to pack specific some sub-directories contents, and extract them with original permission. In the following sample, it extract the directory a with wrong permission, I prefer it keep the same as original foo. How to do it? What's parameter should I add to tar when archive or extract?
create test sample files :
foo ~/tmp $ mkdir -p a/b ; touch a/f1 a/b/f2
create archive file :
foo ~/tmp $ tar cf a.tar a/b
show tar file content :
foo ~/tmp $ tar tvf a.tar
drwxr-xr-x foo/op 0 2018-05-11 10:33 a/b/
-rw-r--r-- foo/op 0 2018-05-11 10:33 a/b/f2
extract with root user :
foo ~/tmp $ mkdir c
foo ~/tmp $ sudo tar -C c -xpf a.tar
foo ~/tmp $ find c -ls
4743788 4 drwxr-xr-x 3 foo op 4096 May 11 10:34 c
4743789 4 drwxr-xr-x 3 root root 4096 May 11 10:34 c/a
4743790 4 drwxr-xr-x 2 foo op 4096 May 11 10:33 c/a/b
4727961 0 -rw-r--r-- 1 foo op 0 May 11 10:33 c/a/b/f2
What I expect is
4743789 4 drwxr-xr-x 3 foo op 4096 May 11 10:34 c/a
I think it is a correct behavior. Your a.tar does not contain directory c/a so it has to be created during the unpacking. That is done under context of the user you are unpacking the archive with - root.
The tar option -p does preserve the rights to directories but only the existing in the archive.
Note: I'm unpacking it with user smalltalk.
Option when you have already created archive and want to remove files from already created archive
I think you could pack all with:
tar cf a.tar a
which would then produce
tar tvf a.tar
drwxrwxr-x smalltalk/smalltalk 0 2018-05-11 10:01 a/
drwxrwxr-x smalltalk/smalltalk 0 2018-05-11 10:01 a/b/
-rw-rw-r-- smalltalk/smalltalk 0 2018-05-11 10:01 a/b/f2
-rw-rw-r-- smalltalk/smalltalk 0 2018-05-11 10:01 a/f1
Then you can simply delete the directories/files you don't want in this case a/f1:
tar -vf a.tar --delete a/f1
You will get:
tar tvf a.tar
drwxrwxr-x smalltalk/smalltalk 0 2018-05-11 10:01 a/
drwxrwxr-x smalltalk/smalltalk 0 2018-05-11 10:01 a/b/
-rw-rw-r-- smalltalk/smalltalk 0 2018-05-11 10:01 a/b/f2
Since now the directory a/ is part of the packed file you will get correct result when unpacking:
find c -ls
149236 0 drwxrwxr-x 3 smalltalk smalltalk 14 May 11 10:14 c
17208669 0 drwxrwxr-x 3 smalltalk smalltalk 14 May 11 10:01 c/a
36340903 0 drwxrwxr-x 2 smalltalk smalltalk 15 May 11 10:01 c/a/b
36333667 0 -rw-rw-r-- 1 smalltalk smalltalk 0 May 11 10:01 c/a/b/f2
First Edit - Second option -- exclude before packaging
Is to create a tar file without the directory in the first place with --exclude option:
For example:
tar cf a.tar a --exclude=a/f1
Which would create:
tar tvf a.tar
drwxrwxr-x smalltalk/smalltalk 0 2018-05-11 10:01 a/
drwxrwxr-x smalltalk/smalltalk 0 2018-05-11 10:01 a/b/
-rw-rw-r-- smalltalk/smalltalk 0 2018-05-11 10:01 a/b/f2
Then the extracted directory would have correct rights.
Second Edit - adding top most directory
based on comment (many files) - adding directory after the archive has been already created
If you have many files I think the best solution is to add the directory to your already created archive.
You would then create the file as you did in your question:
tar cf a.tar a/b
Then you would add the top directory (--no-recursion option excludes the sub-directories):
tar -rvf a.tar --no-recursion a/
You will then get file with the added directory:
tar tvf a.tar
drwxrwxr-x smalltalk/smalltalk 0 2018-05-11 10:01 a/b/
-rw-rw-r-- smalltalk/smalltalk 0 2018-05-11 10:01 a/b/f2
drwxrwxr-x smalltalk/smalltalk 0 2018-05-11 10:01 a/
Then you will get correct extraction:
find c -ls
68333351 0 drwxrwxr-x 3 smalltalk smalltalk 14 May 11 11:01 c
87520961 0 drwxrwxr-x 3 smalltalk smalltalk 14 May 11 10:01 c/a
149236 0 drwxrwxr-x 2 smalltalk smalltalk 15 May 11 10:01 c/a/b
824154 0 -rw-rw-r-- 1 smalltalk smalltalk 0 May 11 10:01 c/a/b/f2
Separate the archive into two steps, the first step archive as previous
tar cf a.tar a/b
The second step to append the directory without recursion
tar -rf a.tar --no-recursion a
Related
According to gnu documentation, to add one or more archives to the end of another archive, I can use the ‘--concatenate’ operation.
But in my testing, I found that I can't add more than one file at a time.
# ls -al
total 724
drwxr-xr-x. 3 root root 60 Oct 14 17:40 .
dr-xr-xr-x. 32 root root 4096 Oct 14 16:28 ..
-rw-r--r--. 1 root root 245760 Oct 14 18:07 1.tar
-rw-r--r--. 1 root root 245760 Oct 14 18:07 2.tar
-rw-r--r--. 1 root root 245760 Oct 14 18:07 3.tar
# tar tvf 1.tar
-rw-r--r-- root/root 238525 2021-10-14 17:28 1.txt
# tar tvf 2.tar
-rw-r--r-- root/root 238525 2021-10-14 17:29 2.txt
# tar tvf 3.tar
-rw-r--r-- root/root 238525 2021-10-14 17:29 3.txt
It appears that it only picked up the first parameter and ignored that rest
# tar -A -f 1.tar 2.tar 3.tar
# tar tvf 1.tar
-rw-r--r-- root/root 238525 2021-10-14 17:28 1.txt
-rw-r--r-- root/root 238525 2021-10-14 17:29 2.txt
As described in an excellent and comprehensive Super User answer,
this is a known bug in gnu tar (reported in August 2008)
I have noticed a strange behavior in UNIX systems:
I'm standing in /noob/
I have a symbolic link to a folder (A# -> /B/C/D/A)
I enter the folder via my symlink (cd A)
pwd says /noob/A/
In /B/C/D/A/ i have a file abc which I can see now.
I want to copy it to /noob/
I type cp abc ..
I type cd ..
I end up in /noob/ which is empty - but the file ended up in /B/C/D/ ???
How come this ambiguity as to where cp and cd points when given .. as argument? I find it confusing. Can anyone explain it in terms I'll understand? (=simple)
All the best, and please forgive a UNIX-noob a stupid question. Lasse
First let's have a look at how cd command does behave by looking at the help menu. What we are looking for is option -L (the default behavior) and option -P
$ help cd cd: cd [-L|[-P [-e]] [-#]] [dir]
Change the shell working directory.
...
...
Options:
-L force symbolic links to be followed: resolve symbolic links in
DIR after processing instances of `..'
-P use the physical directory structure without following symbolic
links: resolve symbolic links in DIR before processing instances
of `..'
...
...
Important section
The default is to follow symbolic links, as if `-L' were specified.
`..' is processed by removing the immediately previous pathname component
back to a slash or the beginning of DIR.
Exit Status:
...
As you can see the default behavior of cd is not what you think it is since he will manipulate the $PWD variable accessed by pwd command in his own way, at each step you can run pwd command or do an echo $PWD to see how it reacts with the different cd commands hereunder.
Let's play with cd command:
We start from the following folder, with a sym link:
[/home/arobert/test/noob] >
ls -ltra
total 8
drwxrwxr-x 5 arobert arobert 4096 5月 11 09:48 ..
lrwxrwxrwx 1 arobert arobert 26 5月 11 09:48 A -> /home/arobert/link/B/C/D/A
drwxrwxr-x 2 arobert arobert 4096 5月 11 10:03 .
USAGE EXAMPLES:
[/home/arobert/test/noob] >
cd A
[/home/arobert/test/noob/A] >
cd ..
[/home/arobert/test/noob] >
cd -L A
[/home/arobert/test/noob/A] >
cd ..
[/home/arobert/test/noob] >
cd -P A
[/home/arobert/link/B/C/D/A] >
cd -P ..
[/home/arobert/link/B/C/D] >
cd /home/arobert/test/noob/
[/home/arobert/test/noob] >
cd A
[/home/arobert/test/noob/A] >
cd -P ..
[/home/arobert/link/B/C/D] >
Now let's play with readlink and cp command:
Let's say we have entered the symlink that points to A -> /home/arobert/link/B/C/D/A in which we have a file a
[/home/arobert/test/noob/A] >
ls -ltra
total 8
drwxrwxr-x 3 arobert arobert 4096 5月 11 09:55 ..
-rw-rw-r-- 1 arobert arobert 0 5月 11 10:10 a
drwxrwxr-x 2 arobert arobert 4096 5月 11 10:10 .
from this folder let's look at where does point . and .. by using readlink -f command:
[/home/arobert/test/noob/A] >
readlink -f .
/home/arobert/link/B/C/D/A
[/home/arobert/test/noob/A] >
readlink -f ..
/home/arobert/link/B/C/D
By consequence, when you run from the location /home/arobert/test/noob/A equivalent to /home/arobert/link/B/C/D/A the command cp a .. the file will be moved to /home/arobert/link/B/C/D as .. points to it.
What you can do now:
Use absolute path with your cp command to avoid bad surprise.
Call the command from /home/arobert/test/noob/ directory using
For example:
[/home/arobert/test/noob] >
cp A/a .
as readlink -f . points to the correct folder
[/home/arobert/test/noob] >
readlink -f .
/home/arobert/test/noob
Result:
[/home/arobert/test/noob] >
ls -ltra
total 8
drwxrwxr-x 5 arobert arobert 4096 5月 11 09:48 ..
lrwxrwxrwx 1 arobert arobert 26 5月 11 09:48 A -> /home/arobert/link/B/C/D/A
-rw-rw-r-- 1 arobert arobert 0 5月 11 10:13 a
drwxrwxr-x 2 arobert arobert 4096 5月 11 10:13 .
This question already has answers here:
How to move files and directories excluding one specific directory to this directory
(3 answers)
Closed 5 years ago.
I have two directories dir1 and dir2. I need to move the content of folder dir1 to dir2 except one folder dir1/src.
I tried this
mv !(src) dir1/* dir2/
But it dosn't work, it still displays this error
bash: !: event not found
Maybe you are looking for something like this?
The answer to my question there states that what you are trying to to is achievable by using the extglob bash shell option. You can turn it on by executing shopt -s extglob or by adding that command to your ~/.bashrc and relogin. Afterwards you can use the function.
To use your example of moving everything from dir1 except dir1/src to dir2, this should work:
mv -vt dir2/ dir1/!(src)
Example output:
$ mkdir -pv dir1/{a,b,c,src} dir2
mkdir: created directory 'dir1'
mkdir: created directory 'dir1/a'
mkdir: created directory 'dir1/b'
mkdir: created directory 'dir1/c'
mkdir: created directory 'dir1/src'
mkdir: created directory 'dir2'
$ ls -l dir1/
total 16
drwxrwxr-x 2 dw dw 4096 Apr 7 13:30 a
drwxrwxr-x 2 dw dw 4096 Apr 7 13:30 b
drwxrwxr-x 2 dw dw 4096 Apr 7 13:30 c
drwxrwxr-x 2 dw dw 4096 Apr 7 13:30 src
$ ls -l dir2/
total 0
$ shopt -s extglob
$ mv -vt dir2/ dir1/!(src)
'dir1/a' -> 'dir2/a'
'dir1/b' -> 'dir2/b'
'dir1/c' -> 'dir2/c'
$ ls -l dir1/
total 4
drwxrwxr-x 2 dw dw 4096 Apr 7 13:30 src
$ ls -l dir2/
total 12
drwxrwxr-x 2 dw dw 4096 Apr 7 13:30 a
drwxrwxr-x 2 dw dw 4096 Apr 7 13:30 b
drwxrwxr-x 2 dw dw 4096 Apr 7 13:30 c
More information about extglob can be found here.
I have the following line
tar -c -v -z -f "$ARCHIVE_PATH/$3_$fileYear$fileMonth.tar.gz" -C "$ARCHIVE_PATH/tmp" .
where
$ARCHIVE_PATH = /opt/colorado/archive/
$3 = IMPORT
$fileYear = 2014
$fileMonth = 06
so the line creates a .tar.gz file called IMPORT_201406.tar.gz in /opt/colorado/archive/ from the files located in /opt/colorado/archive/tmp/
however when i use tar -ztvf "opt/colorado/archive/IMPORT_201406.tar.gz" i see this
-rwxr-xr-x root/root 27 2014-06-04 14:20 ./afile.txt
drwxr-xr-x root/root 0 2014-06-04 14:08 ./opt/
drwxr-xr-x root/root 0 2014-06-04 14:08 ./opt/colorado/
drwxr-xr-x root/root 0 2014-06-04 14:08 ./opt/colorado/archive/
drwxrwxr-x [USER]/[USER] 0 2014-06-04 14:09 ./opt/colorado/archive/tmp/
-rwxr-xr-x root/root 712 2014-06-04 14:20 ./twofile.txt
-rwxr-xr-x root/root 383 2014-06-04 14:20 ./random.cvs
-rwxr-xr-x root/root 27 2014-06-04 14:20 ./helloworld.sh
-rwxr-xr-x root/root 7938 2014-06-04 14:20 ./helloworld.py
from my understanding, if i didn't have -C the /opt/colorado/archive/tmp/ would have been added to every file so adding -C tells tar to move to that directory first, i can see in the list of files however why is the /opt/colorado/archive/tmp/ folder added and is there a way to remove it?
It's adding the directory to the archive because you asked it to. Specifically, you told tar to archive the directory ., so that's what it does. Computers tend to be literal.
If you don't want the directory archived, you'll have to pass the filenames in the directory to tar. Here's one way to do that:
(cd "$ARCHIVE_PATH/tmp"; ls) |
tar -cvzf "$ARCHIVE_PATH/$3_$fileYear$fileMonth.tar.gz" -C "$ARCHIVE_PATH/tmp" -T-
Alternatively, you can execute tar from the directory with the files:
cd "$ARCHIVE_PATH/tmp"; tar -cvzf "$ARCHIVE_PATH/$3_$fileYear$fileMonth.tar.gz" *
I don't know why you get the directory included with its full path, rather than just ., and why it also includes the parent directories in the archive. The version of tar on my system (tar (GNU tar) 1.26) doesn't seem to do that.
This command will generate the tar file:
tar -cvf new.tar.gz my_dir_to_tar/
where new.tar.gz = tar file name
and my_dir_to_tar/ = directory to make a tar
I created a script to exact all tar.gz file it is composed of 5 tar files and the last tar file should remain tar it should not be extracted. So my problem is it will all extract the tar.gz file.
while true; do
for f in *.tar.gz; do
case $f in '*.tar.gz') exit 0;; esac
tar zxf "$f"
rm -v "$f"
done
done
What is the correct way to my solve problem?
You say:
I created a script to exact all tar.gz file it is composed of 5 tar files and the last tar file should remain tar it should not be extracted.
I can see two interpretations for that:
There are five files with the .tar.gz extension in a directory. The first four of those files should be extracted and removed; the fifth should be left unextracted.
There is one .tar.gz file which contains 5 .tar files. The first four of those .tar files should be extracted from the .tar.gz; the fifth should be left unextracted.
There are many ways to deal with each scenario. I'm assuming that the tar file names do not contain spaces or newlines or other oddball characters (which is plausible). If you have oddball file names, it is probably simplest to sanitize them first. Amongst other things, these assumptions mean that you could safely use ls output. However, it is still best not to do so. I am going to assume that 5 is a fixed and magic number.
Scenario 1
i=0
for tarfile in *.tar.gz
do
$(($i++))
[ $i = 5 ] && break
tar -xf "$tarfile"
done
This counts the files it extracts, and stops after the count reaches 5 (so it only extracts the first 4 files).
Scenario 2
tarfiles=$(tar -tf *.tar.gz | sed '$d')
tar -xf *.tar.gz $tarfiles
This collects the list of tar files contained inside the compressed tar file, and deletes the last one listed. It then requests tar to extract the remaining files — so the directory will contain the original .tar.gz file and all the files except the last extracted from the .tar.gz file. If the extracted files are tar files, you can then extract those individually:
for tarfile in $tarfiles
do tar -xf "$tarfile"
done
If you want to remove the extracted tar files too, you can do that.
Demo of Scenario 2
$ mkdir junk
$ cp *.c junk
$ cd junk
$ ls
bo.c zigzag.c
$ for i in {1..5}; do tar -cf tarfile-$i.tar *.c; done
$ ls -l
total 144
-rw-r--r-- 1 jleffler staff 546 May 7 22:34 bo.c
-rw-r--r-- 1 jleffler staff 10752 May 7 22:35 tarfile-1.tar
-rw-r--r-- 1 jleffler staff 10752 May 7 22:36 tarfile-2.tar
-rw-r--r-- 1 jleffler staff 10752 May 7 22:36 tarfile-3.tar
-rw-r--r-- 1 jleffler staff 10752 May 7 22:36 tarfile-4.tar
-rw-r--r-- 1 jleffler staff 10752 May 7 22:36 tarfile-5.tar
-rw-r--r-- 1 jleffler staff 7305 May 7 22:34 zigzag.c
$ tar -czf tarfile-N.tar.gz *.tar
$ ls -l
total 152
-rw-r--r-- 1 jleffler staff 546 May 7 22:34 bo.c
-rw-r--r-- 1 jleffler staff 10752 May 7 22:35 tarfile-1.tar
-rw-r--r-- 1 jleffler staff 10752 May 7 22:36 tarfile-2.tar
-rw-r--r-- 1 jleffler staff 10752 May 7 22:36 tarfile-3.tar
-rw-r--r-- 1 jleffler staff 10752 May 7 22:36 tarfile-4.tar
-rw-r--r-- 1 jleffler staff 10752 May 7 22:36 tarfile-5.tar
-rw-r--r-- 1 jleffler staff 2787 May 7 22:36 tarfile-N.tar.gz
-rw-r--r-- 1 jleffler staff 7305 May 7 22:34 zigzag.c
$ tar -tvf tarfile-N.tar.gz
-rw-r--r-- 0 jleffler staff 10752 May 7 22:35 tarfile-1.tar
-rw-r--r-- 0 jleffler staff 10752 May 7 22:36 tarfile-2.tar
-rw-r--r-- 0 jleffler staff 10752 May 7 22:36 tarfile-3.tar
-rw-r--r-- 0 jleffler staff 10752 May 7 22:36 tarfile-4.tar
-rw-r--r-- 0 jleffler staff 10752 May 7 22:36 tarfile-5.tar
$ rm *.tar *.c
$ ls -l
total 8
-rw-r--r-- 1 jleffler staff 2787 May 7 22:36 tarfile-N.tar.gz
$ tarfiles=$(tar -tf *.tar.gz | sed '$d')
$ echo $tarfiles
tarfile-1.tar tarfile-2.tar tarfile-3.tar tarfile-4.tar
$ tar -xf *.tar.gz $tarfiles
$ ls -l
total 104
-rw-r--r-- 1 jleffler staff 10752 May 7 22:35 tarfile-1.tar
-rw-r--r-- 1 jleffler staff 10752 May 7 22:36 tarfile-2.tar
-rw-r--r-- 1 jleffler staff 10752 May 7 22:36 tarfile-3.tar
-rw-r--r-- 1 jleffler staff 10752 May 7 22:36 tarfile-4.tar
-rw-r--r-- 1 jleffler staff 2787 May 7 22:36 tarfile-N.tar.gz
$ for file in $tarfiles; do echo $file; tar -tvf $file; rm $file; done
tarfile-1.tar
-rw-r--r-- 0 jleffler staff 546 May 7 22:34 bo.c
-rw-r--r-- 0 jleffler staff 7305 May 7 22:34 zigzag.c
tarfile-2.tar
-rw-r--r-- 0 jleffler staff 546 May 7 22:34 bo.c
-rw-r--r-- 0 jleffler staff 7305 May 7 22:34 zigzag.c
tarfile-3.tar
-rw-r--r-- 0 jleffler staff 546 May 7 22:34 bo.c
-rw-r--r-- 0 jleffler staff 7305 May 7 22:34 zigzag.c
tarfile-4.tar
-rw-r--r-- 0 jleffler staff 546 May 7 22:34 bo.c
-rw-r--r-- 0 jleffler staff 7305 May 7 22:34 zigzag.c
$ rm -f *
$ cd ..
$ rmdir junk
$
The problem is you are using this command to extract the file:
tar zxf "$f"
And what that does is decompress the gzip file & then extract tar the archive all at once. So all tarred contents would be extracted. Instead, what you want to do is use gzip -d instead:
gzip -d "$f"
That will only decompress the gzip. As for the logic of only extracting the contents of the first four tar archives but not the fifth, it seems like there is more code logic you are not sharing?