Naming a tar file with date - linux

I want to name a tar file with the date, but what i try isn't working, see the code below for better explanation, and what a i have to do?
date=$( date +%Y-%m-%d_%H:%M:%S )
tar -czf ${imageName-$date}.tar.gz ${imageBasename} 2>>$ERRORLOG
The image name is a variable that stores a name of an image, like: win-ser-rdp
I had tried too:
tar -czf ${imageName}:${date}.tar.gz ${imageBasename} 2>>$ERRORLOG
Didn't work out too :(
Thanks to all the help :D

The problem is the : in the file name, which is treated specially by GNU tar.
On my own system, where I have a directory named foo, I get this:
$ tar zcf foo:42.tar.gz foo
tar (child): Cannot connect to foo: resolve failed
tar: Child returned status 128
tar: Error is not recoverable: exiting now
$
There doesn't seem to be any way to escape the : character -- but you can inhibit the special interpretation with the --force-local option (thanks to Daniel Serodio for pointing this out).
Colon characters in file names, though they're legal, are typically a bad idea anyway. For example, you probably wouldn't be able to access the file via scp.
Here's the relevant section from the GNU tar manual (info tar and go to section 6.1):
To specify an archive file on a device attached to a remote machine,
use the following:
--file=hostname:/dev/file-name
tar will set up the remote connection, if possible, and prompt you for
a username and password. If you use --file=#hostname:/dev/file-name,
tar will attempt to set up the remote connection using your username
as the username on the remote machine.
When I make file names that include a timestamp, I usually use YYYY-MM-DD-HHMMSS
$(date +%F-%H%M%S)
or you can use YYYY-MM-DD_HH_MM_SS if you prefer
$(date +%F-%H_%M_%S)

Your filenames and variables are incorrect:
${imageName-$date}
Should be
${imageName}${date}
and
${imageName}:${date}.tar.gz
is going to be interpreted as an NFS location because of the : in the name, e.g. server:filename. Replace the : with a - or other non-special character.

try avoiding all colons in the tarfilename, as this will make tar attempt to output to a remote file:
date=$( date +%Y%m%d_%H%M%S )
tar -czf ${imageName}-${date}.tar.gz ${imageBasename} 2>>${ERRORLOG}

Related

Using pseudo to retain xattrs when extracting tar

I'm trying the following:
use pseudo to pack an archive (bz2) which has files with security xattr set.
use pseudo again to unpack the archive and keep the security xattr of the files.
Unfortunately, the extraction fails with the following message coming from pseudo:
$ pseudo tar -cjpf myarchive.tar.bz2 --xattrs --xattrs-include='*' myFile
# checked the contents in the meantime and looked good
$ pseudo tar -C unpack-folder/ -xpjf myarchive.tar.bz2 --xattrs --xattrs-include='*'
Warning: PSEUDO_PREFIX unset, defaulting to /home/user/tmp/.
got *at() syscall for unknown directory, fd 4
unknown base path for fd 4, path myFile
couldn't allocate absolute path for 'myFile'.
tar: myFile: Cannot open: Bad address
tar: Exiting with failure status due to previous errors
pseudo: 1.9.0
tar: 1.34
Do you have any idea what could be the problem or have another idea on how to be able to preserve the xattr of the files when extracting the contents of the archive?

Fatal: Failed to read password file: open ~/RPI/filename.sec: no such file or directory. But it exist [linux]

Trying to run .sh file. with --password "~/RPI.filename"in it. Should read the filename but getting an error. Even though the file exists
error:
Fatal: Failed to read password file: open ~/RPI/filename.sec: no such file or directory
list of directories and files:
~/RPI$ l
g.json m1/ m2/ filename.sec startm1.sh*
I have used it before and I know the method works but no idea what on earth is happening here.
Maybe your script is running in sh instead of bash ? I think "~" is bash syntax.
There are 1½ issues here...
First, quoting a tilde prevents tilde expansion; compare ls ~/RPI.filename and ls "~/RPI.filename"
=> leave the tilde unquoted if you don't want a literal '~'
Second, is it ~/RPI.filename or ~/RPI/filename?

How to compress multiple folders with certain name?

I have the following folder,
(Project) [Usr#hpc FOB]$ ls
exec_train.sh FOB_RE2250_BS4ES025.py network_checkpoint_FOB_RE2250_BS2ES05
FOB_RE1150.py FOB_RE2250_BS4ES05.py network_checkpoint_FOB_RE2250_BS2ES1
FOB_RE1200.py FOB_RE2250_BS4ES1.py network_checkpoint_FOB_RE2250_BS4ES025
FOB_RE2250_BS05ES1.py FOB_RE2250.py network_checkpoint_FOB_RE2250_BS4ES05
FOB_RE2250_BS05ES2.py FOB_RE50.py network_checkpoint_FOB_RE2250_BS4ES1
FOB_RE2250_BS1ES1.py network_checkpoint_FOB_RE2250_BS05ES1
FOB_RE2250_BS2ES05.py network_checkpoint_FOB_RE2250_BS05ES2
FOB_RE2250_BS2ES1.py network_checkpoint_FOB_RE2250_BS1ES1
How do I compress the all the network_checkpoint_FOB.... into one .tar.gz archive?
I know I could manually use $ tar -czf FOB.tar.gz network_checkpoint_FOB_RE2250_BS1ES1 network_checkpoint_FOB_RE2250_BS05ES1 ... but this seams cumbersome. I think there should be a way to use string matching but I haven't been able to find a clear concise solution.
You can use wildcard character * in Bash:
$ tar -czf FOB.tar.gz network_checkpoint_FOB*
Bash automatically expands network_checkpoint_FOB* expression to space separated matching file/folder names.

How to specify the tar final structure

I have this structure:
release/folder1/file1
release/folder2/file2
...
release/folderN/fileN
I want to include all those folders (folder1, folder2 ... folderN) in a tar file.
The key is that I want these folders to be in the final tar within another directory named MYAPP so when you open the tar you can see this:
MYAPP/folder1/file1
MYAPP/folder2/file2
...
MYAPP/folderN/fileN
How can I achieve this without renaming the original "release" directory and/or creating new directories.
Is this possible to achive just in the tar process?
Thanks
Add
--transform=s#^release/#MYAPP/#
to your tar command line.
The argument of the --transform command line is a command that is passed to sed together with the file path before it is stored in the archive (use tar -tf to show the names of the files stored in the archive).
The command s#^release/#MYAPP/# tells sed to search (s) release/ at the beginning of the string (^) and replace it with MYAPP/.
The / at the end of the search and replace strings is needed to be sure the complete name of the component is release (to not replace release.txt). The # character is just a regex delimiter. Usually / is used as a regex delimiter but we prefer to use a different delimiter here to avoid the need to escape / (because it is used in the search and replace strings).
Read more in the documentation of tar and sed.

zip command not working

I am trying to zip a file using shell script command. I am using following command:
zip ./test/step1.zip $FILES
where $FILES contain all the input files. But I am getting a warning as follows
zip warning: name not matched: myfile.dat
and one more thing I observed that the file which is at last in the list of files in a folder has the above warning and that file is not getting zipped.
Can anyone explain me why this is happening? I am new to shell script world.
zip warning: name not matched: myfile.dat
This means the file myfile.dat does not exist.
You will get the same error if the file is a symlink pointing to a non-existent file.
As you say, whatever is the last file at the of $FILES, it will not be added to the zip along with the warning. So I think something's wrong with the way you create $FILES. Chances are there is a newline, carriage return, space, tab, or other invisible character at the end of the last filename, resulting in something that doesn't exist. Try this for example:
for f in $FILES; do echo :$f:; done
I bet the last line will be incorrect, for example:
:myfile.dat :
...or something like that instead of :myfile.dat: with no characters before the last :
UPDATE
If you say the script started working after running dos2unix on it, that confirms what everybody suspected already, that somehow there was a carriage-return at the end of your $FILES list.
od -c shows the \r carriage-return. Try echo $FILES | od -c
Another possible cause that can generate a zip warning: name not matched: error is having any of zip's environment variables set incorrectly.
From the man page:
ENVIRONMENT
The following environment variables are read and used by zip as described.
ZIPOPT
contains default options that will be used when running zip. The contents of this environment variable will get added to the command line just after the zip command.
ZIP
[Not on RISC OS and VMS] see ZIPOPT
Zip$Options
[RISC OS] see ZIPOPT
Zip$Exts
[RISC OS] contains extensions separated by a : that will cause native filenames with one of the specified extensions to be added to the zip file with basename and extension swapped.
ZIP_OPTS
[VMS] see ZIPOPT
In my case, I was using zip in a script and had the binary location in an environment variable ZIP so that we could change to a different zip binary easily without making tonnes of changes in the script.
Example:
ZIP=/usr/bin/zip
...
${ZIP} -r folder.zip folder
This is then processed as:
/usr/bin/zip /usr/bin/zip -r folder.zip folder
And generates the errors:
zip warning: name not matched: folder.zip
zip I/O error: Operation not permitted
zip error: Could not create output file (/usr/bin/zip.zip)
The first because it's now trying to add folder.zip to the archive instead of using it as the archive. The second and third because it's trying to use the file /usr/bin/zip.zip as the archive which is (fortunately) not writable by a normal user.
Note: This is a really old question, but I didn't find this answer anywhere, so I'm posting it to help future searchers (my future self included).
eebbesen hit the nail in his comment for my case (but i cannot vote for comment).
Another possible reason missed in the other comments is file exceeding the file size limit (4GB).
I converted my script for unix environment using dos2unix command and executed my script as ./myscript.sh instead bash myscript.sh.
I just discovered another potential cause for this. If the permissions of the directory/subdirectory don't allow the zip to find the file, it will report this error. Actually, if you run a chmod -R 444 on the directory, and then try to zip it, you will reproduce this error, and also have a "stored 0%" report, like this:
zip warning: name not matched: borrar/enviar
adding: borrar/ (stored 0%)
Hence, try changing the permissions of the file. If you are trying to send them through email, and those email filters (like Gmail's) invent silly filters of not sending executables, don't forget that making permissions very strict when making zip compression can be the cause of the error you are reporting, of "name not matched".
spaces are not allowed:
it would fail if there are more than one files(s) in $FILES unless you put them in loop
I also encountered this issue. In my case, the line separate is CRLF in my zip shell script which causes the problem. Using LF fixed it.

Resources