zip command not working - linux

I am trying to zip a file using shell script command. I am using following command:
zip ./test/step1.zip $FILES
where $FILES contain all the input files. But I am getting a warning as follows
zip warning: name not matched: myfile.dat
and one more thing I observed that the file which is at last in the list of files in a folder has the above warning and that file is not getting zipped.
Can anyone explain me why this is happening? I am new to shell script world.

zip warning: name not matched: myfile.dat
This means the file myfile.dat does not exist.
You will get the same error if the file is a symlink pointing to a non-existent file.
As you say, whatever is the last file at the of $FILES, it will not be added to the zip along with the warning. So I think something's wrong with the way you create $FILES. Chances are there is a newline, carriage return, space, tab, or other invisible character at the end of the last filename, resulting in something that doesn't exist. Try this for example:
for f in $FILES; do echo :$f:; done
I bet the last line will be incorrect, for example:
:myfile.dat :
...or something like that instead of :myfile.dat: with no characters before the last :
UPDATE
If you say the script started working after running dos2unix on it, that confirms what everybody suspected already, that somehow there was a carriage-return at the end of your $FILES list.
od -c shows the \r carriage-return. Try echo $FILES | od -c

Another possible cause that can generate a zip warning: name not matched: error is having any of zip's environment variables set incorrectly.
From the man page:
ENVIRONMENT
The following environment variables are read and used by zip as described.
ZIPOPT
contains default options that will be used when running zip. The contents of this environment variable will get added to the command line just after the zip command.
ZIP
[Not on RISC OS and VMS] see ZIPOPT
Zip$Options
[RISC OS] see ZIPOPT
Zip$Exts
[RISC OS] contains extensions separated by a : that will cause native filenames with one of the specified extensions to be added to the zip file with basename and extension swapped.
ZIP_OPTS
[VMS] see ZIPOPT
In my case, I was using zip in a script and had the binary location in an environment variable ZIP so that we could change to a different zip binary easily without making tonnes of changes in the script.
Example:
ZIP=/usr/bin/zip
...
${ZIP} -r folder.zip folder
This is then processed as:
/usr/bin/zip /usr/bin/zip -r folder.zip folder
And generates the errors:
zip warning: name not matched: folder.zip
zip I/O error: Operation not permitted
zip error: Could not create output file (/usr/bin/zip.zip)
The first because it's now trying to add folder.zip to the archive instead of using it as the archive. The second and third because it's trying to use the file /usr/bin/zip.zip as the archive which is (fortunately) not writable by a normal user.
Note: This is a really old question, but I didn't find this answer anywhere, so I'm posting it to help future searchers (my future self included).

eebbesen hit the nail in his comment for my case (but i cannot vote for comment).
Another possible reason missed in the other comments is file exceeding the file size limit (4GB).

I converted my script for unix environment using dos2unix command and executed my script as ./myscript.sh instead bash myscript.sh.

I just discovered another potential cause for this. If the permissions of the directory/subdirectory don't allow the zip to find the file, it will report this error. Actually, if you run a chmod -R 444 on the directory, and then try to zip it, you will reproduce this error, and also have a "stored 0%" report, like this:
zip warning: name not matched: borrar/enviar
adding: borrar/ (stored 0%)
Hence, try changing the permissions of the file. If you are trying to send them through email, and those email filters (like Gmail's) invent silly filters of not sending executables, don't forget that making permissions very strict when making zip compression can be the cause of the error you are reporting, of "name not matched".

spaces are not allowed:
it would fail if there are more than one files(s) in $FILES unless you put them in loop

I also encountered this issue. In my case, the line separate is CRLF in my zip shell script which causes the problem. Using LF fixed it.

Related

The system cannot find the file specified - WinError 2

Upon looping a directory to delete txt files ONLY - a message is returned indicating The System cannot find the file specified: 'File.txt'.
I've made sure the txt files that I'm attempting to delete exist in the directory I'm looping. I've also checked my code and to make sure it can see my files by printing them in a list with the print command.
import os
fileLoc = 'c:\\temp\\files'
for files in os.listdir(fileLoc):
if files.endswith('.txt'):
os.unlink(files)
Upon initial execution, I expected to see all txt files deleted except for other non-txt files. The actual result was an error message "FileNotFoundError: [WinError 2] The system cannot find the file specified: 'File.txt'.
Not sure what I'm doing wrong, any help would be appreciated.
It isn't found because the the path you intended to unlink is relative to fileLoc. In fact with your code, the effect is to unlink the file relative to the current working directory. If there were *.txt files
in the cwd then the code would have unfortunate side-effects.
Another way to look at it:
Essentially, by analogy, in the shell what you're trying to do is equivalent to this:
# first the setup
$ mkdir foo
$ touch foo/a.txt
# now your code is equvalent to:
$ rm *.txt
# won't work as intended because it removes the *.txt files in the
# current directory. In fact the bug is also that your code would unlink
# any *.txt files in the current working directory unintentionally.
# what you intended was:
$ rm foo/*.txt
The missing piece was the path to the file in question.
I'll add some editorial: The Old Bard taught us to "when in doubt, print variables". In other words, debug it. I don't see from the OP an attempt to do that. Just a thing to keep in mind.
Anyway the new code:
Revised:
import os
fileLoc = 'c:\\temp\\files'
for file in os.listdir(fileLoc):
if file.endswith('.txt'):
os.unlink(os.path.join(fileLoc,file))
The fix: os.path.join() builds a path for you from parts. One part is the directory (path) where the file exists, aka: fileLoc. The other part is the filename, aka file.
os.path.join() makes a whole valid path from them using whatever OS directory separator is appropriate for your platform.
Also, might want to glance through:
https://docs.python.org/2/library/os.path.html

How do I use Nagios to monitor a log file that generates a random ID

This the log file that I want to monitor:
/test/James-2018-11-16_15215125111115-16.15.41.111-appserver0.log
I want Nagios to read it this log file so I can monitor a specific string.
The issue is with 15215125111115 this is the random id that gets generated
Here is my script where the Nagios is checking for the Logfile path:
Veriables:
HOSTNAMEIP=$(/bin/hostname -i)
DATE=$(date +%F)
..
CHECK=$(/usr/lib64/nagios/plugins/check_logfiles/check_logfiles
--tag='failorder' --logfile=/test/james-${date +"%F"}_-${HOSTNAMEIP}-appserver0.log
....
I am getting the following output in nagios:
could not find logfile /test/James-2018-11-16_-16.15.41.111-appserver0.log
15215125111115 This number is always generated randomly but I don't know how to get nagios to identify it. Is there a way to add a variable for this or something? I tried adding an asterisk "*" but that didn't work.
Any ideas would be much appreciated.
--tag failorder --type rotating::uniform --logfile /test/dummy \
--rotation "james-${date +"%F"}_\d+-${HOSTNAMEIP}-appserver0.log"
If you add a "-v" you can see what happens inside. Type rotating::uniform tells check_logfiles that the rotation scheme makes no difference between current log and rotated archives regarding the filename. (You frequently find something like xyz..log). What check_logfile does is to look into the directory where the logfiles are supposed to be. From /test/dummy it only uses the directory part. Then it takes all the files inside /test and compares the filenames with the --rotation argument. Those files which match are sorted by modification time. So check_logfiles knows which of the files in question was updated recently and the newest is considered to be the current logfile. And inside this file check_logfiles searches the criticalpattern.
Gerhard

Is there a way to undo a batch-rename of file extensions?

Ok so I kinda dropped the ball. I was trying to understand how things work. I had a few html files on my computer that I was trying to rename as txt files. This was strictly a learning exercise. Following the instructions I found here using this code:
for file in *.html
do
mv "$file" "${file%.html}.txt"
done
produced this error:
mv: rename *.html to *.txt: No such file or directory
Long story short I ended up going rogue and renaming the html files, as well as a lot of other non html files as txt files. So now I have files labeled like
my_movie.mp4.txt
my_song.mp3.txt
my_file.txt.txt
This may be a really dumb question but.. Is there a way to check if a file has two extensions and if yes remove the last one? Or any other way to undo this mess?
EDIT
Doing this find . -name "*.*.txt" -exec echo {} \; | cat -b seems to tell me what was changed and where it is located. The cat -b part is not necessary but I like it. This still doesn't fix what I broke though.
I'm not sure if terminal can check for extensions "twice", but you can check for . in every name an if there's more than one occurence of ., then your file has more extensions. Then you can cut the extension off with finding first occurence of . in a string when going backwards... or last one if checking characters in string in a normal way.
I have a faster option for you if you can use python. You can strip the extension with:
for file in list_of_files:
os.rename(file,os.path.splitext(file)[0])
which can give you from your file.txt.txt your file.txt
Example:
You wrote that your command tells you what has changed, so just take those changed files and dump them into a file(path to file per line). Then you can easily run this:
with open('<path to list>') as f:
list_of_files = f.readlines()
for file in list_of_files:
os.rename(file.strip('\n'), os.path.splitext(file.strip('\n'))[0])
If not, then you'd need to get the list from python:
import os
results = []
for root, folder, filenames in os.walk(<your path to folder>):
for filename in filenames:
if filename.endswith('.txt.txt'):
results.append(os.path.join(root, filename))
With this you got a list of files ending with .txt.txt like this <your folder>\\<path_to_file>.
Get a path to your directory used in os.walk() without folder's name(it's already in list) so it'll be like this:
e.g. os.walk('/home/me/directory') -> path='/home/me/' and res is item already in a list, which looks like directory/...
for res in results:
path = '' # set the path here
file = os.path.join(path,r)
os.rename(file, os.path.splitext(file)[0])
Depending on what files you want to find change .txt.txt in if filename.endswith('...') to whatever you like and os.rename() will take file's name without extension which in your case means it strips the additional extension you don't want to have.

bash script zip filename parsing strangely

I'm trying to zip various files together (one of the included files is actually a zip itself) and name the resulting zip based on a handful of bash variables defined earlier. One of the variables used in the zip file name is being parsed from a #define in a config.h file. I successfully parsed together a .zip with the correct name, but when I tried to implement the same zip script in a slightly different situation I get erroneous zip names.
In Windows explorer, the erroneous zip name looks something like X1276N~E.ZIP
In linux the zip appears with the intended name except with a question mark (which I've come to understand to be some sort of placeholder). i.g. foo-stuff-bar-9.1b?.zip
My current code trying to zip a file with name foo-stuff-bar-9.1b.zip:
foo_name=$1
bar_name=$2
rev_number=$(grep define[[:space:]]*SOME_NUMBER $directory/config.h | awk '{print $3;}'| tr -d '/"')
archive_name="$foo_name"-stuff-"$bar_name"-9."$rev_number"
zip "$archive_name".zip file1 file2 backup1.zip file3
So "foo_name" and "bar_name" are strings coming from the terminal when the script is run, "rev_number" is being parsed from config.h, and I'm formatting it all into "archive_name" before using it in the zip command.
I've tried all sorts of variations of quotation marks and brackets and I get the same weird name name no matter what I try. I'm not sure where my error is being caused as I'm parsing from many sources. Any advice is much appreciated.
Per Marc B's suggestion, I piped the string to xxd -b to look at each character byte by byte. It appeared as though I was accidentally parsing a character at the end of $archive_name when scraping the config.h file.
I was able to fix this by just piping my string through tr -d "[:cntrl:]" to remove the any control characters that would give weird file names.

"Unable to open image" error when using ImageMagick's Filename References

I'm using ImageMagick to do some image processing from the commandline, and would like to operate on a list of files as specified in foo.txt. From the instructions here: http://www.imagemagick.org/script/command-line-processing.php I see that I can use Filename References from a file prefixed with #. When I run something like:
montage #foo.txt output.jpg
everything works as expected, as long as foo.txt is in the current directory. However, when I try to access bar.txt in a different directory by running:
montage /some_directory/#bar.txt
output2.jpg
I get:
montage: unable to open image
/some_directory/#bar.txt: No such file
or directory # blob.c/OpenBlob/2480.
I believe the issue is my syntax, but I'm not sure what to change it to. Any help would be appreciated.
Quite an old entry but it seems relatively obvious that you need to put the # before the full path:
montage #/some_directory/bar.txt output2.jpg
As of ImageMagick 6.5.4-7 2014-02-10, paths are not supported with # syntax. The # file must be in the current directory and identified by name only.
I haven't tried directing IM to pull the list of files from a file, but I do specify multiple files on the command line like this:
gm -sOutputFile=dest.ext -f file1.ppm file2.ppm file3.ppm
Can you pull the contents of that file into a variable, and then let the shell expand that variable?

Resources