I am rather inexperienced in linux,
I have to check in bash script if some zip file is empty - i.e zip contains no files.
I found this code:
if ! zipinfo ${filetotransfer} | tail -n 1 | grep '^0 ' >/dev/null ; then # we have empty zip file!
echo " zip empty"
rm $filetotransfer
exit 0
fi
But it removes file both if zip is empty or not.
Is there any way to check it?
You can just check file size is 22 with stat or md5sum or zip header
# by checking file size
% stat -f%z a.zip
22
% xxd a.zip
0000000: 504b 0506 0000 0000 0000 0000 0000 0000 PK..............
0000010: 0000 0000 0000 ......
# with md5sum
$ md5sum a.zip
76cdb2bad9582d23c1f6f4d868218d6c a.zip
# or by checking zip header
% [ `head -n22 a.zip | tr -d '\0-\6'` = "PK" ] && echo 1
1
You can check the error status of zipinfo -t
f=test.zip
if zipinfo -t "$f" > /dev/null
then
echo "not empy"
else
echo "empty"
fi
Related
File size check
file_path="/home/d-vm/"
cd $file_path
# file_path directory
file=($(cat video.txt | xargs ls -lah | awk '{ print $9}'))
# get name in video.txt --- two files for example VID_141523.mp4 VID_2_141523.mp4
minimumsize=1
actualsize=$(wc -c <"$file")
if [ $actualsize -ge $minimumsize ]; then
echo $file size $actualsize bytes
else
echo error $file size 0 bytes
fi
VID_141523.mp4 file corrupted during conversion. its size 0 bytes
Script output---- error VID_20220709_141523.mp4 size 0 bytes
video.txt
VID_141523.mp4
VID_2_141523.mp4
How to add this construct to the loop ?
It should check all files in the list video.txt
To read a file size stat is the best for use-case here. On a linux machine with GNU stat
$ stat --printf="%s" one
4
This can be looped like
#!/bin/bash
while read -r file ; do
if [ "$(stat --printf="%s" "$file")" -gt 0 ]
then
echo "$file" yes
else
echo "$file" no
fi
done < video.txt
This is too complicated approach in my opinion. Just use find
$ touch zero
$ echo one > one
$ ll
total 8
-rw-r--r-- 1 sverma wheel 4B Jul 11 16:20 one
-rw-r--r-- 1 sverma wheel 0B Jul 11 16:20 zero
one is a 4 byte file. Use -size predicate. Here + means greater than, -type f means only files
$ find . -type f -size +0 -print
./one
You can add a filter for names with something like
$ find . -name '*.mp4' -type f -size +0
You probably mean - "how to loop instead of this: file=($(cat video.txt | xargs ls -lah | awk '{ print $9}'))" (which is pretty horrible by itself (*)).
You should just use a while loop:
cat video.txt | while read file; do
# something with $file
done
Also, please use stat -c%s FILE instead of wc (which, BTW, also takes a file - you can use wc -c FILE instead of using input redirection), as that looks just at the file system information to check the size, instead of loading and counting each byte.
(*) doing ls -lah and then awk '{print$9}' is the same as just doing ls, but there are other issues with this code that is very not bash idiomatic.
I am having a hard time with the following bash script:
basically what the script does is receives a directory and then it searches in all of the folders that are in the directory for files that end with .log. after that it should print to the stdout all the lines from those files sorted by the date they were written in.
my script is this:
#!/bin/bash
find . -name ".*log" | cat *.log | sort --stable --reverse --key=2,3
when i run the script it does return the list but the sort doesnt work properly. my guess is because in some files there are \n which makes it start a new line.
is there a way to ignore the \n that are in the file while still having each line return on a new line?
thank you!
xxd command output:
ise#ise-virtual-machine:~$ xxd /home/ise/Downloads/f1.log
00000000: 3230 3139 2d30 382d 3232 5431 333a 3333 2019-08-22T13:33
00000010: 3a34 342e 3132 3334 3536 3738 3920 4865 :44.123456789 He
00000020: 6c6c 6f0a 576f 726c 640a 0032 3032 302d llo.World..2020-
00000030: 3031 2d30 3154 3131 3a32 323a 3333 2e31 01-01T11:22:33.1
00000040: 3233 3435 3637 3839 206c 6174 650a 23456789 late.
ise#ise-virtual-machine:~$ xxd /home/ise/Downloads/f2.log
00000000: 3230 3139 2d30 392d 3434 5431 333a 3434 2019-09-44T13:44
00000010: 3a32 312e 3938 3736 3534 3332 3120 5369 :21.987654321 Si
00000020: 6d70 6c65 206c 696e 650a mple line.
ise#ise-virtual-machine:~$ xxd /home/ise/Downloads/f3.log
00000000: 3230 3139 2d30 382d 3232 5431 333a 3333 2019-08-22T13:33
00000010: 3a34 342e 3132 3334 3536 3738 3920 4865 :44.123456789 He
00000020: 6c6c 6f0a 576f 726c 6420 320a 0032 3032 llo.World 2..202
00000030: 302d 3031 2d30 3154 3131 3a32 323a 3333 0-01-01T11:22:33
00000040: 2e31 3233 3435 3637 3839 206c 6174 6520 .123456789 late
00000050: 320a 2.
Given that the entries in the log file are terminated with \0 (NUL), find, sed and sort can be combined:
find . -name '*.log' | xargs sed -z 's/\n//g' | sort -z --key=2,3 --reverse
By assuming each record in the file starts with the date and the option --key=2,3 is not necessary, please try:
find . -name "*.log" -exec cat '{}' \; | sort -z | xargs -I{} -0 echo "{}"
The final command xargs .. echo .. will be necessary to print properly the null-terminated lines.
If you still require --key option, please modify the code as you like. I'm not aware how the lines look like as of now.
[UPDATE]
According to the provided information by the OP, I assume the format of the log files
will be:
Each record starts with the date in "yyyy-mm-ddTHH:MM:SS.nanosec" format
and a simple dictionary order sort can be applied.
Each record ends with "\n\0" except for the last record of the file
which ends just with "\n".
Each record may contain newline character(s) in the middle as a part
of the record for the line folding purpose.
Then how about:
find . -name "*.log" -type f -exec cat "{}" \; -exec echo -ne "\0" \; | sort -z
echo -ne "\0" appends a null character to the last record of a file.
Otherwise the record will be merged to the next record of another file.
The -z option to sort treats the null character as a record separator.
No other option to sort will be required so far.
Result with the posted input by the OP:
2019-08-22T13:33:44.123456789 Hello
World
2019-08-22T13:33:44.123456789 Hello
World 2
2019-09-44T13:44:21.987654321 Simple line
2020-01-01T11:22:33.123456789 late
2020-01-01T11:22:33.123456789 late 2
It still keeps the null character "\0" at the end of each record.
If you want to trim it off, please add the tr -d "\0" command
at the end of the pipeline as:
find . -name "*.log" -type f -exec cat "{}" \; -exec echo -ne "\0" \; | sort -z | tr -d "\0"
Hope this helps.
I have mulitple directories and for each directory i want to list the total count of files grouped by their extensions, like follows:
/Documents
2 zip 20 M
3 tar 50 M
5 png 10 K
/Desktop
5 txt 10 K
7 png 5 M
10 jpg 5 M
The following script is my approach, but it's output differs from what i expect:
#!/bin/bash
find . -maxdepth 1 -type d -print0 | while read -d '' -r dir; do
num=$(find $dir find . -type f | sed -n 's/..*\.//p' | sort | uniq -c);
printf "%5d files in directory %s\n" "$num" "$dir";
done
Script Output:
7 csv
1 csv#
2 doc
1 docx
9 html
24 jpg
33 js
20 mp3
2 mp4
9 pdf
101 png
3 ppt
128 swf
3 xml
4 xsd
26 zip
From your script, I get the idea that you just want to go one level deep. So, why not try something like this:
for dir in * ; do
if [ -d "$dir" ] ; then
echo "$dir"
extlist=`ls $dir | sed 's/.*\.//' | uniq`
for ext in $extlist ; do
qty=`qty=`ls "$dir/*$ext"| wc -l`
space=`du -ch "$dir/*$ext"| sed -n 's/ *total//p'`
echo "$qty $ext $space"
done
fi
done
Is this what you're after?
$ mkdir Desktop Documents
$ for d in D*; do for e in zip jpg foo; do for i in $(seq 1 $((RANDOM/3000))); do touch $d/$i.$e; done; done; done
$ declare -A a; for d in D*; do a=(); for f in $d/*.*; do (( a[${f##*.}]++ )); done; printf '%s: ' "$d"; declare -p a; done;
Desktop: declare -A a=([jpg]="3" [foo]="5" [zip]="3" )
Documents: declare -A a=([jpg]="6" [foo]="2" [zip]="2" )
You can parse the counting array as you see fit at the end of each iteration of the outer loop.
I'm making a bash script in which I need to print a number while it's incremented like this:
0000
0001
0002
0003
0004
I have made this but is not working:
#!/bin/bash
i=0
pass[0]=0
pass[1]=0
pass[2]=0
pass[3]=0
for i in $(seq 1 9)
pass[3]="$i"
echo ${pass[*]}
done
I paste the script on cli and i get this.
$ ~ #!/bin/bash
$ ~ i=0
$ ~ pass[0]=0
$ ~ pass[1]=0
$ ~ pass[2]=0
$ ~ pass[3]=0
$ ~ for i in $(seq 1 9)
> pass[3]="$i"
bash: error sintáctico cerca del elemento inesperado `pass[3]="$i"'
$ ~ echo ${pass[*]}
0 0 0 0
$ ~ done
bash: error sintáctico cerca del elemento inesperado `done'
$ ~
Use this pure bash script:
for ((i=0; i<10; i++)); do
printf "%04d\n" $i
one
OUTPUT:
0000
0001
0002
0003
0004
0005
0006
0007
0008
0009
#!/bin/bash
i=0
pass[0]=0
pass[1]=0
pass[2]=0
pass[3]=0
for i in $(seq 1 9)
do
pass[3]="$i"
echo ${pass[*]}
done
did you forget 'do'
For those of you who like expansions, you can also do:
printf "%s\n" {0001..0009}
or
printf "%.4d\n" {1..9}
No loop!
You can store in an array thus:
$ myarray=( {0001..0009} )
$ printf "%s\n" "${myarray[#]}"
0001
0002
0003
0004
0005
0006
0007
0008
0009
$ echo "${myarray[3]}"
0004
You can do the formatting with seq:
seq -w 0000 0010
(if you don't like the {0000..0010} notation, which is more efficient but doesn't allow parameter substitution.)
The below script gives this error:
rm: illegal option -- 4
rm: illegal option -- 5
rm: illegal option -- 4
rm: illegal option -- 3
rm: illegal option -- 2
the script:
#!/bin/bash
keep_no=$1+1
cd "/mydirec/"
rm -rf `ls | sort -nr | tail +$keep_no`
I would like the script to accept an argument (of num of direcs to keep) then remove all directories (including their containing files) except for the (number passed in the script - ordering by the numerical direc names in descending order).
ie if /mydirec/ contains these direc names:
53
92
8
152
77
and the script is called like: bash del.sh 2
then /mydirec/ should contains these direcs (as it removes those that aren't the top 2 in desc order):
152
92
Can someone please help with the syntax?
Should read:
rm -rf `ls | sort -nr | tail -n +$keep_no`
But it is good practice not to parse ls output. Use find instead.
#!/bin/bash
keep_no=$(( $1+1 ))
directory="./mydirec/"
cd $directory
rm -rf `find . -maxdepth 1 -mindepth 1 -type d -printf '%f\n'| sort -nr | tail -n +$keep_no`
cd -
#!/bin/bash
if [[ -z "$1" ]]; then
echo "syntax is..."
exit 1
fi
keep_no=$(( $1 + 1 ))
cd "/mydirec/"
IFS='
'; # record separator: only enter inside single quotes
echo rm -rf $(ls | sort -nr | tail +$keep_no)
Verify the output of the script manually, then execute the script through sh:
./your_script.sh | sh -x
If you want to leave two directories (not to delete) you need to calculate total number of directories. And xargs utility is more convenient way to pass a list of arguments to rm.
#!/bin/bash
dir="/yourdir"
total_no=`ls | wc -l`
keep_no=$(( $total_no - $1 ))
ls | sort -nr | tail -n $keep_no | xargs rm -rf