I have a folder with lots of files named with a continuing number and some text, but some numbers are missing. I want to write all missing numbers into a file.
Here is what I got so far:
#!/bin/bash
for (( c=23457; c<=24913; c++ ))
do
files=$(printf %q kassensystem/documents/"${c}")
ret=$(ls $files*)
echo "$ret" >> ./out.log
done
The output looks like that:
all existing files are written into file, all errors into console. I want exactly the other way. All errors (ls: ..file not found) written into the file!
I tried to use the complete command ls $files* | grep -v 'kasse*', but then I only get a file with empty lines.
Thanks for your help!
exec 4>out.log # open output file just once, not once per write
for (( c=23457; c<=24913; c++ )); do
files=( kassensystem/documents/"$c"* ) # glob into an array
[[ -e $files ]] || echo "$c" >&4 # log if first file in array doesn't exist
done
Related
I have written a small script that will take the users input and then generate the md5sum values for it
count = 0
echo "Enter number of records"
read number
while [ $count -le $number ]
do
echo "Enter path"
read path
echo "file name"
read file_name
md5sum $path"/"$filename #it shows the md5sum value and path+filename
((count++))
done
How can I pass these values ( path,file name, and md5sums ) to CSV file. ( assuming the user chooses to enter more than 1 record)
The output should be like
/c/training,sample.txt,34234435345346549862123454651324 #placeholder values
/c/file,text.sh,4534534534534534345345435342342
Interactively prompting for the number of files to process is just obnoxious. Change the script so it accepts the files you want to process as command-line arguments.
#!/bin/sh
md5sum "$#" |
sed 's%^\([0-9a-f]*\) \(\(.*\)/\)?\([^/]*\)$%\3,\4,\1%'
There are no Bash-only constructs here, so I switched the shebang to /bin/sh; obviously, you are still free to use Bash if you like.
There is a reason md5sum prints the checksum before the path name. The reordered output will be ambiguous if you have file names which contain commas (or newlines, for that matter). Using CSV format is actually probably something you should avoid if you can; Unix tools generally work better with simpler formats like tab-delimited (which of course also breaks if you have file names with tabs in them).
Rather than prompting the user for both a path to a directory and the name of a file in that directory, you could prompt for a full path to the file. You can then extract what you need from that path using bash string manipulations.
#!/bin/bash
set -euo pipefail
function calc_md5() {
local path="${1}"
if [[ -f "${path}" ]] ; then
echo "${path%/*}, ${path##*/}, $(md5sum ${path} | awk '{ print $1 }')"
else
echo "
x - Script requires path to file.
Usage: $0 /path/to/file.txt
"
exit 1
fi
}
calc_md5 "$#"
Usage example:
$ ./script.sh /tmp/test/foo.txt
/tmp/test, foo.txt, b05403212c66bdc8ccc597fedf6cd5fe
I'd like to write a short and simple script, that searches for a file using a specivic filter, and checks the age of that file. I want to write a short output and an error-code. This should be accessible for an NRPE-Server.
The script itself works, but I only have a problem when the file does not exist. This happens with that command:
newestfile=$(ls -t $path/$filter | head -1)
When the files exist, everything works as it should. When there nothing matches my filter, I get the output (I changed the filter to *.zip to show):
ls: cannot access '/backup/*.zip': No such file or directory
But I want to get the following output and then just exit the script with code 1:
there are no backups with the filter *.zip in the directory /backup
I am pretty sure this is a very easy problem but I just don't know whats wron. By the way, I am still "new" to bash scripts.
Here is my whole code:
#!/bin/bash
# Set the variables
path=/backup
filter=*.tar.gz
# Find the newest file
newestfile=$(ls -t $path/$filter | head -1)
# check if we even have a file
if [ ! -f $newestfile ]; then
echo "there are no backups with the filter $filter in the directory $path"
exit 1
fi
# check how old the file is that we found
if [[ $(find "$newestfile" -mtime +1 -print) ]]; then
echo "File $newestfile is older than 24 hours"
exit 2
else
echo "the file $newestfile is younger than 24 hours"
exit 0
fi
Actually, with your code you should also get an error message bash: no match: /backup/*.zip
UPDATE: Fixed the proposed solution, and the missing quotes in the original solution:
I suggest the following approach:
shopt -u failglob # Turn off error from globbing
pathfilter="/backup/*.tar.gz" # Quotes to avoid the wildcards to be expanded here already
# First see whether we have any matching files
files=($pathfilter)
if [[ ! -e ${#files[0]} ]]
then
# .... No matching files
else
# Now you can safely fetch the newest file
# Note: This does NOT work if you have filenames
# containing newlines
newestfile=$(ls -tA $pathfilter | head -1)
fi
I don't like using ls for this task, but I don't see an easy way in bash to do it better.
I'm trying to concatenate a bunch of files into a string so I can use them for a function.
As a test script I'm trying to do this:
#!/bin/bash
for line in $(cat list.txt)
do
x=" "
A=$A$line$x
done
echo "$A"
mv "$A" ./stuff
but I'm getting the error:
mv: cannot stat ‘x.dat y.dat z.dat ’: No such file or directory
but they are most definitely there
can I get some advice please?
This solution will handle file names with spaces too.
#!/bin/bash
mapfile -t lines < list.txt
echo "${lines[#]}"
mv "${lines[#]}" ./stuff/
It reads the entire contents of the file into an array variable, displays the content of the entire array, and finally uses those values in the mv command
Change the last line to mv $A ./stuff
That should work with files that do not have space in their names.
I am still a newbie in shell scripting and trying to come up with a simple code. Could anyone give me some direction here. Here is what I need.
Files in path 1: /tmp
100abcd
200efgh
300ijkl
Files in path2: /home/storage
backupfile_100abcd_str1
backupfile_100abcd_str2
backupfile_200efgh_str1
backupfile_200efgh_str2
backupfile_200efgh_str3
Now I need to delete file 300ijkl in /tmp as the corresponding backup file is not present in /home/storage. The /tmp file contains more than 300 files. I need to delete the files in /tmp for which the corresponding backup files are not present and the file names in /tmp will match file names in /home/storage or directories under /home/storage.
Appreciate your time and response.
You can also approach the deletion using grep as well. You can loop though the files in /tmp checking with ls piped to grep, and deleting if there is not a match:
#!/bin/bash
[ -z "$1" -o -z "$2" ] && { ## validate input
printf "error: insufficient input. Usage: %s tmpfiles storage\n" ${0//*\//}
exit 1
}
for i in "$1"/*; do
fn=${i##*/} ## strip path, leaving filename only
## if file in backup matches filename, skip rest of loop
ls "${2}"* | grep -q "$fn" &>/dev/null && continue
printf "removing %s\n" "$i"
# rm "$i" ## remove file
done
Note: the actual removal is commented out above, test and insure there are no unintended consequences before preforming the actual delete. Call it passing the path to tmp (without trailing /) as the first argument and with /home/storage as the second argument:
$ bash scriptname /path/to/tmp /home/storage
You can solve this by
making a list of the files in /home/storage
testing each filename in /tmp to see if it is in the list from /home/storage
Given the linux+shell tags, one might use bash:
make the list of files from /home/storage an associative array
make the subscript of the array the filename
Here is a sample script to illustrate ($1 and $2 are the parameters to pass to the script, i.e., /home/storage and /tmp):
#!/bin/bash
declare -A InTarget
while read path
do
name=${path##*/}
InTarget[$name]=$path
done < <(find $1 -type f)
while read path
do
name=${path##*/}
[[ -z ${InTarget[$name]} ]] && rm -f $path
done < <(find $2 -type f)
It uses two interesting shell features:
name=${path##*/} is a POSIX shell feature which allows the script to perform the basename function without an extra process (per filename). That makes the script faster.
done < <(find $2 -type f) is a bash feature which lets the script read the list of filenames from find without making the assignments to the array run in a subprocess. Here the reason for using the feature is that if the array is updated in a subprocess, it would have no effect on the array value in the script which is passed to the second loop.
For related discussion:
Extract File Basename Without Path and Extension in Bash
Bash Script: While-Loop Subshell Dilemma
I spent some really nice time on this today because I needed to delete files which have same name but different extensions, so if anyone is looking for a quick implementation, here you go:
#!/bin/bash
# We need some reference to files which we want to keep and not delete,
# let's assume you want to keep files in first folder with jpeg, so you
# need to map it into the desired file extension first.
FILES_TO_KEEP=`ls -1 ${2} | sed 's/\.pdf$/.jpeg/g'`
#iterate through files in first argument path
for file in ${1}/*; do
# In my case, I did not want to do anything with directories, so let's continue cycle when hitting one.
if [[ -d $file ]]; then
continue
fi
# let's omit path from the iterated file with baseline so we can compare it to the files we want to keep
NAME_WITHOUT_PATH=`basename $file`
# I use mac which is equal to having poor quality clts
# when it comes to operating with strings,
# this should be safe check to see if FILES_TO_KEEP contain NAME_WITHOUT_PATH
if [[ $FILES_TO_KEEP == *"$NAME_WITHOUT_PATH"* ]];then
echo "Not deleting: $NAME_WITHOUT_PATH"
else
# If it does not contain file from the other directory, remove it.
echo "deleting: $NAME_WITHOUT_PATH"
rm -rf $file
fi
done
Usage: sh deleteDifferentFiles.sh path/from/where path/source/of/truth
I am very new in programing scripts-.
I have a lot of zip files in a directory. I want to extract them replacing the name of the inside file by the zip file, with the correct extension. Error reporting if there is more than one file, excep if is "remora.txt" inside.
The file "remora.txt" was an ini file for the zip, and I wont use it any more, but is in a lot of my zip files.
Example 1.
ZIp file: maths.zip,
Inside it has:
- "tutorial in maths.doc"
- "remora.txt"
Action:
So the script should erase or deprease "remora.txt" and extract "tutorial in maths.doc" under the name maths.doc
Example 2.
ZIp file: geo.zip,
Inside it has:
- "excersices for geometry.doc"
- "geometry.doc"
- "remora.txt"item
Action:
It should out put "I found more than a file in geo.zip"
I am
Using linux, ubuntu 12
I have done this script, but is not working.
#!/bin/bash
#
# Linux Shell Scripting Tutorial 1.05r3, Summer-2002
#
for archive in *.zip # First I read the zip file
do
((i++))
unzip -Z1 $archive | while read line; # I read all the files in the ZIP
do
line=( ${line//,/ } )
inside[$a]=("${line[#]}") # Here I assigne the name of the file to an array
((a++))
done
If ( $a > 2) then
echo " Too much files in file $archive "
fi
If ($a <= 2)
then
if (inside[0]!= "remora.txt")
then unzip -p $archive > $(printf "%s" $archive).doc
fi
if (inside[1]!= "remora.txt")
then unzip -p $archive > $(printf "%s" $archive).doc
fi
fi
done
Try writing scripts incrementally. Instead of writing 20 statements and then trying to debug them all at once, write one statement at a time and test to make sure it works before writing the next one.
If you run e.g.
If ( $a > 2) then
echo " Too much files in file $archive "
fi
by itself, you'll see that it doesn't work. You then know more specifically what the problem is, and you can look up something like "bash if variable greater than" on Google or Stackoverflow.
Check out the bash tag wiki for more helpful tips on debugging and asking about code.
Things you'll find includes:
if has to be lower case
You need line feed or semicolon before then
To see if a variable is greater than, use [[ $a -gt 2 ]].
To see if an array element does not equal, use [[ ${inside[0]} != "remora.txt" ]]
Pipelines cause subshells. Use while read ...; do ...; done < <(somecommand) instead.