Is there a way to pass multiple values into a CSV file, based on the output of a linux script - linux

I have written a small script that will take the users input and then generate the md5sum values for it
count = 0
echo "Enter number of records"
read number
while [ $count -le $number ]
do
echo "Enter path"
read path
echo "file name"
read file_name
md5sum $path"/"$filename #it shows the md5sum value and path+filename
((count++))
done
How can I pass these values ( path,file name, and md5sums ) to CSV file. ( assuming the user chooses to enter more than 1 record)
The output should be like
/c/training,sample.txt,34234435345346549862123454651324 #placeholder values
/c/file,text.sh,4534534534534534345345435342342

Interactively prompting for the number of files to process is just obnoxious. Change the script so it accepts the files you want to process as command-line arguments.
#!/bin/sh
md5sum "$#" |
sed 's%^\([0-9a-f]*\) \(\(.*\)/\)?\([^/]*\)$%\3,\4,\1%'
There are no Bash-only constructs here, so I switched the shebang to /bin/sh; obviously, you are still free to use Bash if you like.
There is a reason md5sum prints the checksum before the path name. The reordered output will be ambiguous if you have file names which contain commas (or newlines, for that matter). Using CSV format is actually probably something you should avoid if you can; Unix tools generally work better with simpler formats like tab-delimited (which of course also breaks if you have file names with tabs in them).

Rather than prompting the user for both a path to a directory and the name of a file in that directory, you could prompt for a full path to the file. You can then extract what you need from that path using bash string manipulations.
#!/bin/bash
set -euo pipefail
function calc_md5() {
local path="${1}"
if [[ -f "${path}" ]] ; then
echo "${path%/*}, ${path##*/}, $(md5sum ${path} | awk '{ print $1 }')"
else
echo "
x - Script requires path to file.
Usage: $0 /path/to/file.txt
"
exit 1
fi
}
calc_md5 "$#"
Usage example:
$ ./script.sh /tmp/test/foo.txt
/tmp/test, foo.txt, b05403212c66bdc8ccc597fedf6cd5fe

Related

Passing argument into shell script as a form of txt file

I would like to know how to access the contents of a variety of txt files by passing arguments into shell scripts. I'll have different files and I'm expecting to execute with this command:
./script.sh FileA.txt
What should I put into my shell script so that I can access and manipulate the contents of the files?
I tried this but it outputs 0:
echo "$#"
I also tried these, but both output nothing:
for i in $1
do
echo "$i"
done
echo "$1"
To sum up the contents see this link to understand bash arguments more https://tecadmin.net/tutorial/bash-scripting/bash-command-arguments/ . Also as #Barmar said, to iterate through a list of arguments of unknown quantity use for i in "$#" .
edit
and as #Barmar said, $1 is simply the name of the argument. So echoing $1 will just echo the name.
I don't understand your question fully. Lets assume you have list of file names in a text file "FileA.txt".
And you wanted to run some commands for each file in the "FileA.txt" file.
Can you try below:
for i in `cat $1`
do
echo $i
done

Can I get the name of the file currently being read in a for loop?

I want to write a script that takes a word as an argument and searches the current and sub directories' files for the word. if it is found in any of the files it should echo out a message containing the file name and the line the word is found on.
this is what I have so far, but I can't find a way to actually store the file name of the file being read or the line number..
word=$1
for var in $(grep -R "$word *")
do
filename=$(find . -type f -name "*") ------- //this doesnt work
linenmbr=$(grep -n "$ord" file) ----------- //this doesnt work
echo found $word in $filename on line number $linenmbr
done
In bash, any time you are looping, you want to avoid calling utilities (e.g. grep and find) within the loop. That is horribly inefficient because it will spawn a separate subshell for every utility every iteration. (which for 10 iterations -- that is 20 additional subshells, it adds up quick) So in your case, you call grep to feed the loop, and then spawn a separate subshell calling grep again within the loop as well as spawning a separate subshell for find.
You should think of a way to only call grep (or a utility that will provide the needed information) only once, and then parse the output.
If you did want to use grep, then calling grep -rn within a process substitution which is used to feed a while loop is probably as good as you are going to get. You can then use the bash builtin parameter expansions to isolate the filename and line-numbers which will be about as efficient as bash could get, e.g.
#!/bin/bash
[ -z "$1" ] && { ## validate at least 1 input given
printf "error: insufficient input.\nusage: %s srch_term\n" "${0##*/}"
exit 1
}
while read -r line; do ## read each line of grep output
fn="${line%%:*}" ## isolate filename
no="${line#*:}" ## remove filename
no="${no%%:*}" ## isolate number
printf "found %s in %s on line number %d\n" "$1" "$fn" "$no"
done < <(grep -rn "$1") ## grep in process substitution
Choosing A More Efficient Method
If you can accomplish what you are attempting with one of the stream editing tools, e.g. awk or sed, you are likely to be able to isolate the wanted information an order of magnitude faster. For example, using awk and setting globstar you could do something similar to the following:
#!/bin/bash
shopt -s globstar ## set globstar
[ -z "$1" ] && { ## validate at least 1 input given
printf "error: insufficient input.\nusage: %s srch_term\n" "${0##*/}"
exit 1
}
## find all matching files and line numbers
awk -v word="$1" '/'$1'/ {
print "found",word,"in",FILENAME,"on line number",FNR; next
}' **/* 2>/dev/null
Give both a try and let me know if you have further questions.
If you want to compare and ensure both are producing the same output, you can use diff to confirm, e.g.
$ diff <(grepscript.sh | sort) <(awkscript.sh | sort)
(if no difference is reported, the output is the same)

Get current directory (not full path) with filename only when sub folder is present in Linux bash

I have prepared a bash script to get only the directory (not full path) with file name where file is present. It has to be done only when file is located in sub directory.
For example:
if input is src/email/${sub_dir}/Bank_Casefeed.email, output should be ${sub_dir}/Bank_Casefeed.email.
If input is src/layouts/Bank_Casefeed.layout, output should be Bank_Casefeed.layout. I can easily get this using basename command.
src/basefolder is always constant. In some cases (after src/email(basefolder) directory), sub_directories will be there.
This script will work. I can use this script (only if module is email) to get output. but script should work even if sub directory is present in other modules. Maybe should I count the directories? if there are more than two directories (src/basefolder), script should get sub directories. Is there any better way to handle both scenarios?
#!/bin/bash
filename=`basename src/email/${sub_dir}/Bank_Casefeed.email`
echo "filename is $filename"
fulldir=`dirname src/email/${sub_dir}/Bank_Casefeed.email`
dir=`basename $fulldir`
echo "subdirectory name: $dir"
echo "concatenate $filename $dir"
Entity=$dir/$filename
echo $Entity
Using shell parameter expansion:
sub_dir='test'
files=( "src/email/${sub_dir}/Bank_Casefeed.email" "src/email/Bank_Casefeed.email" )
for f in "${files[#]}"; do
if [[ $f == *"/$sub_dir/"* ]]; then
echo "${f/*\/$sub_dir\//$sub_dir\/}"
else
basename "$f"
fi
done
test/Bank_Casefeed.email
Bank_Casefeed.email
I know there might be an easier way to do this. But I believe you can just manipulate the input string. For example:
#!/bin/bash
sub_dir='test'
DIRNAME1="src/email/${sub_dir}/Bank_Casefeed.email"
DIRNAME2="src/email/Bank_Casefeed.email"
echo $DIRNAME1 | cut -f3- -d'/'
echo $DIRNAME2 | cut -f3- -d'/'
This will remove the first two directories.

How can we increment a string variable within a for loop

#! /bin/bash
for i in $(ls);
do
j=1
echo "$i"
not expected Output:-
autodeploy
bin
config
console-ext
edit.lok
need Output like below if give input 2 it should print "bin" based on below condition, but I want out put like Directory list
1.)autodeploy
2.)bin
3.)config
4.)console-ext
5.)edit.lok
and if i like as input:- 2 then it should print "bin"
Per BashFAQ #1, a while read loop is the correct way to read content line-by-line:
#!/usr/bin/env bash
enumerate() {
local line i
i=0
while IFS= read -r line; do
((++i))
printf '%d.) %s\n' "$i" "$line"
done
}
ls | enumerate
However, ls is not an appropriate tool for programmatic use; the above is acceptable if the results of ls are only for human consumption, but not if they're going to be parsed by a machine -- see Why you shouldn't parse the output of ls(1).
If you want to list files and let the user choose among them by number, pass the results of a glob expression to select:
select filename in *; do
echo "$filename" && break
done
I don't understand what you mean in your question by like Directory list, but following your example, you do not need to write a loop:
ls|nl -s '.)' -w 1
If you want to avoid ls, you can do the following (but be careful - this only works if the directory entries do not contain white spaces (because this would make fmt to break them into two lines):
echo *|fmt -w 1 |nl -s '.)' -w 1

What do these lines of Unix/Linux do?

I am a Unix/Linux shell script newbie and I have been asked to look at a script which contains the lines below. The following details in this question are vague but the person who wrote this code left no documentation and has since demised. Can anyone advise what they actually do?
There are two specific pieces of code. The first is simply line source polys.sh where polys.sh is a text file with contents:
failure="020o 040a"
success="002[a-d] 003[a-r] 004[a-s] 005[a-u]
Representing various parameters, I think, to do with the calculations the shell script performs. The nature of the calculations is, I am told, not important because the aim is to just get the script running.
The second piece of code is below and the relevant lines are delimited by Start and Stop comments. What I can tell you is that: $arg1 is blank, $opt1 is also blank, $poly is the path and name of a text file and ./search I believe to be a folder.
if [ $search == "yes" ]
then
# Search stage for squares containing zeros
#
# Start.
output="$outputs/search/"`basename $poly`
./search $opt1 $arg1 < $poly 2>&1 | tee $output
if tail -n1 $output | grep -v "success"
# End.
then
echo "SEARCH FAILURE" >> $output
continue
fi
# Save approximations
#
echo -n "SEARCH SUCCESS " >> $output
cat /tmp/iters >> $output
cp /tmp/zeros $inputs/search/`basename $poly`
else
echo "No search"
fi
EDIT Initial disclaimer as advised by Mr. Charles Duffy:
The below explanations assume you won't hit expansion-related bugs; please correct your code as advised by shellcheck.net to be assured that these explanations are correct
source polys.sh includes the code from the script polys.sh, which is a file in the same folder as the file sourcing it (hence just the filename, without its path).
Within that file:
failure="020o 040a"
success="002[a-d] 003[a-r] 004[a-s] 005[a-u]"
are two variable declarations; the variable $failure is set to "020o 040a" and $success to "002[a-d] 003[a-r] 004[a-s] 005[a-u]". As the file was sourced, these two variables are available in your script (do echo "$failure" and echo "$success" to see for yourself).
output="$outputs/search/`basename $poly`" has two parts to explain:
"$outputs/search/"
sets the variable $output to "$outputs/search/", i.e., to the value of the variable $outputs, appended by the string "/search/".,
`basename $poly`
anything in backticks is a command substitution, which interprets and runs the command returning its output, and the command basename $poly gets the base file or folder name from $poly, if it is a file path (e.g., basename $poly for poly="/dev/file.txt" yields file.txt); the output is appended as a string. to "$outputs/search/".
./search $opt1 $arg1 < $poly 2>&1 | tee $output is two commands, separated by a pipe |:
./search $opt1 $arg1 < $poly 2>&1
runs the executable file ./search (./ is shorthand for the current script's directory) with two arguments, $opt1 and $opt2 variables. $poly is the variable name which should represent a file path, of which the file path has its content redirected to the command (using <). The output of all errors (stderr, as 2) is redirected (>) to the standard output (stdout, or &2, the ampersand represents this is a file descriptor, not a file path, otherwise it would redirect output to a file named 2).
tee $output
tee pipes outputs stdin to stdout and to arguments as file paths. So tee "/home/nick/output" would save the stdin to a file at "/home/nick/output", as well as the stdout.
if tail -n1 $output | grep -v "success"
tail -n1 $output
gets the last line of the file at the "$output" variable's value.
grep -v "success"
searches for any non-match (-v inverts the match) in the last line from tail -n1 of "success" in a line (e.g., if the last line is "fail", it would pass the if statement as it does not contain "success")

Resources