When I try to run this script this error appears : operating extra /home/ubuntu/Desktop/Destino/, and I do not know why , someone help me please.
#!/bin/bash
input="/home/ubuntu/Desktop/Output/SAIDA.txt"
dt=`date +"%Y%m%d%H%M%S"`
layout='C'
if [ -e "$input" ] ; then
header=$(head -n 1 $input)
export header
tail -n +2 $input | split -l 99 -d --additional-suffix=.txt \ --filter='{ printf %s\\n "$header"; cat; }' >/home/ubuntu/Desktop/Destino/$FILE - NOMENCLATURA_${dt}_
for arquivo in ´Is/home/ubuntu/Desktop/*.txt´
do
NOME= ´cat $arquivo | cut -d "." -f1´
touch/home/ubuntu/Desktop/Destino/$NOME.cfg
echo $dt > $NOME.cfg
echo $layout > $NOME.cfg
done
else
echo "The input file does not exist."
fi
You have some strange quote characters in your script. To substitute the output of a command, wrap it with $() or backticks, not ´ characters.
for arquivo in ´Is/home/ubuntu/Desktop/*.txt´
I guess Is was meant to be ls, but you left out the space after it. But there's no need to parse the output of ls, just use the wildcard directly.
for arquivo in /home/ubuntu/Desktop/*.txt
On this line:
tail -n +2 $input | split -l 99 -d --additional-suffix=.txt \ --filter='{ printf %s\\n "$header"; cat; }' >/home/ubuntu/Desktop/Destino/$FILE - NOMENCLATURA_${dt}_
you need to put the output filename in quotes because of the spaces.
tail -n +2 $input | split -l 99 -d --additional-suffix=.txt \ --filter='{ printf %s\\n "$header"; cat; }' >"/home/ubuntu/Desktop/Destino/$FILE - NOMENCLATURA_${dt}_"
Also, the FILE variable is not set, you need to assign that earlier.
On this line:
NOME= ´cat $arquivo | cut -d "." -f1´
you're again using the wrong type of quotes to capture the output of the command. Also, you must not have a space between = and the value you want to assign. It should be:
NOME=$(cat $arquivo | cut -d "." -f1)
There's no need to do export header. The variable is only being used in this script, not in any child processes.
Related
I am writing a function in a BASH shell script, that should return lines from csv-files with headers, having more commas than the header. This can happen, as there are values inside these files, that could contain commas. For quality control, I must identify these lines to later clean them up. What I have currently:
#!/bin/bash
get_bad_lines () {
local correct_no_of_commas=$(head -n 1 $1/$1_0_0_0.csv | tr -cd , | wc -c)
local no_of_files=$(ls $1 | wc -l)
for i in $(seq 0 $(( ${no_of_files}-1 )))
do
# Check that the file exist
if [ ! -f "$1/$1_0_${i}_0.csv" ]; then
echo "File: $1_0_${i}_0.csv not found!"
continue
fi
# Search for error-lines inside the file and print them out
echo "$1_0_${i}_0.csv has over $correct_no_of_commas commas in the following lines:"
grep -o -n '[,]' "$1/$1_0_${i}_0.csv" | cut -d : -f 1 | uniq -c | awk '$1 > $correct_no_of_commas {print}'
done
}
get_bad_lines products
get_bad_lines users
The output of this program is now all the comma-counts with all of the line numbers in all the files,
and I suspect this is due to the input $1 (foldername, i.e. products & users) conflicting with the call to awk with reference to $1 as well (where I wish to grab the first column being the count of commas for that line in the current file in the loop).
Is this the issue? and if so, would it be solvable by either referencing the 1.st column or the folder name by different variable names instead of both of them using $1 ?
Example, current output:
5 6667
5 6668
5 6669
5 6670
(should only show lines for that file having more than 5 commas).
Tried variable declaration in call to awk as well, with same effect
(as in the accepted answer to Awk field variable clash with function argument)
:
get_bad_lines () {
local table_name=$1
local correct_no_of_commas=$(head -n 1 $table_name/${table_name}_0_0_0.csv | tr -cd , | wc -c)
local no_of_files=$(ls $table_name | wc -l)
for i in $(seq 0 $(( ${no_of_files}-1 )))
do
# Check that the file exist
if [ ! -f "$table_name/${table_name}_0_${i}_0.csv" ]; then
echo "File: ${table_name}_0_${i}_0.csv not found!"
continue
fi
# Search for error-lines inside the file and print them out
echo "${table_name}_0_${i}_0.csv has over $correct_no_of_commas commas in the following lines:"
grep -o -n '[,]' "$table_name/${table_name}_0_${i}_0.csv" | cut -d : -f 1 | uniq -c | awk -v table_name="$table_name" '$1 > $correct_no_of_commas {print}'
done
}
You can use awk the full way to achieve that :
get_bad_lines () {
find "$1" -maxdepth 1 -name "$1_0_*_0.csv" | while read -r my_file ; do
awk -v table_name="$1" '
NR==1 { num_comma=gsub(/,/, ""); }
/,/ { if (gsub(/,/, ",", $0) > num_comma) wrong_array[wrong++]=NR":"$0;}
END { if (wrong > 0) {
print(FILENAME" has over "num_comma" commas in the following lines:");
for (i=0;i<wrong;i++) { print(wrong_array[i]); }
}
}' "${my_file}"
done
}
For why your original awk command failed to give only lines with too many commas, that is because you are using a shell variable correct_no_of_commas inside a single quoted awk statement ('$1 > $correct_no_of_commas {print}'). Thus there no substitution by the shell, and awk read "$correct_no_of_commas" as is, and perceives it as an undefined variable. More precisely, awk look for the variable correct_no_of_commas which is undefined in the awk script so it is an empty string . awk will then execute $1 > $"" as matching condition, and as $"" is a $0 equivalent, awk will compare the count in $1 with the full input line. From a numerical point of view, the full input line has the form <tab><count><tab><num_line>, so it is 0 for awk. Thus, $1 > $correct_no_of_commas will be always true.
You can identify all the bad lines with a single awk command
awk -F, 'FNR==1{print FILENAME; headerCount=NF;} NF>headerCount{print} ENDFILE{print "#######\n"}' /path/here/*.csv
If you want the line number also to be printed, use this
awk -F, 'FNR==1{print FILENAME"\nLine#\tLine"; headerCount=NF;} NF>headerCount{print FNR"\t"$0} ENDFILE{print "#######\n"}' /path/here/*.csv
I have been busting my head all day long without coming up with a sucessfull solution.
Setup:
We have Linux RHEL 8.3 and a file, script.sh
There is an enviroment variable set by an application with a dynamic string in it.
export PROGARM_VAR="abc10,def20,ghi30"
The delimiter is always "," and the values inside vary from 1 to 20.
Inside the script I have defined 20 variables which take the values
using "cut" command I take each value and assign it to a variable
var1=$(echo $PROGARM_VAR | cut -f1 -d,)
var2=$(echo $PROGARM_VAR | cut -f2 -d,)
var3=$(echo $PROGARM_VAR | cut -f3 -d,)
var4=$(echo $PROGARM_VAR | cut -f4 -d,)
etc
In our case we will have:
var1="abc10" var2="def20" var3="ghi30" and var4="" which is empty
The loop must take each variable, test if its not empty and execute 10 pages of code using the tested variable. When it reaches an empty variable it should break.
Could you give me a hand please?
Thank you
Just split it with a comma. There are endless possibilities. You could:
10_pages_of_code() { echo "$1"; }
IFS=, read -a -r vars <<<"abc10,def20,ghi30"
for i in "${vars[#]}"; do 10_pages_of_code "$i"; done
or:
printf "%s" "abc10,def20,ghi30" | xargs -n1 -d, bash -c 'echo 10_pages_of_code "$1"' _
A safer code could use readarray instead of read to properly handle newlines in values, but I doubt that matters for you:
IFS= readarray -d , -t vars < <(printf "%s" "abc10,def20,ghi30")
You could also read in a stream up:
while IFS= read -r -d, var || [[ -n "$var" ]]; do
10_pages_of_code "$var"
done < <(printf "%s" "abc10,def20,ghi30")
But still you could do it with cut... just actually write a loop and use an iterator.
i=0
while var=$(printf "%s\n" "$PROGARM_VAR" | cut -f"$i" -d,) && [[ -n "$var" ]]; do
10_pages_of_code "$var"
((i++))
done
or
echo "$PROGRAM_VAR" | tr , \\n | while read var; do
: something with $var
done
I have a parameter file (param.env) having the following content.
MY_PARAM=com:27}WMV\)pviZN
also, a bash file where I am fetching the value of MY_PARAM and writing into a random file.
#!/bin/bash
value=$(grep "^MY_PARAM=" param.env | cut -d '=' -f2-)
value1=$(cat param.env | grep "^MY_PARAM" | sed 's/=/ /' | awk '{print $2}')
echo $value
echo $value1
printf '%s\n' "$value"
Output:
com:27}WMV\)pviZN
com:27}WMV\)pviZN
com:27}WMV\)pviZN
However, I am expecting \ to be escaped and should not be part of the output.
I am also not allowed to edit the param.env.
Expected output:
com:27}WMV)pviZN
You could source the file, then the string will behave as if you'd assigned it like that in an interactive shell:
$ (. param.env; echo "$MY_PARAM")
com:27}WMV)pviZN
I've put the commands in a subshell so they don't pollute the environment.
I want to add some users who are in this file like:
a b
c d
e f
firstname lastname always
#!/bin/bash
Lines=$(cat newusers.txt | wc -l)
first=$(cat newusers.txt | awk '{print $1}')
last=$(cat newusers.txt | awk '{print $2}')
#test
echo $Lines;
echo $first;
echo $last;
until [ -z $1]; then
useradd - m -d /home/$1 -c "$1 + $2" $1
fi
before loop it works fine but I can't add newline.
The echo shows a c e and second for lastname b d f.
I tried to add newline in but it doesn't works.
What can i use for this?
Because I guess I can't add the user because of the newline problem.
I also searched on stackoverflow to find out a way to check if the user already exists by /dev/null but which variable do i have to use for it?
It's easier to process the file line by line:
while read first last ; do
useradd -m -d /home/"$first" -c "$fist + $last" "$first"
done < newusers.txt
I do not understand what you mean to do by your code, but if you want to read the file line by line and get the values of different fields then you can use the following code snippet:
#!/bin/bash
filename="newusers.txt"
while read -r line
do
fn=$( echo "$line" |cut -d" " -f1 )
ln=$( echo "$line" |cut -d" " -f2 )
echo "$fn $ln"
done < "$filename"
Note: You cannot add users the way you want to using bash script; since you will be prompted for password which must be supplied using tty you can use expect to program it; or use system calls.
The same way it’s possible to write a file that autoextracts itself, I’m looking for a way to autorun a program within a script (or whatever it needs). I want the program part of the script, because I just want one file. It’s actually a challenge: I have a xz compressed program, and I wanna be able to run it without any intervention of the xz program by the user (just a ./theprogram).
Any idea?
Autorun after doing what? Login? Call it in ~/.bashrc. During boot? Write an appropriate /etc/init.d/yourprog and link it to the desired runlevel. Selfextract? Make it a shell archive (shar file). See the shar utility, http://linux.die.net/man/1/shar
Sorry but I was just thinking... Something like this would not work?
(I am assuming it is a script...)
#!/bin/bash
cat << 'EOF' > yourfile
yourscript
EOF
chmod +x yourfile
./yourfile
Still, it's pretty hard to understand exactly what you are trying to do... it seems to me that the "autorun" is pretty similar to a "call the program from within the script"..
I had written a script for this. This should help:
#!/bin/bash
set -e
payload=$(cat $0 | grep --binary-files=text -n ^PAYLOAD: | cut -d: -f1 )
filaname=`head $0 -n $payload | tail -n 1 | cut -d: -f2-`
tail -n +$(( $payload + 1 )) $0 > /tmp/$filaname
set +e
#Do whatever with the payload
exit 0
#Command to add payload:
#read x; ls $x && ( cp 'binary_script.sh' ${x}_binary_script.sh; echo $x >> ${x}_binary_script.sh; cat $x >> ${x}_binary_script.sh )
#Note: Strictly NO any character after "PAYLOAD:", not even newline...
PAYLOAD:
Sample usage:
Suppose myNestedScript.sh contains below data:
#!/bin/bash
echo hello world
Then run
x=myNestedScript.sh; ls $x && ( cp 'binary_script.sh' ${x}_binary_script.sh; echo $x >> ${x}_binary_script.sh; cat $x >> ${x}_binary_script.sh )
It will generate below file, which you can directly execute. Upon executing below file, it will extract myNestedScript.sh to /tmp & run that script.
#!/bin/bash
set -e
payload=$(cat $0 | grep --binary-files=text -n ^PAYLOAD: | cut -d: -f1 )
filaname=`head $0 -n $payload | tail -n 1 | cut -d: -f2-`
tail -n +$(( $payload + 1 )) $0 > /tmp/$filaname
set +e
chmod 755 /tmp/$filaname
/tmp/$filaname
exit 0
PAYLOAD:myNestedScript.sh
#!/bin/bash
echo hello world