I am trying to convert some of my ksh93 scripts to bash in cygwin. I have found 2 things right now that give me trouble. The first is a function I put in .bashrc was not recognized in the script. I put the function in the script to get around that. The second is that it won't glob like it does in ksh93. Setting extglob didn't seem to help. Here is what I have done:
#! /bin/bash
rep() {
perl -E 'say "'"$1"'" x '$2
# seq -s"$1" $2|tr -d '[:digit:]'
}
# added these 2 lines for testing
shopt -s extglob
shopt extglob
ziptext="Monthly files for $(date --date="$(date +%Y-%m-15) -1 month" +'%B %Y')"
equals4=$(rep = $((${#ziptext} + 6)))
equals="$(rep = ${#ziptext})"
spaces="$(rep ' ' ${#ziptext})"
# added these 2 lines for testing
ls -l 20[0-9][0-9]' Monthly Data - review.xlsx'
pwd
echo "
$equals4
= $equals =
= $spaces =
= $ziptext =
= $spaces =
= $equals =
$equals4\n\n\n" | zip -9 -u -z \
20[0-9][0-9]' Monthly Data - review.xlsx' \
20[0-9][0-9]' Monthly Tables - review.xlsx'
The result is:
extglob on
ls: cannot access '20[0-9][0-9] Monthly Data - review.xlsx': No such file or directory
/cygdrive/c/reports
zip warning: 20[0-9][0-9] Monthly Data - review.xlsx not found or empty
zip warning: name not matched: 20[0-9][0-9] Monthly Tables - review.xlsx
From the shell, doing
ls 20[0-9][0-9]' Monthly Data - review.xlsx' \
20[0-9][0-9]' Monthly Tables - review.xlsx'
Results in
'2019 Monthly Data - Review.xlsx'* '2019 Monthly Tables - Review.xlsx'*
What setting am I missing out on to get this to work like it did in ksh93?
Your filename has capital R Review while your glob uses lowercase review.
Your local shell most likely has nocaseglob enabled to do case insensitive globbing. If this used to work on ksh93, it probably had a similar option enabled by default as well.
In bash you have to enable it explicitly in a script with shopt -s nocaseglob
Related
This is my very first shell script for a Unix class, this is one of the scripts I hope to submit for my final. However there are a few kinks I cannot seem to clear up, it seems to be arithmetic operation errors, and I can't seem to figure it out. Please be kind! thank you so much for your time.
lightgreen=`echo -en "\e[92m"
echo What are the values of a, b \& c?
LIGHTRED=`echo -en "\e[101m"
echo a value:
read a
echo b value:
read b
echo c value:
read c
discrim=$(($b**2 - 4*$a*$c))
sqrtd=$((sqrt($discrim) | bc ))
echo test $sqrtd
echo ${lightgreen}The discriminant is:${discrim}
#xone=$((( -$b + sqrt$discrim) / (2 * $a) | bc ))
#xtwo=$((( -$b - sqrt$discrim) / (2 * $a) | bc ))
xone=$((echo (-1*$b + sqrt($discrim)) / (2*$a) | bc ))
xtwo=$((echo (-1*$b - sqrt($discrim)) / (2*$a) | bc ))
echo ${lightgreen}The discriminant is:${discrim}
#if [$discrim -lt 0 ]
# echo $LIGHTRED There are no real solutions.
#
#
#
echo The two solutions are $xone $xtwo
I have tried to mess with the syntax a good amount, I'm not sure if it's the parentheses that mess me up or the sqrt function, I have tried to incorporate | bc but to no avail. Any help is greatly appreciated! :)
Don't hesitate to call man bash, man bc manual pages.
Use https://www.shellcheck.net/ to check your shell scripts.
Shellcheck also exists on command line and in Visual Studio Code with extension.
#! /usr/bin/env bash
# The first line is very important to now the name of the interpreter
# Always close " , ' , or ` sequences with same character
# Do not use old `...` syntax, replaced by $(...)
# Here, use $'...', to assign value with \... sequences
lightgreen=$'\e[92m'
lightred=$'\e[101m'
normal=$'\e[0m'
# It's better to put phrase between "..." or '...'
echo "What are the values of a, b & c?"
# Use read -p option to specify prompt
# Use read -r option to not act backslash as an escape character
read -p "a value: " -r a
read -p "b value: " -r b
read -p "c value: " -r c
# With bash only, it's only possible to use integer values
# discrim=$(($b**2 - 4*$a*$c))
# use bc instead
discrim=$(bc -l <<<"$b^2 - 4*$a*$c")
# The syntax:
# bc <<<"..."
# is equivalent to:
# echo "..." | bc
# but without pipe (|)
# Close the color change with normal return
echo "${lightgreen}The discriminant is: ${discrim}${normal}"
if [[ "${discrim:0:1}" == "-" ]]; then
echo "${lightred}There are no real solutions${normal}"
# ... complex ...
else
sqrtd=$(bc -l <<<"sqrt($discrim)")
echo "sqrt($discrim)=$sqrtd"
xone=$(bc -l <<<"(-1*$b + $sqrtd) / (2*$a)")
xtwo=$(bc -l <<<"(-1*$b - $sqrtd) / (2*$a)")
echo "The two solutions are: $xone and $xtwo"
fi
I'm writing a script to send SQL output in mail, but it is not executing successfully and is not generating the output I want.
The query generates two columns with multiple rows. How can I generate the output in table format as below?
Below is my code:
#!/bin/bash
ORACLE_HOME= **PATH
export ORACLE_HOME
PATH=$PATH:$ORACLE_HOME/bin
export PATH
TNS_ADMIN= ** PATH
export TNS_ADMIN
today=$(date +%d-%m-%Y)
output=$(sqlplus -S user/pass#service <<EOF
set heading off;
SELECT distinct list_name ,max(captured_dttm) as Last_received FROM db.table1
group by list_name having max(captured_dttm) <= trunc(sysdate - interval '2' hour);
EOF)
if [ -z "$output" ];
then
echo"its fine"
exit
else
echo "
Dear All,
Kindly check we've not received the list for last 2 hour : $output
Regards,
Team" | mailx -S smtp=XX.XX.X.XX:XX -s "URGENT! Please check list FOR $today" user#abc.com
fi
When using a here document, the closing string can't be followed by anything but a newline. Move the closing parenthesis to the next line:
output=$(sqlplus -S user/pass#service <<EOF
...
EOF
)
I'm trying to load the data from linux file(which contains duplicates,and the data is unloaded from source table) to a table.
mylinux file properties:
$ file -bi myfile_I.out
application/octet-stream; charset=binary
before loading the data to a table.I should delete the duplicates from the linux file.
My approach to delete the duplicates:
Unloaded the data from source table to temp file (TempeEX.out)
from TempEX.out file performed sort -u function and deleted the
duplicates and the final unique data records loading to myfile_I.out
Finally load the myfile_I.out data to a target_table
I am facing the issue in STEP 2 {Unable to delete the complete duplicates from TempEX.out file}
#------------------------------------------------------------------#
#- Delete the duplicates from TempEX.out write the unique data-----#
#------------------to myfile_I.out----------------------------------#
echo -e "Eliminate the duplicates from the ${FILE_PATH}/TempEX.out
file" >> ${LOG}
sort -u ${FILE_PATH}/TempEX.out > ${DEST_PATH}/myfile_I.out
echo -e "Unique records successfully written into
${DEST_PATH}/myfile_I.out" >> ${LOG}
count=0
while read
do
((count=$count+1))
done <${DEST_PATH}/myfile_I.out
echo -e "Total No of unique records in ${DEST_PATH}/myfile_I.out:"
${count} "\n" >> $LOG
#-----------------------------------------------------------------#
Actual Results:
Counts:
$wc -l TempEX.out myfile_I.out
196466 TempEX.out -->#File Contains duplicate records#
196460 myfile_I.out-->#Unique records after my approach(sort -u)#
392926 total
I did some sort functions to know the duplicates present in myfile_I.out
Duplicate record count in TempEX.out file
$ cut -d'^X' -f1,6,10 TempEX.out|sort|uniq -d|wc -l
5
Duplicate record count in myfile_I.out file
$ cut -d'^X' -f1,6,10 myfile_I.out|sort|uniq -d|wc -l
1
Got which records(on primary_key) having duplicates in TempEX.out file
$ cut -d'^X' -f1,6,10 TempEX.out|sort|uniq -d|cat
701234567 412345678 19
701234568 412345677 18
709875641 412345859 17
701234569 425984031 21
701234570 409845216 20
Got which records(on primary_key) having duplicates in myfile_I.out file
$ cut -d'^X' -f1,6,10 myfile_I.out|sort|uniq -d|cat
709875641 412345859 17
Expected Results:
To eliminate the duplicates from TempEX.out file an load the unique data to myfile_I.out.
sort -u TempEX.out > myfile_I.out /*cant resolving the issue*/
Can we do something like this?(perform up on primary keys)
sort -u -f1,6,10 TempEX.out > myfile_I.out
Here is a little script that might help. It won't modify the original file with the new data, but create a new file to load (I always prefer to keep the original in case of errors). It's doing its verification on the primary key, but will ensure that in case of a duplicate primary key that the other columns are also the same. The rational being that, even if you don't mention it, there could be modification of existent data or errors from the input system. Anyways, the script will send those lines in a different file for the user review.
It's written in the comments, but just to be sure, no field in any column should have a value with blank space.
#!/bin/ksh
TIMESTAMP=$(date +"%Y%m%d%H%M")
#No sense to do anything if the files are not readable.
if [[ ! -r $1 || ! -r $2 ]]; then
print "ERROR - You must provide 2 parameters : 1 = path/filename of DB content 2 = path/filename of New Data"
exit
fi
#Declaring 2 associative matrix
typeset -A TableDB
typeset -A DataToAdd
#Opening the different files. 3 and 4 for reading and 5 and 6 for writting.
#File handlers :
# 3 for the data from the DB,
# 4 for the new data to add,
# 5 to write the new data to load (unique and new),
# 6 to write the data in problem (same primary key but with different values)
exec 3<$1
exec 4<$2
exec 5>Data2Load_${TIMESTAMP}.txt
exec 6>Data2Verify_${TIMESTAMP}.txt
#Loading the 2 matrix with their data.
#Here it is assumed that no field in any column contain blank spaces.
#Working with only 3 columns as in the example
while read -u3 a b c && read -u4 d e f; do
TableDB[$a]=( $a $b $c )
DataToAdd[$d]=( $d $e $f )
done
#Checking for duplicate and writting only the new one to load without the lines in possible errors
for i in ${!DataToAdd[#]}; do
if [[ -z ${TableDB[$i]} ]]; then
print -u5 "${DataToAdd[$i][0]} ${DataToAdd[$i][1]} ${DataToAdd[$i][2]}"
elif [[ ${DataToAdd[$i][1]} != ${TableDB[$i][1]} || ${DataToAdd[$i][2]} != ${TableDB[$i][2]} ]]; then
print -u6 "${DataToAdd[$i][0]} ${DataToAdd[$i][1]} ${DataToAdd[$i][2]}"
fi
done
#closing the different files
exec 3>&-
exec 4>&-
exec 5>&-
exec 6>&-
Hope it helps !
I was changed directory name.
In this directory thousands of files.
Some projects use this files, projects have got symlinks on it.
How to find all symlinks, which have got folder name in their address?
how to change all this symlinks to another path in automatic mode?
if 2 only bash scripting with deleting and creating new - i will do it, but may be you know more easy way?
It's a bit complicated, but it can be done with find, readlink, a check to test whether the symlink is relative or not, and sed to get rid of .. in path names (copied 1:1 from this answer).
(Note that most convenient methods (such as readlink -f) are not available due to the symlinks targets not existing anymore.)
Assuming your old path is /var/lib/old/path:
oldpath='/var/lib/old/path';
find / -type l -execdir bash -c 'p="$(readlink "{}")"; if [ "${p:0:1}" != "/" ]; then p="$(echo "$(pwd)/$p" | sed -e "s|/\./|/|g" -e ":a" -e "s|/[^/]*/\.\./|/|" -e "t a")"; fi; if [ "${p:0:'${#oldpath}'}" == "'"$oldpath"'" ]; then ...; fi;' \;
Now replace the ... from above with ln -sf (-f to override the existing link).
Assuming your new path is /usr/local/my/awesome/new/path:
oldpath='/var/lib/old/path';
newpath='/usr/local/my/awesome/new/path';
find / -type l -execdir bash -c 'p="$(readlink "{}")"; if [ "${p:0:1}" != "/" ]; then p="$(echo "$(pwd)/$p" | sed -e "s|/\./|/|g" -e ":a" -e "s|/[^/]*/\.\./|/|" -e "t a")"; fi; if [ "${p:0:'${#oldpath}'}" == "'"$oldpath"'" ]; then ln -sf "'"$newpath"'${p:'${#oldpath}'}" "{}"; fi;' \;
Note that oldpath and newpath have to be absolute paths.
Also note that this will convert all relative symlinks to absolute ones.
It would be possible to keep them relative, but only with a lot of effort.
Breaking it down
For those of you who care what that one-line-inferno actually means:
find - a cool executable
/ - where to search, in this case the system root
-type l - match symbolic links
-execdir - for every match run the following command in the directory of the matched file:
bash - well, bash
-c - execute the following string (leading and trailing ' removed):
p="$(readlink "{}")"; - starting with the most inner:
" - start a string to make sure no expansion happens
{} - placeholder for the matched file's name (feature of -execdir)
" - end the string
readlink ... - find out where the symlink points to
p="$(...)" - and store the result in $p
if [ "${p:0:1}" != "/" ]; then - if the first character of $p is / (i.e. the symlink is absolute), then...
p="$(echo "$(pwd)/$p" | sed -e "s|/\./|/|g" -e ":a" -e "s|/[^/]*/\.\./|/|" -e "t a")"; - convert the path to an absolute one:
$(pwd) - the current directory (where the matched file lies, because we're using -execdir)
/$p - append a slash and the target of the symlink to the path of the working directory
echo "$(pwd)/$p" | - pipe the above to the next command
sed ... - resolve all ..'s, see here
p="$(...)" and store the result back into $p.
fi; - end if
if [ "${p:0:'${#oldpath}'}" == "'"$oldpath"'" ]; - if $p starts with $oldpath
${p:0:'${#oldpath}'} - substring of $p, starting at position 0, with length of $oldpath:
${#oldpath} - length of variable $oldpath
'...' - required because we're inside a '-quoted string
then - then...
ln -sf - link symbolically and override existing file, with arguments:
"'"$newpath"'${p:'${#oldpath}'}" - replace the $oldpath part of $p with $newpath (actually remove as many characters from $p as $oldpath long is, and prepend $newpath to it):
" - start a string
' - end the '-string argument to bash -c
" - append a "-string to it (in which variable expansion happens), containing:
$newpath - the value of $newpath
" - end the "-string argument to bash -c
' - append a '-string to it, containing:
${p: - a substring of p, starting at:
' - end the argument to bash -c
${#oldpath} - append the length of $oldpath to it
' - append another '-string to it
} - end substring
" - end string
"{}" - the link file, whose path stays the same
fi; - end if
\; - delimiter for -execdir
I have many data files in this format:
-1597.5421
-1909.6982
-1991.8743
-2033.5744
But I would like to merge them all into one data file with each original data file taking up one row with spaces in between so I can import it in excel.
-1597.5421 -1909.6982 -1991.8743 -2033.5744
-1789.3324 -1234.5678 -9876.5433 -9999.4321
And so on. Each file is named ALL.ene and every directory in my working directory contains it. Can someone give me a quick fix? Thanks!
:edit. Each file has 11 entries. Those were just examples.
for i in */ALL.ene
do
echo $(<$i)
done > result.txt
Assumptions:
I assume all your data files are of this format:
<something1><newline>
<something2><newline>
<something3><newline>
So for example, if the last newline is missing, the following script will miss the field corresponding to <something3>.
Usage: ./merge.bash -o <output file> <input file list or glob>
The script appends to any existing output files from previous runs. It also does not make any assumptions to how many fields of data every input file has. It blindly puts every line into a line in the output file separated by spaces.
#!/bin/bash
# set -o xtrace # uncomment to debug
declare output
[[ $1 =~ -o$ ]] && output="$2" && shift 2 || { \
echo "The first argument should always be -o <output>";
exit -1; }
declare -a files=("${#}") row
for file in "${files[#]}";
do
while read data; do
row+=("$data")
done < "$file"
echo "${row[#]}" >> "$output"
row=()
done
Example:
$ cat data1
-1597.5421
-1909.6982
-1991.8743
-2033.5744
$ cat data2
-1789.3324
-1234.5678
-9876.5433
-9999.4321
$ ./merge.bash -o test data{1,2}
$ cat test
-1597.5421 -1909.6982 -1991.8743 -2033.5744
-1789.3324 -1234.5678 -9876.5433 -9999.4321
This is what coreutils paste is good at, try:
paste -s data_files*