String Concatenation error in Cygwin bash shell - cygwin

I am having problems concatenate two strings in BASH (I am using Cygwin)
When I am doing it step by step in the cygwin window, it works.
i.e by defining dt=2012-12-31 and c=.txt explicitly and then concatenating in filename=${dt}${c}.
It doesn't seem to work when i am running it through my script where these variables are defined by cutting and assigning values from content of a file.
Though the variables are assigned with the same values as above, the concatenation in this case doesn't work.
instead of 2012-12-31.txt i am getting .txt-12-31 as result.
The code is:
for x in {0..11}
do
IFS=$'\n'
filename=date_list.txt
file=($(<"$filename"))
IFS=$'\t\n'
dt=${file[$x]}
echo $dt
for y in {0..85}
do
IFS=$'\n'
filename=SQL_Mnemonics.txt
file=($(<"$filename"))
IFS=$'\t\n'
Mn=${file[$y]}
for k in {3..502}
do
IFS=$'\n'
c=.txt
filename=${dt}${c}
file=($(<"$filename"))
IFS=$'\t\n'
echo ${file[$k]} > temp_file.txt
cusip=`cut -c11-19 temp_file.txt`
result=$(sh ctest.sh $Mn, $dt, $cusip)
echo "$result" > tmp1.txt
t1=`cut -c18-40 tmp1.txt`
echo $t1 | sed 's/[[:space:]]//g' > temp_file.txt
cat tst.txt | sed 's/-----//g' >> ForFame/${Mn}.${dt}.txt
done
done
done

Related

A strange error in shell script while combining two string variables

I will post my script here
#!/bin/tcsh
echo 'Running'
set fileN = '2021-02-07-0448-04S.JKH_RR.SAC'
set fileE = '2021-02-07-0448-04S.JKH_RR_BHE.SAC'
set compR=BHR
set compT=BHT
set compR_name=BHR.SAC
set compT_name=BHT.SAC
set fileN_rot = `echo $fileN | awk '{split($0,a,".SAC"); print a[1]}'`
set fileE_rot = `echo $fileE | awk '{split($0,a,".SAC"); print a[1]}'`
echo 'output1'
echo $fileN
echo $fileE
echo 'output2'
echo $fileN_rot
echo $fileE_rot
echo 'output3'
echo $fileE_rot-$compR_name
echo $fileN_rot-$compT_name
The output is:
Running
output1
2021-02-07-0448-04S.JKH_RR_BHN.SAC 2021-02-07-0448-04S.JKH_RR_BHE.SAC
output2
2021-02-07-0448-04S.JKH_RR_BHN
2021-02-07-0448-04S.JKH_RR_BHE
output3
2021-02-07-0448-04S.JKH_RR_BHN
-BHR.SAC
2021-02-07-0448-04S.JKH_RR_BHE-BHT.SAC
echo $fileE_rot-$compR_name giving wrong output.
Here the out is copy-pasted from the output file,so -BHR.SAC showing in new line.
But in shell terminal it is showing -BHR.SAC07-0448-04S.JKH_RR_BHN.
I find it strange.
Looks like you have some control chars in your strings. Run cat -Ev script to see them and if you see ^Ms in the output then read Why does my tool output overwrite itself and how do I fix it? for how to deal with them.
Don't write scripts in [t]csh, though, as it wasn't designed for that. Writing a script in csh is like digging a hole with a toothbrush - sure you CAN kinda get there in the end but there are better alternatives. See https://www.google.com/search?q=google+csh+why+not.
Having said that, it's not obvious why you're trying to manipulate text in any shell. Shells exist to manipulate (create/destroy) files and processes and sequence calls to tools. The people who invented shell also invented tools such as awk for shell to call when appropriate to manipulate text. So, here is how to really write a shell script to do what you want (the shell part is to call awk to manipulate the text):
$ cat tst.sh
#!/usr/bin/env bash
awk '
BEGIN {
print "Running"
fileN = "2021-02-07-0448-04S.JKH_RR.SAC"
fileE = "2021-02-07-0448-04S.JKH_RR_BHE.SAC"
compR = "BHR"
compT = "BHT"
compR_name = "BHR.SAC"
compT_name = "BHT.SAC"
fileN_rot = fileN
sub(/\.SAC$/,"",fileN_rot)
fileE_rot = fileE
sub(/\.SAC$/,"",fileE_rot)
print "output1"
print fileN
print fileE
print "output2"
print fileN_rot
print fileE_rot
print "output3"
print fileE_rot "-" compR_name
print fileN_rot "-" compT_name
}
'
$ ./tst.sh
Running
output1
2021-02-07-0448-04S.JKH_RR.SAC
2021-02-07-0448-04S.JKH_RR_BHE.SAC
output2
2021-02-07-0448-04S.JKH_RR
2021-02-07-0448-04S.JKH_RR_BHE
output3
2021-02-07-0448-04S.JKH_RR_BHE-BHR.SAC
2021-02-07-0448-04S.JKH_RR-BHT.SAC
or if there really was some reason to want to do it directly in a shell (e.g. this code is in some loop manipulating files named based on these variables) then:
$ cat tst.sh
#!/usr/bin/env bash
fileN='2021-02-07-0448-04S.JKH_RR.SAC'
fileE='2021-02-07-0448-04S.JKH_RR_BHE.SAC'
compR='BHR'
compT='BHT'
compR_name='BHR.SAC'
compT_name='BHT.SAC'
fileN_rot="${fileN%*.SAC}"
fileE_rot="${fileE%*.SAC}"
echo 'output1'
echo "$fileN"
echo "$fileE"
echo 'output2'
echo "$fileN_rot"
echo "$fileE_rot"
echo 'output3'
echo "${fileE_rot}-${compR_name}"
echo "${fileN_rot}-${compT_name}"
$ ./tst.sh
output1
2021-02-07-0448-04S.JKH_RR.SAC
2021-02-07-0448-04S.JKH_RR_BHE.SAC
output2
2021-02-07-0448-04S.JKH_RR
2021-02-07-0448-04S.JKH_RR_BHE
output3
2021-02-07-0448-04S.JKH_RR_BHE-BHR.SAC
2021-02-07-0448-04S.JKH_RR-BHT.SAC

Bash IFS, getting zapped in function call

Platform CentOS Linux release 7.6.1810, working in bash.
GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)
This is an idiom I've seen recommended for parsing text in bash in general and in particular for returning multiple values from a function.
IFS=":" read A B <<< $(echo ONE:TWO)
I'm getting unexpected behaviour when I call a function, yyy in the example here
IFS=":" read Y1 Y2 <<< $(yyy)
where yyy itself also wants to do a similar call.
The effect is that that within yyy() even though I explicitly specify the IFS
IFS=":" read C1 C2 <<< $( echo "A:B" )
The fields are parsed, but both values are assigned to C1, it gets the value "A B". If the function is called in isolation it works as expected.
This is a test case, distilled down from a much larger script. I want to know what is happening with IFS here. In the failure case (the second example below) setting IFS=":" in the caller somehow cause the result fields to be aggregated. The first and third calls to yyy() below work as expected, output shown after the code.
#!/bin/bash
debug() { echo "$1" 1>&2 ; }
yyy() {
debug "in yyy"
# why are the two values assigned to A here if the caller specified IFS?
IFS=":" read A B <<< $(echo ONE:TWO)
debug "A=$A"
debug "B=$B"
echo "$A:$B"
}
# this works as expected
read Y1 Y2 <<< $(yyy)
echo -e "===\n"
# this cause the read in yyy() to aggregate
IFS=":" read Y1 Y2 <<< $(yyy)
echo -e "===\n"
# This is a workaround that enables yyy() to work correctly
# But why do I need to do this?
OUT="$(yyy)"
IFS=":" read Y1 Y2 <<< $(echo $OUT)
This is the output
in yyy
A=ONE B=TWO
===
in yyy
A=ONE TWO B=
===
in yyy
A=ONE B=TWO
Note that in the second case A gets the value ONE TWO
This seems to be a bug in bash-4.2 as discussed here, IFS incorrectly splitting herestrings in bash 4.2. Should work on the versions above that.
These are the results on the same version as you have - GNU bash, version 4.2.46(2). When I ran the function yyy in debug mode ( by setting set -x in prompt ).
++ IFS=:
++ read A B
+++ echo ONE:TWO
++ debug 'A=ONE TWO'
++ echo 'A=ONE TWO'
A=ONE TWO
++ debug B=
++ echo B=
B=
++ echo 'ONE TWO:'
The above is snippet of the output from the debug mode output. As you can see when the echo ONE:TWO is printed as a result of the command substitution, no word splitting is expected to happen because the line doesn't contain any character of the default IFS value (space/tab or a newline)
So you would expect reading the the whole string with IFS=: expected to split the string and put the values in the constituent variables A and B, but somehow the : character is lost and a string ONE TWO is stored as the first variable value.
Look at the output of the function execution in GNU bash, version 4.4.12(1) which exhibits the right behavior.
++ IFS=:
++ read A B
+++ echo ONE:TWO
++ debug A=ONE
++ echo A=ONE
A=ONE
++ debug B=TWO
++ echo B=TWO
B=TWO
++ echo ONE:TWO
There have been lot of IFS related bugs up to version 4.4.0 bash/CHANGES. So a personal recommendation is to upgrade your bash version to a more recent stable one. Also see Trying to split a string into two variables
Similar bug on version 4.4.0(1)-release
You would expect the ONE:TWO to be unmodified when the $(..) is expanded because for reasons mentioned earlier. But here too the delimit character is lost and the variable A is set to ONE TWO
IFS=":" read A B <<< $(echo ONE:TWO)
echo "$A"
ONE TWO
Surprisingly the above code works on 4.2.46(2), which means the 4.4.0(1) broke a functionality which used to work in the earlier releases.

Shell Script: value too great for base (error token is "16#?")

I am new to shell script. I am working with Hex values and writing a simple script for substraction. Here is my script:
#!/bin/bash
var1=“0x0001”
var2=“0x0005”
var3=“$(( 16#$var2 - 16#$var1 ))”
echo “Diference $var3”
I am getting this error :
line 6: 16#?: value too great for base (error token is "16#?")
Could you please let me know where my mistake is?
$ var1=0x0001
$ var2=0x0005
$ var3=$(( $var2 - $var1 ))
$ echo "Diference $var3"
Diference 4
Assign the hex values without double quotes(i.e not as strings).
Since you have already put a 0x there is no need for 16#
To conver the answer back to hex you can use:
printf '%x' $num
Here is an example:
$ var1=0x19
$ var2=0xA
$ var3=$(( $var1 - $var2 ))
$ echo $var3
15
$ printf '%x\n' $var3
f
$ var3=$(printf '%x' $var3)
$ echo $var3
f
16# and 0x are redundant, and mutually exclusive. The problem is that, due to the 16#, Bash thinks the x is trying to be a digit in a base-16 number (whereas it's only valid in base 34 or higher). Just drop either the 16# or the 0x, and it'll work.

Printing IFS values from file

Script:
#!/bin/ksh
FILENAME=$1
while read RECORD VALUE
do
echo ${RECORD} ${VALUE} "X"
done <"$FILENAME"
input file:
A 1
B 2
The output of script:
X1
X2
If I remove from echo "x", e.g.
echo ${RECORD} ${VALUE}
I am getting
A 1
B 2
what is wrong?
Update:
If I do
echo "X" ${RECORD} ${VALUE}
it prints correctly:
X A 1
X B 2
and :
echo ${RECORD} "X"
also prints correctly, so i am guessing the issues is with VALUE that maybe contains return carriage symbol (as input file was created on windows)
adding this inside the loop:
VALUE=`echo $VALUE| tr -d '\r'`
solved the issue, if you have a better solution you are more than welcome.
There is a parameter expansion operator you can use to remove a character from the end of a value, if it is present.
VALUE=${VALUE%$'\r'}
This is handled in-shell, without needing to start a new process.

How to take variable from printf output and define as value of two other variables

I have a .txt input file that is the product of a printf defining each line as POV(n)="sequenceX,yearY"
cat output.PA
POV01="SEQ010,FY15"
POV02="SEQ010,FY16"
POV03="SEQ020,FY15"
POV04="SEQ020,FY16"
How can I source this file so that I can export each POV as the variable value of sequence and fy, respectively for the given line?
export POV(n)="$seq,$fy"
the printf I have used to get tho this point is as follows:
cat step1
while read -r seq fy; do
printf 'POV%02d="%s,%s"\n' ${counter} ${seq} ${fy}
(( counter = counter + 1 ))
done <test_scenario_02.txt > output.PA
If I source output.PA I get the following:
./step2
POV00=YEAR,
POV01=SEQ010,FY15
POV02=SEQ010,FY16
POV03=SEQ020,FY15
POV04=SEQ020,FY16
POV05=SEQ030,FY15
POV06=SEQ030,FY16
POV07=SEQ030,FY15
POV08=SEQ030,FY16
POV09=SEQ040,FY15
POV10=SEQ040,FY16
POV11=SEQ050,FY15
POV12=SEQ050,FY16
$ cat step2
. ./output.PA
set | grep "^POV"
It is not at all clear what you want, but it seems like you are trying to create an array variable that holds all the value in output.PA. You probably don't need to do that, but this should work:
$ pov=($(sed -e 's/[^"]*"//' -e 's/"$//' output.PA))
$ echo ${pov[0]}
SEQ010,FY15
$ echo ${pov[1]}
SEQ010,FY16
$ echo ${pov[2]}
SEQ020,FY15

Resources