I'm trying to use bash string operators on a constant string. For instance, you can do the following on variable $foo:
$ foo=a:b:c; echo ${foo##*:}
c
Now, if the "a:b:c" string is constant, I would like to have a more concise solution like:
echo ${"a:b:c"##*:}
However, this is not valid bash syntax. Is there any way to perform this?
[The reason I need to do this (rather than hardcoding the result of the substitution, ie. "c" here) is because I have a command template where a "%h" placeholder is replaced by something before running the command; the result of the substitution is seen as a constant by bash.]
That's not possible using parameter expansion.
You could use other commands for this like sed/awk/expr.
but I don't see the requirement for this.
You could just do:
tmp=%h
echo ${tmp##*:}
Or if speed is not an issue, and you don't want to clutter the current environment with unneeded variables:
(tmp=%h; echo ${tmp##*:})
Anyway, you'd be better off using the command template to do the string manipulation or using something simple like cut:
# get third filed delimited by :
$ cut -d: -f3<<<'a:b:c'
c
Or more sophisticated like awk or sed:
#get last field separated by ':'
$ awk -F: '{print $NF}'<<<'a:b:c'
c
$ sed 's/.*:\([^:]*\)/\1/'<<<'a:b:c'
c
Depends on what you need.
You could use expr to get a similar result:
$ expr match "a:b:c" '.*:\(.*\)'
c
You may be able to use Bash regex matching:
pattern='.*:([^:]+)$'
[[ "a:b:c" =~ $pattern ]]
echo "${BASH_REMATCH[1]}"
But why can't you do your template substitution into a variable assignment, then use the variable in the parameter expansion?
Related
I could not get the regex to convert only the key from a key value pair from camel case to underscore sting.
The expressions like sed -E 's/\B[A-Z]/_\U&/g' converts the full value, but I would like to limit the conversion only to the key here.
$ echo UserPoolId="eu-west-1_6K6Q2bT9c" | sed -E 's/\B[A-Z]/_\U&/g'
User_Pool_Id=eu-west-1_6_K6_Q2b_T9c
but i would like to get User_Pool_Id=eu-west-1_6K6Q2bT9c
With GNU awk for the 3rd arg to match() and gensub():
$ echo 'UserPoolId="eu-west-1_6K6Q2bT9c"' |
awk 'match($0,/([^=]+=)"(.*)"/,a) { $0=gensub(/([[:lower:]])([[:upper:]])/,"\\1_\\2","g",a[1]) a[2]} 1'
User_Pool_Id=eu-west-1_6K6Q2bT9c
I don't know if it's what you'd want for this case or not but anyway:
$ echo 'UserPoolID="eu-west-1_6K6Q2bT9c"' |
awk 'match($0,/([^=]+=)"(.*)"/,a) { $0=gensub(/([[:lower:]])([[:upper:]])/,"\\1_\\2","g",a[1]) a[2]} 1'
User_Pool_ID=eu-west-1_6K6Q2bT9c
Note that ID remains as _ID and isn't converted to _I_D.
If you have only one = sign and you want to modify the camel case before the = sign, with GNU sed you can iterate until all substitutions are done:
echo UserPoolId="eu-west-1_6K6Q2bT9c" | sed -E ':a;s/([a-z])([A-Z].*=.*)/\1_\2/;ta'
User_Pool_Id=eu-west-1_6K6Q2bT9c
:a sets label a, ta branches to label a if the previous s command substituted something. The s command in the loop inserts a _ between a lower case and an upper case before the equal sign.
In your example this will first insert a _ between User and Pool, and then between Pool and Id.
Doing this in sed is somewhat challenging because you need a more complex regex and a more complex script. Perhaps a better solution would be to use the shell's substitution facilities to isolate the part you want to operate on.
string='UserPoolId="eu-west-1_6K6Q2bT9c"'
prefix=${string%%=*}
suffix=${string#"$prefix"}
sed -E -e 's/\B[A-Z]/_\U&/g' -e "s/\$/$suffix/" <<<"$prefix"
Bash also has built-in parameter expansion to convert the first character of a string to upper case, but perhaps this is sufficient to solve your immediate problem.
This might work for you (GNU sed):
sed 's/=/&\n/;h;s/\B[[:upper:]]/_&/g;G;s/\n.*\n//' file
Introduce a newline after the = and copy the result to the hold space.
Insert underscores in the required places.
Append the copy to the current line and remove the middle, leaving the answer.
I'm new to sed and have been trying to use it with no luck yet in this case.
I'm reading through a log file and I store the prior line into a variable so that I can extract out the date if needed.
variable string example:
jcl/jclnt.log-[05/06/20 16:42:52.964]:jclnt ST:
I'm only wanting the date and timestamp in the square brackets. I want to ignore the characters before and after. The date and time format are always the same length and format. I can match on it with a regex, just not sure how to extract it from a variable into a new variable with only the data inside the square brackets.
I tried something like this:
priordate= echo "$prior" | awk -F'[][]' '{print $2}'
But that didn't work.
It should work if you remove the space before your echo.
echo "jcl/jclnt.log-[05/06/20 16:42:52.964]:jclnt ST:" | awk -F'[][]' '{print $2}'
05/06/20 16:42:52.964
and then make the statement thus: priordate=$(echo ...)
You can use Bash's native regular expression matching. This is a quick and dirty regular expression that just relies on capturing whatever is between [ and ]. You can certainly make it more specific if necessary.
#!/bin/bash
s="jcl/jclnt.log-[05/06/20 16:42:52.964]:jclnt ST:"
pattern="\[(.*)\]"
if [[ "${s}" =~ $pattern ]]
then
date_time="${BASH_REMATCH[1]}"
fi
echo "${date_time}"
Output:
05/06/20 16:42:52.964
I have a variable in a linux bash ".sh" script
$data="test_1"
now I want to create a new variable ($name) that contains only the part of $data before the underscore, so
$name="test"
I thought of doing this with sed
name=$(echo "$dataset" | sed 's/_.*//');
but this doesn't seem to work. What am I doing wrong?
No need to call an external process(sed). Instead you can use shell's parameter substitution like this:
$ data="test_1"
$ echo "${data%%_*}"
test
${var%%Pattern} Remove from $var the longest part of Pattern that matches the back end(from the right) of $var.
${var%Pattern} for removing shortest pattern
More info on parameter substitution can be found here.
You can store it in a variable like this:
$ name="${data%%_*}"
$ echo "$name"
test
Lets say i have a nested for loop:
for i in $test
do
name=something
for j in $test2
do
name2=something
jj=$j | sed s/'tRap\/tRapTrain'/'BEEML\/BEEMLTrain'/g
if [ name == name2 ]
then
qsub scrip.sh $i $j $jj
fi
done
done
Now the problem occurs when i try to rename the variable $j into variable $jj. I only get empty values back for submitting the script within the if statement. Is there another way to rename variables so that i can pass them through to the if part of the code?
PS. i tried 3 for loops but this makes the script awfully slow.
Your problem is piping the assignment into sed. Try something like
jj=$(echo $j | sed s/'tRap\/tRapTrain'/'BEEML\/BEEMLTrain'/g)
This uses command substitution to assign jj.
This is not correct:
jj=$j | sed s/'tRap\/tRapTrain'/'BEEML\/BEEMLTrain'/g
In order to assign the output of a command to a variable you need to use command substitution like this:
jj=$(sed s/'tRap\/tRapTrain'/'BEEML\/BEEMLTrain'/g <<< "$j")
You may not even have to use sed because bash has in-built string replacement. For example, the following will replace foo with bar in the j variable and assign it to jj:
jj=${j//foo/bar}
There is also a problem with your if-statement. It should be:
if [ "$name" == "$name2" ]
A tiny little thing:
Sed treats the first character after the action selector as the field separator.
Knowing this you can translate your expresion:
sed s/'tRap\/tRapTrain'/'BEEML\/BEEMLTrain'/g
into:
sed s%'tRap/tRapTrain'%'BEEML/BEEMLTrain'%g
So you don't have to worry about scaping your slashes when substituting paths. I normally use '%', but feel free to use any other character. I think the optimal approach would be using a non printable character:
SEP=$'\001' ; sed s${SEP}'tRap/tRapTrain'${SEP}'BEEML/BEEMLTrain'${SEP}g
I have a string variable x=tmp/variable/custom-sqr-sample/test/example
in the script, what I want to do is to replace all the “-” with the /,
after that,I should get the following string
x=tmp/variable/custom/sqr/sample/test/example
Can anyone help me?
I tried the following syntax
it didnot work
exa=tmp/variable/custom-sqr-sample/test/example
exa=$(echo $exa|sed 's/-///g')
sed basically supports any delimiter, which comes in handy when one tries to match a /, most common are |, # and #, pick one that's not in the string you need to work on.
$ echo $x
tmp/variable/custom-sqr-sample/test/example
$ sed 's#-#/#g' <<< $x
tmp/variable/custom/sqr/sample/test/example
In the commend you tried above, all you need is to escape the slash, i.e.
echo $exa | sed 's/-/\//g'
but choosing a different delimiter is nicer.
The tr tool may be a better choice than sed in this case:
x=tmp/variable/custom-sqr-sample/test/example
echo "$x" | tr -- - /
(The -- isn't strictly necessary, but keeps tr (and humans) from mistaking - for an option.)
In bash, you can use parameter substitution:
$ exa=tmp/variable/custom-sqr-sample/test/example
$ exa=${exa//-/\/}
$ echo $exa
tmp/variable/custom/sqr/sample/test/example