sed command with dynamic variable and spaces - not retaining spaces - linux

I am trying to insert a variable value into file from Jenkinsfile using shell script in the section
Variable value is dynamic. I am using sed.
Sed is working fine but it is not retaining the white spaces that the variable have at the beginning.
ex:
The value of >> repoName is " somename"
stage('trying sed command') {
steps {
script {
sh """
#!/bin/bash -xel
repo='${repoName}'
echo "\$repo"
`sed -i "5i \$repo" filename`
cat ecr.tf
"""
}
}
}
current output:
names [
"xyz",
"ABC",
somename
"text"
]
Expected output:
names [
"xyz",
"ABC",
somename
"text"
]
How do i retain the spaces infront of the variable passing from sed

With
$ cat filename
names [
"xyz",
"ABC",
"text"
]
$ repo=somename
we can do:
sed -E "3s/^([[:blank:]]*).*/&\\n\\1${repo},/" filename
names [
"xyz",
"ABC",
somename,
"text"
]
That uses capturing parentheses to grab the indentation from the previous line.
if $repo might contain a value with slashes, you can tell the shell to escape them with this (eye-opening) expansion
repo='some/name'
sed -E "3s/^([[:blank:]]*).*/&\\n\\1${repo//\//\\\/},/" filename
names [
"xyz",
"ABC",
some/name,
"text"
]

I used 1 sed statement to add the content first to the file and then another sed statement for just adding spaces. This fixed my issue. All day i was trying to fit in one command did not work probably from Jenkins and shell usage. But using 2 sed commands as a workaround i was able to finish my task

The i command skips over spaces after the i command to find the text to insert. You can put the text on a new line, with a backslash before the newline, to have the initial whitespace preserved.
stage('trying sed command') {
steps {
script {
sh """
#!/bin/bash -xel
repo='${repoName}'
echo "\$repo"
`sed -i "5i \\\\\\
\$repo" filename`
cat ecr.tf
"""
}
}
}
I've tested this from a regular shell command line, I hope it will also work in the Jenkins recipe.

Related

sed command to add a line after some pattern

I have a file (content below)
name = [
"victor",
"linda",
"harris",
"sandy"
]
Now how to i add a command (shell using sed or awk) and i need below output
name = [
"NEW INPUT HERE",
"victor",
"linda",
"harris",
"sandy"
]
I have tried multiple ways but not able to achieve it. Few of them what i tried
sed '2a'$'\n'' "NEW INPUT HERE", ' filename
I am able to get add it but not giving new line after my input
Some sed are finicky, but each of the following should work:
$ sed -e '1a\
"NEW INPUT HERE",
' input-file
$ sed -e $'1a\\\n "NEW INPUT HERE",\n' input-file # Bashism
Using sed
$ sed '2{h;s/[[:alpha:]][^"]*/NEW INPUT HERE/};2G' input_file
name = [
"NEW INPUT HERE",
"victor",
"linda",
"harris",
"sandy"
]
Works for me with GNU sed:
sed '2i\ "NEW INPUT HERE",'
or
sed '1a\ "NEW INPUT HERE",'
This will just use for the new line whatever indenting you already use for the existing 2nd line:
$ awk -v new='"NEW INPUT HERE",' 'NR==2{orig=$0; sub(/[^[:space:]].*/,""); print $0 new; $0=orig} 1' file
name = [
"NEW INPUT HERE",
"victor",
"linda",
"harris",
"sandy"
]
Another option is to match the [ and the next line.
Then capture the newline and leading spaces in group 1.
In the replacement use your text surrounded by double quotes and a backreference to group 1 to keep the indenting the same.
sed -E '/\[/{N;/(\n[[:blank:]]+)/{s//\1"NEW INPUT HERE",\1/}}' file
Output
name = [
"NEW INPUT HERE",
"victor",
"linda",
"harris",
"sandy"
]
awk -v str='"NEW INPUT HERE",' '
/name/{
print;
getline; # get first element (victor)
le=length($0)-length($1); # calculate first element ident
printf "%*s%s \n%s\n", le, " ", str, $0;
next
}1' file
name = [
"NEW INPUT HERE",
"victor",
"linda",
"harris",
"sandy"
]
This might work for you (GNU sed):
sed '2{h;s/\S.*/"NEW INPUT HERE",/p;g}' file
On line 2, make a copy of the line, substitute the required string starting where existing text starts indented, print the amended line and reinstate the original line.
Another solution using a regexp for the address line:
sed '/name = \[/{n;h;s/\S.*/"NEW INPUT HERE",/p;g}' file

How do I grep and replace string in bash

I have a file which contains my json
{
"type": "xyz",
"my_version": "1.0.1.66~22hgde",
}
I want to edit the value for key my_version and everytime replace the value after third dot with another number which is stored in a variable so it will become something like 1.0.1.32~22hgde. I am using sed to replace it
sed -i "s/\"my_version\": \"1.0.1.66~22hgde\"/\"my_version\": \"1.0.1.$VAR~22hgde\"/g" test.json
This works but the issue is that my_version string doesn't remain constant and it can change and the string can be something like this 1.0.2.66 or 2.0.1.66. So how do I handle such case in bash?
how do I handle such case?
You write a regular expression to match any possible combination of characters that can be there. You can learn regex with fun with regex crosswords online.
Do not edit JSON files with sed - sed is for lines. Consider using JSON aware tools - like jq, which will handle any possible case.
A jq answer: file.json contains
{
"type": "xyz",
"my_version": "1.0.1.66~22hgde",
"object": "can't end with a comma"
}
then, replacing the last octet before the tilde:
VAR=32
jq --arg octet "$VAR" '.my_version |= sub("[0-9]+(?=~)"; $octet)' file.json
outputs
{
"type": "xyz",
"my_version": "1.0.1.32~22hgde",
"object": "can't end with a comma"
}

Adding the curly braces in begning and end of output

I have command that gives me following output:
'First' : 'abc',
'Second' :'xyz',
'Third' :'lmn'
Requirement here is to convert this output into valid json format.
So I replaced all ' to " using sed :
<command> | sed "s/'/\"/g"
"First" : "abc",
"Second" :"xyz",
"Third" :"lmn"
Now I also need to add { in the begining and end of the output how can I do that.
Any other thoughts are also welcome.
sed -z "s/[[:space:]]*'\([^']*\)'[[:space:]]*:[[:space:]]*'\([^']*\)'[[:space:]]*/"'"\1":"\2",/g; s/,$//; s/^/{/; s/$/}/'
First match the '<this>' : '<and this>'
Then convert each such sequences into "<this>":"<and this>",
Remove trailing comma.
Add { } in front of it.
-z is a GNU extension to parse it all as one line. Alternatively you could remove newlines before passing to sed.
|sed -e '1s/^/{/' -e "s/'/"/g" -e '$s/$/}/' does the work.

How to transform a string (with a fixed delimiter) into an array to be appended to a JSON?

I've some string inputted by user such as:
read -p "define module tags (example: TAG1, TAG2): " -r module_tags
if [ "$module tags" = "" ]; then module tags="TAG1, TAG2"; fi
which are tags separated by ,
Than, I need to append these tags in a JSON array field:
{
"modules": [
{
"tags": "<user-custom-tags>"
}
]
}
I would do it in this way:
args='.modules[0].tags = '$module_tags''
tmp=$(mktemp -u)
jq --indent 4 "$args" $plugin_file > "$tmp" && mv "$tmp" $plugin_file
But for this, I need to transform the input TAG1, TAG2 to [ "TAG1", "TAG2" ]
How would you do this?
for tag in $module_tags_array
This splits the value of $module_tags_array on IFS, i.e. you are looping over first TAG1, and then TAG2, not the list of tags.
What you are describing can easily be accomplished with
module_tags="[ $module_tags_array ]"
Notice also that the proper way to echo the value of the variable is with quotes:
echo "$module_tags"
unless you specifically require the shell to perform whitespace tokenization and wildcard expansion on the unquoted value. See also When to wrap quotes around a shell variable?
However, a more natural and obvious solution is to actually use an array to store the values.
tags=("TAG1" "TAG2")
printf "\"%s\", " "${tags[#]}" | sed 's/^/[/;s/, $/]/'
The printf produces a string like
"TAG1", "TAG2",
which we then massage into a JSON array expression with a quick bit of sed postprocessing.
Not using bashism, but using jq command line JSON parser:
<<< "TAG1, TAG2" jq -Rc 'split(", ")'
["TAG1","TAG2"]
-R is for raw input string
-c is for compact JSON output (on 1 line)
As expected, split function turns the input string into different part and put them into a JSON array.
Use the "Search and Replace" feature of Bash's parameter expansion:
input="TAG1, TAG2"
output='[ "'${input//, /\", \"}'" ]'
printf "%s\n" "$output"
But be aware that this is not a proper way to quote.

Use sed in groovy to replace a value in json file

I have a json file with the below content:
"containerDefinitions": [
{
"image": "***.dkr.ecr.us-east-1.amazonaws.com/xyz"
}
]
Now i want to replace the url for image with a new value. So in my jenkins scripted groovy file, i store this existing url value under some variable and then enter the new url value appended by build number. So i try to do the following:
newimageurl="\"***.dkr.ecr.us-east-1.amazonaws.com/xyz:v_$BUILD_NUMBER\""
oldimageurl="\"***.dkr.ecr.us-east-1.amazonaws.com/xyz\""
sed -i -e 's#'"$oldimageurl"'#'"$newimageurl"'#' ./myfile.json
But it ends with error both for syntax for newimageurl for the v_$BUILD_NUMBER and then for sed command.
How to resolve this?
When defining the string variable in groovy, you don't need to add the double quotes. Also, you have an error in the interpolation. You'll do:
newimageurl = "***.dkr.ecr.us-east-1.amazonaws.com/xyz:v_${BUILD_NUMBER}"
oldimageurl = "***.dkr.ecr.us-east-1.amazonaws.com/xyz"
In you sed command as well (also, you have to put it inside an sh command):
sh """
sed -i -e 's#${oldimageurl}#${newimageurl}#' ./myfile.json
"""

Resources