I have a file:
"tags": "['PNP']"
Clearly "[ is wrong, it must ot be "tags" : ['PNP']
So I wanna to replace with sed:
sed -i "1,$ s/"[/[/g" file.json
However it told me that it is not match
How can I do it?
You can do
sed 's/"\[/\[/; s/\]"/\]/' file.json
The brackets [] are special characters in basic regular expressions, so you need to escape them.
On input:
"tags": "['PNP']"
This outputs:
"tags": ['PNP']
Related
I am trying to insert a variable value into file from Jenkinsfile using shell script in the section
Variable value is dynamic. I am using sed.
Sed is working fine but it is not retaining the white spaces that the variable have at the beginning.
ex:
The value of >> repoName is " somename"
stage('trying sed command') {
steps {
script {
sh """
#!/bin/bash -xel
repo='${repoName}'
echo "\$repo"
`sed -i "5i \$repo" filename`
cat ecr.tf
"""
}
}
}
current output:
names [
"xyz",
"ABC",
somename
"text"
]
Expected output:
names [
"xyz",
"ABC",
somename
"text"
]
How do i retain the spaces infront of the variable passing from sed
With
$ cat filename
names [
"xyz",
"ABC",
"text"
]
$ repo=somename
we can do:
sed -E "3s/^([[:blank:]]*).*/&\\n\\1${repo},/" filename
names [
"xyz",
"ABC",
somename,
"text"
]
That uses capturing parentheses to grab the indentation from the previous line.
if $repo might contain a value with slashes, you can tell the shell to escape them with this (eye-opening) expansion
repo='some/name'
sed -E "3s/^([[:blank:]]*).*/&\\n\\1${repo//\//\\\/},/" filename
names [
"xyz",
"ABC",
some/name,
"text"
]
I used 1 sed statement to add the content first to the file and then another sed statement for just adding spaces. This fixed my issue. All day i was trying to fit in one command did not work probably from Jenkins and shell usage. But using 2 sed commands as a workaround i was able to finish my task
The i command skips over spaces after the i command to find the text to insert. You can put the text on a new line, with a backslash before the newline, to have the initial whitespace preserved.
stage('trying sed command') {
steps {
script {
sh """
#!/bin/bash -xel
repo='${repoName}'
echo "\$repo"
`sed -i "5i \\\\\\
\$repo" filename`
cat ecr.tf
"""
}
}
}
I've tested this from a regular shell command line, I hope it will also work in the Jenkins recipe.
I have a file which contains my json
{
"type": "xyz",
"my_version": "1.0.1.66~22hgde",
}
I want to edit the value for key my_version and everytime replace the value after third dot with another number which is stored in a variable so it will become something like 1.0.1.32~22hgde. I am using sed to replace it
sed -i "s/\"my_version\": \"1.0.1.66~22hgde\"/\"my_version\": \"1.0.1.$VAR~22hgde\"/g" test.json
This works but the issue is that my_version string doesn't remain constant and it can change and the string can be something like this 1.0.2.66 or 2.0.1.66. So how do I handle such case in bash?
how do I handle such case?
You write a regular expression to match any possible combination of characters that can be there. You can learn regex with fun with regex crosswords online.
Do not edit JSON files with sed - sed is for lines. Consider using JSON aware tools - like jq, which will handle any possible case.
A jq answer: file.json contains
{
"type": "xyz",
"my_version": "1.0.1.66~22hgde",
"object": "can't end with a comma"
}
then, replacing the last octet before the tilde:
VAR=32
jq --arg octet "$VAR" '.my_version |= sub("[0-9]+(?=~)"; $octet)' file.json
outputs
{
"type": "xyz",
"my_version": "1.0.1.32~22hgde",
"object": "can't end with a comma"
}
I have a huge json file, i will copy a part of it :
"panels": [
"targets": [
{
"alias": "First Prod",
"dimensions": {
"Function": "robot-support-dev-create-human"
},
}
{
"alias": "Second Prod",
"dimensions": {
"Function": "robot-support-prototype-dev-beta-activate-human"
},
}
{
"alias": "third Prod",
"dimensions": {
"Function": "robot-support-dev-jira-kill-human"
},
}
{
"alias": "Somehting",
"dimensions": {
"Robotalias": "default",
"RobotName": "Robot-prod-prototype",
"Operation": "Fight"
},
]
]
I want to perform a Regex on Function each time it contains the robot-support-dev to robot-support-prod-...
sed -i ' s/"robot-support-([a-zA-Z0-9_]+)-dev-([^"]+)"/robot-support-\\1-prod-\\2/g;'
This is what i did but there's something wrong with my regex maybe
You can't match a json with a regex. The "right way(TM)" would be to first extract Function using a json aware tool like jq, then modify it with sed, then insert it back using json aware tool.
Sed by default uses basic regex, so you need to change ( ) + into \( \) \{1,\} (the \+ is a GNU extension) or with GNU sed just use sed -E to use extended regex. Also the \\1 would be interpreted as 2 characters \ with 1, you want to use \1 with a single \. But anyway, your regex is just invalid and does not match what you want (I guess). Also the " are missing on the right side in the replacement string, so your command would just remove the ". Just substitute what you need, try:
sed 's/"robot-support-dev-/"robot-support-prod-/g;'
As #KamilCuk pointed out, there are some issues with the regex you are currently using. I think that if the occurrences you give as an example are the only possibilities, it would work if you match these groups:
sed -i 's/"robot-support\(-\|-.*-\)dev-\(.*\)"/"robot-support\1prod-\2"/g'
As pointed out elsewhere on this page, using sed for this type of problem is, at best, fraught with danger. Since the question has been tagged with jq, it should be pointed out that jq is an excellent match for this type of problem. In particular, a trivial solution can be obtained using the filter sub of arity 2 or 3, i.e. sub/3: sub("FROM"; "TO"; "g").
You might also wish to use walk so that you don't have to be concerned about where exactly the "Function" keys occur, e.g.
walk( if type == "object" and (.Function|type) == "string"
then .Function |= sub( "robot-support-(?<a>([^-]+-)?)dev-"; "robot-support-\(.a)prod-"; "g")
else . end)
I have a curl command which returns me this kind of json formated text
[{"id": "nUsrLast//device control", "name": "nUsrLast", "access": "readonly", "value": "0", "visibility": "visible", "type": "integer"}]
I would like to get the value of the field value.
Can someone give me a simple awk or grep command to do so ?
Here is an awk
awk -v RS="," -F\" '/value/ {print $4}' file
0
How does it work?
Setting RS to , it breaks line to some like this:
awk -v RS="," '{$1=$1}1' file
[{"id": "nUsrLast//device control"
"name": "nUsrLast"
"access": "readonly"
"value": "0"
"visibility": "visible"
"type": "integer"}]
Then /value/ {print $4} prints field 4 separated by "
You could use grep with oP parameters,
$ echo '[{"id": "nUsrLast//device control", "name": "nUsrLast", "access": "readonly", "value": "0", "visibility": "visible", "type": "integer"}]' | grep -oP '(?<=\"value\": \")[^"]*'
0
From grep --help,
-P, --perl-regexp PATTERN is a Perl regular expression
-o, --only-matching show only the part of a line matching PATTERN
Pattern Explanation:
(?<=\"value\": \") Lookbehind is used to set or place the matching marker. In our case, regex engine places the matching marker just after to the string "value": ".
[^"]* Now it matches any character except " zero or more times. When a " is detected then the regex engine would stop it's matching operation.
This solution isn't grep or awk but chances are pretty good your system has perl on it, and this is the best solution thus far:
echo <your_json> | perl -e '<STDIN> =~ /\"value\"\s*:\s*\"(([^"]|\\")*)\"/; print $1;'
It handles the possibility of a failed request by ensuring there is a trailing " character. It also handles backslash-escaped " symbols in the string and whitespace between "value" and the colon character.
It does not handle JSON broken across multiple lines, but then none of the other solutions do, either.
\"value\"\s*:\s*\" Ensures that we're dealing with the correct field, then
(([^"]|\\")*) Captures the associated valid JSON string
\" Makes sure the string is properly terminated
Frankly, you're better off using a real JSON parser, though.
I have this script:
#!/bin/sh
local name="url"
local line='{ "name": "url", "value": "http:\/\/www.example.com\/dir1\/page1" }'
local name2="protocol"
local line2='{ "name": "protocol", "value": "ftp" }'
sed -i "/\<$name2\>/s/.*/$line2/" /mydir/myfile
sed -i "/\<$name\>/s/.*/$line/" /mydir/myfile
myfile contain:
{ "name": "url", "value": "http:\/\/www.example.com\/dir2\/page2" }
{ "name": "url2", "value": "http:\/\/www.example.net/page" }
{ "name": "protocol", "value": "http" }
I detect a problem with / symbol in value field with my sed command. How to fix this error?
Since your files have / all over the place its better to use an alternate regex delimiter; it is supported by sed.
sed -i "/\<$name2\>/s|.*|$line2|" /mydir/myfile
sed -i "/\<$name\>/s|.*|$line|" /mydir/myfile
Use a different separator like this:
sed -i "/\<$name2\>/s%.*%$line2%" /mydir/myfile
For example, this is a dump I did just now:
printf "eins x eins\nzwei x zwei\nabc x abc\ndef x def\n" | sed "/abc/s%x%y%"
It prints:
eins x eins
zwei x zwei
abc y abc
def x def
As you can see, just the line containing abc gets modified by the s command.
Instead of the % sign you can use any character which does not appear in the value you want to replace. Keep that in mind. You also can use _ or ⌴ or ; or a letter, a number, whatever you like. It just mustn't appear in the value.