What's the best way to find and replace the values of all keys in a file without precising the keys ?
Example :
originalFile.txt ------------------------------------> newFile.txt
key1=a key1=$key1
key2=b key2=$key2
key3=c key3=$key3
I came up with this but I'm not able to do it dynamically for any new added key :
sed '/^key1/s/=.*$/= $key1/' newFile.txt
awk -vFS="=" -v OFS="=" '$2="$"$1' originalFile.txt
Using sed: using back referencing capture the left of = in \1 and right of = in \2:
sed -r 's/(^[^=]+)*=(.*)/\1=$\1/g' originalFile.txt
key1=$key1
key2=$key2
key3=$key3
This should do it:
sed -re 's/^(key[[:digit:]]+)=.*$/\1=\$\1/' originalFile.txt
It works because (since the -r options is used) a pair of parentheses captures its match. In the substitution pattern, the \1 reproduces the text matched.
this script extract the password from env and include it in the new file:
while read line; do
password=$(echo $line | sed "s/\(.*\)=.*/\1/g")
newpassword="$(printenv $password)"
if [ -n "$newpassword" ]; then
echo "$password=$newpassword"
else
echo $line
fi
done <originalFile.txt >newFile.txt
Related
I would like to achieve this procedure.
I have this command:
sed -i -e 's/few/asd/g' /usr/local/sbin/.myappenv
This command works if in the file .myappenv there is a few string text. But this command fails to simply create the asd text wheter or not the few is found.
So, if there is the few text, it should replace it, if few text is missing, just create it on the first line.
EDIT_:
It should create the string on the first line of the document .myappenv, it should only be created if there is no coincidence.
The file .myappenv should contain =>
asd
If the file is already populated with few just replace it =>
asd
if there is the few text, it should replace it, if few text is missing, just create it on the first line.
That's an if.
if <file has text>; then <replace text>; else <add text to first line>; fi
or in bash:
file=/usr/local/sbin/.myappenv
if grep -q few "$file"; then
sed 's/few/asd/' "$file"
else
{
echo asd
cat "$file"
} > "$file".tmp
mv "$file".tmp "$file"
fi
How to test if string exists in file with Bash? https://unix.stackexchange.com/questions/99350/how-to-insert-text-before-the-first-line-of-a-file and https://mywiki.wooledge.org/BashGuide/TestsAndConditionals . You might interest yourself in some automation methods, like ansible lineinfile or chezmoi depending on the goal.
Using a single awk you can use this:
awk 'FNR == NR {if (/few/) {n = 1; nextfile}; next}
FNR == 1 && !n {print "asd"} n {gsub(/few/, "asd")} 1' file file
This might work for you (GNU sed):
sed 'H;$!d;x;/few/!s/^/asd/;t;s//asd/;s/.//' file
Make a copy of the file in the hold space (first line will have a prepended newline).
At the end of the file, if few is not matched, insert the literal asd.
Otherwise, replace the first occurrence of few with asd and remove the introduced newline at the start of the file.
To replace all occurrences of few with asd use:
sed 'H;$!d;x;/few/!s/^/asd/;t;s//asd/g;s/.//' file
Alternative:
grep -q few file && sed 's/few/asd/' file || (echo asd; cat file)
Need help in making a sed script to find and replace user input along with single quotes. Input file admins.py:
Script:
read adminsid
while [[ $adminsid == "" ]];
do
echo "You did not enter anything. Please re-enter AdminID"
read adminsid
done
## Please enter Admin's ID
9999999999,8888888888,1111111111
## Script To Replace ADMIN_IDS = [] to ADMIN_IDS = ['9999999999,8888888888,1111111111'] in file
sed -i "s|ADMIN_IDS = \[.*\]|ADMIN_IDS = ['$adminsid']|g" $file
## Current results:
ADMIN_IDS = ['9999999999,8888888888,1111111111']
## Expected results:
ADMIN_IDS = ['9999999999','8888888888','1111111111']
Assign the variable to the data
adminsid=9999999999,8888888888,1111111111
Then use sed -e (script) option to add the quoting, and square brackets.
echo "$adminsid" | sed -e "s/,/\',\'/g" -e "s/^/[\'/" -e "s/$/\']/"
or to apply changes to a file (filename in $file):
sed -i "$file" -e "s/,/\',\'/g" -e "s/^/[\'/" -e "s/$/\']/"
You can do this with awk too:
Suppose you have assigned the variable as :
adminsid=9999999999,8888888888,1111111111
Then the solution:
echo "$adminsid"| awk -F"," -v quote="'" -v OFS="','" '$1=$1 {print "["quote $0 quote"]"}'
-F"," -v OFS="','" :: Replacing separator (,) with (',')
print "["quote $0 quote"]" :: Add single quotes(') and ([) and (]) to the begin and end of line
This might work for you (GNU sed & bash):
<<<"$adminsid" sed 's/[^,]\+/'\''&'\''/g;s/.*/[&]/'
Surround all non-comma characters by single quotes and then surround the entire string by square brackets.
Replace the , with ',' in the variable and add characters at the beginning and at the end.
sed "s/.*/['&']/" <<< "${adminsid//,/','}"
echo "('${adminsid//,/\\',\\'}')"
I have a log file in format like this :
pseudo=thierry33 pseudoConcat=thierry33
pseudo=i love you pseudoConcat=i love you
I want to return all the strings which are between pseudo and pseudoConcat, my desired output is :
thierry33
i love you
How can I do this using sed or awk? I'm trying for a few days in vain.
Thanks.
With sed:
sed -r 's/pseudo=(.*[^ ]) +pseudoConcat.*/\1/'
Explanation:
use GNU option -r to allow +, () without backslashes
capture string after pseudo= with ()
string should end with a non-space [^ ]
before spaces and pseudoConcat +pseudoConcat
use 1st captured group \1 as a replacement
With GNU grep:
grep -oP '(?<=pseudo=).*?(?= *pseudoConcat)' file
Output without trailing spaces:
thierry33
i love you
With bash:
while read -r line; do [[ $line =~ pseudo=(.*?[^\ ])\ *pseudoConcat ]] && echo "${BASH_REMATCH[1]}"; done < file
I want to split a text with comma , not space in for foo in list. Suppose I have a CSV file CSV_File with following text inside it:
Hello,World,Questions,Answers,bash shell,script
...
I used following code to split it into several words:
for word in $(cat CSV_File | sed -n 1'p' | tr ',' '\n')
do echo $word
done
It prints:
Hello
World
Questions
Answers
bash
shell
script
But I want it to split the text by commas not spaces:
Hello
World
Questions
Answers
bash shell
script
How can I achieve this in bash?
Set IFS to ,:
sorin#sorin:~$ IFS=',' ;for i in `echo "Hello,World,Questions,Answers,bash shell,script"`; do echo $i; done
Hello
World
Questions
Answers
bash shell
script
sorin#sorin:~$
Using a subshell substitution to parse the words undoes all the work you are doing to put spaces together.
Try instead:
cat CSV_file | sed -n 1'p' | tr ',' '\n' | while read word; do
echo $word
done
That also increases parallelism. Using a subshell as in your question forces the entire subshell process to finish before you can start iterating over the answers. Piping to a subshell (as in my answer) lets them work in parallel. This matters only if you have many lines in the file, of course.
I think the canonical method is:
while IFS=, read field1 field2 field3 field4 field5 field6; do
do stuff
done < CSV.file
If you don't know or don't care about how many fields there are:
IFS=,
while read line; do
# split into an array
field=( $line )
for word in "${field[#]}"; do echo "$word"; done
# or use the positional parameters
set -- $line
for word in "$#"; do echo "$word"; done
done < CSV.file
kent$ echo "Hello,World,Questions,Answers,bash shell,script"|awk -F, '{for (i=1;i<=NF;i++)print $i}'
Hello
World
Questions
Answers
bash shell
script
Create a bash function
split_on_commas() {
local IFS=,
local WORD_LIST=($1)
for word in "${WORD_LIST[#]}"; do
echo "$word"
done
}
split_on_commas "this,is a,list" | while read item; do
# Custom logic goes here
echo Item: ${item}
done
... this generates the following output:
Item: this
Item: is a
Item: list
(Note, this answer has been updated according to some feedback)
Read: http://linuxmanpages.com/man1/sh.1.php
& http://www.gnu.org/s/hello/manual/autoconf/Special-Shell-Variables.html
IFS The Internal Field Separator that is used for word splitting
after expansion and to split lines into words with the read
builtin command. The default value is ``''.
IFS is a shell environment variable so it will remain unchanged within the context of your Shell script but not otherwise, unless you EXPORT it. ALSO BE AWARE, that IFS will not likely be inherited from your Environment at all: see this gnu post for the reasons and more info on IFS.
You're code written like this:
IFS=","
for word in $(cat tmptest | sed -n 1'p' | tr ',' '\n'); do echo $word; done;
should work, I tested it on command line.
sh-3.2#IFS=","
sh-3.2#for word in $(cat tmptest | sed -n 1'p' | tr ',' '\n'); do echo $word; done;
World
Questions
Answers
bash shell
script
You can use:
cat f.csv | sed 's/,/ /g' | awk '{print $1 " / " $4}'
or
echo "Hello,World,Questions,Answers,bash shell,script" | sed 's/,/ /g' | awk '{print $1 " / " $4}'
This is the part that replace comma with space
sed 's/,/ /g'
For me, use array split is simpler ref
IN="bla#some.com;john#home.com"
arrIN=(${IN//;/ })
echo ${arrIN[1]}
Using readarray(mapfile):
$ cat csf
Hello,World,Questions,Answers,bash shell,script
$ readarray -td, arr < csf
$ printf '%s\n' "${arr[#]}"
Hello
World
Questions
Answers
bash shell
script
I am wrinting a shell script and have a variable like this: something-that-is-hyphenated.
I need to use it in various points in the script as:
something-that-is-hyphenated, somethingthatishyphenated, SomethingThatIsHyphenated
I have managed to change it to somethingthatishyphenated by stripping out - using sed "s/-//g".
I am sure there is a simpler way, and also, need to know how to get the camel cased version.
Edit: Working function derived from #Michał's answer
function hyphenToCamel {
tr '-' '\n' | awk '{printf "%s%s", toupper(substr($0,1,1)), substr($0,2)}'
}
CAMEL=$(echo something-that-is-hyphenated | hyphenToCamel)
echo $CAMEL
Edit: Finally, a sed one liner thanks to #glenn
echo a-hyphenated-string | sed -E "s/(^|-)([a-z])/\u\2/g"
a GNU sed one-liner
echo something-that-is-hyphenated |
sed -e 's/-\([a-z]\)/\u\1/g' -e 's/^[a-z]/\u&/'
\u in the replacement string is documented in the sed manual.
Pure bashism:
var0=something-that-is-hyphenated
var1=(${var0//-/ })
var2=${var1[*]^}
var3=${var2// /}
echo $var3
SomethingThatIsHyphenated
Line 1 is trivial.
Line 2 is the bashism for replaceAll or 's/-/ /g', wrapped in parens, to build an array.
Line 3 uses ${foo^}, which means uppercase (while ${foo,} would mean 'lowercase' [note, how ^ points up while , points down]) but to operate on every first letter of a word, we address the whole array with ${foo[*]} (or ${foo[#]}, if you would prefer that).
Line 4 is again a replace-all: blank with nothing.
Line 5 is trivial again.
You can define a function:
hypenToCamel() {
tr '-' '\n' | awk '{printf "%s%s", toupper(substr($0,0,1)), substr($0,2)}'
}
CAMEL=$(echo something-that-is-hyphenated | hypenToCamel)
echo $CAMEL
In the shell you are stuck with being messy:
aa="aaa-aaa-bbb-bbb"
echo " $aa" | sed -e 's/--*/ /g' -e 's/ a/A/g' -e 's/ b/B/g' ... -e 's/ *//g'
Note the carefully placed space in the echo and the double space in the last -e.
I leave it as an exercise to complete the code.
In perl it is a bit easier as a one-line shell command:
perl -e 'print map{ $a = ucfirst; $a =~ s/ +//g; $a} split( /-+/, $ARGV[0] ), "\n"' $aa
For the records, here's a pure Bash safe method (that is not subject to pathname expansion)—using Bash≥4:
var0=something-that-is-hyphenated
IFS=- read -r -d '' -a var1 < <(printf '%s\0' "${var0,,}")
printf '%s' "${var1[#]^}"
This (safely) splits the lowercase expansion of var0 at the hyphens, with each split part in array var1. Then we use the ^ parameter expansion to uppercase the first character of the fields of this array, and concatenate them.
If your variable may also contain spaces and you want to act on them too, change IFS=- into IFS='- '.