i have a problem with my sed command in my sh script.
I already tried without -i, replacing my file with a backup using > and cp, removing the simples quotes and trying with double after s/ and /g.
echo '=> Enabling Cryptodisk in GRUB'
sed -i 's/#GRUB_ENABLE_CRYPTODISK=y/GRUB_ENABLE_CRYPTODISK=y/g' /etc/default/grub #Enabling Cryptodisk in GRUB
echo '=> Adding Preload_modules in GRUB'
sed -i 's/GRUB_PRELOAD_MODULES="part_gpt part_msdos"/GRUB_PRELOAD_MODULES="part_gpt part_msdos luks cryptodisk"/g' /etc/default/grub #Adding Preload_modules in GRUB
echo '=> Adding Linux CMDLINE in GRUB'
GUIDMAPPER=$(blkid | grep ^/dev/mapper | awk -F "\"" '{print $2}') #Get mapper GUID
GUIDSDA3=$(blkid | grep ^/dev/sda3 | awk -F "\"" '{print $2}') #Get sda3 GUID
sed -i 's/GRUB_CMDLINE_LINUX=""/GRUB_CMDLINE_LINUX="cryptdevice=${GUIDMAPPER}=${GUIDSDA3}:cryptroot root=/dev/mapper/cryptroot crypto=whirlpool:aes-xts-plain64:512:0:"/g' /etc/default/grub #Adding Linux CMDLINE in GRUB
The error is triggered for the last sed call, everything else work fine.
sed -i 's/GRUB_CMDLINE_LINUX=""/GRUB_CMDLINE_LINUX="cryptdevice=${GUIDMAPPER}=${GUIDSDA3}:cryptroot root=/dev/mapper/cryptroot crypto=whirlpool:aes-xts-plain64:512:0:"/g' /etc/default/grub
Tested so far... :
sed -i 's/GRUB_CMDLINE_LINUX=""/GRUB_CMDLINE_LINUX="cryptdevice='"$GUIDMAPPER"'='"$GUIDSDA3"':cryptroot root=/dev/mapper/cryptroot crypto=whirlpool:aes-xts-plain64:512:0:"/g' /etc/default/grub #Adding Linux CMDLINE in GRUB
Result
Thanks (sorry for my bad english)
Edit :
It works by escaping '\' in replacement.
sed -i 's/GRUB_CMDLINE_LINUX=""/GRUB_CMDLINE_LINUX="cryptdevice='"$GUIDMAPPER"'='"$GUIDSDA3"':cryptroot root=\/dev\/mapper\/cryptroot crypto=whirlpool:aes-xts-plain64:512:0:"/g' /etc/default/grub #Adding Linux CMDLINE in GRUB
Thanks to Wiktor Stribiżew
Related
sed -i 's/1.1.1.1/ `hostname -I | cut -f1 -d " "`/g' file.txt
Not able to overwrite IP address using sed command in a given file. How to run this (hostname -I | cut -f1 -d " ") command with sed command?
This might work for you (GNU sed):
sed -i '/1\.1\.1\.1/{s//$(hostname -I | cut -f1 -d " ")/;s/.*/echo "&"/e}' file
Place the commands between $(...) and then echo the line using the evaluate flag of the substitution command.
Just use a variable.
hn=$(hostname -I | cut -f1 -d" ") && sed -i "s/1\.1\.1\.1/$hn/g" file
Sed's e and s///e work on the entire pattern space, so it's just more trouble than it's worth.
I've 95 files that looks like :
2019-10-29-18-00/dev/xx;512.00;0.4;/var/x/xx/xxx
2019-10-29-18-00/dev/xx;512.00;0.68;/xx
2019-10-29-18-00/dev/xx;512.00;1.84;/xx/xx/xx
2019-10-29-18-00/dev/xx;512.00;80.08;/opt/xx/x
2019-10-29-18-00/dev/xx;20480.00;83.44;/var/x/x
2019-10-29-18-00/dev/xx;3584.00;840.43;/var/xx/x
2019-10-30-00-00/dev/xx;2048.00;411.59;/
2019-10-30-00-00/dev/xx;7168.00;6168.09;/usr
2019-10-30-00-00/dev/xx;3072.00;1036.1;/var
2019-10-30-00-00/dev/xx;5120.00;348.72;/tmp
2019-10-30-00-00/dev/xx;20480.00;2033.19;/home
2019-10-30-12-00;/dev/xx;5120.00;348.72;/tmp
2019-10-30-12-00;/dev/hd1;20480.00;2037.62;/home
2019-10-30-12-00;/dev/xx;512.00;0.43;/xx
2019-10-30-12-00;/dev/xx;3584.00;794.39;/xx
2019-10-30-12-00;/dev/xx;512.00;0.4;/var/xx/xx/xx
2019-10-30-12-00;/dev/xx;512.00;0.68;/xx
2019-10-30-12-00;/dev/xx;512.00;1.84;/var/xx/xx
2019-10-30-12-00;/dev/xx;512.00;80.08;/opt/xx/x
2019-10-30-12-00;/dev/xx;20480.00;83.44;/var/xx/xx
2019-10-30-12-00;/dev/x;3584.00;840.43;/var/xx/xx
For some lines I've 2019-10-29-18-00/dev and for some other lines, I've 2019-10-30-12-00;/dev/
I want to add the ; before the /dev/ where it is missing, so for that I use this sed command :
sed 's/\/dev/\;\/dev/'
But How I can apply this command for each lines where the ; is missing ? I try this :
for i in $(cat /home/xxx/xxx/xxx/*.txt | grep -e "00/dev/")
do
sed 's/\/dev/\;\/dev/' $i > $i
done
But it doesn't work... Can you help me ?
Could you please try following with GNU awkif you are ok with it.
awk -i inplace '/00\/dev\//{gsub(/00\/dev\//,"/00;/dev/")} 1' *.txt
sed solution: Tested with GNU sed for few files and it worked fine.
sed -i.bak '/00\/dev/s/00\/dev/00\;\/dev/g' *.txt
This might work for you (GNU sed & parallel):
parallel -q sed -i 's#;*/dev#;/dev#' ::: *.txt
or if you prefer:
sed -i 's#;*/dev#;/dev#' *.txt
Ignore lines with ;/dev.
sed '/;\/dev/{p;d}; s^/dev^;/dev^'
The /;\/dev/ check if the line has ;/dev. If it has ;/dev do: p - print the current line and d - start from the beginning.
You can use any character with s command in sed. Also, there is no need in escaping \;, just ;.
How I can apply this command for each lines where the ; is missing ? I try this
Don't edit the same file redirecting to the same file $i > $i. Think about it. How can you re-write and read from the same file at the same time? You can't, the resulting file will be in most cases empty, as the > $i will "execute" first making the file empty, then sed $i will start running and it will read an empty file. Use a temporary file sed ... "$i" > temp.txt; mv temp.txt "$i" or use gnu extension -i sed option to edit in place.
What you want to do really is:
grep -l '00/dev/' /home/xxx/xxx/xxx/*.txt |
xargs -n1 sed -i '/;\/dev/{p;d}; s^/dev^;/dev^'
grep -l prints list of files that match the pattern, then xargs for each single one -n1 of the files executes sed which -i edits files in place.
grep for filtering can be eliminated in your case, we can accomplish the task with a single sed command:
for f in $(cat /home/xxx/xxx/xxx/*.txt)
do
[[ -f "$f" ]] && sed -Ei '/00\/dev/ s/([^;])(\/dev)/\1;\2/' "$f"
done
The easiest way would be to adjust your regex so that it's looking a bit wider than '/dev/', e.g.
sed -i -E 's|([0-9])/dev|\1;/dev|'
(note that I'm taking advantage of sed's flexible approach to delimiters on substitute. Also, -E changes the group syntax)
Alternatively, sed lets you filter which lines it handles:
sed -i '/[0-9]\/dev/ s/\/dev/;/dev/'
This uses the same substitution you already have but only applied on lines that match the filter regex
This question already has answers here:
Combining two sed commands
(2 answers)
Closed 7 years ago.
I Have to Operate the following sed commands over a file to remove some lines in that file. How can I make this a shell scipt with the file name as a variable. Or is there any simple way to do this as a shell script
sed -i '/^Total/d' delhi222517.txt
sed -i '/^CBSE/d' delhi222517.txt
sed -i '/^Keyword wise/d' delhi222517.txt
sed -i '/^wise/d' delhi222517.txt
sed -i '/^Select A/d' delhi222517.txt
sed -i '/^Enter A/d' delhi222517.txt
sed -i '/^(Keyword/d' delhi222517.txt
sed -i '/^State Name/d' delhi222517.txt
sed -i '/^SNo/d' delhi222517.txt
sed -i '/^Disclaimer/d' delhi222517.txt
sed -i '/^provided/d' delhi222517.txt
sed -i '/^at$/d' delhi222517.txt
sed -i '/^Designed/d' delhi222517.txt
sed -i '/^National/d' delhi222517.txt
sed -i '/^$/d' delhi222517.txt
sed -i '/^\t$/d' delhi222517.txt
sed -i '/^\s$/d' delhi222517.txt
sed -i '/^ /d' delhi222517.txt
sed -i '/^ /d' delhi222517.txt
sed -i 's/^\([0-9]\)/--\1/g' delhi222517.txt
The variable thing is easy:
F=delhi222517.txt
sed -i '/^Total/d' "$F"
...
Or if you want to pass the name of the file as argument to your script:
F="$1"
sed -i '/^Total/d' "$F"
...
But it is better to use the sed options to call it only once. You can use:
sed -i \
-e '/^Total/d' \
-e '/^CBSE/d' \
-e '/^Keyword wise/d' \
... \
delhi222517.txt
Or you can write a file with the full script:
sed -i -f script.sed delhi222517.txt
Or if you feel geek enough, you can use the standard input:
sed -i -f - delhi222517.txt << EOF
/^Total/d
/^CBSE/d
/^Keyword wise/d
...
EOF
On the Command Line
On the command line, you can separate sed commands with semi-colons or with multiple expression arguments. As generic examples:
# Using Semi-Colons
sed -i 's/foo/bar/; s/baz/quux/' infile
# Using Multiple Expressions
sed -i -e 's/foo/bar/' -e 's/baz/quux/' infile
Write a Full-Fledged Sed Script
In general, though, if your commands are numerous, stop using one-liners and build a full-fledged sed script. For example, you could create a file named /tmp/foo.sed containing the following commands from your question:
/^Total/d
/^CBSE/d
/^Keyword wise/d
/^wise/d
/^Select A/d
/^Enter A/d
/^(Keyword/d
/^State Name/d
/^SNo/d
/^Disclaimer/d
/^provided/d
/^at$/d
/^Designed/d
/^National/d
/^$/d
/^\t$/d
/^\s$/d
/^ /d
/^ /d
s/^\([0-9]\)/--\1/g
Then invoke your commands all at once. For example, using GNU sed:
infile='delhi222517.txt'
script='/tmp/foo.sed'
sed --in-place --file="$script" "$infile"
Well you can put them in a a shell script like this:
#!/bin/bash
# some sanity checks
file="$1"
sed -i '/^Total/d' "$file"
sed -i '/^CBSE/d' "$file"
sed -i '/^Keyword wise/d' "$file"
sed -i '/^wise/d' "$file"
#.. more sed commands
btw your various sed commands can be combined into 1 or fewer sed command using reges like:
sed -r -i '/^(Total|CBSE)/d' "$file"
If your script is not going to run any program other than sed your file, then this may be the cleanest way to set it up:
#!/bin/sed -f # <- run the file passed to the program from the command-line
/^Total/Id # /I is the case-insensitive flag, replacing sed -i
/^CBSE/Id
/^Keyword wise/Id
/^wise/Id
/^Select A/Id
/^Enter A/Id
...
Make the above script executable, and then just pass it the filename that you wish to convert:
./mysedscript delhi222517.txt
Using awk you can do all in go:
file=delhi222517.txt
awk '!/^(Total|CBSE|Keyword wise|wise)/' "$file"
I could do the following in python to clean and strip unwanted whitespaces, but can it be done just through the terminal by other means like sed , grep or something?
outfile = open('textstripped.txt','w+','utf8')
for i in open('textfile.txt','r','utf8'):
print>>outfile, i.strip()
Using perl on the command line:
perl -lpe 's/^\s+//; s/\s+$//' file.txt > stripped.txt
This solution is based on sed man page:
sed 'y/\t/ /;s/^ *//;s/ *$//' input > output
http://www.gnu.org/software/sed/manual/sed.html#Centering-lines
Description:
y\t/ / replaces tabs with spaces
s/^ *// removes leading spaces
s/ *$// removes trailing spaces
$ cat input.txt | sed 's/^[ \t]*//;s/[ \t]*$//' > output.txt
This gets rid of the leading and trailing white spaces..
EDIT: sed -e "s/^[ \t]+//; s/[ \t]+$//" -i .bk input.txt
This does in place file editing, and saves backup to input.txt.bk (and saves a process as some suggested)
sed -E "s/(^[ \t]+|[ \t]+$)//" < input > output
Or if you have a GNU-compliant version of SED:
sed -E "s/^\s+|\s+$//g" < in > out
If you have a Mac, I recommed getting homebrew and installing gnu-sed.
Then, alias sed=gsed.
I have been trying quite a few ways with no luck. I have a file named test.txt that has some lorem ipsum and the text [staging: production] I simply want to add a few lines that I have saved in a variable in before it.
If you could explain where I have gone wrong with any of the below it would be much appreciated!
#!/bin/bash
test="lala\
kjdsh"
sed '/^#$/{N; /[staging: production]/ i \
<Location /cgis> \
</Location>\
}' ./test.txt
sed -i -e 's/\[staging\: production\]/\$test/g' ./test.txt
#sed -i 's/Lorem/beautiful/g' test.txt
#awk -v data=$test '{A[NR]=$0}/\[staging\: production\]/{ print data }' test.txt > testfile.txt
#read -a text <<<$(cat test.txt)
#echo ${#text[#]}
#for i in ${text[#]};
#do
# echo -n $i;
# sleep .2;
#done
#ed -s test.txt <<< $'/\[staging\: production\]/s/lalalala/g\nw'
#awk -v data=$test '/\(/\[staging\: production\]\)/ { print data }' test.txt > testfile.txt
# && mv testfile.txt test.txt
#sed -i -e '/\(\[staging\: production\]\)/r/$test\1/g' test.txt
#sed "/\(\[staging\: production\]\)/s//$test\1/g" test.txt
sed -i -e 's/\[staging\: production\]/\$test/g' ./test.txt
won't work because inside singe quotes BASH will not expand \$test.
Therefore you don't need to escape the $.
If you want to substitute with the contents of the variable $test do:
sed -i -e 's/\[staging: production\]/'$test'/g' ./test.txt
You also do not need to escape :
To insert before your pattern works for me this way:
sed -i -e '/\[staging: production\]/ i '$test'' ./test.txt
However to preserve the linebreak inside the variable I needed to define:
test="lala\nkjdsh"
Please note the \n to encode the linebreak.
Try it in perl, it seems to work fine:
perl -pe '{$rep="what\nnow"; s/(\[foo foo2\])/$rep$1/}' file
This might work for you (GNU sed):
test="lala\\
kjdsh"
sed '/\[staging: production\]/i\'"$test" test.txt
N.B. \\ in the variable and the variable is surrouded by "'s in the sed command.