Could someone explain this command for me:
cat | sed -e 's,%,$,g' | sudo tee /etc/init.d/dropbox << EOF
echo "Hello World"
EOF
What does the "sed" command do?
sed is the Stream EDitor. It can do a whole pile of really cool things, but the most common is text replacement.
The s,%,$,g part of the command line is the sed command to execute. The s stands for substitute, the , characters are delimiters (other characters can be used; /, : and # are popular). The % is the pattern to match (here a literal percent sign) and the $ is the second pattern to match (here a literal dollar sign). The g at the end means to globally replace on each line (otherwise it would only update the first match).
Here sed is replacing all occurrences of % with $ in its standard input.
As an example
$ echo 'foo%bar%' | sed -e 's,%,$,g'
will produce "foo$bar$".
It reads Hello World (cat), replaces all (g) occurrences of % by $ and (over)writes it to /etc/init.d/dropbox as root.
sed is a stream editor. I would say try man sed.If you didn't find this man page in your system refer this URL:
http://unixhelp.ed.ac.uk/CGI/man-cgi?sed
Related
I want to trim a string from one character, the last /, to either : or #, which ever appears first. An example would be:
https://www.example.com/?client=safari/this-text:not-this:or_this
would be trimmed to:
this-text
and
https://www.example.com/?client=safari/this-text#not-this:or_this
would be trimmed to:
this-text
I know I can trim text in bash from a specific character to another character, but is there a way to trim from one character to either of 2 characters?
Use grep like so: grep -Po '^.*/\K[^:#]*'
Examples:
echo 'https://www.example.com/?client=safari/this-text:not-this:or_this' | grep -Po '^.*/\K[^:#]*'
or:
echo 'https://www.example.com/?client=safari/this-text#not-this:or_this' | grep -Po '^.*/\K[^:#]*'
Output:
this-text
Here, grep uses the following options:
-P : Use Perl regexes.
-o : Print the matches only, 1 match/line, not the entire lines.
The regex ^.*/\K[^:#]* does the following:
^.*/ : Match from the beginning of the string (^) all the way up to the last slash ('/').
\K : Pretend that the match started at this position.
[^:#]* : zero or more occurrences (greedy) of any characters except : or #. This matches either until the end of the line, or until the next : or #, whichever comes first.
SEE ALSO:
grep manual
NOTE:
This works with GNU grep, which may need to be installed, depending on your system. For example, to install GNU grep on macOS, see this answer: https://apple.stackexchange.com/a/357426/329079
With a little Bash function:
trim() {
local str=${1##*/}
printf '%s\n' "${str%%[:#]*}"
}
This first trims everything up to and including the last /, then everything starting from the first occurrence of : or #.
In use:
$ trim 'https://www.example.com/?client=safari/this-text:not-this:or_this'
this-text
$ trim 'https://www.example.com/?client=safari/this-text#not-this:or_this'
this-text
Another way is to use sed: sed -e 's,^.*/,,' -e 's,[:#].*$,,'.
First -e command (s/regex/replacement/) removes text from the start to the last /, then the second -e removes from : or # to the end of the text.
echo 'https://www.example.com/?client=safari/this-text:not-this:or_this' | sed -e 's,^.*/,,' -e 's,[:#].*$,,'
this-text
Would like to insert line number at specific location in file
e.g.
apple
ball
should be
(1) apple
(2) ball
Using command
sed '/./=' <FileName>| sed '/./N; s/\n/ /'
It generates
1 Apple
2 Ball
1st solution: This should be an easy task for awk.
awk '{print "("FNR") "$0}' Input_file
2nd solution: With pure sed as per OP's attempt try:
sed '=' Input_file | sed 'N; s/^/(/;s/\n/) /'
Easy to do with perl instead:
perl -ne 'print "($.) $_"' foo.txt
If you want to modify the file in-place instead of just printing out the numbered lines on standard output:
perl -ni -e 'print "($.) $_"' foo.txt
Many ways are there to insert line numbers in a file
some of them are :-
1.Using cat command
cat -n file.txt > newfile.txt
2.Using nl command
nl -b a file.txt
Awk and perl both are very usefull and powerfull. But if, like me, you are reluctant to learn yet another programming language, you can complete this task with the bash commands you probably know already.
With bash you can
increment a sequence number n: $((++n))
read all lines from a file foo into a variable l: while read -r l;do ...;done <foo, where the option -r serves to treat backslashes as just characters.
print formatted output to a line: printf "plain text %i %s\n" number string
Now suppose you want to enclose your sequence number in parentheses, and format them to 8 digits with leading zeroes, then you combine all this to get:
n=0;while read -r l;do printf "(%08i) %s\n" $((++n)) "$l";done <foo >numberedfoo
Note that you do not need to initialize the variable n to use it as a sequence number further on. But if you experiment with this command a few times without reinitializing n, your lines will be numbered from where your previous try stopped incrementing.
Finally, if you don't like the C-like formatting syntax of printf, just use plain echo, and leave the formatting to bash variable expansion. Here is how to format a number like in the command above (do type a space before the -, and a ; before the echo) :
nformat="0000000$n"; echo "(${nformat: -8}) ...";
This question already has answers here:
Using different delimiters in sed commands and range addresses
(3 answers)
Closed 6 years ago.
I'm trying to replace text in a file with the output of another command. Unfortunately, the outputted text contains characters bash expands. For example, I'm running the following script to change the file (somestring references output that would break the sed command):
#!/bin/bash
somestring='$6$sPnfj/lnXwZVrec7$fCnL9uy1oWIMZduInKTHBAxhsQxGCsBpm2XfVFFqDPHKidrd93yfjbYvKgYexXHVcvkKdu9lbfy16Ek5GvKy/1'
sed '0,/^title/s/^title*/'"$somestring"'\n&/' $HOME/example.txt
sed fails with this error:
sed: -e expression #1, char 30: unknown option to `s'
I think bash is substuting the contents of $somestring when building the sed command, but is then trying to expand the resulting text. I can't put the entire sed script in single quotes, I need bash to expand it the first time, just not the second. Any suggestions? Thanks
here the forward slash / is the problem. If it's the only issue you can set sed to use a different delimiter.
for example
$ somestring="abc/def"; echo xxx | sed 's/xxx/'"$somestring"'/'
sed: -e expression #1, char 11: unknown option to `s'
$ somestring="abc/def"; echo xxx | sed 's_xxx_'"$somestring"'_'
abc/def
you also need to worry about & and \ chars and escape them if can appear in the replacement text.
If you can't control the the replacement string, either you have to sanitize with another sed script or, alternatively use r command to read it from a file. For example,
$ seq 5 | sed -e '/3/{r replace' -e 'd}'
1
2
3slashes///1ampersand&and2backslashes\\end
4
5
where
$ cat replace
3slashes///1ampersand&and2backslashes\\end
You have several errors here:
the string somestring has characters that are significative for sed command (the most important being '/' that you are using as a delimiter) You can escape it, by substituting it with a previous
somestring=$(echo "$somestring" | sed -e 's/\//\\\//g')
that will convert your / chars to \/ sequences.
you are using sed '0,/^title/s/^title*/'"$somestring"'\n&/' $HOME/example.txt which is looking to substitute the string titl followed by any number of e characters by that $somestring value, followed by a new line and the original one. Unfortunately, sed(1) doesn't allow you to use newline characters in the pattern substitution side of the s command, but you can afford the result by using the i command with a text consisting of you pattern (preceding any new line by a \ to interpret it as literal):
Finally the script leads to:
#!/bin/bash
somestring='$6$sPnfj/lnXwZVrec7$fCnL9uy1oWIMZduInKTHBAxhsQxGCsBpm2XfVFFqDPHKidrd93yfjbYvKgYexXHVcvkKdu9lbfy16Ek5GvKy/1'
somestring=$(echo "$somestring" | sed -e 's/\//\\\//g')
sed '/^title/i\
'"$somestring\\
" $HOME/example.txt
If your shell is Bash, you can use parameter substitution to replace the problematic /:
somestring="{somestring//\//\\/}"
That looks scary, but is easier to understand if you look at the version that replaces x with __:
somestring="${somestring//x/__}"
It might be easier to use (say) underscore as the delimiter for your sed s command, and then the substitution above would be
somestring="${somestring//_/\\_}"
If you already have backslashes, you'll need to first replace those:
somestring="${somestring//\\/\\\\}"
somestring="{somestring//\//\\/}"
If there were other characters that needed escaping (e.g. on the search side of s///), then you could extend the above appropriately.
This URL provides the cleanest answer:
Command to escape a string in bash
printf "%q" "$someVariable"
will escape any characters you need escaped for you.
I have a file like
key\0value\n
akey\0value\n
key2\0value\n
I have to create a script that take as argument a word. I have to return every lines having a key exactly the same than the argument.
I tried
grep -aF "$key\x0"
but grep seems to do not understand the \x0 (\0 same result). Futhermore, I have to check that the line begins with "$key\0"
I only can use sed grep and tr and other no maching commands
To have the \0 taken into account try :
grep -Pa "^key\x0"
it works for me.
Using sed
sed will work:
$ sed -n '/^key1\x00/p' file
key1value
The use of \x00 to represent a hex character is a GNU extension to sed. Since this question is tagged linux, that is not a problem.
Since the null character does not display well, one might (or might not) want to improve the display with something like this:
$ sed -n 's/^\(akey\)\x00/\1-->/p' file
akey-->value
Using sed with keys that contain special characters
If the key itself can contain sed or shell active characters, then we must escape them first and then run sed against the input file:
#!/bin/bash
printf -v script '/^%s\\x00/p' "$(sed 's:[]\[^$.*/]:\\&:g' <<<"$1")"
sed -n "$script" file
To use this script, simply supply the key as the first argument on the command line, enclosed in single-quotes, of course, to prevent shell processing.
To see how it works, let's look at the pieces in turn:
sed 's:[]\[^$.*/]:\\&:g' <<<"$1"
This puts a backslash escape in front of all sed-active characters.
printf -v script '/^%s\\x00/p' "$(sed 's:[]\[^$.*/]:\\&:g' <<<"$1")"
This creates a sed command using the escaped key and stores it in the shell variable script.
sed -n "$script" file
This runs sed using the shell variable script as the sed command.
Using awk
The question states that awk is not an acceptable tool. For completeness, though, here is an awk solution:
$ awk -F'\x00' -v k=key1 '$1 == k' file
key1value
Explanation:
-F'\x00'
awk divides the input up into records (lines) and divides the records up into fields. Here, we set the field separator to the null character. Consequently, the first field, denoted $1, is the key.
-v k=key1
This creates an awk variable, called k, and sets it to the key that we are looking for.
$1 == k
This statement looks for records (lines) for which the first field matches our specified key. If a match is found, the line is printed.
I have long file with the following list:
/drivers/isdn/hardware/eicon/message.c//add_b1()
/drivers/media/video/saa7134/saa7134-dvb.c//dvb_init()
/sound/pci/ac97/ac97_codec.c//snd_ac97_mixer_build()
/drivers/s390/char/tape_34xx.c//tape_34xx_unit_check()
(PROBLEM)/drivers/video/sis/init301.c//SiS_GetCRT2Data301()
/drivers/scsi/sg.c//sg_ioctl()
/fs/ntfs/file.c//ntfs_prepare_pages_for_non_resident_write()
/drivers/net/tg3.c//tg3_reset_hw()
/arch/cris/arch-v32/drivers/cryptocop.c//cryptocop_setup_dma_list()
/drivers/media/video/pvrusb2/pvrusb2-v4l2.c//pvr2_v4l2_do_ioctl()
/drivers/video/aty/atyfb_base.c//aty_init()
/block/compat_ioctl.c//compat_blkdev_driver_ioctl()
....
It contains all the functions in the kernel code. The notation is file//function.
I want to copy some 100 files from the kernel directory to another directory, so I want to strip every line from the function name, leaving just the filename.
It's super-easy in python, any idea how to write a 1-liner in the bash prompt that does the trick?
Thanks,
Udi
cat "func_list" | sed "s#//.*##" > "file_list"
Didn't run it :)
You can use pure Bash:
while read -r line; do echo "${line%//*}"; done < funclist.txt
Edit:
The syntax of the echo command is doing the same thing as the sed command in Eugene's answer: deleting the "//" and everything that comes after.
Broken down:
"echo ${line}" is the same as "echo $line"
the "%" deletes the pattern that follows it if it matches the trailing portion of the parameter
"%" makes the shortest possible match, "%%" makes the longest possible
"//*" is the pattern to match, "*" is similar to sed's ".*"
See the Parameter Expansion section of the Bash man page for more information, including:
using ${parameter#word} for matching the beginning of a parameter
${parameter/pattern/string} to do sed-style replacements
${parameter:offset:length} to retrieve substrings
etc.
here's a one liner in (g)awk
awk -F"//" '{print $1}' file
Here's one using cut and rev
cat file | rev | cut -d'/' -f2-| rev