Now I want to find whether there are lines containing '\' character. I tried grep "\\" XXFile but it hints "Trailing Backslash". But when I tried grep '\\' XXFile it is OK. Could anyone explain why the first case cannot run? Thanks.
The difference is in how the shell treats the backslashes:
When you write "\\" in double quotes, the shell interprets the backslash escape and ends up passing the string \ to grep. Grep then sees a backslash with no following character, so it emits a "trailing backslash" warning. If you want to use double quotes you need to apply two levels of escaping, one for the shell and one for grep. The result: "\\\\".
When you write '\\' in single quotes, the shell does not do any interpretation, which means grep receives the string \\ with both backslashes intact. Grep interprets this as an escaped backslash, so it searches the file for a literal backslash character.
If that's not clear, we can use echo to see exactly what the shell is doing. echo doesn't do any backslash interpretation itself, so what it prints is what the shell passed to it.
$ echo "\\"
\
$ echo '\\'
\\
You could have written the command as
grep "\\\\" ...
This has two pairs of backslashes which bash will convert to two single backslashes. This new pair will then be passed to grep as an escaped backslash getting you what you want.
Related
I'm having a directory with many files having special characters and spaces. I want to perform an operation with all these files so I'm trying to store all filenames in a list.txt and then run the command with this list.
The special characters in my list are & []'.
So basically I want to use sed to replace each occurence with \ + the character in question.
E.g. : filename .txt => filename\ .txt etc...
The thing is I have trouble handling apostrophes.
Here is my command as of now :
ls | sed 's/\ /\\ /g' | sed 's/\&/\\&/g' | sed "s/\'/\\'/g" | sed 's/\[/\\[/g' | sed 's/\]/\\]/g'
At first I had issues with, I believe, the apostrophes in the string command in conflict with the apostrophes surrounding the string. So I used double quotes instead, but it still doesn't work.
I've tried all these and nothing worked :
sed "s/\'/\\'/g" (escaping the apostrophe)
sed "s/'/\'/g" (escaping nothing)
sed "s/'/\\'/g" (escaping the backslash)
sed 's/"'"/\"'"/g' (double quoting single quote)
As a disclaimer, I must say, I'm completely new to sed. I just run my first sed command today, so maybe I'm doing something wrong I didn't realize.
PS : I've seen those thread, but no answer worked for me :
https://unix.stackexchange.com/questions/157076/how-to-remove-the-apostrophe-and-delete-the-space
How to replace to apostrophe ' inside a file using SED
This may do:
cat file
avbadf
test&rr
more [ yes
this ]
and'df
sed -r 's/(\x27|&|\[|\])/\\\1/g' file
avbadf
test\&rr
more \[ yes
this \]
and\'df
\x27 is equal to singe quote '
\x22 is equal to double quote "
Whoops, I found the answer to my question. Here is the working input :
sed "s/'/\\\'/g"
This will effectively replace any ' with \'.
However I'm having trouble understanding exactly what's happening here.
So if I understand correctly, we are escaping the backslash and the apostrophe in the replacement string. Now, if somebody could answer some those, I would be grateful :
Why don't we need to escape the first quote (the one in the pattern to find) ?
Why do we have to escape the backslash whereas for the other characters, there's no need ?
Why do we need to escape the second quote (the one in the replacement string) ?
I think all of your sed matches actually need that replacement pattern. This one seems to work for all examples:
ls | sed "s/\ /\\\ /g" | sed "s/\&/\\\&/g" | sed "s/\[/\\\[/g" | sed "s/\]/\\\]/g" | sed "s/'/\\\'/g"
So it is s/regex/replacement/command and 'regex' and 'replacement' have different sets of special characters.
The only one that's different is s/'/\\\'/g and there only because I don't believe there is any special ' character on the regex expression. There is some obscure \' special character in the replacement expression, for matching buffer ends in multi-line mode, accord to the docs. That might be why it needs an escape in the replacement side, but not in the regex side.
For example, \5 is a special character in the replacement expression, so to replace:
filename5.txt -> filename\5.txt
You would also need, as with apostrophe:
sed "s/5/\\\5/g"
It probably has to do with the mysterious inner works of sed parsing, it might read from right to left or something.
Please try the following:
sed 's/[][ &'\'']/\\&/g' file
By using the same example by #Jotne, the result will be:
gavbadf
gtest\&rr
gmore\ \[\ yes
gthis\ \]
gand\'df
[How it works]
The regex part in the sed s command above just defines a character
class of & []', which should be escaped with a backslash.
The right square bracket ] does not need escaping when put
immediately after the left square bracket [.
The obfuscating part will be the handling of a single quote.
We cannot put a single quote within single quotes even if we escape it.
The workaround is as follows: Say we have an assignment str='aaabbb'.
To put a single quote between "aaa" and "bbb", we can say as
str='aaa'\''bbb'.
It may look puzzling but it just concatenates the three sequences;
1) to close the single-quoted string as 'aaa'.
2) to put a single quote with an escaping backslash as \'.
3) to restart the single-quoted string as 'bbb'.
Hope this helps.
I am trying to grep the string
"SNTCHDCS06-Filesystem D:\\ Label:Data Serial Number f8271450"
from a csv file, but somehow I failed miserably.
I understand that I need to add two backlashes on top of the two backslashes (one for shell, one for bash), but it doesn't work after that.
The below command works.
[root#nagiospdc01 folder]# grep -e "^SNTCHDCS06-Filesystem D:\\\\" in/masterlist.csv
SNTCHDCS06-Filesystem D:\\ Label:Data Serial Number f8271450,SNTCHDCS06,10.24.64.210,Active Directory,AD Server,UCS,Filesystem D:\\ Label:Data Serial Number f8271450,Windows Team,0,XM_OPS_WIN,Windows Team,Y,Y,N,N,Y,Y,Y,Y,Y,N,N,Y,Y,ITOC
When I try to grep for the space after that, grep doesn't work and simply fails.
[root#nagiospdc01 folder]# grep -e "^SNTCHDCS06-Filesystem D:\\\\ " in/masterlist.csv
Appreciate if someone can enlighten me on the correct grep syntax and command.
Use single quotes,
grep -e '^SNTCHDCS06-Filesystem D:\\ Label:Data Serial Number f8271450'
When using double quotes \\ gets converted to a single \ by bash. However bash does not look inside single quotes.
From the Bash manual:
3.1.2.2 Single Quotes
Enclosing characters in single quotes (') preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.
3.1.2.3 Double Quotes
Enclosing characters in double quotes (") preserves the literal value of all characters within the quotes, with the exception of $, `, \, and, when history expansion is enabled, !. The characters $ and ` retain their special meaning within double quotes (see Shell Expansions). The backslash retains its special meaning only when followed by one of the following characters: $, `, ", \, or newline. Within double quotes, backslashes that are followed by one of these characters are removed. Backslashes preceding characters without a special meaning are left unmodified. A double quote may be quoted within double quotes by preceding it with a backslash. If enabled, history expansion will be performed unless an ! appearing in double quotes is escaped using a backslash. The backslash preceding the ! is not removed.
The special parameters * and # have special meaning when in double quotes (see Shell Parameter Expansion).
The -F option in grep allows to search for fixed strings. Also the single quotes helps to search for the exact string.
-F, --fixed-strings :
Interpret PATTERN as a list of fixed strings, separated by newlines, any of which is to be matched.
grep -F 'SNTCHDCS06-Filesystem D:\\ Label:Data Serial Number f8271450' masterlist.csv
*SNTCHDCS06-Filesystem D:\\ Label:Data Serial Number f8271450*
Can anyone help me understand the meaning of
grep "[0-9]\{2\}" filename
It is a non-shell matacharacter command.
grep needs the curly braces backslashed in quantifiers.
grep '[0-9]\{2\}' # n.b. the single quotes
In double quotes, \{ is expanded as {, you need to backslash the backslash to get it through.
I'm desperately trying to replace a forward slash (/) with double backslash enclosed in double quotes ("\\")
but
a=`echo "$var" | sed 's/^\///' | sed 's/\//\"\\\\\"/g'`
does not work, and I have no idea why. It always replaces with just one backslash and not two
When / is part of a regular expression that you want to replace with the s (substitute) command of sed, you can use an other character instead of slash in the command's syntax, so you write, for example:
sed 's,/,\\\\,g'
above , was used instead of the usual slash to delimit two parameters of the s command: the regular expression describing part to be replaced and the string to be used as the replacement.
The above will replace every slash with two backslashes. A backslash is a special (quoting) character, so it must be quoted, here it's quoted with itself, that's why we need 4 backslashes to represent two backslashes.
$ echo /etc/passwd| sed 's,/,\\\\,g'
\\etc\\passwd
How about this?
a=${var//\//\\\\}
Demo in a shell:
$ var=a/b/c
$ a=${var//\//\\\\}
$ echo "$a"
a\\b\\c
Another way of doing it: tr '/' '\'
$ var=a/b/c
$ echo "$var"
a/b/c
$ tr '/' '\' <<< "$var"
a\b\c
When I am running this
echo 'a\\b' | grep '\\'
it correctly identifies the backslashes
but when I used the double quotes
echo 'a\\b' | grep "\\"
It is not running and it is returning a trailing backlash error, I can't figure out why that happens. It should work exactly similar to single quotes as I have escaped the backslashes.
When using double quotes the \\ gets evaluated into \. Single quotes leaves things as-is. Do an echo "\\" to see what I mean.
There are multiple places where backslash escapes can be processed:
In your shell: This happens when you use double quotes and not single quotes. Note that this is a very limited processing and escapes like "\n" will not work. This means when you do echo "a\\b" echo will get a\b as first argument while when you're executing echo 'a\\b' echo will get a\\b as first argument.
In the program you're calling: grep will parse the input as POSIX regular expression which has its own set of escapes. echo may or may not handle escape codes by default. My /bin/echo will not process escape codes by default, my bash bulitin echo also won't but my zsh builtin echo will. If a certain behaviour is wanted this can be specified by -e to enable escapes or by -E to disable escapes.
So if you're calling grep "\\" the shell will process the \\ escape sequence and pass \ to grep which will try to parse that as a regular expression and correclty complains about a trailing backslash.
So when you use double quotes you have to double escape everything resulting in the rather clumsy echo 'a\\b' | grep "\\\\".
To get the least amount of escaping you can use echo 'a\\b' | grep '\\' or even echo 'a\\b' | grep -F '\', where the -F flag tells grep to interpret the pattern as verbatim string and not as a regular expression. If you happen to have an echo that processes escapes by default (this will result in echo 'a\\b' printing a\b) you also need to add -E to the echo command.
grep does a bit of basic regular expression matching.
When in double quotes, the shell parses \\, so \\ will resolve to a single backslash as a parameter.
Additionally, grep does a bit of basic regular expression matching, so to match a single backslash you need to pass two backslashes to grep.
So by calling grep "\\" you actually get grep \, which does not parse as a regular expression, therefore makes grep fail.
To correctly match only double backslashes, do grep -F '\\' (which means fixed string matching instead of regex), grep '\\\\' or grep "\\\\\\\\".