I am writing a bash shell script to output the suffixes of filenames.
In this case I use:
sed 's|.*\.||'
So the output is e.g.:
png
exe
c
But what do I do if the file name has no suffix and therefore no dot? My output should be "no suffix", but I don't know how to do this with sed.
EDIT
What I've already tried:
Directory:
abc.x
abc.y
abc
Input:
find . -type f | sed -E 's/^[^.]+$/no suffix/; s/.*\.//'
Output:
x
y
/abc
Use 2 consecutive substitutions:
sed -E 's/^[^.]+$/no suffix/; s/.+\.//'
One in awk. First some test material that was not provided:
$ cat foo
this.foo
that.bar
nothing
The awk:
$ awk '{n=split($0,a,".");print (n>1?a[n]:"no suffix")}' foo
foo
bar
no suffix
$ cat file
abc.x
abc.y
abc
$ awk -F'.' '{print (NF>1 ? $NF : "no suffix")}' file
x
y
no suffix
How about
sed '/.*\./s///;t;s/.*/no suffix/'
The regex matches lines with a dot. On those lines, we perform a substitution. If a substitution occurred, we are done. Otherwise, perform the other substitution.
The use of an empty regex in the substitution pattern uses the previous pattern. The t command branches if a substitution occurred; without an argument, we branch to the end of the script. (Otherwise, you can set a label with :label and branch to that with tlabel.)
You can accomplish the same with the POSIX shell parameter expansions without invoking separate utilities. For example, to test whether a file contains a '.' you can simply use test, e.g.:
[ "$i" = "${i%.*}" ]
See Posix Programmer's Manual - Shell Command Language - Parameter Expansion
If it tests TRUE, then no extension is present, otherwise, you can use an additional parameter expansion to obtain the extension itself, e.g.
[ "$i" = "${i%.*}" ] && echo "$i - no suffix" || echo "$i - ${i##*.}"
(note: you would need an additional test to exclude .foo (e.g. dotfiles), but that is left to you)
Wrap that in a loop and exclude directory files and you can test every file within a directory or use read within a loop and pipe a list of names to it. For example, looping over the files in a directory would results in:
...
ftlcdfil.c - c
geany-plugin_build_w_gtk+2.patch - patch
geany-plugins.spec - spec
geany.spec - spec
geany_build_w_gtk+2.patch - patch
getfl - no suffix
gtkw_save_test.txt - txt
gtkwrite-master.zip - zip
helloleap - no suffix
helloleap.c - c
jnl - no suffix
messages - no suffix
opensuse_15.0_1s_delay.svg - svg
...
Using Perl
/tmp> ls ab*
abc abc.x abc.y
/tmp> perl -e ' print /\./? "$_\n" : "no suffix\n" for(glob("ab*")) '
no suffix
abc.x
abc.y
/tmp>
Related
I have a directory that includes a lot of java files, and in each file I have a class variable:
String system = "x";
I want to be able to create a bash script which I execute in the same directory, which will go to only the java files in the directory, and replace this instance of x, with y. Here x and y are a word. Now this may not be the only instance of the word x in the java script, however it will definitely be the first.
I want to be able to execute my script in the command line similar to:
changesystem.sh -x -y
This way I can specify what the x should be, and the y I wish to replace it with. I found a way to find and print the line number at which the first instance of a pattern is found:
awk '$0 ~ /String system/ {print NR}' file
I then found how to replace a substring on a given line using:
awk 'NR==line_number { sub("x", "y") }'
However, I have not found a way to combine them. Maybe there is also an easier way? Or even, a better and more efficient way?
Any help/advice will be greatly appreciated
You may create a changesystem.sh file with the following GNU awk script:
#!/bin/bash
for f in *.java; do
awk -i inplace -v repl="$1" '
!x && /^\s*String\s+system\s*=\s*".*";\s*$/{
lwsp=gensub(/\S.*/, "", 1);
print lwsp"String system = \""repl"\";";
x=1;next;
}1' "$f";
done;
Or, with any awk:
#!/bin/bash
for f in *.java; do
awk -v repl="$1" '
!x && /^[[:space:]]*String[[:space:]]+system[[:space:]]*=[[:space:]]*".*";[[:space:]]*$/{
lwsp=$0; sub(/[^[:space:]].*/, "", lwsp);
print lwsp"String system = \""repl"\";";
x=1;next
}1' "$f" > tmp && mv tmp "$f";
done;
Then, make the file executable:
chmod +x changesystem.sh
Then, run it like
./changesystem.sh 'new_value'
Notes:
for f in *.java; do ... done iterates over all *.java files in the current directory
-i inplace - GNU awk feature to perform replacement inline (not available in a non-GNU awk)
-v repl="$1" passes the first argument of the script to the awk command
!x && /^\s*String\s+system\s*=\s*".*";\s*$/ - if x is false and the record starts with any amount of whitespace (\s* or [[:space:]]*), then String, any 1+ whitespaces, system, = enclosed with any zero or more whitesapces, and then a " char, then has any text and ends with "; and any zero or more whitespaces, then
lwsp=gensub(/\S.*/, "", 1); puts the leading whitespace in the lwsp variable (it removes all text starting with the first non-whitespace char from the line matched)
lwsp=$0; sub(/[^[:space:]].*/, "", lwsp); - same as above, just in a different way since gensub is not supported in non-GNU awk and sub modifies the given input string (here, lwsp)
{print "String system = \""repl"\";";x=1;next}1 - prints the String system = " + the replacement string + ";, assigns 1 to x, and moves to the next line, else, just prints the line as is.
You don't need to pre-compute the line number. The whole job can be done by one not-too-complicated sed command. You probably do want to script it, though. For example:
#!/bin/bash
[[ $# -eq 3 ]] || {
echo "usage: $0 <context regex> <target regex> <replacement text>" 1>&2
exit 1
}
sed -si -e "/$1/ { s/\\<$2\\>/$3/; t1; p; d; :1; n; b1; }" ./*.java
That assumes that the files to modify are java source files in the current working directory, and I'm sure you understand the (loose) argument check and usage message.
As for the sed command itself,
the -s option instructs sed to treat each argument as a separate stream, instead of operating as if by concatenating all the inputs into one long stream.
the -i option instructs sed to modify the designated files in-place.
the sed expression takes the default action for each line (printing it verbatim) unless the line matches the "context" pattern given by the first script argument.
for lines that do match the context pattern,
s/\\<$2\\>/$3/ - attempt to perform the wanted substitution
the \< and \> match word start and end boundaries, respectively, so that the specified pattern will not match a partial word (though it can match multiple complete words if the target pattern allows)
t1 - if a substitution was made, then branch to label 1, otherwise
p; d - print the current line and immediately start the next cycle
:1; n; b1 - label 1 (reachable only by branching): print the current line and read the next one, then loop back to label 1. This prints the remainder of the file without any more tests or substitutions.
Example usage:
/path/to/replace_first.sh 'String system' x y
It is worth noting that that does expose the user to some details of seds interpretation of regular expressions and replacement text, though that does not manifest for the example usage.
Note that that could be simplified by removing the context pattern bit if you are sure you want to modify the overall first appearance of the target in each file. You could also hard-code the context, the target pattern, and/or the replacement text. If you hard-code all three then the script would no longer need any argument handling or checking.
I have a directory /user/reports under which many files are there, one of them is :
report.active_user.30092018.77325.csv
I need output as number after date i.e. 77325 from above file name.
I created below command to find a value from file name:
ls /user/reports | awk -F. '/report.active_user.30092018/ {print $(NF-1)}'
Now, I want current date to be passed in above command as variable and get result:
ls /user/reports | awk -F. '/report.active_user.$(date +'%d%m%Y')/ {print $(NF-1)}'
But not getting required output.
Tried bash script:
#!/usr/bin/env bash
_date=`date +%d%m%Y`
active=$(ls /user/reports | awk -F. '/report.active_user.${_date}/ {print $(NF-1)}')
echo $active
But still output is blank.
Please help with proper syntax.
As #cyrus said you must use double quotes in your variable assignment because simple quote are use only for string and not for containing variables.
Bas use case
number=10
string='I m sentence with or wihtout var $number'
echo $string
Correct use case
number=10
string_with_number="I m sentence with var $number"
echo $string_with_number
You can use simple quote but not englobe all the string
number=10
string_with_number='I m sentence with var '$number
echo $string_with_number
Don't parse ls
You don't need awk for this: you can manage with the shell's capabilities
for file in report.active_user."$(date "+%d%m%Y")"*; do
tmp=${file%.*} # remove the extension
number=${tmp##*.} # remove the prefix up to and including the last dot
echo "$number"
done
See https://www.gnu.org/software/bash/manual/bashref.html#Shell-Parameter-Expansion
Given a list of paths separated by a single space:
/home/me/src/test /home/me/src/vendor/a /home/me/src/vendor/b
I want to remove the prefix /home/me/src/ so that the result is:
test vendor/a vendor/b
For a single path I would do: ${PATH#/home/me/src/} but how do I apply it to this series?
You can use // to replace all occurrences of substring. Replace it with null string to remove them.
$ path="/home/me/src/test /home/me/src/vendor/a /home/me/src/vendor/b"
$ echo ${path//\/home\/me\/src\/}
test vendor/a vendor/b
Reference: ${parameter/pattern/string} in Bash reference manual
Using shell parameter expansion doesn't seem to be the solution for this, since it would remove everything up to / from a given point is useful, as nu11p01n73R's answer reveals.
For clarity, I would use sed with the syntax sed 's#pattern#replacement#g':
$ str="/home/me/src/test /home/me/src/vendor/a /home/me/src/vendor/b"
$ sed 's#/home/me/src/##g' <<< "$str"
test vendor/a vendor/b
Like always a grep solution from my side :
echo 'your string' | grep -Po '^/([^ /]*/)+\K.+'
Please note that the above regex do this for any string like /x/y/z/test ... But if you are interested only in replacing /home/me/src/, try the following :
echo 'your string' | grep -Po '^/home/me/src/\K.+' --color
a lot of files I download have crap/spam in their filenames, e.g.
[ www.crap.com ] file.name.ext
www.crap.com - file.name.ext
I've come up with two ways for dealing with them but they both seem pretty clunky:
with parameter expansion:
if [[ ${base_name} != ${base_name//\[+([^\]])\]} ]]
then
mv -v "${dir_name}/${base_name}" "${dir_name}/${base_name//\[+([^\]])\]}" &&
base_name="${base_name//\[+([^\]])\]}"
fi
if [[ ${base_name} != ${base_name//www.*.com - /} ]]
then
mv -v "${dir_name}/${base_name}" "${dir_name}/${base_name//www.*.com - /}" &&
base_name="${base_name//www.*.com - /}"
fi
# more of these type of statements; one for each type of frequently-encountered pattern
and then with echo/sed:
tmp=`echo "${base_name}" | sed -e 's/\[[^][]*\]//g' | sed -e 's/\s-\s//g'`
mv "${base_name}" "{tmp}"
I feel like the parameter expansion is the worse of the two but I like it because I'm able to keep the same variable assigned to the file for further processing after the rename (the above code is used in a script that's called for each file after the file download is complete).
So anyway I was hoping there's a better/cleaner way to do the above that someone more knowledgeable than myself could show me, preferably in a way that would allow me to easily reassign the old/original variable to the new/renamed file.
Thanks
Two answer: using perl rename or using pure bash
As there are some people who dislike perl, I wrote my bash only version
Renaming files by using the rename command.
Introduction
Yes, this is a typical job for rename command which was precisely designed for:
man rename | sed -ne '/example/,/^[^ ]/p'
For example, to rename all files matching "*.bak" to strip the
extension, you might say
rename 's/\.bak$//' *.bak
To translate uppercase names to lower, you'd use
rename 'y/A-Z/a-z/' *
More oriented samples
Simply drop all spaces and square brackets:
rename 's/[ \[\]]*//g;' *.ext
Rename all .jpg by numbering from 1:
rename 's/^.*$/sprintf "IMG_%05d.JPG",++$./e' *.jpg
Demo:
touch {a..e}.jpg
ls -ltr
total 0
-rw-r--r-- 1 user user 0 sep 6 16:35 e.jpg
-rw-r--r-- 1 user user 0 sep 6 16:35 d.jpg
-rw-r--r-- 1 user user 0 sep 6 16:35 c.jpg
-rw-r--r-- 1 user user 0 sep 6 16:35 b.jpg
-rw-r--r-- 1 user user 0 sep 6 16:35 a.jpg
rename 's/^.*$/sprintf "IMG_%05d.JPG",++$./e' *.jpg
ls -ltr
total 0
-rw-r--r-- 1 user user 0 sep 6 16:35 IMG_00005.JPG
-rw-r--r-- 1 user user 0 sep 6 16:35 IMG_00004.JPG
-rw-r--r-- 1 user user 0 sep 6 16:35 IMG_00003.JPG
-rw-r--r-- 1 user user 0 sep 6 16:35 IMG_00002.JPG
-rw-r--r-- 1 user user 0 sep 6 16:35 IMG_00001.JPG
Full syntax for matching SO question, in safe way
There is a strong and safe way using rename utility:
As this is perl common tool, we have to use perl syntax:
rename 'my $o=$_;
s/[ \[\]]+/-/g;
s/-+/-/g;
s/^-//g;
s/-\(\..*\|\)$/$1/g;
s/(.*[^\d])(|-(\d+))(\.[a-z0-9]{2,6})$/
my $i=$3;
$i=0 unless $i;
sprintf("%s-%d%s", $1, $i+1, $4)
/eg while
$o ne $_ &&
-f $_;
' *
Testing rule:
touch '[ www.crap.com ] file.name.ext' 'www.crap.com - file.name.ext'
ls -1
[ www.crap.com ] file.name.ext
www.crap.com - file.name.ext
rename 'my $o=$_; ...
...
...' *
ls -1
www.crap.com-file.name-1.ext
www.crap.com-file.name.ext
touch '[ www.crap.com ] file.name.ext' 'www.crap.com - file.name.ext'
ls -1
www.crap.com-file.name-1.ext
[ www.crap.com ] file.name.ext
www.crap.com - file.name.ext
www.crap.com-file.name.ext
rename 'my $o=$_; ...
...
...' *
ls -1
www.crap.com-file.name-1.ext
www.crap.com-file.name-2.ext
www.crap.com-file.name-3.ext
www.crap.com-file.name.ext
... and so on...
... and it's safe while you don't use -f flag to rename command: file won't be overwrited and you will get an error message if something goes wrong.
Renaming files by using bash and so called bashisms:
I prefer doing this by using dedicated utility, but this could even be done by using pure bash (aka without any fork)
There is no use of any other binary than bash (no sed, awk, tr or other):
#!/bin/bash
for file;do
newname=${file//[ \]\[]/.}
while [ "$newname" != "${newname#.}" ] ;do
newname=${newname#.}
done
while [ "$newname" != "${newname//[.-][.-]/.}" ] ;do
newname=${newname//[.-][.-]/-};done
if [ "$file" != "$newname" ] ;then
if [ -f $newname ] ;then
ext=${newname##*.}
basename=${newname%.$ext}
partname=${basename%%-[0-9]}
count=${basename#${partname}-}
[ "$partname" = "$count" ] && count=0
while printf -v newname "%s-%d.%s" $partname $[++count] $ext &&
[ -f "$newname" ] ;do
:;done
fi
mv "$file" $newname
fi
done
To be run with files as argument, for sample:
/path/to/my/script.sh \[*
Replacing spaces and square bracket by dot
Replacing sequences of .-, -., -- or .. by only one -.
Test if filename don't differ, there is nothing to do.
Test if a file exist with newname...
split filename, counter and extension, for making indexed newname
loop if a file exist with newname
Finaly rename the file.
Take advantage of the following classical pattern:
job_select /path/to/directory| job_strategy | job_process
where job_select is responsible for selecting the objects of your job, job_strategy prepares a processing plan for these objects and job_process eventually executes the plan.
This assumes that filenames do not contain a vertical bar | nor a newline character.
The job_select function
# job_select PATH
# Produce the list of files to process
job_select()
{
find "$1" -name 'www.*.com - *' -o -name '[*] - *'
}
The find command can examine all properties of the file maintained by the file system, like creation time, access time, modification time. It is also possible to control how the filesystem is explored by telling find not to descend into mounted filesystems, how much recursions levels are allowed. It is common to append pipes to the find command to perform more complicated selections based on the filename.
Avoid the common pitfall of including the contents of hidden directories in the output of the job_select function. For instance, the directories CVS, .svn, .svk and .git are used by the corresponding source control management tools and it is almost always wrong to include their contents in the output of the job_select function. By inadvertently batch processing these files, one can easily make the affected working copy unusable.
The job_strategy function
# job_strategy
# Prepare a plan for renaming files
job_strategy()
{
sed -e '
h
s#/www\..*\.com - *#/#
s#/\[^]]* - *#/#
x
G
s/\n/|/
'
}
This commands reads the output of job_select and makes a plan for our renaming job. The plan is represented by text lines having two fields separated by the character |, the first field being the old name of the file and the second being the new computed file of the file, it looks like
[ www.crap.com ] file.name.1.ext|file.name.1.ext
www.crap.com - file.name.2.ext|file.name.2.ext
The particular program used to produce the plan is essentially irrelevant, but it is common to use sed as in the example; awk or perl for this. Let us walk through the sed-script used here:
h Replace the contents of the hold space with the contents of the pattern space.
… Edit the contents of the pattern space.
x Swap the contents of the pattern and hold spaces.
G Append a newline character followed by the contents of the hold space to the pattern space.
s/\n/|/ Replace the newline character in the pattern space by a vertical bar.
It can be easier to use several filters to prepare the plan. Another common case is the use of the stat command to add creation times to file names.
The job_process function
# job_process
# Rename files according to a plan
job_process()
{
local oldname
local newname
while IFS='|' read oldname newname; do
mv "$oldname" "$newname"
done
}
The input field separator IFS is adjusted to let the function read the output of job_strategy. Declaring oldname and newname as local is useful in large programs but can be omitted in very simple scripts. The job_process function can be adjusted to avoid overwriting existing files and report the problematic items.
About data structures in shell programs
Note the use of pipes to transfer data from one stage to the other: apprentices often rely on variables to represent such information but it turns out to be a clumsy choice. Instead, it is preferable to represent data as tabular files or as tabular data streams moving from one process to the other, in this form, data can be easily processed by powerful tools like sed, awk, join, paste and sort — only to cite the most common ones.
You can use rnm
rnm -rs '/\[crap\]|\[spam\]//g' *.ext
The above will remove [crap] or [spam] from filename.
You can pass multiple regex pattern by terminating them with ; or overloading the -rs option.
rnm -rs '/[\[\]]//g;/\s*\[crap\]//g' -rs '/crap2//' *.ext
The general format of this replace string is /search_part/replace_part/modifier
search_part : regex to search for.
replace_part : string to replace with
modifier : i (case insensitive), g (global replace)
uppercase/lowercase:
A replace string of the form /search_part/\c/modifier will make the selected part of the filename (by the regex search_part) lowercase while \C (capital \C) in replace part will make it uppercase.
rnm -rs '/[abcd]/\C/g' *.ext
## this will capitalize all a,b,c,d in the filenames
If you have many regex patterns that need to be dealt with, then put those patterns in a file and pass the file with -rs/f option.
rnm -rs/f /path/to/regex/pattern/file *.ext
You can find some other examples here.
Note:
rnm uses PCRE2 (revised PCRE) regex.
You can undo an unwanted rename operation by running rnm -u
P.S: I am the author of this tool.
If you are using Ubunntu/Debian os use rename command to rename multiple files at time.
If you want to use something not depending on perl, you can use the following code (let's call it sanitizeNames.sh). It is only showing a few cases, but it's easily extensible using string substitution, tr (and sed too).
#!/bin/bash
ls $1 |while read f; do
newfname=$(echo "$f" \
|tr -d '\[ ' \ # Removing opened square bracket
|tr ' \]' '-' \ # Translating closing square bracket to dash
|tr -s '-' \ # Squeezing multiple dashes
|tr -s '.' \ # Squeezing multiple dots
)
newfname=${newfname//-./.}
if [ -f "$newfname" ]; then
# Some string magic...
extension=${newfname##*\.}
basename=${newfname%\.*}
basename=${basename%\-[1-9]*}
lastNum=$[ $(ls $basename*|wc -l) ]
mv "$f" "$basename-$lastNum.$extension"
else
mv "$f" "$newfname"
fi
done
And use it:
$ touch '[ www.crap.com ] file.name.ext' 'www.crap.com - file.name.ext' '[ www.crap.com ] - file.name.ext' '[www.crap.com ].file.anothername.ext2' '[www.crap.com ].file.name.ext'
$ ls -1 *crap*
[ www.crap.com ] - file.name.ext
[ www.crap.com ] file.name.ext
[www.crap.com ].file.anothername.ext2
[www.crap.com ].file.name.ext
www.crap.com - file.name.ext
$ ./sanitizeNames.sh *crap*
$ ls -1 *crap*
www.crap.com-file.anothername.ext2
www.crap.com-file.name-1.ext
www.crap.com-file.name-2.ext
www.crap.com-file.name-3.ext
www.crap.com-file.name.ext
I'm using sed for updating my JSON configuration file in the runtime.
Sometimes, when the pattern doesn't match in the JSON file, sed still exits with return code 0.
Returning 0 means successful completion, but why does sed return 0 if it doesn't find the proper pattern and update the file? Is there a workaround for that?
as #cnicutar commented, the return code of a command means if the command was executed successfully. has nothing to do with the logic you implemented in the codes/scripts.
so if you have:
echo "foo"|sed '/bar/ s/a/b/'
sed will return 0 but if you write some syntax/expression errors, or the input/file doesn't exist, sed cannot execute your request, sed will return 1.
workaround
this is actually not workaround. sed has q command: (from man page):
q [exit-code]
here you can define exit-code as you want. For example '/foo/!{q100}; {s/f/b/}' will exit with code 100 if foo isn't present, and otherwise perform the substitution f->b and exit with code 0.
Matched case:
kent$ echo "foo" | sed '/foo/!{q100}; {s/f/b/}'
boo
kent$ echo $?
0
Unmatched case:
kent$ echo "trash" | sed '/foo/!{q100}; {s/f/b/}'
trash
kent$ echo $?
100
I hope this answers your question.
edit
I must add that, the above example is just for one-line processing. I don't know your exact requirement. when you want to get exit 1. one-line unmatched or the whole file. If whole file unmatching case, you may consider awk, or even do a grep before your text processing...
This might work for you (GNU sed):
sed '/search-string/{s//replacement-string/;h};${x;/./{x;q0};x;q1}' file
If the search-string is found it will be replaced with replacement-string and at end-of-file sed will exit with 0 return code. If no substitution takes place the return code will be 1.
A more detailed explanation:
In sed the user has two registers at his disposal: the pattern space (PS) in which the current line is loaded into (minus the linefeed) and a spare register called the hold space (HS) which is initially empty.
The general idea is to use the HS as a flag to indicate if a substitution has taken place. If the HS is still empty at the end of the file, then no changes have been made, otherwise changes have occurred.
The command /search-string/ matches search-string with whatever is in the PS and if it is found to contain the search-string the commands between the following curly braces are executed.
Firstly the substitution s//replacement-string/ (sed uses the last regexp i.e. the search-string, if the lefthand-side is empty, so s//replacement-string is the same as s/search-string/replacement-string/) and following this the h command makes a copy of the PS and puts it in the HS.
The sed command $ is used to recognise the last line of a file and the following then occurs.
First the x command swaps the two registers, so the HS becomes the PS and the PS becomes the HS.
Then the PS is searched for any character /./ (. means match any character) remember the HS (now the PS) was initially empty until a substitution took place. If the condition is true the x is again executed followed by q0 command which ends all sed processing and sets the return code to 0. Otherwise the x command is executed and the return code is set to 1.
N.B. although the q quits sed processing it does not prevent the PS from being reassembled by sed and printed as per normal.
Another alternative:
sed '/search-string/!ba;s//replacement-string/;h;:a;$!b;p;x;/./Q;Q1' file
or:
sed '/search-string/,${s//replacement-string/;b};$q1' file
These answers are all too complicated. What is wrong with writing a bit of shell script that uses grep to figure out if the thing you want to replace is there then using sed to replace it?
grep -q $TARGET_STRING $file
if [ $? -eq 0 ]
then
echo "$file contains the old site"
sed -e "s|${TARGET_STRING}|${NEW_STRING}|g" ....
fi
For 1 line of input. To avoid repeating the /pattern/:
When s succeeds to substitute, use t to jump conditionally to a label, e.g. x. Otherwise use q to quit with an exit code, e.g. 100:
's/pattern/replacement/;tx;q100;:x'
Example:
$ echo 1 > one
$ < one sed 's/1/replaced-it/;tx;q1;:x'
replaced-it
$ echo $?
0
$ < one sed 's/999/replaced-it/;tx;q100;:x'
1
$ echo $?
100
https://www.gnu.org/software/sed/manual/html_node/Branching-and-flow-control.html
We have the answer above but it took some time for me work out what is happening. I am trying to provide a simple explanation for basic user of sed like me.
Lets consider the example:
echo "foo" | sed '/foo/!{q100}; {s/f/b/}'
Here we have two sed commands. First one is '/foo/!{q100}' This command actually check the pattern matching and return exist code 100 if no match. Consider following examples, -n is used to silent the output so we only get exist code.
This example foo matches so exit code return is 0
echo "foo" | sed -n '/foo/!{q100}'; echo $?
0
This example input is foo and we try match boo so no match and exit code 100 is returned
echo "foo" | sed -n '/boo/!{q100}'; echo $?
100
So if my requirement is only to check a pattern match or not I can use
echo "<input string>" | sed -n '/<pattern to match>/!{q<exit-code>}'
More examples:
echo "20200206" | sed -n '/[0-9]*/!{q100}' && echo "Matched" || echo "No Match"
Matched
echo "20200206" | sed -n '/[0-9]{2}/!{q100}' && echo "Matched" || echo "No Match"
No Match
Second command is '{s/f/b/}' is to replace the f in foo with b which I used many times.
Below is the pattern we use with sed -rn or sed -r.
The entire search and replace command ("s/.../.../...") is optional. If the search and replace is used, for speed and having already matched $matchRe, we use as fast a $searchRe value as possible, using . where the character does not need to be re-verified and .{$len} for fixed length sections of the pattern.
The return value for none found is $notFoundExit.
/$matchRe/{s/$searchRe/$replacement/$options; Q}; q$notFoundExit
For the following reasons:
No time wasted testing for both matched and unmatched case
No time wasted copying to or from buffers
No superfluous branches
Reasonable flexibility
Varying the case of Q commands will vary the behavior depending on when the exit should occur. Behaviors involving the application of Boolean logic to a multiple line input requires more complexity in the solution.
For any number of input lines:
sed --quiet 's/hello/HELLO/;t1;b2;:1;h;:2;p;${g;s/..*//;tok;q1;:ok}'
Fills hold space on match, and checks it after the last line.
Returns status 1 if no match in file.
s/hello/HELLO - substitution to check for
t1 - jump to label 1 if substitution succeeded
b2 - jump to label 2 unconditionally
:1 - label 1
h - copy pattern to hold space (when substitution succeeded)
:2 - label 2
p - print pattern space, unconditionally
${ ... } - match last line, evaluate block inside
g - copy hold space into pattern space (non-empty if first substitution succeded before)
s/..*// - dummy substitution, to set branch-flag
tok - jump to label ok (if dummy substitution succeeded on non-empty hold space)
q1 - exit with error status 1
:ok - label ok
As we already know, when sed fails to match then it simply returns its input string - no error has occurred. It is true that a difference between the input and output strings implies a match, but a match does not imply a difference in the strings; after all sed could have simply matched all of the input characters.
The flaw is created in the following example
h=$(echo "$g" | sed 's/.*\(abc[[:digit:]]\).*/\1/g')
if [ ! "$h" = "$g" ]; then
echo "1"
else
echo "2"
fi
where g=Xabc1 gives 1, while setting g=abc1 gives 2; yet both of these input strings are matched by sed! So, it can be hard to determine whether sed has matched or not. A solution:
h=$(echo "fix${g}ed" | sed 's/.*\(abc[[:digit:]]\).*/\1/g')
if [ ! "$h" = "fix${g}ed" ]; then
echo "1"
else
echo "2"
fi
in which case the 1 is printed if-and-only-if sed has matched.
I had wanted to truncate a file by quitting when the match was found (and exclude the matching line). This is handy when a process that adds lines at the end of the file may be re-run. "Q;Q1" didn't work but simply "Q1" did, as follows:
if sed -i '/text I wanted to find/Q1' file.txt
then
insert blank line at end of file + new lines
fi
insert just the new lines without the blank line