Say I have the below line in file named "logs_test":
Sample input:
"at 10947 usecs after Tue Feb 23 18:29:46 2021 [119] init: Event=populatedonRestart"
I wanted to find a string between "at" and "usecs" and add the string before "2021" in the above line
sample output:
"at 10947 usecs after Tue Feb 23 18:29:46 10947 2021 [119] init: Event=populatedonRestart"
sed command to find a string between two matching patterns:
sed "s/at//;s/usecs.*//“ <file_name>
sed command to add a string before a pattern:
sed 's/2021/string &/g' <file_name>
How can I accomplish two tasks using one sed command? Is there were to use the sed command inside sed to do this ?
This will do it for your example (with GNU sed):
sed 's/^\(at \)\(.* \)\(usecs.*\)\(2021.*\)/\1\2\3\2\4/' your_file
The ways it's working is as follows:
I remember the stuff in between \( and \) (these are called capture groups)
I break the string into 4 capture groups, the 2nd capture group is the
number you care about. And I put the groups back together and use the 2nd
capture group twice: /\1\2\3\2\4/
Once you've confirmed it does what you want, you could add the -i to do the
replacement in-place:
sed -i 's/^\(at \)\(.* \)\(usecs.*\)\(2021.*\)/\1\2\3\2\4/' your_file
Related
Let's suppose a file with following content named file.txt is used as a source.
12345
ABC-12
XCD-13
BCD-12345
Some text on this line
*+-%&
What I am looking for with sed would be to get only ABC-12 line and remove hyphen so the desired output would be ABC12
So far I have been able to get numbers only with following command:
sed "s/[^0-9]//g" file.txt
Output
12345
12
13
12345
I got the closest with grep with command:
grep -oP 'ABC-\b\d{2}\b' file.txt
Output
ABC-12
How should this be constructed with sed and also hyphen removed from the output?
Note also that numbers after ABC can be considered as changing like a variable so the idea would be to search for "ABC and numbers following it after hyphen" instead of searching for "ABC-12" directly.
idea would be to search for ABC and numbers following it after hyphen
You may use this sed for this:
sed '/^ABC-[0-9][0-9]$/!d; s/-//' file
ABC12
Here:
/^ABC-[0-9][0-9]$/!d: Searches input for a line that starts with ABC followed by a hyphen followed by 2 digits and end. All non-matching lines are removed due to !d as command.
s/-//p: Removes - from the match
Update:
As per comment below if ABC- text is not at the start then use:
sed -nE 's/.*(ABC)-([0-9][0-9]).*/\1\2/p' file
You can use
sed -En '/^.*(ABC)-([0-9]+).*/{s//\1\2/p;q}'
Details:
-En - E enables POSIX ERE and n suppresses default line output
^.*(ABC)-([0-9]+).* - start of string (here, line), then any text, then ABC captured into Group 1, -, then one or more digits (captured into Group 2) and then the rest of the line
{s//\1\2/p;q} - if there was a match
s//\1\2/p - take the same pattern and replace with Group 1 + Group 2 text, and print the result of the substitution
q - quit (you only need the first match here).
See the online demo:
#!/bin/bash
s='ABC-12
XCD-13
BCD-12345
Some text on this line
*+-%&'
sed -En '/^.*(ABC)-([0-9]+).*/{s//\1\2/p;q}' <<< "$s"
Output:
ABC12
I have scenario where i want to replace the digit or string after equal to = to null [ blank ]
Without creating new file need to replace in the same file
Data in file :
name=vilas
age=21
code=1345
Need to replace the digit after code=1345 to code=
I have tried this but stuck
sed -i 's/^code=$/code=/1g' file.txt
Note : The value after code= is going to be dynamic need to use regex pattern match which i am not good with
var1=456
$ sed -Ei "s/(code=).*/\1$var1/" file
name=vilas
age=21
code=456
Based on your question:
On MacOS and FreeBSD
sed -i '' -E 's/^code=[[:digit:]]+$/code=/g' test1.txt
On CentOS and Debian
sed -i -E 's/^code=[[:digit:]]+$/code=/g' test1.txt
You can update the character class to fit your needs.
where
-i Edit files in-place
-E Interpret regular expressions as extended (modern) regular expressions rather than basic regular expressions (BRE's).
s The substitute command
g Apply the replacement to all matches to the regexp, not just the first.
GNU docs: https://www.gnu.org/software/sed/manual/html_node/Character-Classes-and-Bracket-Expressions.html
SED docs: https://www.gnu.org/software/sed/manual/sed.html
I am trying to remove the Who: ,What: ,When: ,Where: and the proceeding space from each line in a text file. Below is a example:
Who: Tester1+Password
What: Authentication Success
When: Tues March 20, 2015 08:15:02 UTD
Where: 198.192.1.2
If you want to remove anything upto first :, then you can do:
sed -re 's/(^[^:]+: )(.*)/\2/' file
Tester1+Password
Authentication Success
Tues March 20, 2015 08:15:02 UTD
198.192.1.2
As Glenn suggested we can avoid capture groups completely by removing the portion we don't need.
sed 's/[^:]\+: //' file
Use cut to exclude the first space-separated field:
cut -d " " -f 2- file
sed 's/^[^ ]* *//' YourFile
assuming only these 4 word are possible and as your sample
Put the pattern which matches What, where, when , who inside a capturing group followed by a colon. Then replace the matched chars with an empty string. Add i flag at the last if you want do a case-insensitive match.
$ sed 's/^Wh\(ere\|en\|at\|o\):[[:blank:]]*//' file
Tester1+Password
Authentication Success
Tues March 20, 2015 08:15:02 UTD
198.192.1.2
For general case, you could use
sed 's/^[^[:blank:]]\+[[:blank:]]\+//' file
In looking for a command to delete a line (or lines) from a text file that contain a certain string.
For example
I have a text file as follows
Sat 21-12-2014,10.21,78%
Sat 21-12-2014,11.21,60%
Sun 22-12-2014,09.09,21%
I want to delete all lines that have "21-12-2014" in them.
I'm not able to find a solution that works.
According to #twalberg there is more three alternate solution for this question, which I'm explaining is as follows for future reader of this question for more versatile solutions:
With grep command
grep -v 21-12-2014 filename.txt
explanations:
-v is used to find non-matching lines
With awk command
awk '! /21-12-2014/' filename.txt
explanations:
! is denoting it will print all other lines that contain match of the string. It is not operator signify ignorance.
With sed command
sed -e '/21-12-2014/d' < filename.txt
explanations:
-e is signify scripted regex to be executed
d is denoting delete any match
< is redirecting the input file content to command
Try doing this :
sed -i.bak '/21-12-2014/d' *
A bit of explanations :
sed : the main command line, put the mouse pointer on sed
-i.bak : replace the file in place and make a backup in a .bak file
// is the regex
d means: delete
I want to search for a pattern "xxxx" in a file and delete 5 lines before this pattern and 6 lines after this match. How can i do this using Sed?
This might work for you (GNU sed):
sed ':a;N;s/\n/&/5;Ta;/xxxx/!{P;D};:b;N;s/\n/&/11;Tb;d' file
Keep a rolling window of 5 lines and on encountering the specified string add 6 more (11 in total) and delete.
N.B. This is a barebones solution and will most probably need tailoring to your specific needs. Questions such as: what if there are multiple string throughout the file? What if the string is within the first five lines or multiple strings are within five lines of each other etc etc etc.
Here's one way you could do it using awk. I assume that you also want to delete the line itself and that the file is small enough to fit into memory:
awk '{a[NR]=$0}/xxxx/{f=NR}END{for(i=1;i<=NR;++i)if(i<f-5||i>f+6)print a[i]}' file
Store every line into the array a. When the pattern /xxxx/ is matched, save the line number. After the whole file has been processed, loop through the array, only printing the lines you want to keep.
Alternatively, you can use grep to obtain the line number first:
grep -n 'xxxx' file | awk -F: 'NR==FNR{f=$1}NR<f-5||NR>f+6' - file
In both cases, the lines deleted will be surrounding the last line where the pattern is matched.
A third option would be to use grep to obtain the line number then use sed to delete the lines:
line=$(grep -nm1 'xxxx' file | cut -d: -f1)
sed "$((line-5)),$((line+6))d" file
In this case I've also added the -m switch so grep exits after finding the first match.
if you know, the line number (what is not difficult to obtain), you can use something like that:
filename="test"
start=`expr $curr_line - 5`
end=`expr $curr_line + 6`
sed "${start},${end}d" $filename (optionally sed -i)
of course, you have to remember about additional conditions like start shouldn't be less than 1 and end greater than number of lines in file.
Another - maybe more easy to follow - solution would be to use grep to find the keyword and the corresponding line:
grep -n 'KEYWORD' <file>
then use sed to get the line number only like this:
grep -n 'KEYWORD' <file> | sed 's/:.*//'
Now that you have the line number simply use sed like this:
sed -i "$(LINE_START),$(LINE_END) d" <file>
to remove lines before and/or after! With only the -i you will override the <file> (no backup).
A script example could be:
#!/bin/bash
KEYWORD=$1
LINES_BEFORE=$2
LINES_AFTER=$3
FILE=$4
LINE_NO=$(grep -n $KEYWORD $FILE | sed 's/:.*//' )
echo "Keyword found in line: $LINE_NO"
LINE_START=$(($LINE_NO-$LINES_BEFORE))
LINE_END=$(($LINE_NO+$LINES_AFTER))
echo "Deleting lines $LINE_START to $LINE_END!"
sed -i "$LINE_START,$LINE_END d" $FILE
Please note that this will work only if the keyword is found once! Adapt the script to your needs!