I want to search for this line:
<script type="text/javascript" src="//example.com/file.php?zoneid=112" async data-cfasync="false"></script>
And remove it/replace it with blank.
What command line could I run on linux to search and replace this term on all files on the server?
I have used a combination of grep and sed
string_to_replace='<script type="text/javascript" src="//example.com/file.php?zoneid=112" async data-cfasync="false"></script>'
First just run the grep statement to make sure you are matching the right string and file. If everything looks good, then run the following command;
grep -Inr "${string_to_replace}" | while read line; do filename=`echo $line | cut -d ":" -f 1`; line_no=`echo $line | cut -d ":" -f 2`; sed "${line_no}d" -i $filename; done
In grep, -I will ignore binaries, -n will print the line number and -r will do a recursive search.
Let me know if something is not working.
You can use sed with regex and command d over the file:
sed -i '/regex/d' file
Next construction could help you to pass filenames with find to sed:
find . | xargs sed -i '/regex/d'
Be careful with debugging. :)
Related
I've 95 files that looks like :
2019-10-29-18-00/dev/xx;512.00;0.4;/var/x/xx/xxx
2019-10-29-18-00/dev/xx;512.00;0.68;/xx
2019-10-29-18-00/dev/xx;512.00;1.84;/xx/xx/xx
2019-10-29-18-00/dev/xx;512.00;80.08;/opt/xx/x
2019-10-29-18-00/dev/xx;20480.00;83.44;/var/x/x
2019-10-29-18-00/dev/xx;3584.00;840.43;/var/xx/x
2019-10-30-00-00/dev/xx;2048.00;411.59;/
2019-10-30-00-00/dev/xx;7168.00;6168.09;/usr
2019-10-30-00-00/dev/xx;3072.00;1036.1;/var
2019-10-30-00-00/dev/xx;5120.00;348.72;/tmp
2019-10-30-00-00/dev/xx;20480.00;2033.19;/home
2019-10-30-12-00;/dev/xx;5120.00;348.72;/tmp
2019-10-30-12-00;/dev/hd1;20480.00;2037.62;/home
2019-10-30-12-00;/dev/xx;512.00;0.43;/xx
2019-10-30-12-00;/dev/xx;3584.00;794.39;/xx
2019-10-30-12-00;/dev/xx;512.00;0.4;/var/xx/xx/xx
2019-10-30-12-00;/dev/xx;512.00;0.68;/xx
2019-10-30-12-00;/dev/xx;512.00;1.84;/var/xx/xx
2019-10-30-12-00;/dev/xx;512.00;80.08;/opt/xx/x
2019-10-30-12-00;/dev/xx;20480.00;83.44;/var/xx/xx
2019-10-30-12-00;/dev/x;3584.00;840.43;/var/xx/xx
For some lines I've 2019-10-29-18-00/dev and for some other lines, I've 2019-10-30-12-00;/dev/
I want to add the ; before the /dev/ where it is missing, so for that I use this sed command :
sed 's/\/dev/\;\/dev/'
But How I can apply this command for each lines where the ; is missing ? I try this :
for i in $(cat /home/xxx/xxx/xxx/*.txt | grep -e "00/dev/")
do
sed 's/\/dev/\;\/dev/' $i > $i
done
But it doesn't work... Can you help me ?
Could you please try following with GNU awkif you are ok with it.
awk -i inplace '/00\/dev\//{gsub(/00\/dev\//,"/00;/dev/")} 1' *.txt
sed solution: Tested with GNU sed for few files and it worked fine.
sed -i.bak '/00\/dev/s/00\/dev/00\;\/dev/g' *.txt
This might work for you (GNU sed & parallel):
parallel -q sed -i 's#;*/dev#;/dev#' ::: *.txt
or if you prefer:
sed -i 's#;*/dev#;/dev#' *.txt
Ignore lines with ;/dev.
sed '/;\/dev/{p;d}; s^/dev^;/dev^'
The /;\/dev/ check if the line has ;/dev. If it has ;/dev do: p - print the current line and d - start from the beginning.
You can use any character with s command in sed. Also, there is no need in escaping \;, just ;.
How I can apply this command for each lines where the ; is missing ? I try this
Don't edit the same file redirecting to the same file $i > $i. Think about it. How can you re-write and read from the same file at the same time? You can't, the resulting file will be in most cases empty, as the > $i will "execute" first making the file empty, then sed $i will start running and it will read an empty file. Use a temporary file sed ... "$i" > temp.txt; mv temp.txt "$i" or use gnu extension -i sed option to edit in place.
What you want to do really is:
grep -l '00/dev/' /home/xxx/xxx/xxx/*.txt |
xargs -n1 sed -i '/;\/dev/{p;d}; s^/dev^;/dev^'
grep -l prints list of files that match the pattern, then xargs for each single one -n1 of the files executes sed which -i edits files in place.
grep for filtering can be eliminated in your case, we can accomplish the task with a single sed command:
for f in $(cat /home/xxx/xxx/xxx/*.txt)
do
[[ -f "$f" ]] && sed -Ei '/00\/dev/ s/([^;])(\/dev)/\1;\2/' "$f"
done
The easiest way would be to adjust your regex so that it's looking a bit wider than '/dev/', e.g.
sed -i -E 's|([0-9])/dev|\1;/dev|'
(note that I'm taking advantage of sed's flexible approach to delimiters on substitute. Also, -E changes the group syntax)
Alternatively, sed lets you filter which lines it handles:
sed -i '/[0-9]\/dev/ s/\/dev/;/dev/'
This uses the same substitution you already have but only applied on lines that match the filter regex
I need to convert my all url relative path
https://www.example.com/image/abc.gif to /image/abc.gif
i tried this command but not worked for https:\\ section . How can i use https\\ in this command .
grep -rl "http://www.example.com" /root/ | xargs sed -i 's/http://www.example.com//g'
The following will do the job:
echo "https://www.example.com/image/abc.gif" | sed 's/https:\/\/www\.example\.com//g'
OUTPUT
/image/abc.gif
You can use regular expressions, the -r indicates extended regexes. The ? says 0 or 1 occurrence of 's'.
echo "http://www.example.com/image/abc.gif" | sed -r 's/https?:\/\/www\.example\.com//g'
I have got the following Problem.
I´m doing a grep like:
$command = grep -r -i --include=*.cfg 'host{' /omd/sites/mesh/etc/icinga/conf.d/objects
I got the following output:
/omd/sites/mesh/etc/icinga/conf.d/objects/testsystem/test1.cfg:define host{
/omd/sites/mesh/etc/icinga/conf.d/objects/testsystem/test2.cfg:define host{
/omd/sites/mesh/etc/icinga/conf.d/objects/testsystem/test3.cfg:define host{
...
for all *.cfg files.
With exec($command,$array)
I passed the result in an array.
Is it possible to get only the filenames as result of the grep-command.
I have tried the following:
$Command= grep -l -H -r -i --include=*.cfg 'host{' /omd/sites/mesh/etc/icinga/conf.d/objects
but I got the same result.
I know that on the forum a similar topic exists.(How can I use grep to show just filenames (no in-line matches) on linux?), but the solution doesn´t work.
With "exec($Command,$result_array)" I try to get an array with the results.
The mentioned solutions works all, but I can´t get an resultarray with exec().
Can anyone help me?
Yet another simpler solution:
grep -l whatever-you-want | xargs -L 1 basename
or you can avoid xargs and use a subshell instead, if you are not using an ancient version of the GNU coreutils:
basename -a $(grep -l whatever-you-want)
basename is the bash straightforward solution to get a file name without path. You may also be interested in dirname to get the path only.
GNU Coreutils basename documentation
Is it possible to get only the filenames as result of the grep command.
With grep you need the -l option to display only file names.
Using find ... -execdir grep ... \{} + you might prevent displaying the full path of the file (is this what you need?)
find /omd/sites/mesh/etc/icinga/conf.d/objects -name '*.cfg' \
-execdir grep -r -i -l 'host{' \{} +
In addition, concerning the second part of your question, to read the result of a command into an array, you have to use the syntax: IFS=$'\n' MYVAR=( $(cmd ...) )
In that particular case (I formatted as multiline statement in order to clearly show the structure of that expression -- of course you could write as a "one-liner"):
IFS=$'\n' MYVAR=(
$(
find objects -name '*.cfg' \
-execdir grep -r -i -l 'host{' \{} +
)
)
You have then access to the result in the array MYVAR as usual. While I while I was testing (3 matches in that particular case):
sh$ echo ${#MYVAR[#]}
3
sh$ echo ${MYVAR[0]}
./x y.cfg
sh$ echo ${MYVAR[1]}
./d.cfg
sh$ echo ${MYVAR[2]}
./e.cfg
# ...
This should work:
grep -r -i --include=*.cfg 'host{' /omd/sites/mesh/etc/icinga/conf.d/objects | \
awk '{print $1}' | sed -e 's|[^/]*/||g' -e 's|:define$||'
The awk portion finds the first field in it and the sed command trims off the path and the :define.
I need some assistance trying to build up a variable using a list of exclusions in a file.
So I have a exclude file I am using for rsync that looks like this:
*.log
*.out
*.csv
logs
shared
tracing
jdk*
8.6_Code
rpsupport
dbarchive
inarchive
comms
PR116PICL
**/lost+found*/
dlxwhsr*
regression
tmp
working
investigation
Investigation
dcsserver_weblogic_
dcswebrdtEAR_weblogic_
I need to build up a string to be used as a variable to feed into egrep -v, so that I can use the same exclusion list for rsync as I do when egrep -v from a find -ls.
So I have created this so far to remove all "*" and "/" - and then when it sees certain special characters it escapes them:
cat exclude-list.supt | while read line
do
echo $line | sed 's/\*//g' | sed 's/\///g' | 's/\([.-+_]\)/\\\1/g'
What I need the ouput too look like is this and then export that as a variable:
SEXCLUDE_supt="\.log|\.out|\.csv|logs|shared|PR116PICL|tracing|lost\+found|jdk|8\.6\_Code|rpsupport|dbarchive|inarchive|comms|dlxwhsr|regression|tmp|working|investigation|Investigation|dcsserver\_weblogic\_|dcswebrdtEAR\_weblogic\_"
Can anyone help?
A few issues with the following:
cat exclude-list.supt | while read line
do
echo $line | sed 's/\*//g' | sed 's/\///g' | 's/\([.-+_]\)/\\\1/g'
Sed reads files line by line so cat | while read line;do echo $line | sed is completely redundant also sed can do multiple substitutions by either passing them as a comma separated list or using the -e option so piping to sed three times is two too many. A problem with '[.-+_]' is the - is between . and + so it's interpreted as a range .-+ when using - inside a character class put it at the end beginning or end to lose this meaning like [._+-].
A much better way:
$ sed -e 's/[*/]//g' -e 's/\([._+-]\)/\\\1/g' file
\.log
\.out
\.csv
logs
shared
tracing
jdk
8\.6\_Code
rpsupport
dbarchive
inarchive
comms
PR116PICL
lost\+found
dlxwhsr
regression
tmp
working
investigation
Investigation
dcsserver\_weblogic\_
dcswebrdtEAR\_weblogic\_
Now we can pipe through tr '\n' '|' to replace the newlines with pipes for the alternation ready for egrep:
$ sed -e 's/[*/]//g' -e 's/\([._+-]\)/\\\1/g' file | tr "\n" "|"
\.log|\.out|\.csv|logs|shared|tracing|jdk|8\.6\_Code|rpsupport|dbarchive|...
$ EXCLUDE=$(sed -e 's/[*/]//g' -e 's/\([._+-]\)/\\\1/g' file | tr "\n" "|")
$ echo $EXCLUDE
\.log|\.out|\.csv|logs|shared|tracing|jdk|8\.6\_Code|rpsupport|dbarchive|...
Note: If your file ends with a newline character you will want to remove the final trailing |, try sed 's/\(.*\)|/\1/'.
This might work for you (GNU sed):
SEXCLUDE_supt=$(sed '1h;1!H;$!d;g;s/[*\/]//g;s/\([.-+_]\)/\\\1/g;s/\n/|/g' file)
This should work but I guess there are better solutions. First store everything in a bash array:
SEXCLUDE_supt=$( sed -e 's/\*//g' -e 's/\///g' -e 's/\([.-+_]\)/\\\1/g' exclude-list.supt)
and then process it again to substitute white space:
SEXCLUDE_supt=$(echo $SEXCLUDE_supt |sed 's/\s/|/g')
Here's what output looks like, basically:
? RESTRequestParamObj.cpp
? plugins/dupfields2/_DupFields.cpp
? plugins/dupfields2/_DupFields.h
I need to get the filenames from second column and pass them to rm. There's AWK script that goes like awk '{print $2}' but I was wondering if there's another solution.
If you have spaces between the ? and the filename then:
cut -c9-
If they're tabs then:
cut -f2
Placed your output in file
$> cat ./text
? RESTRequestParamObj.cpp
? plugins/dupfields2/_DupFields.cpp
? plugins/dupfields2/_DupFields.h
Edit it with sed
$> cat ./text | sed -r -e 's/(\?[\ \t]*)(.*)/\2/g'
RESTRequestParamObj.cpp
plugins/dupfields2/_DupFields.cpp
plugins/dupfields2/_DupFields.h
Sed in here is matching 2 parts of line -
? with tabs or spaces
Other characters until the end f the line
And then it changes whole line only with second part.
This might work for you:
echo "? RESTRequestParamObj.cpp" | sed -e 's/^\S\+/rm /' | sh
or using GNU sed
echo "? RESTRequestParamObj.cpp"| sed -r 's/^\S+/rm /e'
bash only solution, assuming your output comes from stdin:
while read line; do echo ${line##* }; done
use cut/perl instead
cut -f2 -t'\t'|xargs rm -rf
<your output>|perl -ne '#cols = split /\t/; print $cols[1]'|xargs rm -rf