I'm attempting to find all .PHP files that are in certain depth of a directory (at least 4 levels down, but not more than 5 levels in).
I'm logged into my Centos server with root authority via shell.
The string I want to search for is:
$slides='';
What I have in front of me.. I would expect it to work. I tried to escape the $ with a \ (I thought perhaps it works like regex, needing special chars excluded). I tried without the ='' portion, or tried adding \'\' to that part.. or remove the ='' altogether to simplify. nothing.
find . -maxdepth 5 -mindepth 4 -type f -name ‘*.php’ -print | xargs grep "\$slides=’’" *
I'm already running it under the directory under which I want to recursively search.
Also - I have the filter to look for *.php only but I still get a bunch of directory names in the return with a warning that says grep: [dir_name]: Is a directory
Clearly I am missing something here as far as syntax of grep command goes, or how the filter works here. I use grep more in PHP so this is quite a transition for me!
So you were almost right. The problem looks to have been the grep part of the command
grep "\$slides=''" *
Namely the * was the issue. From the bash manual
After word splitting, unless the -f option has been set (see The Set
Builtin), Bash scans each word for the characters ‘*’, ‘?’, and ‘[’.
If one of these characters appears, and is not quoted, then the word
is regarded as a pattern, and replaced with an alphabetically sorted
list of filenames matching the pattern
When you piped the found files with find into xargs and attempted to grep them with *, grep would have interpreted this as you wanting to find the string $slides='' in a list of filenames/directories returned by the glob *, and you cannot grep directories without supplying the -r flag to grep, so it returned an error.
Instead, what you wanted to do is pipe the found files with find into xargs so it can add the list of filenames to the grep command, as that's what xargs does. From the xargs manual
xargs reads items from the standard input, delimited by blanks (which
can be protected with double or single quotes or a backslash) or
newlines, and executes the command (default is /bin/echo) one or more
times with any initial- arguments followed by items read from
standard input. Blank lines on the standard input are ignored.
Making the correct command
find . -maxdepth 5 -mindepth 4 -type f -name '*.php' -print0 | xargs -0 grep "\$slides=''"
Using the -print0 flag in find, and the -0 flag in xargs, to use NUL as the delimiter, in case any filenames contained newlines.
If you want to use shell_exec from your PHP code, it is a program execution function which allows you to run a command like 'ls -al' in the operating system shell and get the result returned into a variable. Querystrings are not commands you can use in this way.
Do you mean running PHP from the command line so that it runs from the shell, not from the web server:
php -r 'echo "hello world\n";'
If you run PHP 4.3 and above, you can use the PHP Command Line Interface (CLI) which can also execute scripts stored in files. Have a look at the syntax and examples at: http://php.net/features.commandline
I have multiple files in a directory that are in .aiff format, and I would like to convert them to .wav using SoX. I have tried the code on this website, which is as follows
theFiles = `/Users/me/RainbowAiff/*.aiff`;
for eachFile in $theFiles; do v1=${eachFile%*.aiff};
oldFile="$v1.aiff"; newFile = "$v1.wav";
echo oldFile $oldFile; echo newFile $newFile; sox $oldFile $newFile; done
and this website, which is as follows
for i in `/Users/me/RainbowAiff/ *.aiff`; do echo -e "$i"; sox $i $i.wav; echo -e "$i.wav"; done;
but I get an error message in both instances that says "cannot execute binary file". What could be the source of this error?
Incompatible binary files cause this error, see this question.
If the outputs from file sox and uname -a commands tell that there is a discrepancy between the binary file and your operating system, i.e., x86 vs. ARM, you need to find the correct binary for SoX. Although there is no SoX release since 2015, check this and this to find a version that is compatible with your system.
The problem is also related to the file attributes of the sox. In order to give it the executable attribute, simply go to the folder where sox is located and run chmod +x sox command (you need root access first).
The two snippets you linked to looks a little sketchy to me with their use of ls and echo.
echo -e might in fact be what gave you the syntax error, as OSX (BSD) echo does not have an -e option. (type man echo into the terminal to see).
I took the liberty of assembling an alternative one-liner, based around find and its -exec option.
find -E . -maxdepth 1 -iregex '.*\.(aif|aiff)' -exec bash -c '$0 "$1" "${1%.*}.wav"' sox {} \;
To explain it:
find: can find about anything
-E: enables extended regex*
.: starts in current working directory. Can be replaced with relative or absolute path
-maxdepth 1: looks only in the first directory layer. Replace 1 with 2 to also find files within folders, with 3 to also find files within folders within folders …
Removing this option will enable full recursion (find files all the way down)
-iregex: use case-insensitive regular expressions
'.*\.(aif|aiff)': match anything that starts with anything (.) at any length (*), followed by a period (\.) and the strings 'aif' or 'aiff' (or 'AIF', 'AIFF', 'aiFF' …) ((aif|aiff))
-exec bash -c : execute the following in a non-interactive bash shell
Right around here it gets a bit more complicated
'$0 "$1" "${1%.*}.wav"': this is the call to be run inside the bash shell. $0 will expand to the program name. "$1" will expand to the first (and in this case, only) argument, enclosed in double quotes in case of white-spaces. "${1%.*}.wav" will also expand to the first argument, except it will strip away the last period and everything after, before tagging on '.wav' at the end. Effectively replacing the file extension.
sox {}: these are the arguments passed to the shell call. The first (sox) is the program we want to use, referred to as $0 within the call. The second ({}) is whatever file find has found, and referred to as $1 within the shell call
That's quite a mouthful and I'm no expert, so there might be some mistakes in what I've written, though the general outline should be solid.
*This is OSX specific, a less pretty but more portable option would be:
find . -maxdepth 1 \( -iname "*.aif" -o -iname "*.aiff" \) -exec bash -c '$0 "$1" "${1%.*}.wav"' sox {} \;
Somehow, at some point, I accidentally created a file in my home directory named '-s'. It is about 500 kb and I have no idea if it contains important data or not. I cannot figure out any way to do anything with this file, because every command I use to try to view, copy, or move it interprets the filename as an argument.
I've tried putting it in quotes, escaping it with a backslash, a combination of the two, nothing seems to work.
Also, when I first posed this question to my coworkers, we puzzled over it for a while until someone finally overheard and asked "why don't you just rename it?" After I explained to him that cp and mv both think the filename is an argument so it doesn't work, he said "no, not from the command line, do it from Gnome." I sheepishly followed his advice, and it worked. HOWEVER I'm still interested in how you would solve this dilemma if you didn't have a window manager and the command line was the only option.
You can refer to it either using ./-filename or some command will allow you to put it after double dash:
rm -- -filename
You can get rid of it with:
rm ./-s
The rm command (at least under Ubuntu 10.04) even tells you such:
pax#pax-desktop:~$ rm -w
rm: invalid option -- 'w'
Try `rm ./-w' to remove the file `-w'.
Try `rm --help' for more information.
The reason that works is because rm doesn't think it's an option (since it doesn't start with -) but it's still referring to the specific file in the current directory.
You could use --, e.g.:
rm -- -file
Just for fun you could also use/abuse find.
find . -name "-s" -delete
or
find . -name "-s" -exec cat {} \;
besides using rm, if you know a language, you can also use them. They are not affected by such shell warts.
Ruby(1.9+)
$ ruby -rfileutils -e 'FileUtils.rm("-s")'
or
$ ruby -e 'File.unlink("-s")'
I need a way to tell if grep does find something, and ideally pass that return value to an if statement.
Let's say I have a tmp folder (current folder), in that there are several files and sub-folders. I want to search all files named abc for a pattern xyz. The search is assumed to be successful if it finds any occurrence of xyz (it does not matter how many times xyz is found). The search fails if no occurrence of xyz is found.
In bash, it can be done like this:
find . -name "abc" -exec grep "xyz" {} \;
That would show if xyz is found at all. But I'm not sure how pass the result (successful or not) back to an if statement.
Any help would be appreciated.
You can try
x=`find . -name abc | xargs grep xyz`
echo $x
That is, x contains your return value. It is blank when there is no match.
If you want to know that find finds some files abc and that at least one of them contains the string xyz, then you can probably use:
if find . -name 'abc' -type f -exec grep -q xyz {} +
then : All invocations of grep found at least one xyz and nothing else failed
else : One or more invocations of grep failed to find xyz or something else failed
fi
This relies on find returning an exit status for its own operations, and a non-zero exit status of any of the command(s) it executes. The + at the end groups as many file names as find thinks reasonable into a single command line. You need quite a lot of file name (a large number of fairly long names) to make find run grep multiple times. On a Mac running Mac OS X 10.10.4, I got to about 3,000 files, each with about 32 characters in the name, for an argument list of just over 100 KiB, without grep being run multiple times. OTOH, when I had just under 8000 files, I got two runs of grep, with around 130 KiB of argument list for each.
Someone briefly left a comment asking whether the exit status of find is guaranteed. The answer is 'yes' — though I had to modify my description of the exit status a bit. Under the description of -exec, POSIX specifies:
If any invocation [of the 'utility', such as grep in this question] returns a non-zero value as exit status, the find utility shall return a non-zero exit status.
And under the general 'Exit status' it says:
The following exit values shall be returned:
0 — All path operands were traversed successfully.
>0 — An error occurred.
Thus, find will report success as long as none of its own operations fails and as long as none of the grep commands it invokes fails. Zero files found but no failures will be reported as success. (Failures might be lack of permission to search a directory, for example.)
find returns the result of the -exec'd command as its result, just place the command in an if statement:
if [[ -n $(find . -name "abc" -exec grep "xyz" {} \;) ]]
then
# do something
fi
The grep command can be given a -q option that will "quiet" its output. Grep will return a success or a failure on the basis of whether it found anything in the file you pointed it at.
If your shell is capable of it, rather than trying to parse the output of a find command, you might want to try using a for loop instead. For example in bash:
shopt -s globstar
cd /some/directory
for file in **/abc; do
grep -q "xyz" "$file" && echo "Found something in $file"
done
Is this what you're looking for?
You should be able to do this with just grep with --include I believe.
Like this:
if grep -r --include=abc -q xyz; then
: Found at least one match
else
: No matching lines found
fi
I'm trying to copy a bunch of files below a directory and a number of the files have spaces and single-quotes in their names. When I try to string together find and grep with xargs, I get the following error:
find .|grep "FooBar"|xargs -I{} cp "{}" ~/foo/bar
xargs: unterminated quote
Any suggestions for a more robust usage of xargs?
This is on Mac OS X 10.5.3 (Leopard) with BSD xargs.
You can combine all of that into a single find command:
find . -iname "*foobar*" -exec cp -- "{}" ~/foo/bar \;
This will handle filenames and directories with spaces in them. You can use -name to get case-sensitive results.
Note: The -- flag passed to cp prevents it from processing files starting with - as options.
find . -print0 | grep --null 'FooBar' | xargs -0 ...
I don't know about whether grep supports --null, nor whether xargs supports -0, on Leopard, but on GNU it's all good.
The easiest way to do what the original poster wants is to change the delimiter from any whitespace to just the end-of-line character like this:
find whatever ... | xargs -d "\n" cp -t /var/tmp
This is more efficient as it does not run "cp" multiple times:
find -name '*FooBar*' -print0 | xargs -0 cp -t ~/foo/bar
I ran into the same problem. Here's how I solved it:
find . -name '*FoooBar*' | sed 's/.*/"&"/' | xargs cp ~/foo/bar
I used sed to substitute each line of input with the same line, but surrounded by double quotes. From the sed man page, "...An ampersand (``&'') appearing in the replacement is replaced by the string matching the RE..." -- in this case, .*, the entire line.
This solves the xargs: unterminated quote error.
This method works on Mac OS X v10.7.5 (Lion):
find . | grep FooBar | xargs -I{} cp {} ~/foo/bar
I also tested the exact syntax you posted. That also worked fine on 10.7.5.
Just don't use xargs. It is a neat program but it doesn't go well with find when faced with non trivial cases.
Here is a portable (POSIX) solution, i.e. one that doesn't require find, xargs or cp GNU specific extensions:
find . -name "*FooBar*" -exec sh -c 'cp -- "$#" ~/foo/bar' sh {} +
Note the ending + instead of the more usual ;.
This solution:
correctly handles files and directories with embedded spaces, newlines or whatever exotic characters.
works on any Unix and Linux system, even those not providing the GNU toolkit.
doesn't use xargs which is a nice and useful program, but requires too much tweaking and non standard features to properly handle find output.
is also more efficient (read faster) than the accepted and most if not all of the other answers.
Note also that despite what is stated in some other replies or comments quoting {} is useless (unless you are using the exotic fishshell).
Look into using the --null commandline option for xargs with the -print0 option in find.
For those who relies on commands, other than find, eg ls:
find . | grep "FooBar" | tr \\n \\0 | xargs -0 -I{} cp "{}" ~/foo/bar
find | perl -lne 'print quotemeta' | xargs ls -d
I believe that this will work reliably for any character except line-feed (and I suspect that if you've got line-feeds in your filenames, then you've got worse problems than this). It doesn't require GNU findutils, just Perl, so it should work pretty-much anywhere.
I have found that the following syntax works well for me.
find /usr/pcapps/ -mount -type f -size +1000000c | perl -lpe ' s{ }{\\ }g ' | xargs ls -l | sort +4nr | head -200
In this example, I am looking for the largest 200 files over 1,000,000 bytes in the filesystem mounted at "/usr/pcapps".
The Perl line-liner between "find" and "xargs" escapes/quotes each blank so "xargs" passes any filename with embedded blanks to "ls" as a single argument.
Frame challenge — you're asking how to use xargs. The answer is: you don't use xargs, because you don't need it.
The comment by user80168 describes a way to do this directly with cp, without calling cp for every file:
find . -name '*FooBar*' -exec cp -t /tmp -- {} +
This works because:
the cp -t flag allows to give the target directory near the beginning of cp, rather than near the end. From man cp:
-t, --target-directory=DIRECTORY
copy all SOURCE arguments into DIRECTORY
The -- flag tells cp to interpret everything after as a filename, not a flag, so files starting with - or -- do not confuse cp; you still need this because the -/-- characters are interpreted by cp, whereas any other special characters are interpreted by the shell.
The find -exec command {} + variant essentially does the same as xargs. From man find:
-exec command {} +
This variant of the -exec action runs the specified command on
the selected files, but the command line is built by appending
each selected file name at the end; the total number of invoca‐
matched files. The command line is built in much the same way
that xargs builds its command lines. Only one instance of `{}'
is allowed within the command, and (when find is being invoked
from a shell) it should be quoted (for example, '{}') to protect
it from interpretation by shells. The command is executed in
the starting directory. If any invocation returns a non-zero
value as exit status, then find returns a non-zero exit status.
If find encounters an error, this can sometimes cause an immedi‐
ate exit, so some pending commands may not be run at all. This
variant of -exec always returns true.
By using this in find directly, this avoids the need of a pipe or a shell invocation, such that you don't need to worry about any nasty characters in filenames.
With Bash (not POSIX) you can use process substitution to get the current line inside a variable. This enables you to use quotes to escape special characters:
while read line ; do cp "$line" ~/bar ; done < <(find . | grep foo)
Be aware that most of the options discussed in other answers are not standard on platforms that do not use the GNU utilities (Solaris, AIX, HP-UX, for instance). See the POSIX specification for 'standard' xargs behaviour.
I also find the behaviour of xargs whereby it runs the command at least once, even with no input, to be a nuisance.
I wrote my own private version of xargs (xargl) to deal with the problems of spaces in names (only newlines separate - though the 'find ... -print0' and 'xargs -0' combination is pretty neat given that file names cannot contain ASCII NUL '\0' characters. My xargl isn't as complete as it would need to be to be worth publishing - especially since GNU has facilities that are at least as good.
For me, I was trying to do something a little different. I wanted to copy my .txt files into my tmp folder. The .txt filenames contain spaces and apostrophe characters. This worked on my Mac.
$ find . -type f -name '*.txt' | sed 's/'"'"'/\'"'"'/g' | sed 's/.*/"&"/' | xargs -I{} cp -v {} ./tmp/
If find and xarg versions on your system doesn't support -print0 and -0 switches (for example AIX find and xargs) you can use this terribly looking code:
find . -name "*foo*" | sed -e "s/'/\\\'/g" -e 's/"/\\"/g' -e 's/ /\\ /g' | xargs cp /your/dest
Here sed will take care of escaping the spaces and quotes for xargs.
Tested on AIX 5.3
I created a small portable wrapper script called "xargsL" around "xargs" which addresses most of the problems.
Contrary to xargs, xargsL accepts one pathname per line. The pathnames may contain any character except (obviously) newline or NUL bytes.
No quoting is allowed or supported in the file list - your file names may contain all sorts of whitespace, backslashes, backticks, shell wildcard characters and the like - xargsL will process them as literal characters, no harm done.
As an added bonus feature, xargsL will not run the command once if there is no input!
Note the difference:
$ true | xargs echo no data
no data
$ true | xargsL echo no data # No output
Any arguments given to xargsL will be passed through to xargs.
Here is the "xargsL" POSIX shell script:
#! /bin/sh
# Line-based version of "xargs" (one pathname per line which may contain any
# amount of whitespace except for newlines) with the added bonus feature that
# it will not execute the command if the input file is empty.
#
# Version 2018.76.3
#
# Copyright (c) 2018 Guenther Brunthaler. All rights reserved.
#
# This script is free software.
# Distribution is permitted under the terms of the GPLv3.
set -e
trap 'test $? = 0 || echo "$0 failed!" >& 2' 0
if IFS= read -r first
then
{
printf '%s\n' "$first"
cat
} | sed 's/./\\&/g' | xargs ${1+"$#"}
fi
Put the script into some directory in your $PATH and don't forget to
$ chmod +x xargsL
the script there to make it executable.
bill_starr's Perl version won't work well for embedded newlines (only copes with spaces). For those on e.g. Solaris where you don't have the GNU tools, a more complete version might be (using sed)...
find -type f | sed 's/./\\&/g' | xargs grep string_to_find
adjust the find and grep arguments or other commands as you require, but the sed will fix your embedded newlines/spaces/tabs.
I used Bill Star's answer slightly modified on Solaris:
find . -mtime +2 | perl -pe 's{^}{\"};s{$}{\"}' > ~/output.file
This will put quotes around each line. I didn't use the '-l' option although it probably would help.
The file list I was going though might have '-', but not newlines. I haven't used the output file with any other commands as I want to review what was found before I just start massively deleting them via xargs.
I played with this a little, started contemplating modifying xargs, and realised that for the kind of use case we're talking about here, a simple reimplementation in Python is a better idea.
For one thing, having ~80 lines of code for the whole thing means it is easy to figure out what is going on, and if different behaviour is required, you can just hack it into a new script in less time than it takes to get a reply on somewhere like Stack Overflow.
See https://github.com/johnallsup/jda-misc-scripts/blob/master/yargs and https://github.com/johnallsup/jda-misc-scripts/blob/master/zargs.py.
With yargs as written (and Python 3 installed) you can type:
find .|grep "FooBar"|yargs -l 203 cp --after ~/foo/bar
to do the copying 203 files at a time. (Here 203 is just a placeholder, of course, and using a strange number like 203 makes it clear that this number has no other significance.)
If you really want something faster and without the need for Python, take zargs and yargs as prototypes and rewrite in C++ or C.
You might need to grep Foobar directory like:
find . -name "file.ext"| grep "FooBar" | xargs -i cp -p "{}" .
If you are using Bash, you can convert stdout to an array of lines by mapfile:
find . | grep "FooBar" | (mapfile -t; cp "${MAPFILE[#]}" ~/foobar)
The benefits are:
It's built-in, so it's faster.
Execute the command with all file names in one time, so it's faster.
You can append other arguments to the file names. For cp, you can also:
find . -name '*FooBar*' -exec cp -t ~/foobar -- {} +
however, some commands don't have such feature.
The disadvantages:
Maybe not scale well if there are too many file names. (The limit? I don't know, but I had tested with 10 MB list file which includes 10000+ file names with no problem, under Debian)
Well... who knows if Bash is available on OS X?