Is there a way to use shell globbing to identify nested directories?
so if I have dir/dir1/dir2/dir3/dir4/dir5/.. and I have files under all of them, what is the equivalent globbing pattern to match all files under all directories, similar to - for example - ls -R
In Bash 4, with shopt -s globstar, and zsh you can use **/* which will include everything except hidden files. You can do shopt -s dotglob in Bash 4 or setopt dotglob in zsh to cause hidden files to be included.
In ksh, set -o globstar enables it. I don't think there's a way to include dot files implicitly, but I think **/{.[^.],}* works.
Specifically about git (gitignore, gitattributes, and commands that take filenames): if the pattern contains no slash, * wildcards will match deep. If it does contain a slash, git will call fnmatch with the FNM_PATHNAME flag, and simple wildcards won't match slashes. ** to match deep isn't supported. Maybe this kind of deep matching could be more widely supported with a new FNM_STARSTAR flag, and an implementation in glibc, gnulib and other places.
If you want to act on all the files returned by find, rather than just list them, you can pipe them to xargs:
find <directory> -type f | xargs ls
But this is only for commands that don't have a recursive flag.
You may try:
**/*.*
However it'll ignore hidden files (such as .git files). Sometimes it's a life-saver.
Read more at: What expands to all files in current directory recursively? at SO
You can use tree, it will show all folders recursively.
tree <path>
There is no way to do this with vanilla Bash, however most commands accept a -R or --recursive option to tell them to descend into directories.
If you simply want to list all files located anywhere within a directory or its sub-directories, you can use find.
To recursively find files (-type f) with a given directory:
find <directory> -type f
Related
I'm looking to search for files of a specific name, modify the name to the full path and then copy the results to another folder.
Is it possible to update each find result with the full path as the file name; i.e.
./folder/subfolder/my-file.csv
becomes
folder_subfolder_my-file.csv
I am listing the files using the following and would like to script it.
find . -name my-file.csv -exec ls {} \;
Since you're using bash, you can take advantage of globstar and use a for loop:
shopt -s globstar # set globstar option
for csv in **/my-file.csv; do
echo "$csv" "${csv//\//_}"
done
shopt -u globstar # unset the option if you don't want it any more
With globstar enabled, ** does a recursive search (similar to the basic functionality of find).
"${csv//\//_}" is an example of ${var//match/replace}, which does a global replacement of all instances of match (here an escaped /) with replace.
If you're happy with the output, then change the echo to mv.
Just to demonstrate how to do this with find;
find . -type f -exec bash -c '
for file; do
f=${file#./}
cp "$file" "./${f//\//_}"
done' _ {} +
The Bash pattern expansion ${f//x/y} replaces x with y throughout. Because find prefixes each found file with the path where it was found (here, ./) we trim that off in order to avoid doing mv "./file" "._file". And because the slash is used in the parameter expansion itself, we need to backslash the slash we want the shell to interpret literally. Finally, because this parameter expansion syntax is a Bash-only extension, we use bash rather than sh.
Obviously, if you want to rename rather than copy, replace cp with mv.
If your find does not support -exec ... + this needs to be refactored somewhat (probably to use xargs); but it should be supported on any reasonably modern platform.
With perl's rename command ...
$ prename
Usage: rename [-v] [-n] [-f] perlexpr [filenames]
... you can rename multiple files by applying a regular expression. rename also accepts file names via stdin:
find ... | rename -n 's#/#_#g'
Check the results and if they are fine, remove -n.
Regarding zip, I'm looking for a way to ensure a file is added to an archive independent of what has been passed to the archive creation exclusion list.
In my instance, I've developed an application that allows users to specify their own exclusion filter when creating a zip.
However, I need to ensure a couple custom files are always added to the archive, independent of what was specified in the filter.
For example:
Filters Specified: "*.bar"
File I need to add: foo.bar
So if I execute
zip -rq -i foo.bar -x "*.bar"
foo.bar will not be included in the archive.
So it comes down to:
How can one override what is in the exclusion list for the few files I need added?
Alternatively, could one develop an exclusion expression that combines what the user specified something that effectively says omit it all but foo.bar?
A solution I've come up with is to take two passes at it - first create an archive with the exclusion list and then grow the archive by foo.bar, but I'm looking for a way to do it in a single shot.
Thanks
The -i option is used to include only the specified files. With pattern "foo.bar" we only include the file named foo.bar leaving others.
The -x option is used to exclude files. With pattern *.bar we exclude the foo.bar file that we included before.
Also you must consider that the patterns will be resolved as pathname expansion not as POSIX or Perl regular expresion. If you want do patterns more complex you can use extended patterns.
By example:
shopt -s extglob
zip -rq foo.zip . -i ?(foo.bar|!(*.bar))
shopt -u extglob
Or:
shopt -s extglob
zip -rq foo.zip . -x !(!(*.bar)|foo.bar)
shopt -u extglob
But the last commands aren't recursive because the patterns doesn't contemplate subdirectories. Although we use -r option we include only the files in pattern. The manual page tell us about write \ before of * for apply recursively, but not work with all patterns.
Also you can use other commands to get list of files to include.
By example:
zip -rq foo.zip . -i `find ./ -not -name "*.bar" -o -name "foo.bar"`
Or:
zip -r foo.zip . -x `find ./ -name "*.bar" -a -not -name "foo.bar"`
how would I type a file path in ubuntu terminal to include all files in all sub-directories?
If I had a main directory called "books" but had a ton of subdirectories with all sorts of different names containing files, how would I type a path to include all files in all subdirectories?
/books/???
From within the books top directory, you can use the command:
find . -type f
Then, if you wanted to, say run each file through cat, you could use the xargs command:
find . -type f | xargs cat
For more info, use commands:
man find
man xargs
It is unclear what you actually want ... Probably you will get a better solution to your problem, if you ask directly for it, not for one other problem you've come accross trying to circumvent the original problem.
do you mean something like the following?
file */*
where the first * expands for all subdirectories and the second * for all contained files ?
I have chosen the file command arbitrarily. You can choose whatever command you want to run on the files you get shell-expanded.
Also note that directories will also be included (if not excluded by name, e.g. *.png or *.txt).
The wildcard * is not exactly the file path to include all files in all subdirectories but it expands to all files (or directories) matching the wildcard expression as a list, e.g. file1 file2 file3 file4. See also this tutorial on shell expansion.
Note that there may be easy solutions to related problems. Like to copy all files in all subdirectories (cp -a for example, see man cp).
I also like find very much. It's quite easy to generate more flexible search patterns in combination with grep. To provide a random example:
du `find . | grep some_pattern_to_occur | grep -v some_pattern_to_not_occur`
./books/*
For example, assuming i'm in the parent directory of 'books':
ls ./books/*
EDIT:
Actually, to list all the tree recursively you should use:
ls -R ./books/*
I have a folder llvm2.9 in which i ran this command.
$> ctags -R --sort=1 --c++-kinds=+p --fields=+iaS --extra=+q --language-force=C++
This was indexing methods in *.html files also which were present in llvm2.9/docs. I found this out because when i pressed ctrl-] for some class, it went to the html file.
How do i force ctags to use .cpp/.h files alone or ignore a particular directory.
Thanks
You can exclude a filetype using
--exclude='*.html'
If you need to exclude more than just .html files:
You can't comma separate a list inside an exclude option. This doesn't work:
ctags --exclude=*.html,*.js ./*
However, you can pass multiple exclude options:
ctags --exclude=*.html --exclude=*.js ./*
Pass the -V option to help with debugging:
ctags -V --exclude=*.html --exclude=*.js ./*
Gives the output:
Reading initial options from command line
Option: --exclude=*.html
adding exclude pattern: *.html
Option: --exclude=*.js
adding exclude pattern: *.js
The simplest way in vim would be
:!ctags {.,**}/*.{cpp,h}
Explanation: The braces expand to
:!ctags ./*.cpp **/*.cpp **/*.h **/*.h
So it looks for source or header files in the current directory (./) or any nested directory (**/). Note **/ wouldn't match the current directory (it always matches at least 1 sub directory level)
In shell:
find -iname '*.cpp' -o '*.h' -print0 | xargs -0 ctags
Explanation: This recursively finds all .cpp and .h files under the current directory and passes them to ctags on the command line.
The way print0 and -0 work together is to ensure it works correctly with weird filenames (e.g. containing whitespace or even new line characters)
I'll leave the rest of the ctags options for your own imagination :)
PS. For recent bash-es, you can use
shopt -s globstar
ctags {.,**}/*.{cpp,h}
and get much the same behaviour as in vim !
I didn't want to track down every filetype which might get processed in a large project, and I was only interested in Python, so I explicitly only processed python files using ctags --languages=Python .... The list of language names can be seen using ctags --list-languages.
I want to find all the different .gitignore files I have to combine them into one.
I tried something like find */**/.gitignore but that came up with nothing.
All the files are in sub directories of my current dir, as well as the current directory.
I am using Bash on linux
find -name .gitignore
That should do
Since you are using bash:
shopt -s globstar
echo **/.gitignore
From man bash:
globstar
If set, the pattern ** used in a pathname expansion context will match a
files and zero or more directories and subdirectories. If the pattern is
followed by a /, only directories and subdirectories match.
Try this
find /home/user1 -name 'result.out' 2>/dev/null
In your case
find /<your DIR> -name '*.gitignore' 2>/dev/null
This results in
/home/user1/result.out
/home/user1/dir1/result.out
/home/user1/dir2/result.out