I'm having a really hard time finding an answer for this because most people want to find a list of files, then do the xargs.
I have a file with a list of file names. I would like to go through the list of file names in that file and one by one search a single directory (not recursive) and any time it finds the file, move it to a sub folder.
cat test.txt | xargs find . -type f -iname
At this point, I get this error:
find: paths must precede expression: file1
Why don't you use something like:
for i in `cat test.txt`
do
test -e $i && mv <src> <dst>
done
Related
experts, i have many folders and inside the folder there are many sub-folders and the sub-folders contain many files.However, in all the sub-folders one file name is same that is input.ps.Now i want to rename the same input.ps with full path plus file name
so input.ps in all directories should be renamed to home_wuan_data_filess_input.ps
i tried
#!/bin/sh
for file in /home/wuan/data/filess/*.ps
do
mv $file $file_
done
But it does not do the same as i expect,i hope experts will help me.Thanks in advance.
so input.ps in all directories should be renamed to home_wuan_data_filess_input.ps
You may use this find solution:
find /home/wuan/data/filess -type f -name 'input*.ps' -exec bash -c '
for f; do fn="${f#/}"; echo mv "$f" "${fn//\//_}"; done' _ {} +
while read line;
do
fil=${line//\//_}; # Converts all / characters to _ to form the file name
fil=${fil:1}; # Remove the first -
dir=${line%/*}; # Extract the directory
echo "mv $line $dir/$fil"; # Echo the move command
# mv "$line" "$dir/$fil"; # Remove the comment to perform the actual command
done <<< "$(find /path/to/dir -name "input.ps")"
Ok, so file will end up being
/home/wuan/data/filess/input.ps
What we need here is the path, and the full, snake-cased name. Let's start by getting the path:
for f in /home/wuan/data/filess/*.ps; do
path="${f%*/}";
This will match the substring of f up until the last occurrence of /, effectively giving us the path.
Next, we want to snake_case all the things, which is even easier:
for f in /home/wuan/data/filess/*.ps; do
path="${f%*/}";
newname="${f//\//_}"
This replaces all instances of / with _, giving the name you want the new file to have. Now let's put all of this together, and move the file f to path/newname:
for f in /home/wuan/data/filess/*.ps; do
path="${f%*/}";
newname="${f//\//_}"
mv "${f}" "${path}/${newname}"
done
That should do the trick
Here's one of many sites listing some of the bash string manipulations you can use.
Sorry for the delayed update, the power in my building just cut out :)
So for example, I have a text document of a list of file names I may have in a directory. I want to grep or use find to find out if those file names exist in a specific directory and the subdirectories within it. Current I can do it manually via find . | grep filename but that's one at a time and when I have over 100 file names I need to check to see if I have them or not that can be really pesky and time-consuming.
What's the best way to go about this?
xargs is what you want here. The case is following:
Assume you have a file named filenames.txt that contains a list of files
a.file
b.file
c.file
d.file
e.file
and only e.file doesn't exist.
the command in terminal is:
cat filenames.txt | xargs -I {} find . -type f -name {}
the output of this command is:
a.file
b.file
c.file
d.file
Maybe this is helpful.
If the files didn't move, since the last time, updatedb ran, often < 24h, your fastest search is by locate.
Read the filelist into an array and search by locate. In case the filenames are common (or occur as a part of other files), grep them by the base dir, where to find them:
< file.lst mapfile filearr
locate ${filearr[#]} | grep /path/where/to/find
If the file names may contain whitespace or characters, which might get interpreted by the bash, the usual masking mechanisms have to been taken.
A friend had helped me figure it out via find . | grep -i -Ff filenames.txt
I have many files in a single directory, the first few are named:
1A_T
1B_T
2B
3_6A_T
Each file has three columns of data that are tab delimited.
I need to add the name of the file to the top of each column in the file, and do this for all files in the directory. Keeping everything tab delimited is crucial.
Simple suggestions? I've tried several approaches (I'm new to this) from the internet but can't get it to work properly.
Thanks for your help
You can use the following command:
for f in `find . -type f`; do echo $f; sed -i.bak "1 i $f\t$f\t$f" $f; done
just in case test it in a different directory before doing it on your working directory.
Explanations:
find . -type f will get all the files from your directory (and its sub dir) (run this command from the directory where you want to proceed or change . accordingly with the PATH to your dir)
for f in will iterate on all your files you could also use xargs command
sed -i.bak "1 i $f\t$f\t$f" $f; will take a backup of your file then edit it and add before the first line of your file the filename\tfilename\tfilename
TESTED:
REMARK:
beware of spaces in the filenames!
Last but not least, I have taken both of your points in the comment and you can use the following command in order to not have the ./ before the file name on the first line and to process files only in the working directory.
for f in `find . -maxdepth 1 -type f`; do echo `basename $f`; sed -i.bak "1 i `basename $f`\t`basename $f`\t`basename $f`" $f; done
Im trying to look for the text Elapsed time inside a specific log file names vsim.log. Im not familiar with grep, but after some googling I found that grep -r will allow me to do recursively searches and grep -r "Elapsed time" will do recursive searches for that phrase within all files in my directory. According to this link, I can then do grep -r "Elapsed time" ./vsim* to recursively search through the directories for files starting with vsim and look inside those files for Elapsed time. However, when i tried this i get grep: No match. which i know is not true since I know the files exist there with those keywords. What am i messing up?
Continuing from my comment, you can use find to locate the file vsim.log if you do not know its exact location and then use the -execdir option to find to grep the file for the term Elapsed time, e.g.
find path -type f -name "vsim.log" -execdir grep -H 'Elapsed time' '{}' +
That will return the filename along with the matched text which you can simply parse to isolate the filename if desired. You can process all files that match if you anticipate more than one by feeding the results of the find command into a while read -r loop, e.g.
while read -r match; do
# process "$match" as desired
echo "Term 'Elapsed time' found in file ${match%:*}"
done < <(find path -type f -name "vsim.log" -execdir grep -H 'Elapsed time' '{}' +)
Where:
find is the swiss-army knife for finding files on your system
path can be any relative or absolute path to search (e.g. $HOME or /home/dorojevets) to search all files in your home directory
the option -type f tells find to only locate files (see man find for link handling)
the option -name "foo" tell find to only locate files named foo (wildcards allowed)
the -exec and -execdir options allow you to execute the command that follows on each file (represented by '{}')
the grep -H 'Elapsed time' '{}' being the command to execute on each filename
the + being what tells find it has reached the end of the command (\; used with -exec)
finally, the ${match%:*} parameter expansion on the variable $match is used to parse the filename from filename:Elapsed time returned by grep -H (the %:* simply being used to trim everything to the first : from the right of $match)
Give that a try and compare the execution time to a recursive grep of the file tree. What you may be missing in this discussion, is that you use find if you know some part of the filename (or file mod time, or set of permissions, etc) that contains the information you need. It can search millions of files in a file tree vastly quicker than you can recursively grep every single file. If you have no clue what file may contain the needed info -- then use grep and just wait...
Try:
grep -r "Elapsed time" * --include vsim.log
Or this answer Use grep --exclude/--include syntax to not grep through certain files.
The following works just in case if you are using Mac:
To search UUID in *.c files recursively under given folder "/home" use the following:
grep -r "UUID" --include "*.c" /home/
To recursively search UUID in all main.c files in multiple projects under current folder use the following:
grep -r "UUID" --include "main.c" .
In my hierarchy of directories I have many text files called STATUS.txt. These text files each contain one keyword such as COMPLETE, WAITING, FUTURE or OPEN. I wish to execute a shell command of the following form:
./mycommand OPEN
which will list all the directories that contain a file called STATUS.txt, where this file contains the text "OPEN"
In future I will want to extend this script so that the directories returned are sorted. Sorting will determined by a numeric value stored the file PRIORITY.txt, which lives in the same directories as STATUS.txt. However, this can wait until my competence level improves. For the time being I am happy to list the directories in any order.
I have searched Stack Overflow for the following, but to no avail:
unix filter by file contents
linux filter by file contents
shell traverse directory file contents
bash traverse directory file contents
shell traverse directory find
bash traverse directory find
linux file contents directory
unix file contents directory
linux find name contents
unix find name contents
shell read file show directory
bash read file show directory
bash directory search
shell directory search
I have tried the following shell commands:
This helps me identify all the directories that contain STATUS.txt
$ find ./ -name STATUS.txt
This reads STATUS.txt for every directory that contains it
$ find ./ -name STATUS.txt | xargs -I{} cat {}
This doesn't return any text, I was hoping it would return the name of each directory
$ find . -type d | while read d; do if [ -f STATUS.txt ]; then echo "${d}"; fi; done
... or the other way around:
find . -name "STATUS.txt" -exec grep -lF "OPEN" \{} +
If you want to wrap that in a script, a good starting point might be:
#!/bin/sh
[ $# -ne 1 ] && echo "One argument required" >&2 && exit 2
find . -name "STATUS.txt" -exec grep -lF "$1" \{} +
As pointed out by #BroSlow, if you are looking for directories containing the matching STATUS.txt files, this might be more what you are looking for:
fgrep --include='STATUS.txt' -rl 'OPEN' | xargs -L 1 dirname
Or better
fgrep --include='STATUS.txt' -rl 'OPEN' |
sed -e 's|^[^/]*$|./&|' -e 's|/[^/]*$||'
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# simulate `xargs -L 1 dirname` using `sed`
# (no trailing `\`; returns `.` for path without dir part)
Maybe you can try this:
grep -rl "OPEN" . --include='STATUS.txt'| sed 's/STATUS.txt//'
where grep -r means recursive , -l means only list the files matching, '.' is the directory location. You can pipe it to sed to remove the file name.
You can then wrap this in a bash script file where you can pass in keywords such as 'OPEN', 'FUTURE' as an argument.
#!/bin/bash
grep -rl "$1" . --include='STATUS.txt'| sed 's/STATUS.txt//'
Try something like this
find -type f -name "STATUS.txt" -exec grep -q "OPEN" {} \; -exec dirname {} \;
or in a script
#!/bin/bash
(($#==1)) || { echo "Usage: $0 <pattern>" && exit 1; }
find -type f -name "STATUS.txt" -exec grep -q "$1" {} \; -exec dirname {} \;
You could use grep and awk instead of find:
grep -r OPEN * | awk '{split($1, path, ":"); print path[1]}' | xargs -I{} dirname {}
The above grep will list all files containing "OPEN" recursively inside you dir structure. The result will be something like:
dir_1/subdir_1/STATUS.txt:OPEN
dir_2/subdir_2/STATUS.txt:OPEN
dir_2/subdir_3/STATUS.txt:OPEN
Then the awk script will split this output at the colon and print the first part of it (the dir path).
dir_1/subdir_1/STATUS.txt
dir_2/subdir_2/STATUS.txt
dir_2/subdir_3/STATUS.txt
The dirname will then return only the directory path, not the file name, which I suppose it what you want.
I'd consider using Perl or Python if you want to evolve this further, though, as it might get messier if you want to add priorities and sorting.
Taking up the accepted answer, it does not output a sorted and unique directory list. At the end of the "find" command, add:
| sort -u
or:
| sort | uniq
to get the unique list of the directories.
Credits go to Get unique list of all directories which contain a file whose name contains a string.
IMHO you should write a Python script which:
Examines your directory structure and finds all files named STATUS.txt.
For each found file:
reads the file and executes mycommand depending on what the file contains.
If you want to extend the script later with sorting, you can find all the interesting files first, save them to a list, sort the list and execute the commands on the sorted list.
Hint: http://pythonadventures.wordpress.com/2011/03/26/traversing-a-directory-recursively/