I have created a file (search.txt) containing file names of .fasta files I want to copy from the current directory (which also contains many unwanted .fasta files). Is it possible to use this text file to find and copy the matching files in the current directory to a new location?
The search.txt file contains a list of names like this:
name_1
name_2
name_3
I tried to build the search term using find and grep, like this:
find . *.fasta | grep -f search.txt
which is returning output like this for each matching file:
./name_1.fasta
./name_2.fasta
./name_3.fasta
name_1.fasta
name_2.fasta
name_3.fasta
It's finding the correct files, but I'm not sure if this output is useful / can be used to copy these files?
To get only matching filenames from search.txt I would do this:
find . -type f -name '*.fasta' -print0 | grep -zf search.txt | xargs -r0 cp -t target-dir/
It will find all files with the extension .fasta, display only the ones with matching patterns in search.txt, and bulk cp them to target-dir, and each filename is terminated with a nullbyte in case filenames contain newlines.
Using Bash, you can read all the files from the list into an array:
$ mapfile -t files < search.txt
$ declare -p files
declare -a files=([0]="name_1" [1]="name_2" [2]="name_3")
Then, you can append the desired file extension to all array elements:
$ files=("${files[#]/%/.fasta}")
$ declare -p files
declare -a files=([0]="name_1.fasta" [1]="name_2.fasta" [2]="name_3.fasta")
And finally, move them to the desired location:
$ mv "${files[#]}" path/to/new/location
You don't actually need the intermediate step:
mapfile -t files < search.txt
mv "${files[#]/%/.fasta}" path/to/new/location
Related
I have a list of patterns in filenames.txt, and I want to search a folder for filenames containing the names.
patterns.txt:
254b
0284ee
001ty
288qa
I want to search a folder for filenames containing any of these patterns in its filename and copy all found files to a destination directory.
So far i found a solution to view files as follows:
set -f; find ./ -type f \( $(printf -- ' -o - iname *%s*' $(cat patterns.txt) | cut -b4-) \); set +f
I can find all files based on the patterns on my patterns.txt file, but how do I copy them top a newfolder ?
Assuming target folder will not need to maintain the original hierarchy (or that the input directory does not have sub directories), using find, grep, and xargs should work:
find . -type f -print0 |
grep -z -i -F -f patterns.txt |
xargs -0 -s1000 cp -t /new/folder
The sequence has the advantage of bulking the copy - will be efficient for large number of files. Using NUL to separate file name should allow any special character in the file name.
I have a directory containing multiple directories. here is an example of the list of directories:
dir1_out
dir2_out
dir3_out
dir4_out
Each directory contains multiple files.
For example folder1_out contains the following files:
file1
file2
file3
In the same fashion other directories contain several folders.
I would like to add the name of each directory to file name in the corresponding directory.
I would like to have the following result in first directory(dir1_out):
dir1.file1
dir1.file2
dir1.file3
Since I have around 50 directories I would like to write a loop that takes the name of each directory and add the name to the beginning of all subfiles.
Do you have any idea how can I do that in linux.
A simple bash onliner if there aren't too many files is:
for p in */*; do [ -f "$p" ] && mv -i "$p" "${p%/*}/${p/\//.}"; done
This uses parameter expansions to generate new filenames, after checking that we are trying to rename an actual file - See bash manpage descriptions of ${parameter%word} and ${parameter/pattern/string}
If there may be too many files to safely expand them all into a single list:
#!/bin/bash
find . -maxdepth 2 -print |\
while read p; do
p="${p#./}"
mv -i "$p" "${p%/*}/${p/\//.}"
done
I want to search k1.sh, in all folders of my directory
ADU-07 WebInterface-JobTable_files
ADU-07 WebInterface-JobTable.html
ADU-07 WebInterface-SelfTestResults_files
ADU-07 WebInterface-SelfTestResults.html
meas_2016-07-13_20-22-00
meas_2016-07-13_20-25-13
meas_2016-07-13_20-29-43
meas_2016-07-13_20-33-43
meas_2016-07-13_20-37-43
meas_2016-07-13_20-54-43
meas_2016-07-13_21-46-43
When I try
grep -r -e "k1.sh" /home/milenko/SerradoMel/MT06
I got nothing.Why? How to check if my grep supports -r option?
grep checks files' contents, not files' names. You can test this like so:
mkdir -p foo.d
echo "non-matching" >foo.d/k1.sh
echo "k1.sh" >foo.d/matching
grep -r -e k1.sh foo.d
rm -rf -- foo.d
The output of the above is:
foo.d/matching:k1.sh
...where foo.d/matching is the filename which was matched, and k1.sh is the content.
You'll see that it doesn't identify the file named k1.sh (but having the contents non-matching), but instead identifies the file named matching; this is because the documented purpose of grep is to look at files' contents, not files' names.
Instead, to find a file named k1.sh, use:
find /home/milenko/SerradoMel/MT06 -type f -name k1.sh
Question: Do you want to find all files named k1.sh or do you want to find the string k1.sh inside all files?
Answer: #PauloScardine All files named k1.sh,will try with find.
Then grep is the wrong guy for the job. The grep command looks for the string "k1.sh" inside all files in that path - if you want to find all files named "k1.sh" you are looking for the find command instead:
find /home/milenko/SerradoMel/MT06 -name k1.sh
I have an entire directory structure with zip files. I would like to:
Traverse the entire directory structure recursively grabbing all the zip files
I would like to find a specific file "*myLostFile.ext" within one of these zip files.
What I have tried
1. I know that I can list files recursively pretty easily:
find myLostfile -type f
2. I know that I can list files inside zip archives:
unzip -ls myfilename.zip
How do I find a specific file within a directory structure of zip files?
You can omit using find for single-level (or recursive in bash 4 with globstar) searches of .zip files using a for loop approach:
for i in *.zip; do grep -iq "mylostfile" < <( unzip -l $i ) && echo $i; done
for recursive searching in bash 4:
shopt -s globstar
for i in **/*.zip; do grep -iq "mylostfile" < <( unzip -l $i ) && echo $i; done
You can use xargs to process the output of find or you can do something like the following:
find . -type f -name '*zip' -exec sh -c 'unzip -l "{}" | grep -q myLostfile' \; -print
which will start searching in . for files that match *zip then will run unzip -ls on each and search for your filename. If that filename is found it will print the name of the zip file that matched it.
Some have suggested to use ugrep to search zip files and tarballs. To find the zip files that contain a mylostfile file, specify it as a -g glob pattern like so:
ugrep -z -l -g'myLostfile' ''
With the empty regex pattern '' this this recursively searches all files down the working directory, including any zip, tar, cpio/pax archives for mylostfile. If you only want to search the zip files located in the working directory:
ugrep -z -l -g'myLostfile' '' *.zip
I have a database of files in a folder. I wish to sort the files containing *C: into one folder and the files containing *c: into another folder. How can this by achieved?**
I can use *.krn to access every file.
$ grep --help | grep with-matches
-l, --files-with-matches print only names of FILEs containing matches
What now depends on how many files there are and how paranoid you must be about their names. From the simplest
mv $(grep -l pattern files) target
to the most robust
grep -l -Z pattern files | xargs -0 mv -t target-directory --