I am working on a bioinformatics workflow in which the tool in question, 'salmon' creates multiple directories having a 'quant.sf' file. I want to find all 'lnc' entries within these files and save them as 'lnc.sf' for all directories.
I was previously running
cat quant.sf | grep 'lnc' > lnc.sf
in all directories individually that seemed to solve my problem. Now I want to write a script that goes into each directory and generates a lnc.sf file.
I have tried doing
find . -name "quant.sf" | while read A
do
cat $A | grep 'lnc' > lnc.sf
done
But this just creates a concatenated lnc.sf file in the current directory. Any help is highly appreciated.
Thank You!
If all your quant.sf files are at the same hierarchy level, the following should work, assuming a folder structure like month/day/quant.sf:
grep -h 'lnc' */*/quant.sf > lnc.sf
Otherwise, find the files, be aware of using find+read instead of exec or xargs; understand variable expansion with whitespaces, get rid of the redundant cat process, and write the file to the correct directory:
find . -name 'quant.sf' | while IFS= read -r A
do
grep 'lnc' "$A" > "${A%/*}/lnc.sf"
done
If you have GNU find + xargs, use -print0 combined with -0:
find . -name 'quant.sf' -print0 | xargs -0 -n1 sh -c 'grep "lnc" "$1" > "${1%/*}/lnc.sf"' -
Or use -exec of find, which avoids problems with weird files names:
find . -name 'quant.sf' -exec sh -c 'grep "lnc" "$1" > "${1%/*}/lnc.sf"' - ';'
I'm looking to create files with names based on the command output of the previous command i.e. if i run
find . -name *.mp4 | wc -l > filename
So that the output of the amount of files of that type is the filename of the created file.
Here's a solution that renames the file after it has been created:
find . -name *.mp4 | wc -l > filename && mv filename `tail -n 1 filename`
What is happening in this one-liner:
find . -name *mp4 | wc -l > filename : Finds files with mp4 suffix and then counts how many were found and redirects the output to a file named filename
tail -n 1 filename: Outputs the very last line in the file named filename. If you put backticks around it (`tail -n 1 filename`) then that statement is executed and replaced by the text it returns.
mv filename `tail -n 1 filename`: Renames the original file named filename to the executed statement above.
When you combine these with &&, the second statement only runs if the first was successful.
I am trying to find which package.json is causing a dependency warning. It's coming from a sub-module. I have a find command find . -name 'foo' that outputs
/a/very/very/very/long/path/to/a/submodule/.../node_modules/foo
/another/very/very/very/long/path/to/a/submodule/.../node_modules/foo
I want to see what version of foo is included in the parent module's package.json. Something like this
more "$find_output"/../../package.json | grep foo
Figured it out:
find . -name 'foo' -print0 | xargs -0 bash -c 'for path; do more "$path"/../../package.json | grep "foo"; echo $path; done ' bash
cat `find . -name '*.css'`
This will open any css file. I now what do two things.
1) How do I add *.js to this as well. So I want to look inside all css and javascript files.
2) I want to look for any css or image files within those (css or js files) and push those into an array. So I guess look for a .png, .jpg, .gif, .tif, .css and put everything before that until the quote or single quote into an array. I want an array because this command will go into a shell script and after I get all the names of the files that I need I will need to loop through and download those files later.
Any help would be appreciated.
Extra hackery, in case someone needs it:
find ./ -name "*.css" | xargs grep -o -h -E '[A-Za-z0-9:./_-]+\.(png|jpg|gif|tif|css)'| sed -e 's/\.\./{{url here}}/g'|xargs wget
will download every missing resource
Do the command:
find ./ -name "*.css" -or -name "*.js" > fileNames.txt
Then read each line of fileNames.txt in the loop and download them.
Or if you are going to use wget to download the images you could do:
find ./ -name "*.css" -or -name "*.js" | xargs grep '*.png' | xargs wget
May need a little refinement like a cut after the grep but you get the idea
1) simple answer: you can add the names of all .js files to your cat command, by instructing find to find more files:
cat `find . -name '*.css' -or -name '*.js'`
2) a text-searching tool such as grep is probably what you're after:
find . -name '*.css' -or -name '*.js' | xargs grep -o -h -E '[A-Za-z0-9:./_-]+\.(png|jpg|gif|tif|css)'
Note: my grep pattern isn't universal or perfect, but it's a starting example. It matches any string that includes alpha-numeric,colon,dot,slash,underscore or hyphens in it, followed by any one of the given extensions.
The -o option causes grep to output only the parts of the .css/.js files that match the pattern (i.e. only the apparent filenames).
If you want to download them you could add | xargs wget -v to the command, which would instruct wget to fetch all those filenames.
NOTE: this won't work for relative filenames; some other magic will be required (i.e. you'll have to resolve them with respect to the grepped file's location). Perhaps some extra hackery, such as sed or awk.
Also: How often do you see references to TIFFs in your CSS/JS?
This is similar to this question, but I want to include the path relative to the current directory in unix. If I do the following:
ls -LR | grep .txt
It doesn't include the full paths. For example, I have the following directory structure:
test1/file.txt
test2/file1.txt
test2/file2.txt
The code above will return:
file.txt
file1.txt
file2.txt
How can I get it to include the paths relative to the current directory using standard Unix commands?
Use find:
find . -name \*.txt -print
On systems that use GNU find, like most GNU/Linux distributions, you can leave out the -print.
Use tree, with -f (full path) and -i (no indentation lines):
tree -if --noreport .
tree -if --noreport directory/
You can then use grep to filter out the ones you want.
If the command is not found, you can install it:
Type following command to install tree command on RHEL/CentOS and Fedora linux:
# yum install tree -y
If you are using Debian/Ubuntu, Mint Linux type following command in your terminal:
$ sudo apt-get install tree -y
Try find. You can look it up exactly in the man page, but it's sorta like this:
find [start directory] -name [what to find]
so for your example
find . -name "*.txt"
should give you what you want.
You could use find instead:
find . -name '*.txt'
To get the actual full path file names of the desired files using the find command, use it with the pwd command:
find $(pwd) -name \*.txt -print
That does the trick:
ls -R1 $PWD | while read l; do case $l in *:) d=${l%:};; "") d=;; *) echo "$d/$l";; esac; done | grep -i ".txt"
But it does that by "sinning" with the parsing of ls, though, which is considered bad form by the GNU and Ghostscript communities.
DIR=your_path
find $DIR | sed 's:""$DIR""::'
'sed' will erase 'your_path' from all 'find' results. And you recieve relative to 'DIR' path.
Here is a Perl script:
sub format_lines($)
{
my $refonlines = shift;
my #lines = #{$refonlines};
my $tmppath = "-";
foreach (#lines)
{
next if ($_ =~ /^\s+/);
if ($_ =~ /(^\w+(\/\w*)*):/)
{
$tmppath = $1 if defined $1;
next;
}
print "$tmppath/$_";
}
}
sub main()
{
my #lines = ();
while (<>)
{
push (#lines, $_);
}
format_lines(\#lines);
}
main();
usage:
ls -LR | perl format_ls-LR.pl
You could create a shell function, e.g. in your .zshrc or .bashrc:
filepath() {
echo $PWD/$1
}
filepath2() {
for i in $#; do
echo $PWD/$i
done
}
The first one would work on single files only, obviously.
Find the file called "filename" on your filesystem starting the search from the root directory "/". The "filename"
find / -name "filename"
If you want to preserve the details come with ls like file size etc in your output then this should work.
sed "s|<OLDPATH>|<NEWPATH>|g" input_file > output_file
In the fish shell, you can do this to list all pdfs recursively, including the ones in the current directory:
$ ls **pdf
Just remove 'pdf' if you want files of any type.
You can implement this functionality like this
Firstly, using the ls command pointed to the targeted directory. Later using find command filter the result from it.
From your case, it sounds like - always the filename starts with a word
file***.txt
ls /some/path/here | find . -name 'file*.txt' (* represents some wild card search)
In mycase, with tree command
Relative path
tree -ifF ./dir | grep -v '^./dir$' | grep -v '.*/$' | grep '\./.*' | while read file; do
echo $file
done
Absolute path
tree -ifF ./dir | grep -v '^./dir$' | grep -v '.*/$' | grep '\./.*' | while read file; do
echo $file | sed -e "s|^.|$PWD|g"
done