I have lots of files in a directory and I want to list all files in a specific format by creating date order (newer files first)
I can do this with stat and sort in the directory of files:
Commands:
cd /path
stat -c '%.19y %n' * | sort -nr
Output:
2014-01-25 16:44:52 Filename1.txt
2014-01-24 16:34:17 Filename2.txt
It is fine. This is what I want exactly.
But when I try to run outside of the directory, command shows filenames with /path
Commands:
cd /
stat -c '%.19y %n' /path/* | sort -nr
Output:
2014-01-25 16:44:52 /path/Filename1.txt
2014-01-24 16:34:17 /path/Filename2.txt
How can I list file names without path?
Provided you have GNU find, you can use that instead
find /path -printf '%TY-%Tm-%Td %TH:%TM:%.2TS %f\n'
2015-10-12 04:54:24 file
Related
I have several subdirectories within a parent folder, each with a URLs.txt file within the subdirectory.
$ ls
H3K4me1_assay H3K4me2_assay H3K4me3_assay ... +30 or so more *_assay files
Each assay_file contains one URLs.txt file:
$ cat URLs.txt
https://www.encodeproject.org/files/ENCFF052HMX/##download/ENCFF052HMX.bed.gz
https://www.encodeproject.org/files/ENCFF052HMX/##download/ENCFF052HMX.bed.gz
https://www.encodeproject.org/files/ENCFF466DMK/##download/ENCFF466DMK.bed.gz
... +200 or more URLs
Is there any way I can execute a command from the parent folder that reads and curls the URLs.txt file in each subdirectory, and then downloads the file within each subdirectory?
I can cd into each file and run the following commands to download all of the files:
$ cd ~/largescale/H3K4me3_assay
$ ls URL* | xargs -L 1 -d '\n' zcat | xargs -L 1 curl -O -J -L
But I will have to run this command for experiments with +300 folders, so cd'ing in each time isn't really practical.
I have tried to run this, it does download the correct files but within the parent folder rather than the subdirectories. Any idea what I am doing wrong?
$ for i in ./*_assay; do cd ~/largescale/"$i" | ls URL* | xargs -L 1 -d '\n' zcat | xargs -L 1 curl -O -J -L; done
Thanks, Steven
I have a folder called documentaries on my Linux computer.
I have SSH access to seedbox (also Linux).
How do I find out which documentaries I have in both computers?
On seedbox it's a flat file structure. Some documentaries are files, some are folders which contain many files, but all in same folder
For example:
data/lions_botswana.mp4
data/lions serengeti/S01E01.mkv
data/lions serengeti/S01E02.mkv
data/strosek_on_capitalism.mp4
data/something_random.mp4
Locally structure is more organized
documentaries/animals/lions_botswana.mp4
documentaries/animals/lions serengeti/S01E01.mkv
documentaries/animals/lions serengeti/S01E02.mkv
documentaries/economy/strosek_on_capitalism.mp4
documentaries/something_random.mp4
I am not looking for command like diff, I am looking for command like same (opposite of diff) if such command exists.
Based on the answer from Zumo de Vidrio, and my comment:
on one computer
cd directory1/; find | sort > filelist1
on the other
cd directory2/; find | sort > filelist2
copy them in one place an run:
comm -12 filelist1 filelist2
or as a one liner:
ssh user#host 'cd remotedir/; find|sort' | comm -12 - <(cd localdir/; find|sort)
Edit: With multiple folders this would look as follows
on one computer
cd remotedir/; find | sort > remotelist
on the other
cd localdir/subdir1/; find > locallist1
cd -;
cd localdir/subdir2/; find > locallist2
cd -;
#... and so on
sort locallist1 locallist2 > locallistall
copy them in one place an run:
comm -12 remotelist locallistall
or as a (now very long) one liner:
ssh user#host 'cd remotedir/; find|sort' | comm -12 - <({cd localdir/subdir1/; find; cd -; cd localdir/subdir2/; find; cd -; cd localdir/subdir3/; find}|sort)
Export list of remote files to local file by:
ssh user#seedbox 'find /path/to/data -type f -execdir echo {} ";"' > remote.txt
Note: On Linux you've to use absolute path to avoid leading ./ or use with "$PWD"/data.
Then grep the result of find command:
find documentaries/ -type f | grep -wFf remote.txt
This will display only these local files which also exist on remote.
If you would like to generate similar list on local and compare two files, try:
find "$PWD"/documentaries/ -type f -execdir echo {} ';' > local.txt
grep -wFf remote.txt local.txt
However above methods aren't reliable, since one file could have a different size. If files would have the same structure, you could use rsync to keep your files up-to-date.
For more reliable solution, you can use fdupes which can find all files which exist in both directories by comparing file sizes and MD5 signatures.
Sample syntax:
fdupes -r documentaries/ data/
However both directories needs to be accessible locally, so you can always use sshfs tool to mount the remote directory locally. Then you can use fdupes to find all duplicate files. It has also option to remove the other duplicates (-d).
Copy the ls output of each Computer to a same folder and then apply diff over them:
In your computer:
ls -R documentaries/ > documentaries_computer.txt
In seedbox:
ls -R documentaries/ > documentaries_seedbox.txt
Copy both files to a same location and execute:
diff documentaries_computer.txt documentaries_seedbox.txt
You can mount remote folder using sshfs, then you can use diff -r to find the differences between them.
E.g.
sshfs user#seedbox-host:/path/to/documentaries documentaries/
diff -rs /local/path/documentaries/animals documentaries/ | grep identical
diff -rs /local/path/documentaries/economy documentaries/ | grep identical
Can I use ls | grep -c /full/path/to/file to count the occurrences of a file, but while executing the command from a different directory than where the files I'm looking for are?
Let's say I want to look how many .txt files I have in my "results" directory. Can I do something like ls | grep -c /full/path/to/results/*.txt while I'm in another directory?
Although I have .txt files in that directory, I always get a zero when I run the command from another directory :( What's happening? Can I only use ls for the current directory?
You have to use ls <dirname>. Plain ls defaults only to the current directory.
What you are trying to do can be accomplished by find <dir> -name "*.txt" | grep -c txt or find <dir> -name "*.txt" | wc -l
But you can do ls * | grep \.txt$ as well. Please read the manual to find the differences.
grep accepts regular expressions, not glob. /foo/bar/*.txt is a glob. Try /foo/bar/.*\.txt
also ls lists files and directories under your current directory. It will not list the full path. Do some tests, and you will see it easily.
ls may output results in a single line, and this could make your grep -c give an incorrect result. Because grep does line-based matching.
In the current folder I have lots of subfolders and they have files in them. How can I put those folders with no AAA or BBB in the name into a .zip file?
123AAAcc
123.txt
BBB.csv
222BBBss
...
ADFAAA
BBB
adsf.txt
vvBB
111.mov
BBB.avi
I've tried the following and it also excludes vvBB\BBB.avi and ADFAAA\BBB\*. I would like to keep those.
tar -zcvf test.tgz --exclude='*AAA*' --exclude='*BBB*' .
The problem is, how can I have --exclude only work on level 1 subfolder names, but recursively on all filenames and folder names?
Hopefully I have a .zip file containing:
vvBB
111.mov
BBB.avi
I would like to achieve this as one command line. How can I do this?
You can accomplish this behavior by chaining together the following commands:
ls will list all of the files and folders in the current directory.
grep -v "\(.*BBB.*\)\|\(.*AAA.*\)" looks for all names with AAA or BBB (surrounded by anything), then (with -v) excludes them and returns all other results.
xargs tar -cvzf test.tgz will take arguments from a pipe and apply them to tar. All together, you get:
ls | grep -v "\(.*BBB.*\)\|\(.*AAA.*\)" | xargs tar -cvzf test.tgz
The partial results I get are:
$ ls
123AAAcc
222BBBss
ADFAAA
vvBB
$ ls | grep -v "\(.*BBB.*\)\|\(.*AAA.*\)"
vvBB
$ ls | grep -v "\(.*BBB.*\)\|\(.*AAA.*\)" | xargs tar -cvzf test.tgz
a vvBB
a vvBB/111.mov
a vvBB/BBB.avi
I need to list the files on Directory. But only true files, not the folders.
Just couldn't find a way to test if the file is a folder or a directory....
Could some one provide a pice o script for that?
Thanks
How about using find?
To find regular files in the current directory and output a sorted list:
$ find -maxdepth 1 -type f | sort
To find anything that is not a directory (Note: there are more things than just regular files and directories in Unix-like systems):
$ find -maxdepth 1 ! -type d | sort
In bash shell test -f $file will tell you if $file is a file:
if test -f $file; then echo "File"; fi
You can use ls -l | grep ^d -v to realize what you want. I tested it in Redhat 9.0, it lists only the true files, including the Hidden Files.
If u want to get a list the folders on Directory. But only folders, not the true files. You can use ls -l | grep ^d