I have some problem with the Linux grep command, it don't work !!!
I am trying the following test on my Ubuntu system:
I have create the following folder: /home/andrea/Scrivania/prova
Inside this folder I have created a txt file named prova.txt and inside this file I have write the string test and I have save it
In the shell I have first access the folder /home/andrea/Scrivania/prova and so I have launched the grep command in the following way:
~/Scrivania/prova$ grep test
The problem is that the cursor continues to blink endlessly and cannot find NOTHING! Why? What is the problem?
You've not provided files for the grep command to scan
grep "test" *
or for recursive
grep -r "test" *
Because grep searches standard input if no files are given. Try this.
grep test *
You are not running the command you were looking for.
grep test * will look for test in all files in your current directory.
grep test prova.txt will look for test specifically in prova.txt
(grep test will grep the test string in stdin, and will not return until EOF.)
You need to pipe in something to grep - you cant just call grep test without any other arguments as it is actually doing nothing. try grep test *
Another use for grep is to pipe in a command
e.g. This is my home directory:
drwx------+ 3 oliver staff 102 12 Nov 21:57 Desktop
drwx------+ 10 oliver staff 340 17 Nov 18:34 Documents
drwx------+ 17 oliver staff 578 20 Nov 18:57 Downloads
drwx------# 12 oliver staff 408 13 Nov 20:53 Dropbox
drwx------# 52 oliver staff 1768 11 Nov 12:05 Library
drwx------+ 3 oliver staff 102 12 Nov 21:57 Movies
drwx------+ 5 oliver staff 170 17 Nov 10:40 Music
drwx------+ 3 oliver staff 102 20 Nov 19:17 Pictures
drwxr-xr-x+ 4 oliver staff 136 12 Nov 21:57 Public
If i run
l | grep Do
I get the result
drwx------+ 10 oliver staff 340 17 Nov 18:34 Documents
drwx------+ 17 oliver staff 578 20 Nov 18:57 Downloads
remember to pipe the grep command
From grep man page:
Grep searches the named input FILEs (or standard input if no files
are
named, or the file name - is given) for lines containing a match to the
given PATTERN.
If you don't provide file name(s) for it to use, it will try to read from stdin.
Try grep test *
As per GNU Grep 3.0
A file named - stands for standard input. If no input is specified,
grep searches the working directory . if given a command-line
option specifying recursion; otherwise, grep searches standard input.
So for OP's command, without any additional specification, grep tries to search in standard input, which is not actually provided there.
A simple approach is grep -r [pattern], as per the above, to specify recursion with -r and search in current directory and sub-directories.
Also note that wildcard * only includes files, not directories. If used, a prompt might be shown for hint:
grep: [directory_name]: Is a directory
Related
I want to use wget to download multiple files at once in a script using wildcards, like this:
wget -r -nd --no-parent --no-remove-listing $ftpUrl -l1 -A file1*.txt &
wget -r -nd --no-parent --no-remove-listing $ftpUrl -l1 -A file2*.txt &
wget -r -nd --no-parent --no-remove-listing $ftpUrl -l1 -A file3*.txt &
The problem is that wget downloads .listing file everytime and because there are multiple instances running, sometimes the file is being downloaded when another instance is reading it.
Is there a way to lock .listing file or ask wget to not download it (I can do it manually as the first command)? I don't understand how it reads this .listing file since it's not a plain list of the URLs, but rather something like this:
drwxr-xr-x 3 4015 4015 16384 Dec 14 21:23 .
drwxr-xr-x 4 4015 4015 4096 Dec 14 21:23 ..
-rw-r--r-- 1 4015 4015 327 Feb 15 2022 file1-bla.txt
-rw-r--r-- 1 4015 4015 10716 Feb 15 2022 file2-bla.txt
-rw-r--r-- 1 4015 4015 163 Feb 15 2022 file2-bla.txt
If I try to use -i .listing (or even if I rename .listing to list.txt and use -i list.txt) I get an error saying that the URLs are not valid or something.
RFC 959 stipulates that
LIST (LIST) This command causes a list to be sent from the server to the passive DTP. If the pathname specifies a directory or other
group of files, the server should transfer a list of files in the
specified directory. If the pathname specifies a file then the server
should send current information on the file. A null argument implies
the user's current working or default directory. The data transfer is
over the data connection in type ASCII or type EBCDIC. (The user must
ensure that the TYPE is appropriately ASCII or EBCDIC). Since the
information on a file may vary widely from system to system, this
information may be hard to use automatically in a program, but may be
quite useful to a human user.
Observe that it does not impose formal requirement what exactly will be returned.
drwxr-xr-x 3 4015 4015 16384 Dec 14 21:23 .
drwxr-xr-x 4 4015 4015 4096 Dec 14 21:23 ..
-rw-r--r-- 1 4015 4015 327 Feb 15 2022 file1-bla.txt
-rw-r--r-- 1 4015 4015 10716 Feb 15 2022 file2-bla.txt
-rw-r--r-- 1 4015 4015 163 Feb 15 2022 file2-bla.txt
This looks akin to output of ls -lah command, if you are interested in using that to form URL list acceptable by wget then you should take last column (excluding . and ..) and prefix them with URL of FTP server, if that URL is ftp://ftp.example.com and you got list as above then your list of URLs should look as follows
ftp://ftp.example.com/file1-bla.txt
ftp://ftp.example.com/file2-bla.txt
ftp://ftp.example.com/file3-bla.txt
I have lot of files starting with processConfig-. I wanted to rename it to processCfg-. What's the easy way to change the first part of file name to processCfg- in linux?
But I don't want to rename this file processConfig.json since it doesn't match with my prefix.
> ls -lrth
total 467
-rw-r--r-- 1 david staff 9.8K May 26 15:14 processConfig-data-1234.json
-rw-r--r-- 1 david staff 11K May 26 15:14 processConfig-data-8762.json
-rw-r--r-- 1 david staff 4.9K May 26 15:14 processConfig-dataHold-1.json
-rw-r--r-- 1 david staff 6.6K May 26 15:14 processConfig-letter.json
-rw-r--r-- 1 david staff 5.6K May 26 16:44 processConfig-data-90987.json
-rw-r--r-- 1 david staff 284K May 28 18:44 processConfig.json
Like this :
rename -n 's/^processConfig-/processCfg-/' processConfig-*.json
Remove -n switch when the output looks good to rename for real.
man rename
There are other tools with the same name which may or may not be able to do this, so be careful.
The rename command that is part of the util-linux package, won't.
If you run the following command (GNU)
$ file "$(readlink -f "$(type -p rename)")"
and you have a result that contains Perl script, ASCII text executable and not containing ELF, then this seems to be the right tool =)
If not, to make it the default (usually already the case) on Debian and derivative like Ubuntu :
$ sudo apt install rename
$ sudo update-alternatives --set rename /usr/bin/file-rename
If you don't have this command with another distro, search your package manager to install it or do it manually (no deps...)
This tool was originally written by Larry Wall, the Perl's dad.
Sorry if this makes no sense, but I will try to give all the information needed!
I would like to use rsync to copy a range of sequentially numbered files from one folder to another.
I am archiving a DCDM (Its a film thing) and it contains in the order of 600,000 individually numbered, sequential .tif image files (~10mb ea.).
I need to break this up to properly archive onto LTO6 tapes. And I would like to use rsync to prep the folders such that my simple bash .sh file can automate the various folders and files that I want to back up to tape.
The command I normally use when running rsync is:
sudo rsync -rvhW --progress --size only <src> <dest>
I use sudo if needed, and I always test the outcome first with --dry-run
The only way I’ve got anything to work (without kicking out errors) is by using the * wildcard. However, this only does files with the set pattern (eg. 01* will only move files from the range 010000 - 019999) and I would have to repeat for 02, 03, 04 etc..
I've looked on the internet, and am struggling to find an answer that works.
This might not be possible, and with 600,000 .tif files, I can't write an exclude for each one!
Any thoughts as to how (if at all) this could be done?
Owen.
You can check for the file name starting with a digit by using pattern matching:
for file in [0-9]*; do
# do something to $file name that starts with digit
done
Or, you could enable the extglob option and loop over all file names that contain only digits. This could eliminate any potential unwanted files that start with a digit but contain non-digits after the first character.
shopt -s extglob
for file in +([0-9]); do
# do something to $file name that contains only digits
done
+([0-9]) expands to one or more occurrence of a digit
Update:
Based on the file name pattern in your recent comment:
shopt -s extglob
for file in legendary_dcdm_3d+([0-9]).tif; do
# do something to $file
done
Globing is the feature of the shell to expand a wildcard to a list of matching file names. You have already used it in your question.
For the following explanations, I will assume we are in a directory with the following files:
$ ls -l
-rw-r----- 1 5gon12eder staff 0 Sep 8 17:26 file.txt
-rw-r----- 1 5gon12eder staff 0 Sep 8 17:26 funny_cat.jpg
-rw-r----- 1 5gon12eder staff 0 Sep 8 17:26 report_2013-1.pdf
-rw-r----- 1 5gon12eder staff 0 Sep 8 17:26 report_2013-2.pdf
-rw-r----- 1 5gon12eder staff 0 Sep 8 17:26 report_2013-3.pdf
-rw-r----- 1 5gon12eder staff 0 Sep 8 17:26 report_2013-4.pdf
-rw-r----- 1 5gon12eder staff 0 Sep 8 17:26 report_2014-1.pdf
-rw-r----- 1 5gon12eder staff 0 Sep 8 17:26 report_2014-2.pdf
The most simple case is to match all files. The following makes for a poor man's ls.
$ echo *
file.txt funny_cat.jpg report_2013-1.pdf report_2013-2.pdf report_2013-3.pdf report_2013-4.pdf report_2014-1.pdf report_2014-2.pdf
If we want to match all reports from 2013, we can narrow the match:
$ echo report_2013-*.pdf
report_2013-1.pdf report_2013-2.pdf report_2013-3.pdf report_2013-4.pdf
We could, for example, have left out the .pdf part but I like to be as specific as possible.
You have already come up with a solution to use this for selecting a range of numbered files. For example, we can match reports by quater:
$ for q in 1 2 3 4; do echo "$q. quater: " report_*-$q.pdf; done
1. quater: report_2013-1.pdf report_2014-1.pdf
2. quater: report_2013-2.pdf report_2014-2.pdf
3. quater: report_2013-3.pdf
4. quater: report_2013-4.pdf
If we are to lazy to type 1 2 3 4, we could have used $(seq 4) instead. This invokes the program seq with argument 4 and substitutes its output (1 2 3 4 in this case).
Now back to your problem: If you want chunk sizes that are a power of 10, you should be able to extend the above example to fit your needs.
old question i know, but someone may find this useful. the above examples for expanding a range also work with rsync. for example to copy files starting with a, b and c but not d and e from dir /tmp/from_here to dir /tmp/to_here:
$ rsync -avv /tmp/from_here/[a-c]* /tmp/to_here
sending incremental file list
delta-transmission disabled for local transfer or --whole-file
alice/
bob/
cedric/
total: matches=0 hash_hits=0 false_alarms=0 data=0
sent 89 bytes received 24 bytes 226.00 bytes/sec
total size is 0 speedup is 0.00
If you are writing to LTO6 tapes, you should consider including "--inplace" to your command. Inplace is meant for writing to linear filesystems such as LTO
I am a beginner at linux and really enthusiastic to learn the OS. I am trying to launch Firefox(or any other software like Evince) from the command line as follows:
[root#localhost ~]# at 1637
[root#localhost ~]# at> firefox
[root#localhost ~]# at> ^d
The job gets scheduled without any error. But at the specified time it does not run.
I also tried giving the following path:
[root#localhost ~]# at 1637
[root#localhost ~]# at> /usr/bin/firefox
[root#localhost ~]# at> ^d
Still no result. But When I try to use echo to display a text on the screen it appears at the specified time as desired. What might be the issue?
I think you have not set DISPLAY. at will run in separate shell where display is not set.
try the following code.
dinesh:~$ at 2120
warning: commands will be executed using /bin/sh
at> export DISPLAY=:0
at> /usr/bin/firefox > firefox.log 2>&1
at> <EOT>
job 7 at Tue Mar 11 21:20:00 2014
If it is still failing check firefox.log for more information.
1) Its not always recommended to run things as root
2) You can also try ./firefox if you are in the current directory of firefox. In linux you need to pay attention to your path variable. Unless . (the current directory) is in your path you will have to type ./program if the program is in the same directory as you.
Also you need to pay attention to file permissions: In linux you have read-write-eXecute access.
ls -l will do a list of directories and show the file permissions:
drwxr-xr-x 10 user staff 340 Oct 6 2012 GlassFish_Server/
drwx------# 15 jeffstein staff 510 Oct 6 15:01 Google Drive/
drwxr-xr-x 20 jeffstein staff 680 May 14 2013 Kindle/
drwx------+ 67 jeffstein staff 2278 Jan 26 14:22 Library/
drwx------+ 19 jeffstein staff 646 Oct 23 18:28 Movies/
drwx------+ 15 jeffstein staff 510 Jan 3 20:29 Music/
drwx------+ 90 jeffstein staff 3060 Mar 9 20:23 Pictures/
drwxr-xr-x+ 6 jeffstein staff 204 Nov 3 21:16 Public/
drwxr-xr-x 22 jeffstein staff 748 Jan 14 2012 androidTools/
-rwxrwxrwx 1 jeffstein staff 1419 Aug 28 2013 color.sh*
This is an example of ls -l here you can see color.sh has -rwxrwxrwx that means that anybody can read or write or run the file.
Without actually knowing where you installed firefox however I can't be of more help but these are some small pointers which might help.
try finding where firefox is actually installed using "whereis firefox" command.
Then try using that path in at command.
In order to get directions on how to use a command type:
man at
this will display the "manual"
DESCRIPTION
The at and batch utilities read commands from standard input or a speci-
fied file. The commands are executed at a later time, using sh(1).
at executes commands at a specified time;
atq lists the user's pending jobs, unless the user is the superuser;
in that case, everybody's jobs are listed;
atrm deletes jobs;
batch executes commands when system load levels permit; in other words,
when the load average drops below _LOADAVG_MX (1.5), or the value
specified in the invocation of at run.
So obviously you need to schedule a job with at and you can see if it worked with atq
Read the manual and it should help - if i have more time I'll write you a quick example.
I'm just wondering about this problem:
if I can use something like "ls -al | less", less should have the ability of waiting for input from stdin. What I expected to happen is after running "less" command, the program hang up and wait for input(as a consequence of calling gets() or something like that.)
But why in fact it showed an error message "Missing filename ("less --help" for help)" end exited?
Thank you.
less command can check both argc>1 and stdin be associated with a file(not a tty).
The less command is not designed in that matter. When you execute a command in bash it will display all of the information at once. The less command is used to display the STDOUT of a command or the contents of a file one screen at a time.
$ ls -al | less
total 16
drwxrwxr-x 4 hdante hdante 4096 Nov 24 17:11 .
drwxr-xr-x 88 hdante hdante 4096 Mar 24 22:14 ..
drwxrwxr-x 2 hdante hdante 4096 Nov 25 01:55 new
drwxrwxr-x 3 hdante hdante 4096 Nov 24 18:27 old
(END)
It works. Something is wrong with your less. From less manual pages:
http://www.linuxmanpages.com/man1/less.1.php
https://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man1/less.1.html
The manual describes the filename as optional.
Hints to diagnose your problem:
try alias | grep less, to see if the command is being modified
try set | grep LESS, and check the scripts being run by LESSCLOSE and LESSOPEN