My Intro to OS teacher wants me to enter a command that long lists the /etc directory and finds all files modified or created with the size of 4096 bytes.
My answer is simply find /etc -size 4k -ls.
How am I supposed to use the grep command to do this? It seems kind of pointless...
It sure makes no sense but in IT you kind of get used to insane seeming requirements so here is one:
$ ls -l /etc | grep "^-\([^ ]* *\)\{4\}4096 "
-rw-r--r-- 1 root root 4096 Oct 27 2014 wgetrc
^- it's a file (^d for dir etc.)
[^ ]* * field and delimiting space(s)
\(...\)\{4\} four times previous, ie. filesize is in the fifth field
4096 filesize. Notice the trailing space to limit the size exactly to 4096.
Related
There are 3 txt files called
1.txt 2.txt 3.txt
I want to batch copy with the name
1.txt.cp 2.txt.cp 3.txt.cp
using the wildcard *
I entered the command cp *.txt *.txt.cp
but it wasn't working...
cp : target *.txt.cp : is not a directory
what was the problem???
Use: for i in *.txt; do cp "$i" "$i.cp"; done
Example:
$ ls -l *.txt
-rw-r--r-- 1 halley halley 20 out 27 08:14 1.txt
-rw-r--r-- 1 halley halley 25 out 27 08:14 2.txt
-rw-r--r-- 1 halley halley 33 out 27 08:15 3.txt
$ ls -l *.cp
ls: could not access '*.cp': File or directory does not exist
$ for i in *.txt; do cp "$i" "$i.cp"; done
$ ls -l *.cp
-rw-r--r-- 1 halley halley 20 out 27 08:32 1.txt.cp
-rw-r--r-- 1 halley halley 25 out 27 08:32 2.txt.cp
-rw-r--r-- 1 halley halley 33 out 27 08:32 3.txt.cp
$ for i in *.txt; do diff "$i" "$i.cp"; done
$
If you are used to MS/Windown CMD shell, it is important to note that Unix system handle very differently the wild cards. MS/Windows has kept the MS/DOS rule that said that wild cards were not interpreted but were passed to the command. The command sees the wildcard characters and can handle the second * in the command as noting where the match from the first should go, making copy ab.* cd.* sensible.
In Unix (and derivatives like Linux) the shell is in charge of handling the wildcards and it replaces any word containing one with all the possible matches. The good news is that the command has not to care about that. But the downside is that if the current folder contains ab.txt ab.md5 cd.jpg, a command copy ab.* cd.* will be translated into copy ab.txt ab.md5 cd.jpg which is probably not want you would expect...
The underlying reason is Unix shells are much more versatile than the good old MS/DOS inherited CMD.EXE and do have simple to use for and if compound commands. Just look at #Halley Oliveira's answer for the syntax for your use case.
I was working my way through a primer on Shell (Bash) Scripting and had the following doubt :
Why does not the following command print the contents of cp's directory : which cp | ls -l
Does not piping by definition mean that we pass the output of one command to another i.e. redirect the output ?
Can someone help me out ? I am a newbie ..
The output of which is being piped to the standard input of ls. However, ls doesn't take anything on standard input. You want it (I presume) to be passed as a parameter. There are a couple of ways of doing that:
which cp | xargs ls -l
or
ls -l `which cp`
or
ls -l $(which cp)
In the first example the xargs command takes the standard output of the previous previous command and makes each line a parameter to the command whose name immediately follows xargs. So, for instance
find / | xargs ls -l
will do an ls -l on each file in the filesystem (there are some issues with this with peculiarly named files but that's beyond the scope of this answer).
The remaining two are broadly equivalent and use the shell to do this, expanding the output from which into the command line for cp.
It would be,
$ ls -l $(which cp)
-rwxr-xr-x 1 root root 130304 Mar 24 2014 /bin/cp
OR
$ which cp | xargs ls -l
-rwxr-xr-x 1 root root 130304 Mar 24 2014 /bin/cp
To pass the output of one command as parameter of another command, you need to use xargs along with the pipe symbol.
From man xargs
xargs - build and execute command lines from standard input.xargs reads items
from the standard input, delimited by blanks (which can be protected
with double or single quotes or a backslash) or newlines, and executes
the command (default is /bin/echo) one or more times with any initial-
arguments followed by items read from standard input. Blank lines on
the standard input are ignored.
how to do arithmetic operations with the size of the files and directory when they are in different unites like free space is in MB and the file size is in GB
With one preparatory command I am able to fetch the size of the "/home/abc/def" directory in MB. Its 30GB so getting in KB is not a good idea.
mount fssizeMB
===== =======
/home/abc/def 30002
root#abc:/home/abc/def> ls -lrth
total 7.0G
drwxrwxrwx 3 root root 114 Oct 29 2012 file1
drwxr-xr-x 3 root root 103 Nov 22 2012 file2
-rw-r--r-- 1 root root 114 Jan 25 2013 file3
-rw-r--r-- 1 mtc users 3.8G Jul 22 03:02 file4 <------------------- concerned file
-rw-r--r-- 1 mtc users 3.2G Jul 24 22:26 file5
-rw-r--r-- 1 root root 0 Jan 5 20:30 file6
How to turn below logic in script:
If twice the file size of file4 is < free space of "/home/abc/def " then echo success or else failure.
You could use stat or du -Sh to get the size of file (don't use ls for that in a script).
And to browse the files of a folder :
for i in <direcory>/*; do ...; done
Then, you could use test or [ commands (or [[ if you use Bash) to make a comparison (with -ge, -gt, -lt, -le options as arithmetic operators).
See the manpages of each command to get more information.
this would work with percentages, just to give you an idea, you could modify it to deal with MB or GB and so on.
my advice: doing arithmetic operations in bash is not such a good idea, you should work with programming languages that deal with special variable data type, like float or str and so on. bash is simpler and doesn't work so well with arithmetic operations. sure it does your + and -, but when it comes to percentages and floats... not so well.
try python or perl, or try researching something else. and definitely use, as suggested above, du -sh
#!/usr/bin/env bash
#take df -h and iterate trough percentages
#check to see if file system is full more than 80 %
perc="$(df -h | awk '{print $5}'| sed -e 's/[%/ a-z/A-Z].*//g' )"
#echo $perc
for p in $perc
do
if [ $p -gt 30 ] #change 30 to whatever
then
df -h | grep $p
echo -e "$p Exceeded on `hostname`"
fi
done
Most commands have options to show the size using a specific unit.
The -h flag of ls and df are to produce "human readable" format, which is not suitable for arithmetic calculations, as they can be in inconsistent units.
To get the size of a file, use stat, or even wc -c. It's a bad practice to parse the output of ls -l, so don't use that.
If you can get the size of a file consistently in kilobytes, and the size of free space consistently in bytes, not a problem, you can just multiply the size in bytes with 1024 to be able to make comparisons in consistent units.
The specific commands and flags to use will depend on your operating system and the software installed.
Using bash, I have dir of /home/user/logs/
Aug 2 15:34 backup.20120802.tar.gz
Aug 3 00:26 backup.20120803.tar.gz
Aug 4 00:25 backup.20120804.tar.gz
Aug 15 06:39 backup.20120816.tar.gz
This gets updated every few days, but if something goes wrong I want it to automatically restore the most recent backup, how can I use bash only extract the most recent?
ls -t1 /home/user/logs/ | head -1
gives you the most recent modified file in /home/user/logs/.
So you could do:
cd /dir/to/extract
tar -xzf "$(ls -t1 /home/user/logs/ | head -1)"
NOTE:
this assumes that /home/user/logs/ is flat and contains nothing but "*.tar.gz" files
If the time stamps may not always be reliable, try sorting by date.
ls -1 /home/user/logs/backup.*.tar.gz | sort -t . -k2rn | head -1
Ideally, you should not parse the output from ls, but if there are only regularly named files matching the wildcard, it may be the easiest solution; sort expects line-oriented input, anyway, so the task becomes more involved in the general case of completely arbitrary file names. (This may make no sense to you, but it would be perfectly okay as far as Unix is concerned to have a file named backup.20120816.tar.gz(newline)backup.20380401.tar.gz.)
my grep command looks like this
zgrep -B bb -A aa "pattern" *
I would lke to have output as:
file1:line1
file1:line2
file1:line3
file1:pattern
file1:line4
file1:line5
file1:line6
</blank line>
file2:line1
file2:line2
file2:line3
file2:pattern
file2:line4
file2:line5
file2:line6
The problem is that its hard to distinguish when lines corresponding to the first found result end and the lines corresponding to the second found result start.
Note that although man grep says that "--" is added between contiguous group of matches. It works only when multiple matches are found in the same file. but in my search (as above) I am searching multiple files.
also note that adding a new blank line after every bb+aa+1 line won't work because what if a file has less than bb lines before the pattern.
pipe grep output through
awk -F: '{if(f!=$1)print ""; f=$1; print $0;}'
Pipe | any output to:
sed G
Example:
ls | sed G
If you man sed you will see
G Append's a newline character followed by the contents of the hold space to the pattern space.
The problem is that its hard to distinguish when lines corresponding to the first found result end and the lines corresponding to the second found result start.
Note that although man grep says that "--" is added between contiguous group of matches. It works only when multiple matches are found in the same file. but in my search (as above) I am searching multiple files.
If you don't mind a -- in lieu of a </blank line>, add the -0 parameter to your grep/zgrep command. This should allow for the -- to appear even when searching multiple files. You can still use the -A and -B flags as desired.
You can also use the --group-separator parameter, with an empty value, so it'd just add a new-line.
some-stream | grep --group-separator=
I can't test it with the -A and -B parameters so I can't say for sure but you could try using sed G as mentioned here on Unix StackEx. You'll loose coloring though if that's important.
There is no option for this in grep and I don't think there is a way to do it with xargs or tr (I tried), but here is a for loop that will do it (for f in *; do grep -H "1" $f && echo; done):
[ 11:58 jon#hozbox.com ~/test ]$ for f in *; do grep -H "1" $f && echo; done
a:1
b:1
c:1
d:1
[ 11:58 jon#hozbox.com ~/test ]$ ll
-rw-r--r-- 1 jon people 2B Nov 25 11:58 a
-rw-r--r-- 1 jon people 2B Nov 25 11:58 b
-rw-r--r-- 1 jon people 2B Nov 25 11:58 c
-rw-r--r-- 1 jon people 2B Nov 25 11:58 d
The -H is to display file names for grep matches. Change the * to your own file glob/path expansion string if necessary.
Try with -c 2; with printing a context I see grep is separating its found o/p