actual output comes as
$ grep -Hcw count copy_hb_script
copy_hb_script:5
I'm using the below command to get the expected out put but I'm failing
grep -Hcw count copy_hb_script | awk '{print $1}' |xargs ls -ld | awk '{print $8 " " $9 }'
getting out put is
03:49 copy_hb_script
Missing the count of the file, is there any alternate to get the time stamp with count of the file like below
03:49 copy_hb_script:5
You can avoid parsing ls output and use the stat command in bash for detailed file information.
# 'stat -c' produces output as 2016-09-15 16:03:40.655456000 +0530
# Stripping off extra information after the '.' using string-manipulation
# Running the grep with the count together with the previous command
modDate=$(stat -c %y copy_hb_script); echo "${modDate%.*}" "$(grep -Hcw count copy_hb_script)"
Produces an output as
016-09-15 16:03:40 copy_hb_script:5
Related
I'm working on Linux and need to calculate the sum size of some files in a directory.
I've written a bash script named cal.sh as below:
#!/bin/bash
while IFS='' read -r line || [[ -n "$line" ]]; do
echo $line
done<`ls -l | grep opencv | awk '{print $5}'`
However, when I executed this script ./cal.sh, I got an error:
./cal.sh: line 6: `ls -l | grep opencv | awk '{print $5}'`: ambiguous redirect
And if I execute it with sh cal.sh, it seems to work but I will get some weird message at the end of output:
25
31
385758: File name too long
Why does sh cal.sh seem to work? Where does File name too long come from?
Alternatively, you can do:
du -cb *opencv* | awk 'END{print $1}'
option -b will display each file in bytes and -c will print the total size.
Ultimately, as other answers will point out, it's not a good idea to parse the output of ls because it may vary between systems. But it's worth knowing why the script doesn't work.
The ambiguous redirect error is because you need quotes around your ls command i.e.:
while IFS='' read -r line || [[ -n "$line" ]]; do
echo $line
done < "`ls -l | grep opencv | awk '{print $5}'`"
But this still doesn't do what you want. The "<" operator is expecting a filename, which is being defined here as the output of the ls command. But you don't want to read a file, you want to read the output of ls. For that you can use the "<<<" operator, also known as a "here string" i.e.:
while IFS='' read -r line || [[ -n "$line" ]]; do
echo $line
done <<< "`ls -l | grep opencv | awk '{print $5}'`"
This works as expected, but has some drawbacks. When using a "here string" the command must first execute in full, then store the output of said command in a temporary variable. This can be a problem if the command takes long to execute or has a large output.
IMHO the best and most standard method of iterating a commands output line by line is the following:
ls -l | grep opencv | awk '{print $5} '| while read -r line ; do
echo "line: $line"
done
I would recommend against using that pipeline to get the sizes of the files you want - in general parsing ls is something that you should avoid. Instead, you can just use *opencv* to get the files and stat to print the size:
stat -c %s *opencv*
The format specifier %s prints the size of each file in bytes.
You can pipe this to awk to get the sum:
stat -c %s *opencv* | awk '{ sum += $0 } END { if (sum) print sum }'
The if is there to ensure that no input => no output.
I need to display the filesize and the filename. Like this:
4.0K Desktop
I'm extracting these two fields using cut from the ls -l output:
ls -lhS | cut -d' ' -f5,9
Due to multiple spaces in the ls -l output, I'm getting a few erroneous outputs, like:
4.0K 19:54
4.0K 19:55
6
18:39
31
25
How should I fix this?
I need to accomplish this task using pipes only and no bash scripting ( output could be multiple pipes ) and preferably no sed, awk.
If no alternative to sed or awk is available- use of sed is OK.
You can avoid parsing ls output and use the stat command which comes as part of GNU coreutils in bash for detailed file information.
# -c --format=FORMAT
# use the specified FORMAT instead of the default; output a newline after each use of FORMAT
# %n File name
# %s Total size, in bytes
stat -c '%s %n' *
You can use translate character command before using cut.
ls -lhS | tr -s ' ' | cut -d' ' -f 5,9
Or you could just submit to awk:
$ ls -lhS | awk '$0=$5 OFS $9'
ie. replace whole record $0 with fields $5 and $9 separated by output field separator OFS.
I'm trying to parse out certain information from a bash script on Ubuntu
I'm having a bash script execute every x seconds which does write:
forever list
this response from that command looks like this:
info: Forever processes running
data: uid command script forever pid logfile uptime
data: [0] _1b2 /usr/bin/nodejs /home/ubuntu/node/server.js 28968 28970 /root/.forever/_1b2.log 0:0:17:17.233
I want to parse out the location of the logfile /root/.forever/_1b2.log
Any ideas how to accomplish this with bash?
Two of the many awk variations to solve this issue:
# most basic
command | awk 'NR==3{ print $8 }' data
# a little bit more robust:
command | awk '$1=="data:" && $2=="[0]" { print $8 }' data
# ^^^^^^^^^
# here I filter on the "[0]" text, but depending your needs
# you might want to use $3=="_1b2" or $4=="/usr/bin/nodejs"
You could try the below GNU and basic sed commands,
$ command | sed -nr 's/^.* (\S+) [0-9]+:[0-9]+\S+$/\1/p'
/root/.forever/_1b2.log
$ command | sed -n 's/^.* \(\S\+\) [0-9]\+:[0-9]\+\S\+$/\1/p'
/root/.forever/_1b2.log
It prints only the non-space characters which is followed by a space + one or more digits + : symbol + one or more digits.
You could grep the line(s) you're looking for and pipe the output to awk:
forever list | grep "\.log" | awk '{print $4}'
This will find all processes with a .log file, and print the log file's location (4th column).
Assuming the logfile is in /root/.forever directory, you can use grep this regex:
forever list | grep -o "/root/\.forever/.*\.log"
I'm connecting to an exadata and want to get information about "ORACLE_HOME" variable inside them. So i'm using this command:
ls -l /proc/<pid>/cwd
this is the output:
2 oracle oinstall 0 Jan 23 21:20 /proc/<pid>/cwd -> /u01/app/database/11.2.0/dbs/
i need the get the last part :
/u01/app/database/11.2.0 (i dont want the "/dbs/" there)
i will be using this command several times in different machines. So how can i get this substring from whole output?
Awk and grep are good for these types of issues.
New:
ls -l /proc/<pid>/cwd | awk '{print ($NF) }' | sed 's#/dbs/##'
Old:
ls -l /proc/<pid>/cwd | awk '{print ($NF) }' | egrep -o '^.+[.0-9]'
Awk prints the last column of the input which is your ls command and then grep grabs the beginning of that string up the last occurrence of numbers and dots. This is a situational solution and perhaps not the best.
Parsing the output of ls is generally considered sub-optimal. I would use something more like this instead:
dirname $(readlink -f /proc/<pid>/cwd)
I have a number of log files in a directory. I am trying to write a script to search all the log files for a string and echo the name of the files and the line number that the string is found.
I figure I will probably have to use 2 grep's - piping the output of one into the other since the -l option only returns the name of the file and nothing about the line numbers. Any insight in how I can successfully achieve this would be much appreciated.
Many thanks,
Alex
$ grep -Hn root /etc/passwd
/etc/passwd:1:root:x:0:0:root:/root:/bin/bash
combining -H and -n does what you expect.
If you want to echo the required informations without the string :
$ grep -Hn root /etc/passwd | cut -d: -f1,2
/etc/passwd:1
or with awk :
$ awk -F: '/root/{print "file=" ARGV[1] "\nline=" NR}' /etc/passwd
file=/etc/passwd
line=1
if you want to create shell variables :
$ awk -F: '/root/{print "file=" ARGV[1] "\nline=" NR}' /etc/passwd | bash
$ echo $line
1
$ echo $file
/etc/passwd
Use -H. If you are using a grep that does not have -H, specify two filenames. For example:
grep -n pattern file /dev/null
My version of grep kept returning text from the matching line, which I wasn't sure if you were after... You can also pipe the output to an awk command to have it ONLY print the file name and line number
grep -Hn "text" . | awk -F: '{print $1 ":" $2}'