I am working on shellscript with excel sheet. Till now I have done as shown in screenshot by using below command:
bash execution.sh BehatIPOP.xls| awk '/Script|scenario/' | awk 'BEGIN{print "Title\tResult"}1' | awk '0 == NR%2{printf "%s",$0;next;}1' >> BehatIPOP.xls
My requirement is along with the heading Result I want to add(concat) current date also. So I am getting date by using below command:
$(date +"%d-%m-%y %H:%M:%S")
So date will display like this : 25-08-2016 17:00:00
But I am not getting how can use date command in the above mentioned command to achieve heading like below:
| Title | Result # 25-08-2016 17:00:00|
Thanks for any suggestions..
You can pick up the date inside awk and store it in a variable d like this, if that is what you mean:
awk 'BEGIN{cmd="date +\"%d-%m-%y %H:%M:%S\""; cmd |getline d; close(cmd);print "Result # " d}'
Result # 25-08-16 13:44:05
Don't use awk at all for the header, just use date directly:
{ printf "Title\tResult # "; date +"%d-%m-%y %H:%M:%S"; bash execution.sh BehatIPOP.xls |
awk '/Script|scenario/' |
awk '1 == NR%2{printf "%s",$0;next;}1'; } >> BehatIPOP.xls
Note that there's no need for 2 awks, but I'm keeping that here to minimize the diff. Since I've pulled the header out of the awk, the comparison changes from 0==NR%2 to 1==NR%2.
Related
Tried various approaches, but nearest to working one:
Replace multiple spaces with single one
Replace commas(,) in INTERNAL_IP column with Pipe(|)
Remove 4th cloumn (PREEMPTIBLE) as it was causing IPs in INTERNAL_IP cloumn shift under it.
Replace space with comma(,) to prepare a csv file.
But did not work. Gets messed up at PREEMPTIBLE cloumn.
gcloud compute instances list > file1
tr -s " " < file1 > file2 // to replace multiple spaces with single one
sed s/\,/\|/g file2 > file3 // to replace , with pipe
awk '{$4=""; print $0}' file3 // to remove 4th column
sed -e 's/\s\+/,/g' file3 > final.csv
Output of gcloud compute instances list command:
Expected format:
Any help or suggestion is appreciated. Thank you in advance.
Edit:
Attached sample input and expected output files:
sample_input.txt
expected_output.xlsx
csv format is supported in gcloud CLI so everything you are doing can be done without sed/awk maybe with | tail -n +2 if you want to skip the column header :
gcloud compute instances list --format="csv(NAME,ZONE,MACHINE_TYPE,PREEMPTIBLE,INTERNAL_IP,EXTERNAL_IP,STATUS)" > final.csv
Or if you wanted to do something with the data in your bash script:
while IFS="," read -r NAME ZONE MACHINE_TYPE PREMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS
do
echo "NAME=$NAME ZONE=$ZONE MACHINE_TYPE=$MACHINE_TYPE PREMPTIBLE=$PREMPTIBLE INTERNAL_IP=$INTERNAL_IP EXTERNAL_IP=$EXTERNAL_IP STATUS=$STATUS"
done < <(gcloud compute instances list --format="csv(NAME,ZONE,MACHINE_TYPE,PREEMPTIBLE,INTERNAL_IP,EXTERNAL_IP,STATUS)" | tail -n +2 | awk ' BEGIN {print "NAME,ZONE,MACHINE_TYPE,PREMPTIBLE,INTERNAL_IP,EXTERNAL_IP,STATUS"} {print $1","$2","$3","" "","$4","" "","$5}' )
Based on attached files sample input & expected output i have made following change :
Some of the instances for multiple internal IPs and they are
separated by ",". I have replaced that "," with "-" using sed
's/,/-/g' to aviod conflicts with other fields as we are
generating a CSV.
Displaying $4 & $6 in 5th & 7th columns so that they will be aligned
with Column Headers Internal IP Address and Status
cat command_output.txt | grep -v 'NAME' | sed 's/,/-/g' | awk ' BEGIN {print "NAME,ZONE,MACHINE_TYPE,PREMPTIBLE,INTERNAL_IP,EXTERNAL_IP,STATUS"} {print $1","$2","$3","" "","$4","" "","$5}'
I have a directory /user/reports under which many files are there, one of them is :
report.active_user.30092018.77325.csv
I need output as number after date i.e. 77325 from above file name.
I created below command to find a value from file name:
ls /user/reports | awk -F. '/report.active_user.30092018/ {print $(NF-1)}'
Now, I want current date to be passed in above command as variable and get result:
ls /user/reports | awk -F. '/report.active_user.$(date +'%d%m%Y')/ {print $(NF-1)}'
But not getting required output.
Tried bash script:
#!/usr/bin/env bash
_date=`date +%d%m%Y`
active=$(ls /user/reports | awk -F. '/report.active_user.${_date}/ {print $(NF-1)}')
echo $active
But still output is blank.
Please help with proper syntax.
As #cyrus said you must use double quotes in your variable assignment because simple quote are use only for string and not for containing variables.
Bas use case
number=10
string='I m sentence with or wihtout var $number'
echo $string
Correct use case
number=10
string_with_number="I m sentence with var $number"
echo $string_with_number
You can use simple quote but not englobe all the string
number=10
string_with_number='I m sentence with var '$number
echo $string_with_number
Don't parse ls
You don't need awk for this: you can manage with the shell's capabilities
for file in report.active_user."$(date "+%d%m%Y")"*; do
tmp=${file%.*} # remove the extension
number=${tmp##*.} # remove the prefix up to and including the last dot
echo "$number"
done
See https://www.gnu.org/software/bash/manual/bashref.html#Shell-Parameter-Expansion
I am using the paste command as below:
message+=$(paste <(echo "${arr[0]}") <(echo "$starttimeF") <(echo "$ttime") <(echo "$time") | column -t)
echo "$message"
It gives me the following output:
ASPPPLD121 11:45:00 00:00:16 00:02:23
FPASDDF123 11:45:00 00:00:16 00:02:23
ZWASD77F0D 09:04:58 02:40:18 03:51:10
DDPADSDSD5 11:29:41 00:15:35 01:17:33
How do I redirect this to a CSV or an EXCEL FILE?
OR
How do I put it into HTML table?
You have three questions in one. The easiest to solve is the CSV file output:
echo $message | awk 'BEGIN{OFS=","}{$1=$1}1' > mycsvfile.csv
This will also open in excel since excel can read csv files, so maybe that solves the second question?
Awk is opening the file and reading line by line. We set the OFS (Output Field Seperator) to a comma. Then we just set one of the fields equal to itself, which is a just a trick to get awk to process the record, and let it print out the results to the CSV file.
I suppose you could use awk to print out a HTML table too, but it seems like a gross way of doing things:
echo $message | awk 'BEGIN{print "<table>"} {print "<tr>";for(i=1;i<=NF;i++)print "<td>" $i"</td>";print "</tr>"} END{print "</table>"}'
html output awk script stolen from here
if [ -f $FILEPATH3 ]; then
#Will print the Header columns from properties file.
print $header >$CFILEPATH3
#To add rows in output file from input file .
awk -F\" 'BEGIN{OFS=FS;} {for(i=1;i<=NF;i=i+2){gsub(/,/,"~\t",$i);}}1' $FILEPATH3 > $TFILEPATH3
#Removes the footer,header and prints the columns as per mapping by picking column numbers from properties file
cat $TFILEPATH3| sed '1d' | awk 'BEGIN { FS = "~\t"; OFS = ",";}{ DATE = date -d "$'$ms2'" "+%Y%m%d" } { printf "%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s\n", $'$ms1', $DATE,$'$ms3',$'$ms4',$'$ms5',$'$ms6',$'$ms7',$'$ms8’,”MS”,$’$ms10',$'$ms11',$'$ms12',$'$ms13',$'$ms14',$'$ms15',$'$ms16',$'$ms17'}' >> $CFILEPATH3
In the above code, I'm trying to copy the data from the input file to output file. ms1, ms2 are the column positions of a CSV input file.
ms2 is the date with format mm/dd/yyyy and which is considered as variable. We need to convert the variable into YYYYMMDD format and write it into output file
In the script I'm trying to change the date format to YYYYMMDD.. but I'm getting an error.
I think the error is from this code
{ DATE = date -d "$'$ms2'" "+%Y%m%d" }
You are trying to access the variable in single quotes and which wont happen, So try to use the following syntax.
DATE=$(echo $ms2|awk -F '/' '{print $3$2$1}')
cat $TFILEPATH3| sed '1d' | awk -v ms2="$DATE" 'BEGIN { FS = "~\t"; OFS = ",";} { printf "%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s\n", $'$ms1', $ms2,$'$ms3',$'$ms4',$'$ms5',$'$ms6',$'$ms7',$'$ms8’,”MS”,$’$ms10',$'$ms11',$'$ms12',$'$ms13',$'$ms14',$'$ms15',$'$ms16',$'$ms17'}' >> $CFILEPATH3
I have a file with records having timestamp fields that include GMT offset. I want to use the sed command to replace the value on the record to a regular timestamp (without GMT offset).
For example:
`$date -d '2012/11/01 00:50:22 -0800' '+%Y-%m-%d %H:%M:%S'`
returns this value which is what I am looking to do:
2012-11-01 01:50:22
Except I want to perform that operation on every line of this file and apply the date command to the timestamp value. Here is a sample record:
"SB","6GV96644X48128125","","","","T0006",2012/10/03 13:08:43 -0700,"NJ"
Here is my code:
head -1 myfile | sed 's/,[0-9: /\-]\{25\},/,'"`date -d \1 '+%Y-%m-%d %H:%M:%S'`"',/
which doesn't work: it just ignores \1 and replaces the matched pattern with today's date:
"SB","6GV96644X48128125","","","","T0006",2012-11-14 01:00:00,"NJ"
I hoped that \1 would result in the matched patterns be passed to the date function and return a regular timestamp value (as in the example I provided above showing how the date functions applies the GMT offset and returns a regular stimestamp string) and would replace the old value on the record.
I would use awk instead. For example:
awk '{cmd="date -d \""$7"\" \"+%Y-%m-%d %H:%M:%S\"";
cmd | getline k; $7=k; print}' FS=, OFS=, myFile
This will replace the 7th field with the results of running the date command on the original contents of the 7th field.
In sed:
head -1 datefile |
sed '
# handle % in input correctly
s/%/%%/g
# execute date(1) command
s/\(.*,\)\([0-9: /\-]\{25\}\)\(,.*\)/'"date -d '\2' '+\1%Y-%m-%d %H:%M:%S\3'"'/e'
'
This might work for you (GNU sed):
sed -r 's/^(([^,]*,){6})([^,]*)(.*)/printf "%s%s%s" '\''\1'\'' $(date -d '\''\3'\'' '\''+%Y-%m-%d %H:%M:%S'\'') '\''\4'\''/e;q' file
use
head -1 datefile | sed -e 's?\(..\)/\(..\)/\(.... ..:..:..\)?'"date -d '\2/\1/\3' '+%s'"'?e'