I have executed a command that generated data in the form
111111
222222
333333
444444
555555
I have defined a variable in the command and just echo that variable which procedure this data
now I want to store this data into an array, in shell scripting
my_array[0]=111111
my_array[1]=222222
.....
so on
like this
I have tried a couple of things but all of them are storing index[0]. I want to perform some arithmetic operations on this data that why I want it into an array form.
my_array=()
select sum(a1) sum , date(create_timestamp) date from student where date(create_timestamp) >= 'YYYY-MM-DD' and date(create_timestamp) <= 'YYYY-MM-DD' group by date" | sed '1d'| while read sum date; do
# echo $sum
# echo $date
my_array=( "$sum" )
echo $my_array
# echo ${my_array[0]}
done
I think you mean my_array+=( "$sum" ) to add elements rather than overwrite the whole array, but if you trust the output format, why not assign it all at once instead of using a read loop?
mapfile -t my_array < <(
select sum(a1) sum , date(create_timestamp) date
from student
where date(create_timestamp) >= 'YYYY-MM-DD'
and date(create_timestamp) <= 'YYYY-MM-DD'
group by date" | sed '1d'
)
Related
root#node033:~# vi exppass
root#node033:~# bash exppass
exppass: line 7: syntax error: unexpected end of file
root#node033:~# cat exppass
cat /etc/shadow |
while IFS=":" read col1 col2 col3 col4 col5 col6 col8;
do
echo $col3 $col8
if [ $expire -lt $today ];
then
I am trying to remove the expiration passwd using to compare the date and time.
adjust your script to do something along the lines:
if [ $expire -lt $today ]; then
#delete the password
I think $expire above is equivalent to one of the columns you're reading. and you can get today's date by doing something like today=$(date +%s)
If you're happy w/ an awk & bash solution rather than doing the hard lifting in a loop:
awk -F: -v today=$(( $( date "+%s" ) / 86400 )) '$8!=""{print $1, today-$8}' /etc/shadow
Explanation:
-F: defines the input field separator to be a colon.
-v today=$(( $( date "+%s" ) / 86400 )) expresses today's date in days since epoch (which is the format used in /etc/shadow) and assigns it to an awk variable called today.
Now for the awk logic:
$8!="" if the 8th field (Account Expiry) isn't unset, {print $1, today-$8} print the username and the difference between today and the expiry date in days. If the date lies in the past, you get a positive value, if it's in the future, a negative one.
Edit:
Looking at the jumble above it appears that you're trying to check for both password and account expiry:
This should do:
awk -F: -v today=$(( $( date "+%s" ) / 86400 )) '$3!="" || $8!=""{printf "%s\tpw: %s\tacc: %s\n", $1,today - $3, today-$8}' /etc/shadow
The 2nd field now shows the age of the password, the 3rd the account expiration. If this still isn't what you're after you'll need to sit down and rephrase your question so it reflects what you're actually after.
Let's say I have some holiday data (holiday_master.csv) in columns, something like
...
20200320 Vernal Equinox Day
20200429 Showa Day
20200503 Constitution Day
20200505 Green Day
20200720 Children's Day
20200811 Sea Day
...
Given this set of data, I want to find the next closest holiday from the given date.
For example if the input is 20200420, 20200429 Showa Day is expected.
If the input is 20200620, 20200720 Children's Day is expected.
I have a feeling that awk has the necessary functionality to do this, but any solution that works in a bash script is welcome.
Would you please try the bash script:
#!/bin/bash
input="20200428" # or assign to whatever
< "holiday_master.csv" sort -nk1,1 | # sort the csv file by date and pass to the while loop
while read -r date desc; do
if (( date >= input )); then # if the date is greater than or equal to the input
echo "$date" "$desc" # then print the line
break # and exit the loop
fi
done
Assuming no two days will ever have the same date...
DATE=<some desired input date>
awk "{print (\$1 - $DATE"' "\t" $0)}' calendar.txt | sed '/^-/d' | sort | head -n 1 | awk '{$1=""; print $0}'
Explanation
awk "{print (\$1 - $DATE"' "\t" $0)}' calendar.txt: Prepend a column to the input.txt file describing the difference between the desired input date and the date column
sed '/^-/d': Remove all lines beginning with -. Dates with negative differences have already passed.
sort: Sort the remaining entries from least to greatest (based upon the difference column)
head -n 1: Select only the first row (The lowest difference)
awk '{$1=""; print $0}': Print all but the first column
Prettier script version
#!/bin/bash
# Usage: script <Date> <Calendar file>
DATE=${1:--1}
CAL=${2:-calendar.txt}
# Arg check and execute
if[ ! -f $CAL ]
then
echo "File not found: $CAL"
echo "Usage: script <Date> <Calendar file>"
elif [ $DATE -le 0 ]
then
echo "Invalid date: $DATE"
echo "Usage: script <Date> <Calendar file>"
elif [ $(echo "$DATE" | grep -Ewo -- '-?[0-9]+' | wc -l) -eq 0 ]
then
echo "Invalid date: $DATE"
echo "Usage: script <Date> <Calendar file>"
else
awk '{print ($1 - '"$DATE"' "\t" $0)}' $CAL | sed '/^-/d' | sort | head -n 1 | awk '{$1=""; print $0}'
fi
As you use YYYYMMDD format we might compare it just like numbers (note: year is greater than month, month is greater than day). So you can use AWK following way, let:
20200320 Vernal Equinox Day
20200429 Showa Day
20200503 Constitution Day
20200505 Green Day
20200720 Children's Day
20200811 Sea Day
be file named holidays.txt then:
awk 'BEGIN{inputdate=20200420}{if($1>inputdate){print $2;exit}}' holidays.txt
output:
Showa
Explanation: in BEGIN I set inputdate to 20200420 then when line with greater number in 1st column is found I print content of 2nd column and exit (otherwise later dates would be printed too). Note that AWK does automatically parse number when asked to do comparison (> in this case) so you do not have to care about conversion yourself - you could even do inputdate="20200420" and it would work too.
This solution assumes that all dates in file are already sorted.
Using awk and assuming the source data is comma separated:
awk -F, -v dayte="20200420" '
BEGIN {
"date -d "dayte" +%s" | getline dat1
{
{
"date -d "$1" +%s" | getline dat2;
dat3=dat2-dat1;
if (dat3 > 0 )
{
hols[dat3]=$2
}
}
END {
asorti(hols,hols1,"#ind_num_asc");
print hols[hols1[1]]
}
' holiday_master.csv
One liner:
awk -F, -v dayte="20200420" 'BEGIN { "date -d "dayte" +%s" | getline dat1 } { "date -d "$1" +%s" | getline dat2;dat3=dat2-dat1;if (dat3 > 0 ) { hols[dat3]=$2 } } END { asorti(hols,hols1,"#ind_num_asc");print hols[hols1[1]] }' holiday_master.csv
Set the field separator to , and set a variable dayte to the date we wish to check. In the BEGIN block, we pass the dayte variable through to date command via an awk pipe/getline and read the epoch result into the variable dat1. We do the same with the first column on the master file ($1) and read this into dat2. We take the difference between the epoch dates and read the result into dat3. Only if the result is positive (in the future) do we then use dat3 for an index in a "hols" array, with the holiday description as the value. In the END block, we sort the indexes of hols into a news hols1 array basing the sort on ascending, numeric indexes. We then take the first index of the new hols1 array to attain the holiday that is closest to the dayte variable.
Assuming the holiday list file is sorted by date as you have given, the below would work
$ awk -v dt="20200420" ' (dt-$1)<0 { print;exit } ' holiday.txt
20200429 Showa Day
$ awk -v dt="20200620" ' (dt-$1)<0 { print;exit } ' holiday.txt
20200720 Children's Day
$
If the holiday file is not sorted, then you can use below
$ shuf holiday.txt | awk -v dt="20200420" ' dt-$1<0 { a[(dt-$1)*-1]=$0 } END { asort(a); print a[1] } '
20200429 Showa Day
$ shuf holiday.txt | awk -v dt="20200620" ' dt-$1<0 { a[(dt-$1)*-1]=$0 } END { asort(a); print a[1] } '
20200720 Children's Day
I would like to rank log entries by the timestamp of each entry.
let's say my grep result is like this, with each entry having different number of fields and time on different number of columns:
a, 3, time:123
b, time:124, 4
c, time:122, 5
how should I pipe the result such that it looks like this?
c, time:122, 5
a, 3, time:123
b, time:124, 4
Would you try the following:
while IFS= read -r line; do
[[ $line =~ time:([0-9]+) ]] && printf "%s\t%s\n" "${BASH_REMATCH[1]}" "$line"
done < file | sort -n | cut -f 2-
It first extracts the time after the time: substring.
Then it prepends the time before the line using a tab as a delimiter.
It numerically sorts the lines.
Finally it cuts off the 1st field.
A general solution is:
for each line:
detect log format
extract timestamp column based on detected format
convert timestamp into sortable-form
print sortable-form + column delimiter + original line
pipe output of previous stage into something that sorts on the new first column
pipe output of previous stage into something that strips off the new first column
I have a source file with 18 columns in which columns 10 , 11 and 15 are in the format dd/mm/yyyy and all these needs to be converted to yyyy-mm-dd and written to target file along with other columns.
I am aware of date formatting functions on Variables but do not know how to apply the same on few columns in a file.
I don’t have a machine available to test, but consider using awk with a little function since you are doing the same thing 3 times. It will look something like this:
awk ‘
function dodate(in){
split(in,/\//,a) # split existing date into elements of array “a”
return a[3] “-“ a[2] “-“ a[1]
}
{ $10=dodate($10); $11=dodate($11); $15=dodate($15); print }’ yourFile
Reference for awk functions, and split.
If the fields on each line are separated by commas, tell awk that with:
awk -F, ...
Maybe you could use command awk to solve it.
As you have 3 cols contain date (col 10, 11, 15), here I assume a sample string which field seperator is |, col contains date is the 4th col
aa|bb|cc|29/09/2017|dd|ee|ff
use String-Manipulation Functions to extract date, then format it with getline to format it to expected syntax.
command is
echo 'aa|bb|cc|2017-09-29|dd|ee|ff' | awk -F\| 'BEGIN{OFS="|"}{$4=gensub(/([0-9]{1,2})\/([0-9]{1,2})\/([0-9]{4})/,"\\3\\2\\1","g",$4); "date --date=\""$4"\" +\"%F\"" | getline a; $4=a; print $0}'
output is
aa|bb|cc|2017-09-29|dd|ee|ff
Hope to help you.
If you have the dateutils package installed, you can use dateutils.dconv
cat file | dateutils.dconv -S -i "%d/%m/%Y"
-i specify input date format
-S sed mode, process only the matched string and copy the rest
Input File
aa|bb|cc|29/09/2017|dd|ee|ff|02/10/2017|gg
Output
aa|bb|cc|2017-09-29|dd|ee|ff|2017-10-02|gg
I'd use the date command:
while read fmtDate
do
date -d ${fmtDate} "+%Y-%m-%d"
done
I'm new to bash. I am trying to get 2 inputs from the user in the form: DD/MM/YYYY DD/MM/YYYY(day,month,year in one line). Here is what I have tried for dd (I also will need to get MM and YYYY from both inputs):
dd1=read | cut -d'/' -f1 (I tried this with backquotes and it didn't work)
[do something with dd1 ...]
echo $dd1
$dd1 keeps coming up as empty. I could use some pointers (not specific answers) for my homework question. Thanks.
You got it backwards, try like this;
read dd1 && echo $dd1|cut -d'/' -f1
Give this a try. It will allow the user to type the date in and will split it for you on the slashes.
IFS=/ read -r -p "Enter a date in the form DD/MM/YYYY: " dd mm yy
Do you need to do this on one line, or do you want the user to INPUT two dates on one line?
If all you need is for the users to input two dates on the command line, you can do this:
read -p "Enter two dates in YY/MM/DD format: " date1 date2
Then, after the user enters in the two dates, you can parse them to verify that they're in the correct format. You can keep looping around until the dates are correct:
while 1
do
read -p "Enter two dates in 'DD/MM/YYY' format: date1 date2
if [ ! date1 ] -o [ ! date2 ]
then
echo "You need to enter two dates."
sleep 2
continue
if
[other tests to verify date formats...]
break # Passed all the tests
done
If you can input the dates one at a time, you can manipulate the IFS variable to use slashes instead of white spaces as separators.
OLDIFS="$IFS" #Save the original
IFS="/"
read -p "Enter your date in MM/DD/YYYY format: " month day year
IFS="$OLDIFS" #Restore the value of IFS
This could be put inside a while loop like the example above where you could verify the dates were entered in correctly.
Actually, you could do this:
OLDIFS="$IFS" #Save the original
IFS="/ " #Note space before quotation mark!
read -p "Enter two dates in MM/DD/YYYY format: " month1 day1 year1 month2 day2 year2
IFS="$OLDIFS" #Restore the value of IFS
echo "Month #1 = $month1 Day #1 = $day1 Year #1 = $year1"
echo "Month #2 = $month1 Day #2 = $day2 Year #2 = $year2"
And get both dates on the same command line.