My Data Looks like this:
File_A
2015-08-01 00:00 424.9563018750976
2015-08-01 00:01 785.7380434263472
....
2015-08-01 23:59
File_B
2015-08-02 00:01 1.4625096463725384
2015-08-02 00:02 6.0047607076482485
....
2015-08-02 23:59
So the Date Format is: %Y-%m-%d %H:%M
I'd like to plot a series for the whole month but for each day only the timeperiod fom e.g. 09:00 - 15:00.
Here's what I tried:
clear
reset
cd 'C:\Users\SammerP\Desktop\Desktop Calc\AIT ALL\CatchAllDataAIT\data'
set title 'Data of Month' font 'bold verdana,08'
set xdata time
set timefmt '%Y-%m-%d %H:%M'
set format x '%Y-%m-%d %H:%M'
set ylabel 'Value'
set yrange [550:700]
set ytics (550,560,570,580,590,600,610,620,630,640,650,660,670,680,690,700,710,720,730,740,750,760,770,780,790,800)
set xtics rotate by 45 offset -1.5,-1.2
set xrange ['2014-08-01 09:00':'2014-08-31 15:00']
set xtics ('08-01' '2014-08-01 13:00','08-02' '2014-08-02 13:00','08-03' '2014-08-03 13:00','08-04' '2014-08-04 13:00','08-05' '2014-08-05 13:00',\
'08-06' '2014-08-06 13:00','08-07' '2014-08-07 13:00','08-08' '2014-08-08 13:00','08-09' '2014-08-09 13:00','08-10' '2014-08-10 13:00','08-11' '2014-08-11 13:00',\
'08-12' '2014-08-12 13:00','08-13' '2014-08-13 13:00','08-14' '2014-08-14 13:00','08-15' '2014-08-15 13:00','08-16' '2014-08-16 13:00',\
'08-17' '2014-08-17 13:00','08-18' '2014-08-18 13:00','08-19' '2014-08-19 13:00','08-20' '2014-08-20 13:00','08-21' '2014-08-21 13:00',\
'08-22' '2014-08-22 13:00','08-23' '2014-08-23 13:00','08-24' '2014-08-24 13:00','08-25' '2014-08-25 13:00','08-26' '2014-08-26 13:00',\
'08-27' '2014-08-27 13:00','08-28' '2014-08-28 13:00','08-29' '2014-08-29 13:00','08-30' '2014-08-30 13:00','08-31' '2014-08-31 13:00')
set grid
set nokey
plot'File_A.csv' using 1:2 axes x1y1 linewidth 1 lc rgb 'red' w l,\
'File_B' using 1:2 axes x1y1 linewidth 1 lc rgb 'red' w l,
..........
I tried it by plotting only the certain line from each file but than I end up with hole between the days.
I thought that maybe the problem was the xrange setting so I deleted that but the Problem was the same
Is there a way to set the xrange so that Gnuplot plots From Day 1 to 31, every Day from 09:00 from 15:00 and without the empty spaces between the days.
Here's an example of how it looks like now:
That cannot be done automatically with gnuplot. In my opinion, your best option is to not use a time-axis, but a normal numerical axis and provide your own mapping to label the xtics properly and to adjust the gap between the days like you want.
Here is a possible solution, but without having proper data for testing I couldn't verify if I got all steps correct:
gapBetweenDaysInSeconds = 1000
startHour = 9.0
endHour = 15.0
set xtics ("01-08" (13-startHour)*60*60)
set for [i=2:31] xtics add (sprintf("08-%02d", i) ((i-1)*(endHour-startHour) + 13 - startHour)*60*60 + (i-1)*gapBetweenDaysInSeconds)
# for gnuplot 5.0
# gettime(col) = timecolumn(col, '%Y-%m-%d %H:%M')
# for all gnuplot version.
gettime(col) = strptime('%Y-%m-%d %H:%M', strcol(col).' '.strcol(col+1))
files="File_A.csv File_B.csv File_C.csv" # ... to be continued
plot for [file in files] file using (t=gettime(1), tm_hour(t) < startHour || tm_hour(t) >= endHour ? 1/0 : (((tm_mday(t)-1)*(endHour-startHour) + (tm_hour(t)-startHour))*60 + tm_min(t))*60 + tm_sec(t) + (tm_mday(t)-1)*gapBetweenDaysInSeconds):2 lt 1
That assumes, that you have one file per day, and those files are listed, separated by white spaces, in the variable files.
The first call to set xtics sets only a single xtic and deletes all other. Then I use set xtics add to add all following tics in a loop.
For each row, the time stamp is parsed using the gettime user-defined function and assigned to the variable t. In a second step I extract only the day of the mont, hour, minutes and seconds using the tm_* functions and calculate a position on the xaxis given in seconds since the first of the respective month. The gap between two days can be given using the variable gapBetweenDaysInSeconds.
At the moment the month is hardcoded to 08.
That should be a reasonable starting point.
Related
I have a file of data where the first column is time in seconds. The start of time is 0, and onward from there.
I want to plot the data with an x-axis formatted as days:hours:minutes:seconds. T=0 should map to 00:00:00:00. I can't figure out how to get days to start at 00 instead of 01. I have tried the below. I also tried setting xrange to [-86400:173000], but that maps to day 365, not 0. Shouldn't it be common to plot some time-sampled data, that may span days, starting with T=0?
It seems that GnuPlot needs a different set of time format characters for zero-based time plotting, instead of date-based. Unless it already has it and I have missed it.
data
0 0
3600 10
7200 30
21600 50
160000 100
GnuPlot script
set xdata time
set format x "%02j:%H:%M:%S"
set timefmt "%s"
set xrange [0:173000]
plot "data" using 1:2 with lines
I can get you part way there. Gnuplot has a separate set of time formats for relative time. Zero-based and handles positive and negative intervals. It's hard to find in the documentation, but here is a section from "help time_specifiers".
Format Explanation
%tH +/- hours relative to time=0 (does not wrap at 24)
%tM +/- minutes relative to time=0
%tS +/- seconds associated with previous tH or tM field
Examples of time format:
The date format specifiers encode a time in seconds as a clock time
on a particular day. So hours run only from 0-23, minutes from 0-59,
and negative values correspond to dates prior to the epoch
(1-Jan-1970). In order to report a time value in seconds as some
number of hours/minutes/seconds relative to a time 0, use time
formats %tH %tM %tS. To report a value of -3672.50 seconds
set format x # default date format "12/31/69 \n 22:58"
set format x "%tH:%tM:%tS" # "-01:01:12"
set format x "%.2tH hours" # "-1.02 hours"
set format x "%tM:%.2tS" # "-61:12.50"
Using these relative time formats with your sample data I can get as far as:
$data << EOD
0 0
3600 10
7200 30
21600 50
160000 100
EOD
set xtics time format "%tH:%tM:%tS"
set title 'set xtics time format "%tH:%tM:%tS"'
set xrange [0:173000]
plot $data using 1:2 with lp
Now the problem is that there is no equivalent relative day format. Call that a bug or at least a missing feature. Let's take a stab at adding days to the format by hand.
secperday = 3600*24
days(t) = gprintf("%02g:", int(t)/secperday)
hours(t) = strftime("%02tH:%tM:%tS", int(t)%secperday)
# Create ten days worth of tic labels
# Every six hours with no label; once a day with full label
set xtics 6*3600 format ""
do for [i=0:10] {
T = day * secperday
set xtics add ( days(T).hours(T) T )
}
plot $data using 1:2 with lp
As mentioned in the comments above, one workaround would be using week days, which however would limit you to 7 days.
Since 0 seconds correspond to Thursday, 01.01.1970 00:00:00 you have to subtract 4 days = 24*3600*4 seconds to make it a Sunday (=0).
Another strange workaround would be to use multiplot and plot twice, just for the day labels. You have to set a bottom margin to exactly "overplot" the previous plot. There would be still room for fine tuning.
By the way: If the scale is several days then the question is if seconds in the label are actually relevant?
Code:
### timedate days starting from zero
reset session
$Data <<EOD
0 0
3600 10
7200 30
21600 50
160000 100
450000 222
500000 333
EOD
set multiplot layout 2,1
# first workaround, limited to 7 days
set format x "day %1w\n%H:%M:%S" timedate
plot $Data u ($1-24*3600*4):2 w lp pt 7 notitle
# second workaround, using multiplot
set format x "\n%H:%M:%S" timedate
set bmargin 3
plot $Data u 1:2 w lp pt 7 notitle
set multiplot previous
set format x "day %s"
set xrange[GPVAL_X_MIN/86400:GPVAL_X_MAX/86400]
plot $Data u ($1/86400):2 w p ps 0 notitle # point size zero, i.e. invisible
unset multiplot
### end of code
Result:
$data << EOD
1563619139 10
1532083139 9
1500547139 8
1469011139 7
1437388739 6
1405852739 5
1374316739 4
1342780739 3
1311158339 2
1279622339 1
EOD
set terminal png
set xdata time
set timefmt "%s"
set format x '%Y'
unset key
plot '$data' u 1:2
How do I plot values only from say 2015? I tried plot ["2015":] '$data' u 1:2 via the docs but it doesn't work as expected.
I realise I could edit $data, but I don't want to do that.
There is an older an a newer gnuplot syntax for timedata.
The example below uses the newer syntax.
Check help time/date, help timecolumn, and help strptime.
Code:
### time data
reset session
$Data << EOD
1563619139 10
1532083139 9
1500547139 8
1469011139 7
1437388739 6
1405852739 5
1374316739 4
1342780739 3
1311158339 2
1279622339 1
EOD
unset key
set format x "%Y" time
StartTime = strptime("%Y","2015") # 2015-01-01 00:00:00 in seconds after 1970
set xrange[StartTime:]
set xtics StartTime, 3600*24*365 # start time and major tic distance one year in seconds
plot $Data u (timecolumn(1,"%s")):2 w lp pt 7
### end of code
Result:
You must use the same format of timefmt to specify the range.
plot ["1420092000":] '$data' u 1:2
The seconds in Gnuplot are measured starting from 1970.
I calculated the starting time considering 365.25 days in a year, hence 1420092000 s, but using strptime("%Y","2015"), as reported in the other answer, is without doubt more correct and precise.
You can add
set xtics 31557600
to have only major tics corresponding to the beginning of year. The value specified is the time increment between tics that must be given in seconds.
I'm trying to plot the following data
29.07.2012 18:45:04;23.6;54
29.07.2012 18:50:04;22.7;56
29.07.2012 18:55:04;22.2;56
29.07.2012 19:00:04;22.0;56
29.07.2012 19:05:04;21.9;57
29.07.2012 19:10:04;21.8;56
29.07.2012 19:15:04;21.8;54
29.07.2012 19:20:04;21.7;53
29.07.2012 19:25:04;21.7;53
(Date, time, temperature, humidity) in the following style (cropped at the top):
The labels on the x axis are the hour from the time of day and below are the weekdays and the date. I don't have the weekdays in my data file, but I'd like to have the date below the hours.
My plotfile:
set datafile separator ";"
set terminal png size 5280,1024
set output '~/tfd.png'
set xdata time
set timefmt "%d.%m.%Y %H:%M:%S"
set format x "%H"
plot "data.csv" using 1:2 title 'temperatur'
I can think of three method to do this. If you don't have to have those dates on the same axis, the second method is probably the most stable. Both the first and third methods have their advantages and disadvantages. Between those two, the third is possibly the better approach, but it requires more work.
For these examples, in order to make sure that the data would span more than 1 day, I used your same data but added one extra line
31.07.2012 19:30:04;22.7;53
All three methods work with version 5.0.
Method 1 does not line up correctly in version 4.6, but can be made to with one extra command.
Method 2 should work in any reasonably recent version.
Method 3 will not place all date labels in version 4.6 due to an overflow bug in iteration (see here for some explanation), but can be made to work by changing the iteration to place the labels.
Method 1 - Multiplot
We can do this by superimposing the same plot over itself using multiplot and doing the x-axis different each time.
set datafile separator ";"
set xdata time
set timefmt "%d.%m.%Y %H:%M:%S"
# Increase bottom margin to allow room for dates
set bmargin at screen 0.1
set multiplot layout 1,1
# tics starting at 0 every 6 hours showing hour
set xtics 0,60*60*6 format "%H"
plot "data.csv" using 1:2 with lines t "Temperature"
# Tics starting at 0 every 24 hours showing day.month
# moved down by 1 character to be under hours
set xtics 0,60*60*24 format "%d.%m" offset 0,-1
set origin 0,0 # This is not needed in version 5.0
replot
unset multiplot
Other than the difference in the axis labels, the plots must be identical to avoid them not lining up, and it does cause the y-axis labels to be slightly bolded as they are written over themselves.
In version 5.0 the set origin command is not needed, but is needed in version 4.6.
Method 2 - Secondary Axis
If using the secondary axis is acceptable, you could also approach it that way. For example, if the day is shown on the x2 axis and the hour on the x1 axis, we could do
set datafile separator ";"
set xdata time
set x2data time
set timefmt "%d.%m.%Y %H:%M:%S"
set xtics 0,60*60*6 format "%H"
set x2tics 0,60*60*24 format "%d.%m"
plot "data.csv" using 1:2 with lines t "Temperature"
This eliminates some of the problems of the multiplot method, but results in the two data labels being on different axes.
Method 3 - Setting Manual Labels
Finally we can manually set the labels. Fortunately, we can use a loop in the most recent version of gnuplot, so we don't have to issue a separate command for it, but we do have to compute the labels ourselves.
We can use the stats command to compute the labels. The stats command will complain if we give it time data, so we must use it before setting the time mode on, and we must do a little bit of work for computing the day boundaries. To make sure that we are working with the start of each day and not sometime in the middle, we parse the dates into an internal representation (seconds since the Unix Epoch), and round down to the nearest multiple of 86400 (the number of seconds in a day).
Altogether we can do
# in-large margin for date labels
set bmargin at screen 0.1
set datafile separator ";"
# Get first and last day in data file as STATS_min and STATS_max
stats "data.csv" u (floor(strptime("%d.%m.%Y %H:%M:%S",stringcolumn(1))/86400)*86400) nooutput
set xdata time
set timefmt "%d.%m.%Y %H:%M:%S"
set for [i=STATS_min:(STATS_max+86400):86400] label strftime("%d.%m",i) at i,graph 0 center offset 0,char -2
# set xtics every 6 hours
set xtics 0,60*60*6 format "%H"
plot "data.csv" using 1:2 with lines t "Temperature"
We can improve this by numbering the labels if we need to later remove them ((i-STATS_min)/86400+1 will number them 1, 2, 3, etc). Note that like the first method we needed to increase the margin size on the bottom. I added one extra day to the labels to cover the possible rounding up that gnuplot will do on the x-axis.
There is a bug dealing with iteration and integer overflow in version 4.6. To use this solution in 4.6, change
set for [i=STATS_min:(STATS_max+86400):86400] label strftime("%d.%m",i) at i,graph 0 center offset 0,char -2
to
days = (STATS_max-STATS_min)/86400+1
set for [i=0:days] label strftime("%d.%m",i*86400+STATS_min) at (i*86400+STATS_min),graph 0 center offset 0,char -2
i have a csv file from 5 year malware data collected there are 2 columns the dates and the ips every date have 1 or more ips example
1/5/2013 12.234.123
1/5/2013 45.123.566
1/5/2013 100.546.12
1/6/2013 42.153.756
3/4/2014 75.356.258 etc... (every day for 5 years)
now i am trying to get the percentage difference between every month example:
November 2014 - 10%
December 2014 - 15%
i tried to put the percentage on y axis and in x2 axis but im getting some crazy results i am new to gnuplot and im still learning it here is the code i have right now:
set title 'Results Per Month'
set xlabel 'Date'
set ylabel 'Percentage'
set terminal png size 2800,900
set datafile sep ','
set xdata time
set timefmt '%Y/%m/%d'
set xrange['2009/3/22':'2014/12/02']
set xtics 30*24*60*60
set format x '%Y/%m'
set autoscale x2fix
set x2tics
set x2range[0:*]
set format x2 "%g %%"
set xtics nomirror rotate by -90
set grid ytics xtics
set ytics 10
set yrange [0:*]
set term png
set output 'file.png'
plot 'export.csv' using (timecolumn(1) - (tm_mday(timecolumn(1))-1)*24*60*60):(1) smooth frequency w lp pt 7 ps 2 notitle, \
'' using (($1-$2)/$1*100):x2ticlabels(2) axes x2y1 with points ps 2 lw 2
I would suggest you to use some external script for such kind of preprocessing (you can also do this on-the-fly). Yes, you can do this in gnuplot in two steps, but can become quite complicated and requires some more profound knowledge of gnuplot.
Here is a working script, but I won't go into detail about the many different aspects of the actual implementation:
set xdata time
set timefmt '%Y/%m/%d'
set datafile separator ','
set table 'temporaryfile.dat'
set format x '%Y/%m/%d'
plot 'export.csv' using (timecolumn(1) - (tm_mday(timecolumn(1))-1)*24*60*60):(1) smooth frequency
unset table
set y2tics
set ytics nomirror
set timefmt '"%Y/%m/%d"'
set format x '%b %Y'
set xtics rotate by 90 right
set datafile separator white
set yrange[0:*]
x0=x1=0
plot 'temporaryfile.dat' using 1:(strcol(3) eq "i" ? $2 : 1/0) w lp pt 7 ps 2 title 'IP count', \
'' using 1:(x1=x0, x0=$2, strcol(3) eq "i" ? ($0 == 0 || x0 == 0 ? 0 : (x0-x1)/x0 * 100.0) : 1/0) axes x1y2 w lp title 'percentual change'
Basically, first you plot the result data of smooth frequency into a second data file. Then you can plot this, and to the calculations for the percentages.
Please note, that I used a timeformat which corresponds to your test data (and the data of your previous question), which doesn't correspond with what you have in your script! Please pay attention to this.
Also note, that the timefmt before the actual plot must be extended by quote signs which are written around the dates in tmp.dat.
Finally, the strcol(3) eq 'i' is necessary to circumvent a gnuplot bug, which causes a last line to be written with invalid data.
I have some csv data I'm trying to plot in gnuplot.
example:
1,2014-11-07T17:01
2,2014-11-08T11:53
3,2014-11-08T18:50
4,2014-11-09T09:42
5,2014-11-10T11:40
6,2014-11-11T12:34
I'm using
set xtics format "%a"
to show a short day name of the week (Mon, Tue, Wed) which is shown once per day at midnight.
How can I get gnuplot to show the xtics at midday/noon/12pm rather than at midnight?
This is a bit similar to the question mixing date and time on gnuplot xaxis. It's basically about extracting the timestamp for the first noon based on the first date in your data file. Then you can use this timestamp as start for setting the xtics.
The extraction of the first noon is done with stats, strptime and some other time functions:
reset
fmt = "%Y-%m-%dT%H:%M"
set datafile separator ','
stats 'test.txt' using (strptime(fmt, strcol(2))) every ::::0 nooutput
t = int(STATS_min)
t_start_midnight = t - tm_hour(t)*60*60 - tm_min(t)*60 - tm_sec(t)
t_start_noon = t_start_midnight - 12*60*60
Then you can plot your data with
set xdata time
set timefmt fmt
set xtics t_start_noon, 24*60*60
set format x '%a'
plot 'test.txt' using 2:1 with lp pt 7 ps 2 notitle
Note, that stats doesn't work in time mode, and must be invoked before set xdata time.