I have a list a of data points read from file in gnuplot. Is there any way to count the number of negative values in the list, and store the result as a variable?
I could not find anything upon a quick Google search.
Thank you for your help.
There are at least two ways to extract the desired number, it depends how you want to use this number further.
count the negative values with stats (check help stats) and a comparison. $2<0 returns 1 or 0. Hence, the variable STATS_sum will contain the number of negative values.
or alternatively
count the negative values during plotting using the ternary operator (check help ternary). Plot the number into your plot using plotting style with labels.
Script:
### count negative values
reset session
# create some random test data
set table $Data
plot '+' u 1:(rand(0)*10-1) w table
unset table
set key noautotitle
set xzeroaxis lt -1
stats $Data u ($2<0) nooutput
print sprintf("Negative values: %d", STATS_sum)
plot c=0 $Data u 1:($2<0?c=c+1:0,$2) w lp pt 7 lc "red" ti "Data", \
'+' u (0):(10):(sprintf("Negative values: %d",c)) every ::::0 w labels offset 0,-1
### end of script
Result:
Negative values: 6
Related
I wonder whether there is a way to subtract smoothed data from original ones when doing things of the kind:
plot ["17.12.2020 08:00:00":"18.12.2020 20:00:00"] 'data3-17-28.csv1' using 4:5 title 'Sensor 3' with lines, \
'' using 4:5 smooth acsplines
Alternatively I would need to do it externally, of course.
As #Suntory already suggested you can plot smoothed data into a table.
However, keep in mind, the number of datapoints will be determined by set samples, default setting is 100 and the smoothed datapoints will be equidistant. So, if you set samples to the number of your datapoints and your data is equidistant as well, then all should be fine.
Concatenating data line by line is not straightforward in gnuplot, since gnuplot is not intended to do such operations.
The following gnuplot-only solution assumes that you have your data in a datablock $Data without headers and empty lines. If not, you could either plot it with table from file into a table named $Data or use the following approach in the accepted answer of this question: gnuplot: load datafile 1:1 into datablock
If you don't have equidistant data, you need to interpolate data, which is also not straightforward in gnuplot, see: Resampling data with gnuplot
It's up to you: either you use external tools (which might not be platform-independent) or you apply a somewhat cumbersome platform independent gnuplot-only solution.
Code:
### plot difference of data to smoothed data
reset session
$Data <<EOD
1 0
2 13
3 16
4 17
5 11
6 8
7 0
EOD
stats $Data u 0 nooutput # get number of rows or datapoints
set samples STATS_records
set table $Smoothed
plot $Data u 1:2 smooth acsplines
unset table
# put both datablock into one
set print $Difference
do for [i=1:|$Data|] {
print sprintf('%s %s',$Data[i],$Smoothed[i+4])
}
set print
plot $Data u 1:2 w lp pt 7, \
$Smoothed u 1:2 w lp pt 6, \
$Difference u 1:($2-$4) w lp pt 4 lc "red"
### end of code
Result:
If I well understand you would like this :
First write your smooth's data in out.csv file
set table "out.csv" separator comma
plot 'file' u 4:5 smooth acsplines
unset table
Then this line will paste 'out.csv' to file as an appended column.You will maybe need to delete first lines using sed command (sed '1,4d' out.csv)
stats 'file' matrix
Thanks to stats we automatically get the number of column in your original data (STATS_size_x).
plot "< paste -d' ' file out.csv" u 4:($5-$(STATS_size_x+2)) w l
Could you please try this small code on your data.
I would like to draw a line with plots that contain "jumping" values.
Here is an example: when we have plots of sin(x) for several cycles and plot it, unrealistic line will appear that go across from right to left (as shown in following figure).
One idea to avoid this might be using with linespoints (link), but I want to draw it without revising the original data file.
Do we have simple and robust solution for this problem?
Assuming that you are plotting a function, that is, for each x value there exists one and only one corresponding y value, the easiest way to achieve what you want is to use the smooth unique option. This smoothing routine will make the data monotonic in x, then plot it. When several y values exist for the same x value, the average will be used.
Example:
Data file:
0.5 0.5
1.0 1.5
1.5 0.5
0.5 0.5
Plotting without smoothing:
set xrange [0:2]
set yrange [0:2]
plot "data" w l
With smoothing:
plot "data" smooth unique
Edit: points are lost if this solution is used, so I suggest to improve my answer.
Here can be applied "conditional plotting". Suppose we have a file like this:
1 2
2 5
3 3
1 2
2 5
3 3
i.e. there is a backline between 3rd and 4th point.
plot "tmp.dat" u 1:2
Find minimum x value:
stats "tmp.dat" u 1:2
prev=STATS_min_x
Or find first x value:
prev=system("awk 'FNR == 1 {print $1}' tmp.dat")
Plot the line if current x value is greater than previous, or don't plot if it's less:
plot "tmp.dat" u ($0==0? prev:($1>prev? $1:1/0), prev=$1):2 w l
OK, it's not impossible, but the following is a ghastly hack. I really advise you add an empty line in your dataset at the breaks.
$dat << EOD
1 1
2 2
3 3
1 5
2 6
3 7
1 8
2 9
3 10
EOD
plot for [i=0:3] $dat us \
($0==0?j=0:j=j,llx=lx,lx=$1,llx>lx?j=j+1:j=j,i==j?$1:NaN):2 w lp notit
This plots your dataset three times (acually four, there is a small error in there. I guess i have to initialise all variables), counts how often the abscissa values "jump", and only plots datapoints if this counter j is equal to the plot counter i.
Check the help on the serial evaluation operator "a, b" and the ternary operator "a?b:c"
If you have data in a repetitive x-range where the corresponding y-values do not change, then #Miguel's smooth unique solution is certainly the easiest.
In a more general case, what if the x-range is repetitive but y-values are changing, e.g. like a noisy sin(x)?
Then compare two consecutive x-values x0 and x1, if x0>x1 then you have a "jump" and make the linecolor fully transparent, i.e. invisible, e.g. 0xff123456 (scheme 0xaarrggbb, check help colorspec). The same "trick" can be used when you want to interrupt a dataline which has a certain forward "jump" (see https://stackoverflow.com/a/72535613/7295599).
Minimal solution:
plot x1=NaN $Data u 1:2:(x0=x1,x1=$1,x0>x1?0xff123456:0x0000ff) w l lc rgb var
Script:
### plot "folded" data without connecting lines
reset session
# create some test data
set table $Data
plot [0:2*pi] for [i=1:4] '+' u 1:(sin(x)+rand(0)*0.5) w table
unset table
set xrange[0:2*pi]
set key noautotitle
set multiplot layout 1,2
plot $Data u 1:2 w l lc "red" ti "data as is"
plot x1=NaN $Data u 1:2:(x0=x1,x1=$1,x0>x1?0xff123456:0x0000ff) \
w l lc rgb var ti "\n\n\"Jumps\" removed\nwithout changing\ninput data"
unset multiplot
### end of script
Result:
My data is a typical CSV-like text where each line consists of two columns where the first is text and the second a unix timestamp, as follows:
Mom 1441008169
Dad 1442516527
Ken 1441783871
...
<text> <unix-timestamp>
I thought I could arrange the data along a timeline, drawing dots/shapes of color corresponding to the text in the line, at a point along the x axis corresponding to the timestamp. At best I got gnuplot to tell me:
line 0: Need using spec for y time data
when I tell it to:
set ydata time
set timefmt "%s"
plot "-"
<data>
EOF
I want to render a plot using dots, or diamonds or shapes with color corresponding to the text string in first column. In other words, if my text values fall within the set {"Mom", "Dad", "Ken"}, then gnuplot should draw these shapes corresponding to "Mom" in red, "Dad" in green, and "Ken" in blue, or something like that, at points corresponding to their respective timestamps along the x axis.
The challenge for me is to have gnuplot distinguish between the text strings. The data can be thought of as, for instance, incoming calls from a person where the timestamp indicates date and time for the call and text represents the person calling. I thought representing these data as plotted dots/orbs/diamonds/whatever of different color along a time line would be a good idea, visually.
How would I achieve that? I can, optionally, generate some sort of identifier table where the unique text strings are each equated to a unique sequential generated ID, if that helps gnuplot.
I guess what you want is something like this
The x-axis spans the time interval which is specified by your data file (2nd column). Each name (Ken, Mom, Dad) is represented by a different point type (pt) and a specific colour (lc rgb 'color').
You can generate this plot by the following commands (assuming your data file's name is 'test'):
set xdata time
set timefmt "%s"
set format x "%d/%m/%y"
set xtics rotate
unset ytics
set yrange[0:2]
plot 'test' u ($2):(stringcolumn(1) eq "Ken" ? 1 :1/0) w p pt 5 ps 1.5 lc rgb 'blue' t 'Ken',\
'' u ($2):(stringcolumn(1) eq "Dad" ? 1 :1/0) w p pt 7 ps 1.5 lc rgb 'red' t 'Dad',\
'' u ($2):(stringcolumn(1) eq "Mom" ? 1 :1/0) w p pt 3 ps 1.5 lc rgb 'green' t 'Mom'
You can use different point types by assigning different numbers to pt. ps specifies the point size.
Another representation I came up with is the following:
You can generate it with:
plot 'test' u ($2):(1):(stringcolumn(1)) with labels t '',\
'' u ($2):(0.95) with impulses t ''
I hope this answers your question, and it is what you were looking for.
I want to use gnuplot for real time plotting (Data gets appended to file which I use for plotting and I use replot for real time plotting). I also want to put a label for the latest entry which is plotted. So as to get a idea what is the latest value. Is there a way to do this?
If you are on a unixoid system, you can use tail to extract the last line from the file and plot it separately in whatever way you desire. To give a simple example:
plot\
"data.dat" w l,\
"< tail -n 1 data.dat" u 1:2:2 w labels notitle
This will plot the whole of data.dat with lines and the last point with labels, with the label depicting the value.
There is no need to use the Linux command tail, you can simply do it with gnuplot-only, hence platform-independently.
The principle: while plotting the data, you assign the values of column 1 and 2 to variables x0 and y0, respectively.
After the first plot command, x0 and y0 will contain the last values.
With this, you don't have to load the file a second time for extracting the last values.
For the label plotting, use these values and print the label with a sprintf() expression (check help sprintf).
The construct '+' u ... every ::0::0 is just one way of many ways to plot a single data point.
Data: SO28152083.dat
1 5.1
2 2.2
3 3.3
4 1.4
5 4.5
Script: (works with gnuplot 4.4.0, March 2010 or even with earlier versions)
### plot last value as label
reset
FILE = "SO28152083.dat"
set key noautotitle
set offsets 0.5,0.5,1,1
plot FILE u (x0=$1):(y0=$2) w lp pt 7 lc rgb "red" ti "data", \
'+' u (x0):(y0):(sprintf("%g",y0)) every ::0::0 w labels offset 0,1
### end of script
Result:
I would like to reproduce this plot with gnuplot:
My data has this format:
Data
1: time
2: price
3: volume
I tried this:
plot file using 1:2 with lines, '' using 1:3 axes x1y2 with impulses
Which gives a normal time series chart with y1 as price and y2 as volume.
Next, I tried:
plot file using 2:1 with lines, '' using 2:3 axes x1y2 with impulses
Which gives prices series with y1 as time and y2 as volume.
However, I need the price to remain at y1 and volume at x2.
Maybe something like:
plot file using 1:2 with lines,' ' using 2:3 axes y1x2 with impulses
However, that does not give what I want.
Gnuplot has no official way to draw this kind of horizontal boxplots. However, you can use the boxxyerrorbars (shorthand boxxy) to achieve this.
As I don't have any test data of your actual example, I generated a data file from a Gaussian random-walk. To generate the data run the following python script:
from numpy import zeros, savetxt, random
N = 500
g = zeros(N)
for i in range(1, N):
g[i] = g[i-1] + random.normal()
savetxt('randomwalk.dat', g, delimiter='\t', fmt='%.3f')
As next thing, I do binning of the 'position data' (which in your case would be the volume data). For this one can use smooth frequency. This computes the sum of the y values for the same x-values. So first I use a proper binning function, which returns the same value for a certain range (x +- binwidth/2). The output data is saved in a file, because for the plotting we must exchange x and y value:
binwidth = 2
hist(x) = floor(x+0.5)/binwidth
set output "| head -n -2 > randomwalk.hist"
set table
plot 'randomwalk.dat' using (hist($1)):(1) smooth frequency
unset table
unset output
Normally one should be able to use set table "randomwalk.hist", but due to a bug, one needs this workaround to filter out the last entry of the table output, see my answer to Why does the 'set table' option in Gnuplot re-write the first entry in the last line?.
Now the actual plotting part is:
unset key
set x2tics
set xtics nomirror
set xlabel 'time step'
set ylabel 'position value'
set x2label 'frequency'
set style fill solid 1.0 border lt -1
set terminal pngcairo
set output 'randwomwalk.png'
plot 'randomwalk.hist' using ($2/2.0):($1*binwidth):($2/2.0):(binwidth/2.0) with boxxy lc rgb '#00cc00' axes x2y1,\
'randomwalk.dat' with lines lc rgb 'black'
which gives the result (with 4.6.3, depends of course on your random data):
So, for your data structure, the following script should work:
reset
binwidth = 2
hist(x) = floor(x+0.5)/binwidth
file = 'data.txt'
histfile = 'pricevolume.hist'
set table histfile
plot file using (hist($2)):($3) smooth unique
unset table
# get the number of records to skip the last one
stats histfile using 1 nooutput
unset key
set x2tics
set xtics nomirror
set xlabel 'time'
set ylabel 'price'
set x2label 'volume'
set style fill solid 1.0 border lt -1
plot histfile using ($2/2.0):($1*binwidth):($2/2.0):(binwidth/2.0) every ::::(STATS_records-2) with boxxy lc rgb '#00cc00' axes x2y1,\
file with lines using 1:2 lc rgb 'black'
Note, that this time the skipping of the last table entry is done by counting all entries with the stats command, and skipping the last one with every (yes, STATS_records-2 is correct, because the point numbering starts at 0). This variant doesn't need any external tool.
I also use smooth unique, which computes the average value of the , instead of the sum (which is done with smooth frequency).