Specific gnuplot by data grouping - gnuplot

Question

I'm new in gnuplot and sorry that my problem formulation might be unprecise, but I don't know how to find the tools/commnds needed to solve my problem. The code for plotting I would like to integrate in my bash file.
I have a data set like:
285 1 50 7.35092
265 1 50 7.35092
259 1 50 7.35092
258 1 50 7.35092
264 1 50 7.35092
491 5 50 33.97
488 5 50 33.97
495 5 50 33.97
492 5 50 25.1649
495 5 50 33.0725
500 5 50 13.6176
507 5 50 32.2502
489 5 50 33.0725
494 5 50 33.97
491 5 50 33.97
746 10 50 34.6007
746 10 50 34.6007
767 10 50 30.858
745 10 50 34.8789
746 10 50 34.6007
747 10 50 34.6007
758 10 50 34.6007
772 10 50 34.60
I already grouped the data by entering a new line between blocks. I would like to calculate for each block the mean and standard deviation of the 4th column.
Then I would like to plot on the Y axes the mean with the confidence interval (standard deviation) and on the X axes the value from the second column.
Each data block has a unique number in the 2nd column.
Solution: so far I got the values for a point from the first block but while I try to plot I get an error:
#myBash code for plotting.sh
FILEIN=simulationR.txt
rm plotTestR.png
gnuplot << EOF
reset
set terminal png
set output 'plotTestR.png'
set ylabel 'reward'
set xlabel 'Nr of simualtion'
set title 'Simualtio duration'
set grid
stats "$FILEIN" using 4 every :::0::0 nooutput
mean1 = sprintf('%.3f', STATS_mean)
std1 = sprintf('%.3f', STATS_stddev)
stats "$FILEIN" using 2 every :::0::0 nooutput
x1 = sprintf('%.3f', STATS_max)
plot '-' w yerrorbars title std1
x1 mean1 std1
exit
EOF
and the error:
gnuplot> plot '-' w yerrorbars title std1
^
line 1: Bad data on line 1 of file -

Answer 1

Usually, gnuplot isn't made for such data processing tasks. That's best done with an external script, which does the processing and writes to stdout, which can then be feed directly to gnuplot like
plot '< python myscript.py simulationR.txt'
In your example, you can only have fixed data after the plot '-' part, no variable substitution is done here.
However, gnuplot version 5 introduces a new inline data structure, to which you can write your computed values (set print $data).
Note, that the following is a plain gnuplot script, if you want to wrap it in a bash script (which is not necessary, since you can pass variables to a gnuplot script via the command line), then you must escape the $ characters.
FILEIN="simulationR.txt"
system('rm -f plotTestR.png')
reset
set terminal pngcairo
set output 'plotTestR.png'
set ylabel 'reward'
set xlabel 'Nr of simulation'
set title 'Simulation duration'
set grid
set print $data
do for [i=0:2] {
stats FILEIN using 2:4 every :::i::i nooutput
print sprintf("%e %e %e", STATS_max_x, STATS_mean_y, STATS_stddev_y)
}
set autoscale xfix
set offsets 1,1,0,0
plot $data using 1:2:3 w yerrorbars
A further improvement could be to separate two blocks by two blank lines, in which case you can use
stats 'simulationR.txt' using 0 nooutput
to have the number of blocks in the variable STATS_blocks, and you can rewrite the loop as
do for [i=0:STATS_blocks-1] {
stats FILEIN using 2:4 index i nooutput
print sprintf("%e %e %e", STATS_max_x, STATS_mean_y, STATS_stddev_y)
}