I generally find Octave to be the easiest-to-use tool for data evaluation, but cumbersome for data-visualization.
On the other hand I find Gnuplot to be the easiest-to-use tool for data-visualization because of
its very focused nature,
its strong library of demo examples, and
its comprehensive interactive help features.
These advantages are sabotaged by putting any language-wrapper between gnuplot and the user, but pure gnuplot (intentionally) lacks capabilities for doing any non-trivial preprocessing (e.g. numerical calculus).
Octave can use gnuplot as a backend, but in order to preserve the advantages I would prefer directly using gnuplot commands, say:
dx = 0.1;
x = 0:dx:2*pi;
y = sin(x);
y2 = cumsum(y)*dx;
outputFileName = "sin.pdf"
# Contains mockup syntax:
gp set term pdfcairo size 30cm,20cm noenhanced
gp set output '${outputFileName}'
gp plot ${x,y} using 1:2 with lines title "The sin(x) function", \
gp ${x,y2} using 1:2 with lines title "Integrated sin(x) function", \
gp cos(x) title "The cos(x) function"
gprun
Writing a basic implementation of this "gp" utility would be easy enough, though variable interpolation would have to be replaced by explicit gp(["set output '" outputFileName "'"]) at the cost of readability. Not so easy however would be the creation of the ${x,y} table file.
Is there any straight-forward way to obtain similar functionality?
Ironically, some much older versions of octave (up until around 2005, 2006) had this capability (gset, graw, ...) but had them demoted to an internal implementation detail. In some older discussions I find references to e.g. __graw__, which also doesn't exist anymore, and some outdated documentation sites mention gplot, which still exists but with an entirely different purpose.
Honestly, I think you're overthinking it. Creating a temporary script and running it is often even more straightforward and portable than cumbersome libraries (e.g. see my previous similar answer here for an sqlite example )
Using that approach below works well for me, is very readable in my opinion, and does not result in unnecessary files (at least not visibly on your workspace).
dx = 0.1;
x = 0 : dx : 2 * pi;
y = sin( x );
y2 = cumsum( y ) * dx;
outputFileName = "sin.pdf"
% Create Gnuplot Script
GPScriptName = tempname(); gp = fopen( GPScriptName, 'w');
fdisp( gp, 'set term pdfcairo size 30cm,20cm noenhanced' );
fdisp( gp, sprintf( 'set output "%s"', outputFileName ) );
fdisp( gp, 'plot "-" using 1:2 with lines title "The sin(x) function", \' );
fdisp( gp, ' "-" using 1:2 with lines title "Integrated sin(x) function", \' );
fdisp( gp, ' cos(x) title "The cos(x) function"' );
fdisp( gp, [ x(:), y(:) ] );
fdisp( gp, 'e' );
fdisp( gp, [ x(:), y2(:) ] );
fdisp( gp, 'e' );
fdisp( gp, ';' );
fclose( gp )
% Run script with Gnuplot
Cmd = sprintf( 'gnuplot --persist "%s"', GPScriptName );
[Status, Output] = system( Cmd );
I used 'inline' data here, so that I could dump it straight in the script (as suggested here), but if you prefer, you could use the same logic to fdisp them to a separate temporary Data file, and make your Gnuplot script read from that; also you may or may not handle status errors and delete the temp files after the operation (like I did in the linked sqlite example) if you're worried about producing lots of temporary files and taking up space ... but in practice it typically doesn't matter.
I think this approach is fine and gives you more control ... but, if you insist, it should be trivial to wrap those fdisp calls into a gp function, and the system call into a gprun function ...
Related
Consider the following file that I want to plot using gnuplot: Servos20211222_105253.csv
# Date/Time 2021/12/22, 10:52:53
# PonE=0,LsKp=200,LsKi=0,LsKd=250,HsKp=40,HsKi=0,HsKd=130,Sp=800,TDEC=1175137
#
# Rel. Time, currentPos, PosPID, currentSpeed, speedPID, Lag, ServoPos
0.00000,4693184,0,0,0,0,4693184
0.00000,4693184,2300,0,368,0,4693184
0.00391,4693185,2300,12,367,0,4693184
:
:
I would like to:
set the plot title to the date/time from the first comment record.
display the record that starts "# PonE" as a caption.
extract the value for TDEC and plot a horizontal line with the name "Target"
I have some influence over the format of the header records, so if (for example) it would be better that they were not comments but provided in some other way, then that can be done.
It is a common problem to get text values from files using only gnuplot. If you can use OS and shell dependent solutions, I'd suggest to use remove the comments from the file and try something like
set title "`head -1 Servos20211222_105253.csv`"
You can place text anywhere using set label <"label text">, where the label text can be the 2nd line from the file.
You can plot a straight line using plot:
p sin(x), 0.5 title "TDEC"
But instead of 0.5, you need to get the value using shell scripts again, e.g. the cut unix command.
There are ways with gnuplot only, although sometimes a bit cumbersome compared with using tools which you have available on Linux (or comparable tools which you need to install on Windows).
Update: shorter and "simplified" script
One possible gnuplot-only way:
set commentschar to nothing, i.e. ''
assign the columns to variables and/or arrays, e.g. myDate, myTime, P[1..9].
Merge P[1..8] into a multi-line string Params by "mis"-using sum (check help sum)
Convert P[9] into a floating point number TDEC for plotting
Script: (modified the data a bit just for illustration)
### extract values from headers with gnuplot only
reset session
$Data <<EOD
# Date/Time 2021/12/22, 10:52:53
# PonE=0,LsKp=200,LsKi=0,LsKd=250,HsKp=40,HsKi=0,HsKd=130,Sp=800,TDEC=1175137
#
# Rel. Time, currentPos, PosPID, currentSpeed, speedPID, Lag, ServoPos
0.00000,1300000,0,0,0,0,4693184
0.00200,1200000,2300,0,368,0,4693184
0.00391,1100000,2300,12,367,0,4693184
EOD
set datafile separator comma commentschar ''
array P[9] # array to store parameters
stats $Data u ($0==0 ? (myDate=strcol(1)[3:], myTime=strcol(2)) : \
sum [_i=1:9] (P[_i] = _i==1 ? strcol(_i)[3:] : strcol(_i) ,0 )) \
every ::0::1 nooutput
set datafile commentschar # set back to default
Params = P[1]
Params = (sum [_i=2:8] (Params=Params.sprintf("\n%s",P[_i]),0),Params)
set title sprintf("%s %s", myDate, myTime)
TDEC = real(P[9][6:]) # convert to real number
set label 1 at graph 0.02, first TDEC P[9] offset 0,-0.7
set label 2 at graph 0.02, graph 0.85 Params
plot $Data u 1:2 w lp pt 7 title "Data", \
TDEC w l lc "red" title "Target"
### end of script
Result:
Is it possible to iteratively generate datablocks, where the name of the datablock is build up inside the loop?
Let's assume I have three fruits (in reality there are more):
array namelist[3] = ['apple', 'banana', 'pineapple']
I want to create three datablocks with the names $apple_data, $banana_data and $pineapple_data, so I tried the following:
do for [i=1:|namelist|] {
set table '$'.namelist[i]."_data"
plot ...
unset table
}
Unfortunately, instead of datablocks gnuplot created files with these names in the working directory. I guess gnuplot is checking whether the first character after set table is a $?
My second attempt was to remove the apostrophes around $:
set table $.namelist[i]."_data"
But this raised the weird error "Column number or datablock line expected", pointing at the period right after $.
Any ideas on how to achieve that?
The reason for all this is that I read in the banana/apple data files with a lengthy path, apply some lengthy calculations within using, and reuse these for lots of successive stats and plot commands. I would like to avoid having to hard-code and copy-paste the same long path and the cumbersome using command over and over again.
Not sure if I fully understood your detailed intention.
If you only want to avoid typing (or copy pasting) a lengthy path again and again, simply use variables:
FILE1 = 'C:/Dir1/SubDir1/SubSubDir1/SubSubSubDir1/File1'
FILE2 = 'C:/Dir2/SubDir2/SubSubDir2/SubSubSubDir2/File2'
plot FILE1 u 1:2, FILE2 u 1:2
Anyway, you asked for dynamically generated datablocks. One way which comes to my mind is using evaluate, check help evaluate. Check the example below as a starting point, which can probably be simplified.
Code: (simplified thanks to #Eldrad's comment)
### dynamically generate some datablocks
reset session
myNames = 'apple banana pineapple'
myName(i) = word(myNames,i)
N = words(myNames)
set samples 11
do for [i=1:N] {
eval sprintf('set table $%s_data',myName(i))
plot '+' u 1:(rand(0)) w table
unset table
}
plot for [i=1:N] sprintf('$%s_data',myName(i)) w lp pt 7 ti myName(i)
### end of code
Result:
Tracking the battery discharge / charge on my phone. Data shaped like a 'V'. Charge percentage and elapsed time in seconds, like this:
100,0
74,8103
51,15304
24,23407
2,30609
23,33286
55,37360
76,40064
100,44132
Here's how I'm processing n files and plotting them together:
plots=""
my_xrange=0
files=system("ls -1B data/*.csv | sort -k2 -t '-'")
do for [ file in files ] {
this="'".file."' ".'using ($2/3600):1 notitle, '
plots=plots.this
}
eval('plot '.plots)
(happy plot elided because of too many links)
Then the boss comes by and says the "uphills" are distracting, can I plot just the "downhills" pretty please? No worries, stats will give me the minima, so I xrange. Woot!
plots=""
my_xrange=0
files=system("ls -1B *.csv | sort -k2 -t '-'")
do for [ file in files ] {
stats file using ($2/3600):1 prefix "STATS" nooutput
if ( STATS_pos_min_y > my_xrange ) { my_xrange = STATS_pos_min_y }
this="'".file."' ".'using ($2/3600):1 notitle, '
plots=plots.this
}
print sprintf( 'stats: plot xrange is %.2f', my_xrange )
set xrange [0:my_xrange]
eval('plot '.plots)
xrange obliterates some uphills
I know there's got to be a way to just not plot the uphills, so I wind up with using conditional.
this="'".file."' ".'using (($2/3600) <= STATS_pos_min_y ? $1 : 1/0) notitle, '
which does effectively kill the uphills, but does two bad things to the plot:
the X axis values don't match the previous plot
the plot lines are merged on top of one-another
broken things
I need to use $2/3600 because I want things to be expressed in terms of hours. Oh, sure, I could write a script to massage the data before it even gets to gnuplot, but that's admitting defeat.
What have I not understood in my use of using?
The plot command for your whole data (purple line) is
plot file using ($2/3600):1 with linespoints
while that for the conditional data (blue line) is
plot file using ((($2/3600) <= STATS_pos_min_y) ? $1 : 1/0) with linespoints
This using statement selects only those data points for which ($2/3600) <= STATS_pos_min_y. However, you do not specify where on the x-axis those data points should be plotted, so gnuplot assumes it's just the line number in the data file (0,1,2,3,4). You have to specify the position on the x axis just like you did in the unconditional plot:
plot file using ($2/3600):((($2/3600) <= STATS_pos_min_y) ? $1 : 1/0) with linespoints
should do it.
By the way, formatting is rather limited in SO comments. To post code it's usually better to edit the question where you have more formatting options.
I am using a gnuplot plot loop to plot data from several plots together:
filenames = "my data files"
plot for file in filenames file.".txt" \
title file
Right now I'm using title file to set the plot title, but I'd like more control over the plot title without resorting to changing my file names. For example, in pseduocode, I'd like:
files = [first, second, third, fourth]
titles = [One title, second title, third title, fourth title]
plot for [n=1:4] files[n] titles[n]
Note that the titles consist of multiple words, so words(titles,n) is not an option.
Is there another method I can use to give me more flexibility in my titles?
First of all: good news, the 5.0 version has limited support for quoting text parts for use with word and words.
With version 5.0RC3, the following works fine:
titles='"first title text" "second title text"'
plot x title word(titles, 1), 2*x title word(titles, 2)
A second 'hack' would work with the postscript terminal, in case you are using it, and encodes the space inside the title with its octal representation \040:
set terminal postscript eps enhanced
set output 'spaces.eps'
titles='first\040title\040text second\040title\040text'
plot x title word(titles, 1), 2*x title word(titles, 2)
A third version uses a replacement character for the spaces and a shell call to sed to insert the spaces after the splitting:
titles='first#title#text second#title#text'
sub(s) = system(sprintf("echo \"%s\" | sed 's/#/ /g'", s))
plot x title sub(word(titles, 1)), 2*x title sub(word(titles, 2))
You could also setup a function myword, which uses awk or similar to do the splitting directly. But that would probably require some fiddling with quote characters.
This is indeed possible using word(string_with_words, index) :
filenames = "my data files"
description= "one two three"
plot for [n=1:4] word(filenames, i) title word(description, i)
I would like to plot a series of vertical lines in gnuplot at a set interval.
Some information about the plot.
The plot is mainly some data from a .dat file. The gnuplot script is called by a bash scripts which alters the gnu plot script using sed. This is a snipit of the an old bash script (ugly I'm sure).
sed -i 's/C = CONCEHOLD/C = '${$CO}'/g' $GNUPLOTROOT/plotviscosity.plt
gnuplot $GNUPLOTROOT/plotviscosity.plt
mv my-plot.ps $VISCPLOTNAME
sed -i 's/C = '${$CO}'/C = CONCEHOLD/g' $GNUPLOTROOT/plotviscosity.plt
with the . plt file looking like this.
set title "Viscosity vs Time, C = CONCEHOLD, beta = RATHOLD, zeta = ZETAHOLD"
set xlabel "Time"
set ylabel "Viscosity"
plot "viscout.dat" using 3:2 title 'Viscosity'
# Saving to my-plot.ps
load save.plt
#
I would like to add to this plot a series of vertical lines at a set repeating interval. I have found how to plot vertical lines via http://t16web.lanl.gov/Kawano/gnuplot/parametric-e.html
set parametric
const=3
set trange [1:4]
set xrange [0:5]
set yrange [0:5]
plot const,t
I would like to have
const=repititionperiod*i
where i is an integer belonging to (1,calculateduppedlimit].
I could input repititionperiod via sed again and in a similar vain calculateduppedlimit but need some sort of for loop either within gnuplot or a separate gnuplot script that adds a vertical line to the already created plot within a for loop in my bash script.
I can't find any information on loops within gnu plot or adding to a previously created plot.
Any advice gratefully received.
EDIT: Gnuplot does now in fact now support a for loop, you can read about it here
As I understand gnuplot doesn't have a for loop, although you can generate one of sorts as follows:
Make a file "loop.gp" containing
const = const + 1
#... some gnuplot commands ...
if(const<100) reread
then in a gnuplot terminal, or script write,
const = 3; load "loop.gp";
This will give you a simple loop.
(this example is taken from the misc. section of http://t16web.lanl.gov/Kawano/gnuplot/index-e.html)
For your particular answer you might try adding arrows rather than paremetric lines,
eg.
set arrow from const,1 to const,4 nohead
will do much the same thing.
In this case you loop.gp could be
const = const + repititionperiod
#... some gnuplot commands ...
set arrow from const,1 to const,4 nohead
if(const<calculatedupperlimit) reread
and you would run you loop with
const = 1; repititionperiod=2;calculatedupperlimit = 10; load "loop.gp"; replot;
The replot plots the arrows.
If you "just" want the lines and nothing else - then you will need to feed a graph to actually plot (a set of arrows doesn't count). The example you gave could then be used to plot the first vertical line.
hope this helps.
Tom