See this article Enclosed, but not Encrypted.
I have some binary data. I want to perform the gnuplots shown in that article, but using my data.
For a three-dimensional phase-space plot, the sequence a, b, c, d, e,
f, etc. can be used as space coordinates (a-b, b-c, c-d), (b-c, c-d,
d-e), (c-d, d-e, e-f), etc. Patterns in the plot created reveal
recurring relations between subsequent sequences. In this phase plot,
50,000 16-bit random numbers would produce an unstructured cloud of
dots.
I want to do exactly the same kind of thing. I have a binary file (about 10 MB) and I'd like to run it through gnuplot to create the nice gnuplot graphs.
What do I type into gnuplot to make that happen?
Doing a Google search for "phase space plot" and gnuplot doesn't return much. I don't know if that's because the article is a translation from German. I don't think I've found relevant answers in stack exchange sites.
To plot the 3d phase space use the following script, which works like the running average example from the gnuplot page:
reset
back4 = back3 = back2 = back1 = 0
shift(x) = (back4 = back3, back3 = back2, back2 = back1, back1 = x)
samples(x) = $0 < 3 ? NaN : x
set ticslevel 0
# the labels are only for orientation when checking the test data
set xlabel 'xlabel'
set ylabel 'ylabel'
splot 'randomdata.dat' using (shift($1), samples(back4-back3)):(samples(back3-back2)):(samples(back2-back1))
Gnuplot must hold four data values, which are stored in back1 to back4. For every new value, the stored values are shifted with shift. samples takes care that the first three values are not used, but only stored (NaN creates an invalid data point).
To test it, use this file randomdata.dat:
21
15
10
6
3
1
0
This plots four data points at (6,5,4), (5,4,3), (4,3,2), and (3,2,1).
If you have a binary data file with e.g. 16bit numbers, use
splot 'binaryfile' binary format="%ushort" using (shift($1), samples(back4-back3)):(samples(back3-back2)):(samples(back2-back1))
If you need to change the datasize, invoke gnuplot and type show datafile binary datasizes to see which formats are supported.
Related
I'm making a c++ code which prints commands for gnuplot, in order to plot different things faster. The code plots the data already as the data fit as well, but now I'm adding some labels, and I want to print the fit equation, I mean something with this form
f(x) = (a +/- Δa)*x + (b +/- Δb)
I have the following line for printing it
set label 1 at screen 0.22, screen 0.75 sprintf('f(x) = %3.4f*x + %3.4f', a, b)
But, as you can see, there is only a and b values with no errors, I was thinking something like put there in the sprintf function any error related variables (FIT_something) and then have something like
set label 1 at screen 0.22, screen 0.75 sprintf('f(x) = (%3.4f +/- %3.4f)*x + (%3.4f + %3.4f)', a, deltaa, b, deltab)
But I can't find those, my answers are: does those exists? and if the answer is no, is there any way to print the variable errors further just writing it explicitly on the line?
Thanks for your help
Please read the statistical overview section of the gnuplot documentation (help statistical_overview). Keeping in mind the caveats described there, see also the documentation for set fit errorvariables, which I extract below:
If the `errorvariables` option is turned on, the error of each fitted
parameter computed by `fit` will be copied to a user-defined variable
whose name is formed by appending "_err" to the name of the parameter
itself. This is useful mainly to put the parameter and its error onto
a plot of the data and the fitted function, for reference, as in:
set fit errorvariables
fit f(x) 'datafile' using 1:2 via a, b
print "error of a is:", a_err
set label 1 sprintf("a=%6.2f +/- %6.2f", a, a_err)
plot 'datafile' using 1:2, f(x)
If the `errorscaling` option is specified, which is the default, the
calculated parameter errors are scaled with the reduced chi square. This is
equivalent to providing data errors equal to the calculated standard
deviation of the fit (FIT_STDFIT) resulting in a reduced chi square of one.
I want to plot only data points. Now I can plot the points which only considers 1 type of point. But my data contains different column variables. I want to plot different figures with different x and y variables from the data. Suppose I want to plot variable D against variable A or variable E against variable year but I want to plot data points with different colors or different types of points either *, dot, diamond etc. based on suppose, variable pub or variable E. Now for colormap I want to show colormap beside the figure with where the range of the variable value will be shown. For different type of points the point indexes will be suppose another variable E.
Also the 1st data should have a completely different point so that it can be distinguishable. My code actually shows different point for that data but it also plots with others.
Here is the truncated data.
Can anyone help me with that?
My code:
T = readtable('Data.xlsx');
year = T.Year;
pub = T.Publication;
A = T.A;
B = T.B;
C = T.C;
D = T.D;
E = T.F;
% Plot Data
f = figure;
%hold on; grid on, box on;
plot(A, D,'*')
hold on;
plot(A(1), D(1),'d')
It feels like this matlab example should be pretty close to what you want. It is a scatter plot (like your plot(A,D,'*') command), and has a colour scale that varies with a third variable c.
You should then combine this with a hold on command and plotting the first point using a different style suitable to your liking. You could something along the lines of the following (I have not downloaded your data, so I will use the example from the matlab link I provided):
x = linspace(0,3*pi,200); % Independent variable
y = cos(x) + rand(1,200); % Dependent variable
c = linspace(1,10,length(x)); % Colour variable
% Plot all points except the first one using colours in c and size 50:
scatter( x(2:end), y(2:end), 50, c(2:end) );
hold on
% Plot first point differently: twice the size, and with a filled marker:
scatter( x(1), y(1), 100, c(1), 'filled');
legend({'Data','First point'});
hold off
I would like to plot 1000+ curves and display their eye diagram with gnuplot.
Example of eye-diagram example with matlab: http://www.mathworks.fr/fr/help/comm/ref/commscope.eyediagram.html
I can already plot the curves using the script bellow:
gnuplot> plot for [col=1:1000] 'input_dataset1.txt' using 0:col with lines linecolor rgb("#0000ff")
Result: output_image.png
My problem is that when two lines intersects, the intersection has the same color as the line. The eye-diagram should display area with lots of intersections in a different color.
I haven't fould any example of such diagrams made with gnuplot.
Playing with line transparency didn't work: the intersction of two semi-transparent lines is the same color as the line.
Any ideas ?
Thanks,
I worked out a gnuplot-only way to do this, it involves a bit of work and you'll probably have to fine tune the details for your particular problem.
As an example I generated a data file containing values for the function exp(x) and its Taylor expansions from order zero (T^(0)[exp(x)] = 1) to order 3 (T^(3)[exp(x)] = 1 + x + x**2/2. + x**3/6.). This kind of data is suited to this problem because you will have a high data density around the origin, where all the approximations converge to the exact value, and lower data density away from it. It can be generated like this with gnuplot:
set xrange [0:1]
set table
set output "| grep -v '^$' > data"
plot exp(x), 1, 1+x, 1+x+x**2/2., 1+x+x**2/2.+x**3/6.
unset table ; unset output
Note I'm formatting the output so my data file has no blank lines, otherwise gnuplot treats fields separated by blank lines as different data blocks and this eventually messes up the histograms below. This data looks like this (plot "data"):
Now, I create a 2D histogram with this data. It would be extremely helpful if gnuplot offered this feature, but it doesn't, so the task gets a bit tricky. What I will do is create several 1D histograms. For more info on how to generate the latter, check this.
The first thing is to figure out the width along x and y for your bins, xwidth and ywidth, where the number of data points are counted, that is, we divide the data space into a grid where each element measures xwidth by ywidth and is assigned a number equal to the number of data points contained within. The smaller these elements the better resolution your graph will have, but also the more data points you'll need for it to look good. For my data above, this could be something like
xwidth = 0.02
ywidth = 0.05
Now we declare a function to define our 1D bins (details):
bin(x,width)=width*floor(x/width)+width/2.0
and define the number of bins along each direction. Because the xrange for my data is [0:1] and my yrange is [1:2.8], the number of bins would be 50 and 36, respectively. I could use Nx = xrange / xwidth but that would lead to a float Nx and I want an integer. To be safe I do:
Nx = 50
Ny = 36
It might make more sense to define these values the other way around: calculate xwidth as xrange / Nx, in which case you should not have problems with integer/float.
Now I generate the 1D histograms along y, looping over x values:
set output "| grep -v 'u\\|^$' | sed 's/#/\\n#/g' > data2"
set table
plot for [i=0:(Nx-1)] "./data" using \
(bin($2,ywidth)):( i*xwidth <= $1 && (i+1.)*xwidth > $1 ? 1.0 : 0.0) \
smooth freq
unset table ; unset output
Now data2 contains Nx blocks of data, each of them being a scan along y with Ny data points. The value of these data points is the number of data entries in the original data file. As it is, data2 contains 2D data (y, color), which I need to remap to 3D. The x value is given by the data block position, accessible with the every option in gnuplot. To plot this 3-dimensionally I do:
set output "| grep -v 'u\\|^$' | sed 's/#/\\n#/g' > data3"
set table
splot for [i=0:(Nx-1)] "./data2" every :::i::i using \
((i+0.5)*xwidth):1:2
unset table ; unset output
This data3 can now be plotted as a color map:
plot "./data3" with image
which looks like this:
Had I used higher quality data (i.e. with higher resolution) the graph would look nicer. With 2x resolution along each direction, the same looks like below:
I have a large set of data points from x = 1 to x = 10e13 (step size is fixed to about 3e8).
When I try to plot them using a logscale I certainly get an incredible huge point-density towards the end. Of course this affects my output plots since postscript and svg files (holding each and every data point) are getting really big.
Is there a way to tell gnuplot to decrease the data density dynamically?
Sample data here. Shows a straight line using logarithmic x-axis.
Usually, for this kind of plots, one can use a filter function which selects the desired points and discards all others (sets their value to 1/0:
Something like:
plot 'sample.dat' using (filter($1) ? $1 : 1/0):2
Now you must define an appropriate filter function to change the data density. Here is a proposal, with pseudo-data, although you might for sure find a better one, which doesn't show this typical logarithmic pattern:
set logscale x
reduce(x) = x/(10**(floor(log10(x))))
filterfunc(x) = abs(log10(sc)+(log10(x) - floor(log10(x))) - log10(floor(sc*reduce(x))))
filter(x) = filterfunc(x) < 1e-5 ? x : 1/0
set multiplot layout 1,2
sc = 1
plot 'sample.data' using (filter($1)):2 notitle
sc = 10
replot
The variable sc allows to change the density. The result is (with 4.6.5) is:
I did some work inspired by Christoph's answer and able to get equal spacing in log scale. I made a filtering, if you have numbers in the sequence you can simply use Greatest integer function and then find the nearest to it in log scale by comparing the fraction part. Precision is tuned by precision_parameter here.
precision_parameter=100
function(x)=(-floor(precision_parameter*log10(x))+(precision_parameter*log10(x)))
Now filter by using the filter function defined below
density_parameter = 3.5
filter(x)=(function(x) < 1/(log10(x))**density_parameter & function(x-1) > 1/(log10(x))**density_parameter ) ? x : 1/0
set datafile missing "NaN"
Last line helps in plotting with line point. I used x and x-1 assuming the xdata is in arithmetic progression with 1 as common difference, change it accordingly with your data. Just replace x by filter(x) in the plot command.
plot 'sample_data.dat' u (filter($1)):2 w lp
I know how to create a histogram (just use "with boxes") in gnuplot if my .dat file already has properly binned data. Is there a way to take a list of numbers and have gnuplot provide a histogram based on ranges and bin sizes the user provides?
yes, and its quick and simple though very hidden:
binwidth=5
bin(x,width)=width*floor(x/width)
plot 'datafile' using (bin($1,binwidth)):(1.0) smooth freq with boxes
check out help smooth freq to see why the above makes a histogram
to deal with ranges just set the xrange variable.
I have a couple corrections/additions to Born2Smile's very useful answer:
Empty bins caused the box for the adjacent bin to incorrectly extend into its space; avoid this using set boxwidth binwidth
In Born2Smile's version, bins are rendered as centered on their lower bound. Strictly they ought to extend from the lower bound to the upper bound. This can be corrected by modifying the bin function: bin(x,width)=width*floor(x/width) + width/2.0
Be very careful: all of the answers on this page are implicitly taking the decision of where the binning starts - the left-hand edge of the left-most bin, if you like - out of the user's hands. If the user is combining any of these functions for binning data with his/her own decision about where binning starts (as is done on the blog which is linked to above) the functions above are all incorrect. With an arbitrary starting point for binning 'Min', the correct function is:
bin(x) = width*(floor((x-Min)/width)+0.5) + Min
You can see why this is correct sequentially (it helps to draw a few bins and a point somewhere in one of them). Subtract Min from your data point to see how far into the binning range it is. Then divide by binwidth so that you're effectively working in units of 'bins'. Then 'floor' the result to go to the left-hand edge of that bin, add 0.5 to go to the middle of the bin, multiply by the width so that you're no longer working in units of bins but in an absolute scale again, then finally add back on the Min offset you subtracted at the start.
Consider this function in action:
Min = 0.25 # where binning starts
Max = 2.25 # where binning ends
n = 2 # the number of bins
width = (Max-Min)/n # binwidth; evaluates to 1.0
bin(x) = width*(floor((x-Min)/width)+0.5) + Min
e.g. the value 1.1 truly falls in the left bin:
this function correctly maps it to the centre of the left bin (0.75);
Born2Smile's answer, bin(x)=width*floor(x/width), incorrectly maps it to 1;
mas90's answer, bin(x)=width*floor(x/width) + binwidth/2.0, incorrectly maps it to 1.5.
Born2Smile's answer is only correct if the bin boundaries occur at (n+0.5)*binwidth (where n runs over integers). mas90's answer is only correct if the bin boundaries occur at n*binwidth.
Do you want to plot a graph like this one?
yes? Then you can have a look at my blog article: http://gnuplot-surprising.blogspot.com/2011/09/statistic-analysis-and-histogram.html
Key lines from the code:
n=100 #number of intervals
max=3. #max value
min=-3. #min value
width=(max-min)/n #interval width
#function used to map a value to the intervals
hist(x,width)=width*floor(x/width)+width/2.0
set boxwidth width*0.9
set style fill solid 0.5 # fill style
#count and plot
plot "data.dat" u (hist($1,width)):(1.0) smooth freq w boxes lc rgb"green" notitle
As usual, Gnuplot is a fantastic tool for plotting sweet looking graphs and it can be made to perform all sorts of calculations. However, it is intended to plot data rather than to serve as a calculator and it is often easier to use an external programme (e.g. Octave) to do the more "complicated" calculations, save this data in a file, then use Gnuplot to produce the graph. For the above problem, check out the "hist" function is Octave using [freq,bins]=hist(data), then plot this in Gnuplot using
set style histogram rowstacked gap 0
set style fill solid 0.5 border lt -1
plot "./data.dat" smooth freq with boxes
I have found this discussion extremely useful, but I have experienced some "rounding off" problems.
More precisely, using a binwidth of 0.05, I have noticed that, with the techniques presented here above, data points which read 0.1 and 0.15 fall in the same bin. This (obviously unwanted behaviour) is most likely due to the "floor" function.
Hereafter is my small contribution to try to circumvent this.
bin(x,width,n)=x<=n*width? width*(n-1) + 0.5*binwidth:bin(x,width,n+1)
binwidth = 0.05
set boxwidth binwidth
plot "data.dat" u (bin($1,binwidth,1)):(1.0) smooth freq with boxes
This recursive method is for x >=0; one could generalise this with more conditional statements to obtain something even more general.
We do not need to use recursive method, it may be slow. My solution is using a user-defined function rint instesd of instrinsic function int or floor.
rint(x)=(x-int(x)>0.9999)?int(x)+1:int(x)
This function will give rint(0.0003/0.0001)=3, while int(0.0003/0.0001)=floor(0.0003/0.0001)=2.
Why? Please look at Perl int function and padding zeros
I have a little modification to Born2Smile's solution.
I know that doesn't make much sense, but you may want it just in case. If your data is integer and you need a float bin size (maybe for comparison with another set of data, or plot density in finer grid), you will need to add a random number between 0 and 1 inside floor. Otherwise, there will be spikes due to round up error. floor(x/width+0.5) will not do because it will create pattern that's not true to original data.
binwidth=0.3
bin(x,width)=width*floor(x/width+rand(0))
With respect to binning functions, I didn't expect the result of the functions offered so far. Namely, if my binwidth is 0.001, these functions were centering the bins on 0.0005 points, whereas I feel it's more intuitive to have the bins centered on 0.001 boundaries.
In other words, I'd like to have
Bin 0.001 contain data from 0.0005 to 0.0014
Bin 0.002 contain data from 0.0015 to 0.0024
...
The binning function I came up with is
my_bin(x,width) = width*(floor(x/width+0.5))
Here's a script to compare some of the offered bin functions to this one:
rint(x) = (x-int(x)>0.9999)?int(x)+1:int(x)
bin(x,width) = width*rint(x/width) + width/2.0
binc(x,width) = width*(int(x/width)+0.5)
mitar_bin(x,width) = width*floor(x/width) + width/2.0
my_bin(x,width) = width*(floor(x/width+0.5))
binwidth = 0.001
data_list = "-0.1386 -0.1383 -0.1375 -0.0015 -0.0005 0.0005 0.0015 0.1375 0.1383 0.1386"
my_line = sprintf("%7s %7s %7s %7s %7s","data","bin()","binc()","mitar()","my_bin()")
print my_line
do for [i in data_list] {
iN = i + 0
my_line = sprintf("%+.4f %+.4f %+.4f %+.4f %+.4f",iN,bin(iN,binwidth),binc(iN,binwidth),mitar_bin(iN,binwidth),my_bin(iN,binwidth))
print my_line
}
and here's the output
data bin() binc() mitar() my_bin()
-0.1386 -0.1375 -0.1375 -0.1385 -0.1390
-0.1383 -0.1375 -0.1375 -0.1385 -0.1380
-0.1375 -0.1365 -0.1365 -0.1375 -0.1380
-0.0015 -0.0005 -0.0005 -0.0015 -0.0010
-0.0005 +0.0005 +0.0005 -0.0005 +0.0000
+0.0005 +0.0005 +0.0005 +0.0005 +0.0010
+0.0015 +0.0015 +0.0015 +0.0015 +0.0020
+0.1375 +0.1375 +0.1375 +0.1375 +0.1380
+0.1383 +0.1385 +0.1385 +0.1385 +0.1380
+0.1386 +0.1385 +0.1385 +0.1385 +0.1390
Different number of bins on the same dataset can reveal different features of the data.
Unfortunately, there is no universal best method that can determine the number of bins.
One of the powerful methods is the Freedman–Diaconis rule, which automatically determines the number of bins based on statistics of a given dataset, among many other alternatives.
Accordingly, the following can be used to utilise the Freedman–Diaconis rule in a gnuplot script:
Say you have a file containing a single column of samples, samplesFile:
# samples
0.12345
1.23232
...
The following (which is based on ChrisW's answer) may be embed into an existing gnuplot script:
...
## preceeding gnuplot commands
...
#
samples="$samplesFile"
stats samples nooutput
N = floor(STATS_records)
samplesMin = STATS_min
samplesMax = STATS_max
# Freedman–Diaconis formula for bin-width size estimation
lowQuartile = STATS_lo_quartile
upQuartile = STATS_up_quartile
IQR = upQuartile - lowQuartile
width = 2*IQR/(N**(1.0/3.0))
bin(x) = width*(floor((x-samplesMin)/width)+0.5) + samplesMin
plot \
samples u (bin(\$1)):(1.0/(N*width)) t "Output" w l lw 1 smooth freq