I want to fit a function with a dataset using gnuplot.
I use a data set example, in the file "data":
1 2
5 4
6 5
7 8
If I do in gnuplot
>f(x) = a*x+b
>fit f(x) "data" via a,b
It works just good, (and with this example I get a≃0.855 and b≃0.687)
Now what I really want to do is to fit the function floor(a*x+b). So I tried exactly the same way
>f(x) = floor(a*x+b)
>fit f(x) "data" via a,b
And I get the output
Iteration 0
WSSR : 8 delta(WSSR)/WSSR : 0
delta(WSSR) : 0 limit for stopping : 1e-005
lambda : 0
initial set of free parameter values
a = 1
b = 1
Singular matrix in Givens()
error during fit
Googling it didn't help me, I also tried to find if there was some contraindication using fit with floor but again I didn't find anything.
Has someone an idea?
Note : I use Gnuplot 4.6 patchlevel 0, built for Windows 32bit
There is a fundamental problem fitting with floor, which is that your least squares error function is piecewise constant, so when you look for the gradient of the error with respect to your fit parameters you always get zero.
In this example the minimum sum of squares error is exactly 3 for a range of a,b in the neighborhood of .85,1.5
Mathermatica, (which is far more poweful), gives a result 1,1 along with a warning that due to the zero gradient it can not be sure if this is really a minimum.
Related
I'm hoping someone will be able to tell me why the equation that Excel generated is not giving the correct results as it is graphed correctly.
I have some X and Y points that I will list below. I plotted those points in Excel and then plotted the trend line, and had it show me the equation of the trendline. When I take the equation and then plug in the X values I get very different answers back.
X and Y Values
X Y
0 3
3 2
5 1.4
7 1
10 0.5
18 0.1
When I set the intercept to 3, the equation of the trendline is y = 0.0088x5 - 0.1457x4 + 0.8753x3 - 2.224x2 + 1.4798x + 3
Screenshot of Excel window with equation
Any help is greatly appreciated.
I suspect you didn't set up your graph correctly.
Select a single cell in your table
Insert/Scatter (and decide which you want with regard to markers, etc)
Select the line and add Trendline
Set you parameters for the trendline
If you want to get the formula for the trendline from the "show formula" option, be sure to format the trendline label to be numeric with 15 decimals. Otherwise the equation will certainly not work, even if it appears to be correct.
Note that you can obtain the formula directly using the LINEST worksheet function.
=LINEST(Y,X^{1,2,3,4,5}) returns the array:
{0.0000399230399230442,-0.00152188552188569,0.0192991822991846,-0.0840134680134806,-0.217128427128402,2.99999999999999}
The last value in the array is the y-intercept
The slight differences are due to the use of different algorithms for the two methods.
I have a file with a matrix like :
1 2 3
4 5 6
7 8 9
Using gnuplot, I would like to extract the Variable in the 3th row on the 2th column, and store it in a variable called X for example. please how to do that using gnuplot.
Thanks
You can do that within a plot command,
set table "/dev/null"
X=0
X_row=3
X_col=2
plot "file.dat" using (($0==X_row)?(X=column(X_col),X):0)
unset table
To save time the plot command can do something useful at the same time, like... plotting something.
Thanks, It's solved actually using this syntax :
plot u 0:($0==RowIndex?(VariableName=$ColumnIndex):$ColumnIndex)
#RowIndex starts with 0, ColumnIndex starts with 1
print VariableName
It's already explained quite well here :
by #StackJack
I have temporal data, where some time intervals contain only missing values. I want to show explicitely those missing values intervals.
For now, the solution I have is to check whether the value is NaN or not, as such:
plot file_name using 1:(stringcolumn(num_column) eq "NaN" ? 1/0 : column(num_column)) with lines,\
"" using 1:(stringcolumn(num_column) eq "NaN" ? 1000 : 1/0) with points
Which will result in drawing points at y = 1000 instead of the line for missing values, which gives the following result:
However, this is not ideal because a) I need to specify a y value at which to draw the points and b) it's quite ugly, especially when the dataset is longer in time.
I would like to produce something like this instead:
That is, to fill completely this interval with a color (possibly with some transparency unlike my image). Note that in these examples there is only one interval of missing values, bu in reality there can be any number of them on one plot.
We can do some pre-processing to accomplish this. Suppose that we have the following data file, data.txt
1 8
2 6
4 NaN
5 NaN
6 NaN
7 9
8 10
9 NaN
10 NaN
11 6
12 11
and the following python 3 program (obviously, using python is not the only way to do this), process.py1
data = [x.strip().split() for x in open("data.txt","r")]
i = 0
while i<len(data):
if (data[i][1]=="NaN"):
print(data[i-1][0],end=" ") # or use data[i][0]
i+=1
while data[i][1]=="NaN": i+=1
print(data[i][0],end=" ") # or use data[i-1][0]
else: i+=1
This python program will read the data file, and for each range of NaN values, it will output the last good and next good x-coordinates. In the case of the example data file, it outputs 2 7 8 11 which can be used as bounds for drawing rectangles. Now we can do, in gnuplot2
breaks = system("process.py")
set for [i=0:words(breaks)/2-1] object (i+1) rectangle from word(breaks,2*i+1),graph 0 to word(breaks,2*i+2),graph 1 fillstyle solid noborder fc rgb "orange"
Which will draw filled rectangles over this range. It determines how many "blocks" (groups of two values) are in the breaks variable then reads these two at a time using the breaks as left and right bounds for rectangles.
Finally, plotting the data
plot "data.txt" u 1:2 with lines
produces
which shows the filled rectangles over the range of NaN values.
Just to provide a little more applicability, the following awk program, process.awk3 serves the same purpose as the above python program, if awk is available and python isn't:
BEGIN {
started = 0;
last = "";
vals = "";
}
($2=="NaN") {
if (started==0) {
vals = vals " " last;
started = 1;
}
}
($2!="NaN") {
last = $1
if (started==1) {
vals = vals " " last;
started = 0;
}
}
END {
sub(/^ /,"",vals);
print vals;
}
We can use this by replacing the system call above with
breaks = system("awk -f process.awk data.txt")
1 The boundaries are extended to the last and next point to completely fill the gap. If this is not desired, the commented values will cover only the region identified by NaN in the file (4-6 and 8-10 in the example case). The program will not handle NaN values as the first or last data point.
2 I used solid orange for the gaps. Feel free to use any color spec there.
3 The awk program extends the boundaries in the same way as the python program, but takes more modification to get the other behavior. It has the same limitations in not handling NaN values as the first or last data point.
Using two filled curves
A somewhat "hacky" way of doing it is using two filled curves, as such:
plot file_name using 1:(stringcolumn(num_column) eq "NaN" ? 1/0 : column(num_column)) with lines ls 2,\
"" using 1:(stringcolumn(num_column) eq "NaN" ? 0 : 1/0) with filledcurve x1 ls 3,\
"" using 1:(stringcolumn(num_column) eq "NaN" ? 0 : 1/0) with filledcurve x2 ls 3
Both filledcurve must have the same linestyle, so that we get one uniform rectangle.
One filledcurve has x1 as parameter and the other x2, so that one fills above 0 and the other below 0.
You can remove the curve at 0 and make the filling transparent using this:
set style fill transparent solid 0.8 noborder
This is the result:
Note that the dashed line at 0 under the rectangle is a bit glitchy compared to the other dashed lines. Note also that if some rectangles are very small in width, they will look lighter than expected.
According to figure above. this picture is generated from data points in text file. My question is that how can i remove the line at any two points if graph is jumped? (In my picture see that graph is jump about on x~260)
note that my purpose is that i just want to make this graph look like piecewise function that mean line on the middle of graph should not be connected because is jumped.
In gnuplot you can split a line in several parts either when you have an invalid data value somewhere, or an empty line.
For the first situation, you could check inside the using statement, if the difference to the previous point is too large, and invalidate the current point. But that would also make you loose not only the connecting line, but also the first point after the jump:
lim=3
y2=y1=0
plot 'test.dat' using (y2=y1,y1=$2,$1):($0 > 0 && abs(y2-y1) > lim ? 1/0 : y1) with linespoints
The test data file I used is
1 1
2 1.1
3 0.95
4 1
5 5
6 6
7 5.5
8 5.8
9 -2
10 -2.5
11 -4
As you see, the points at x=5 and x=9 are lost.
Alternatively, you can pipe your data through an external tool like awk for the filtering. In this case you can insert an empty line when the difference between two consecutive y-values exceeds some limit:
filter(lim) = 'awk ''{if(NR > 1 && sqrt((y-$2)**2) > '.lim.') print ""; print; y=$2}'' test.dat'
plot '< '.filter(3) using 1:2 with lines
Note, that I used the sqrt((..)**2) only to simulate an abs function, which awk doesn't have.
I was wondering if someone could help with explaining in simple terms what interpolation is and how its used in 3d computer graphics
Simply put: given two points A and B, find a point between them.
For example, if I want to move something along a line from a position x=1 to x=4 in one step:
1-----------------------4
The first step is at location 1, the second step is at location 4, so the object moves instantly from one location to the other. However, if I want the object to take a certain amount of time or number of frames to make the transition, I'll need to refine that by finding intermediate points that are evenly spaced.
If I want the object to take two steps (or frames) to move from 1 to 4,
1-----------X-----------4
I need to calculate what the new point (X) is so I can draw the object there at the appropriate time. In this case, the point X will be
(max-min)
location = min + (current_step) * --------
steps
location is what we're trying to find. min=1, max=4, and in this example steps=2 since we want to divide the span into two steps:
step: location:
0 1
1 2.5
2 4
1------------(2.5)-----------4
If we want to take 4 steps:
step: location:
0 1
1 1.75
2 2.5
3 3.25
4 4
1---(1.75)---(2.5)---(3.25)---4
And so forth. For four steps, the object moves 25% of the total distance per frame. For 10 steps, 10%, etc ad nauseum.
For multiple dimensions (when an object has a 2- or 3-dimensional trajectory), just apply this to each X,Y,Z axis independently.
This is linear interpolation. There are other kinds. As always, Google can help you out.
Other applications include texture mapping, anti-aliasing, image smoothing and scaling, etc., and of course many other uses outside of games and graphics.
Note: a lot of frameworks already provide this. In XNA, for instance, it's Matrix.Lerp.
Interpolation is the smooth adjustment from one thing to another. It is used in animation.
For example, if an object is at location 1, and we want to move it to location 2 over the course of six seconds, we need to slowly interpolate its location between the two endpoints. Interpolation also refers to any search for a location on that path.
Interpolation is the 'guessing' of points based on other points.
for example when you have the points (0,0) and (2,2) you might 'guess' that the point (1,1) also belongs to the set.
The simples application is to deduce a line from two points.
The same thing works in 3 or actually n-dimension.
In 3D graphics it will be used
for animations, to calculate the position of things based on start and end coordinations
calculating lines
gradients
scaling of graphics
and probably many more
General Definition
Interpolation (in mathematics) can be regarded as a transition from one value to another. Interpolation usually uses a value in the 0 to 1 range like a percentage. 0 is the starting value and 1 is the end value. The main purpose of interpolation is to find values in between given values.
Types of Interpolation
There are many types of interpolation used in various programs, the most common being linear interpolation. This type of interpolation is the most simple and straight-forward; It is used to find values in a line segment between two points or numbers. There are also: cubic interpolation, quadratic interpolation, bilinear, trilinear, etc. For more information go here: https://en.wikipedia.org/wiki/Interpolation.
Application in 3D Graphics
Interpolation, especially linear, bilinear and trilinear, is important for computing fragments in geometry (the textures and visuals of the geometry), blending volumetric textures, mip-mapping (a depth of field effect on texture), and lighting (like unreal engine's volumetric lightmaps). The results of the interpolation may vary, but it could potentially yield very realistic results. It is a rather large computation, especially when the interpolation is in 3-dimensions or above (hyperspace).
Example of Interpolation
In 1 Dimension:
n1 = 1
n2 = 2
i = 0.5
n3 = (n1 - n1 * i) + n2 * i
///////////////////////////////////////
n3
├────────┼────────┼────────┼────────┤
1 1.25 1.5 1.75 2
///////////////////////////////////////
In 2 Dimensions:
v1 = {1, 1}
v2 = {1.5, 2}
i = 0.5
d = √((v1.x - v2.x)^2 + (v1.y - v2.y)^2)
v3 = {v1.x + -d * i * ((v1.x - v2.x) / d),v1.y + -d * i * ((v1.y - v2.y) / d)}
///////////////////////////////
2 ┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼
┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼
┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼
┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼
┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼ v2
1.5 ┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─●
┼─┼─┼─┼─┼─┼─┼v3─┼─┼─┼─┼─┼
┼─┼─┼─┼─┼─┼─●─┼─┼─┼─┼─┼─┼
┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼
┼v1─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼
●─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼
1 1.5 2
///////////////////////////////