I am trying to use LINEST in Excel 2013 to get the coefficients for a cubic function but LINEST does not work well with non-linear functions according to this link MS KB828533. Apparently this is because of the way collinearity is handled. A similar question is asked here Quadratic and cubic regression in Excel but it does not address the problem.
Excel's builtin Column Chart | Trendline (3rd degree poly) produces correct coefficients. However, LINEST as well as Data Analysis | Regression both give wrong coefficients.
EDIT: Excel's builtin Column Chart does NOT produce correct coefficients for polynomials. Only use Column Chart trendline for linear data! (please see answer).
This is my data:
x y
2006 7798
2007 8027
2008 9526
2009 11661
2010 16014
2011 18731
2012 23405
2013 25294
2014 28578
I can only get the third coefficent (here x3) using this:
={LINEST(y;(x-AVERAGE(x))^{1,2,3})}
Results:
Coef Chart LINEST
x3 -62.295 -62.295
x2 1098.254 163.834
x1 -2746.214 3564.226
intcpt 9528.659 15467.104
CORRECT x3 correct, rest WRONG
I have also tried a more complex LINEST like this:
={MMULT(LINEST(y;(x-AVERAGE(x))^{1,2,3});
IFERROR(COMBIN({3;2;1;0};{3,2,1,0})*(-AVERAGE(x))^({3;2;1;0}-{3,2,1,0});0))}
But in similar fashion only x3 is correct and the rest is wrong.
Any help is appreciated.
Problem solved. It turns out that using anything else but XY Scatter Plot to calculate regression coefficients for polynomials (or a trendline), will produce wrong coefficients.
Conclusively, do not use Line, Bar and Column charts to calculate regression coefficients for polynomials. The following image shows the difference in calculated coefficients - the top figure uses XY Scatter Plot and produces correct coefficients while bottom figure is created with a Column chart. Both figures uses the same data.
Related
I have two columns of data - first is fractions (goes on x-axis), second is integers (goes on y).
Is there a way to plot one against the other in a histogram type manner so it looks like a spectrum?
Use the "scatter plot" tool. It is also called a "scatter chart" tool. Once you have your chart in place, you can add a trend line to determine the formula for the relationship between x and y values. You will be able to find the formula in terms of y = mx+b. You can also determine the r squared value which will be important in the interpretation of the relationship. There are several options for scatter charts that might lead you off course, so experiment with the tool a bit to discover the correct options for your situation.
I'm trying to use excel to get the coefficient for two financial market spreads using two methods on data series Sprd1 against data series Sprd2:
1) I used scatter plot and simply added a trend line, showing R^2 (0.4052) and Coefficient (0.614). Trend line should be using SLOPE() to get the coefficient...
2) I used =CORREL(Sprd1, Sprd2), showing 0.637; =RSQ(Sprd1, Sprd2), yielding 0.4052.
I understand that the R-sq values should be pretty close. But why would the coefficents differ? I'm trying to look for any difference in terms of excel's embedded methods or assumptions on the trendline and the CORREL.
Thank you very much!
While both RSQ and CORREL work from the same equation
the value returned by RSQ is the square of that result.
i.e. RSQ()=CORREL()^2
SLOPE, on the other hand, does not use (y-MEAN(y))^2, nor does it take a square root of the denominator:
so will give slightly different results, depending on the mean of y
Okay, so I am working with the daily closings of the Dow Jones Industrial Average (DJIA) index from January 1979 to December 1989. I have successfully plotted the time-evolution of the index, but I am stumped as to how to add a trendline (specifically, exponential). You can get the data here:
http://research.stlouisfed.org/fred2/series/DJIA/downloaddata
I just downloaded it to Excel and then imported it to R as a csv file and plotted it.
In addition to that, how can I add a trendline at a specific place? Say, I wanted a trendline from the year 1985 to 1988?
If that's literally what you want to do (I suspect it isn't), then all you have to do is plot and add a curve with the specified function of x:
plot(y~x)
par(new=TRUE)
curve(0.0629*exp(0.0003*x))
If you actually want to fit a curve based on something you do in R, you'll have to generate some kind of model and then plot fitted values from that using basically the same thing but instead of curve you could use lines.
How to specify excel linest weighted polynomial fit formula, something like
LINEST(y*w^0.5,IF({1,0},1,x)*w^0.5,FALSE,TRUE), but this is for linear fit. I'm looking for similar formula for 2nd order and 3rd order polynomial regression fit.
In a reply to the other post in Weighted trendline an approach was already suggested for weighted polynomials. For example for a cubic fit try with CTRL+SHIFT+ENTER in a 4x1 range:
=LINEST(y*w^0.5,(x-1E-99)^{0,1,2,3}*w^0.5,FALSE)
(-1e-99 ensures that 0^0=1). Similar to the linear case for R^2 try:
=INDEX(LINEST((y-SUMPRODUCT(y,w)/SUM(w))*w^0.5,(x-1E-99)^{0,1,2,3}*w^0.5,FALSE,TRUE),3,1)
Derivation
In standard least squares we find the vector b that minimises:|y-Xb|²=(y-Xb)'(y-Xb)
In the weighted case b is chosen to minimise instead: |W(y-Xb)|²=(y-Xb)'W'W(y-Xb)
So the weighted regression is Wy on WX where W'W = W² is the diagonal matrix of the weights.
I'm not great with Math at all and so-so with Excel. I have a requirement to calculate a series of values to apply as an offset for a price schedule. I know what I basically want to achieve with the offsets and how I want them to change across span of values but I don't know how I can plot these values or something close to it in Excel. For an idea of the curve I'm roughly trying to create you can plot these values:
0
20
30
35
38
39
39.5
40
40.5
40.75
41
41.125
41.25
Or if you don't want to use excel you can take a look at this chart image:
Now it was easy to quickly create that by entering some values and tweaking them to get a shape I roughly want, but I need 1000 data points! In other words, instead of the 13 point I provided above I would need 1000. This would be crazy to try and do manually.
I can think of two possible options:
Create the range of values with fewer points, then somehow
extrapolate that data set into more points. Maybe extrapolate isn't
correct but essentially convert 10 points of data into 1000
Interactively draw a curve and then have it plotted to values within
a defined min/max range
I've tried googling for help but I'm just coming up with tons and tons of "how to plot your data" sites - I need to work from the OTHER direction ;)
Thanks for any help or tips.
Like you said, you could draw a curve and plot those points, then put them into some equation solver to get an equation that could describe the curve you're looking for.
But just from looking at your curve above, you might be able to use an equation of the following form.
y = A - (B / x)
To make a curve similar to yours above, you might try...
y = 40 - (5 / x))
You can use a site like https://functionplotter.com/ to plot the function and play with the values of A and B until you have curve that you are satisfied with.
Is this kind of thing you are looking for?
spline interpolation in exel
XlXtrFun.xll
basically you already got 13 y's and 13 implict x's (1 to 13). You provide bunch of "x" values of small increment, like 0.01, 0.02, 0.03, ...,12.999 13.000, to this tool. then this tool will calculate y value for each of the ones. You will plot these as "x y scatter plot"
A simple and very accurate way is to plot the curve as an x-y scatter plot, and then fit a polynomial via Insert Trendline.
In this case a 4th order polynomial provides close to a perfect fit (R2 of .9973)
Y = -0.0139x4 + 0.4819x3 - 6.0607x2 + 33.026x - 26.718
You can also solve this without charts using LINEST, see Walkenbach's guide here. But charts provide the easiest reference