I'm using Fit.Polynomial to fit my data and show by graph, and all works great, but I couldn't find way to present my Y-function to any order. If the function is order 2 so I want it to be presents as y=ax^2+bx+c, and if the order is 3 so function would be y=ax^3+bx^2+cx+d.
Is there any way to do it?
Use Polynomial.Fit which returns Polynomial instead of Fit.Polynomial which returns double[].
var x = new double[] { 1, 2, 3 };
var y = new double[] { 2, 5, 9 };
Console.WriteLine("Fit.Polynomial");
double[] #double = Fit.Polynomial(x, y, 2);
Console.WriteLine(#double);
Console.WriteLine(string.Join(", ", #double));
Console.WriteLine("\nPolynomial.Fit");
Polynomial polynomial = Polynomial.Fit(x, y, 2);
Console.WriteLine(polynomial);
Console.WriteLine("\nPolynomial.Fit");
Polynomial polynomial = Polynomial.Fit(x, y, 2);
Console.WriteLine(polynomial);
Gives
Fit.Polynomial
System.Double[]
0, 1.5, 0.500000000000001
Polynomial.Fit
1.5x + 0.500000000000001x^2
Related
I've download the opencv from https://opencv.org/opencv-demonstrator-gui/ to make some live test on some images.
I found that this filter work perfectly for my needs:
,
I need to code it in my python script, tried to follow this tutorial :https://docs.opencv.org/3.4/d2/d2c/tutorial_sobel_derivatives.html
but I'm unable to find and match setting I need (pre-filtering Deriche, or Schar operator type).
I guess also I should use this syntax:
cv.Sobel(gray, ddepth, 1, 0, ksize=3, scale=scale, delta=delta, borderType=cv.BORDER_DEFAULT)
Thx.
UPDATE
Using this lines I'm close to right result:
scale = 1
delta = 0
ddepth = cv2.CV_16S
src = cv2.imread(image, cv2.IMREAD_COLOR)
src = cv2.GaussianBlur(src, (3, 3), 0)
gray = cv2.cvtColor(src, cv2.COLOR_BGR2GRAY)
grad_x = cv2.Sobel(gray, ddepth, 1, 0, ksize=3, scale=scale, delta=delta, borderType=cv2.BORDER_DEFAULT)
# Gradient-Y
# grad_y = cv.Scharr(gray,ddepth,0,1)
grad_y = cv2.Sobel(gray, ddepth, 0, 1, ksize=3, scale=scale, delta=delta, borderType=cv2.BORDER_DEFAULT)
abs_grad_x = cv2.convertScaleAbs(grad_x)
abs_grad_y = cv2.convertScaleAbs(grad_y)
grad = cv2.addWeighted(abs_grad_x, 0.5, abs_grad_y, 0.5, 0)
You are only doing the X derivative Sobel filter in Python/OpenCV. It is likely you really want the gradient magnitude, not the X directional derivative. To compute the magnitude, you need both the X and Y derivatives and then compute the magnitude. You also like will need to compute as float so as not to get one sided derivatives. You can later convert the magnitude to 8-bit if you want.
gradx = cv2.Sobel(gray,cv2.CV_64F,1,0,ksize=3)
grady = cv2.Sobel(gray,cv2.CV_64F,0,1,ksize=3)
gradmag = cv2.magnitude(gradx,grady)
The Scharr is similar and can be found at https://docs.opencv.org/master/d4/d86/group__imgproc__filter.html#gaa13106761eedf14798f37aa2d60404c9
The program I have here is simulating the velocity of a falling object.
The velocity is calculated by subtracting the y position from time_1 and time_2.
The problem that I have is that the dimensions of array v and array t don't match. Instead of shortening array t I would like to add 0 at the beginning of the v array. So that the graph will show v = 0 at t= 0. Yes, I know it is a small interval and that it does not really matter. But I want to know it for educational purpose.
I'm wondering if i can write the line v = (y[1:] - y[:-1])/0.1 in a from where i keep the dimension.
The ideal thing that would happen is that the array y will be substracted with an array y[:-1] and that this subtraction will happen at the end of the y array so the result will be an array of dimension 101 with a 0 as start value.
I would like to know your thoughts about this.
import matplotlib.pyplot as plt
t = linspace(0,10,101)
g = 9.80665
y = 0.5*g*t*t
v = (y[1:] - y[:-1])/0.1
plt.plot(t,v)
plt.show()
is there a function where i can add a certain value to the beginning of an array? np.append will add it to the end.
Maybe you could just pre-define the length of the result at the beginning and then fill up the values:
import numpy as np
dt = .1
g = 9.80665
t_end = 10
t = np.arange(0,t_end+dt,dt)
y = 0.5*g*t*t
v = np.zeros(t.shape[0])
v[1:] = (y[1:] - y[:-1])/dt
if you simply looking for the append at index function it would be this one:
np.insert([1,2,3,4,5,6], 2, 100)
>> array([ 1, 2, 100, 3, 4, 5, 6])
another possible solution to this would be to use np.append but inverse your order :
import numpy as np
v = np.random.rand(10)
value = 42 # value to append at the beginning of v
value_arr = np.array([value]) # dimensions should be adjust for multidimensional array
v = np.append(arr = value_arr, values = v, axis=0)
and the possible variants following the same idea, using np.concatenate or np.hstack ...
regarding your second question in comments, one solution may be :
t = np.arange(6)
condlist = [t <= 2, t >= 4]
choicelist = [1, 1]
t = np.select(condlist, choicelist, default=t)
I'm running a multi-class Logistic Regression (withLBFGS) with Spark 1.6.
given x and possible labels {1.0,2.0,3.0}
the final model will only output what is the best prediction, say 2.0.
If I'm interested to know what was the second best prediction, say 3.0, how could I retrieve that information?
In NaiveBayes I would use the model.predictProbabilities() function which for each sample would output a vector with all the probabilities for each possible outcome.
There are two ways to do logistic regression in Spark: spark.ml and spark.mllib.
With DataFrames you can use spark.ml:
import org.apache.spark
import sqlContext.implicits._
def p(label: Double, a: Double, b: Double) =
new spark.mllib.regression.LabeledPoint(
label, new spark.mllib.linalg.DenseVector(Array(a, b)))
val data = sc.parallelize(Seq(p(1.0, 0.0, 0.5), p(0.0, 0.5, 1.0)))
val df = data.toDF
val model = new spark.ml.classification.LogisticRegression().fit(df)
model.transform(df).show
You get the raw predictions and probabilities:
+-----+---------+--------------------+--------------------+----------+
|label| features| rawPrediction| probability|prediction|
+-----+---------+--------------------+--------------------+----------+
| 1.0|[0.0,0.5]|[-19.037302860930...|[5.39764620520461...| 1.0|
| 0.0|[0.5,1.0]|[18.9861466274786...|[0.99999999431904...| 0.0|
+-----+---------+--------------------+--------------------+----------+
With RDDs you can use spark.mllib:
val model = new spark.mllib.classification.LogisticRegressionWithLBFGS().run(data)
This model does not expose the raw predictions and probabilities. You can take a look at predictPoint. It multiplies the vectors and picks the class with the highest prediction. The weights are publicly accessible, so you could copy that algorithm and save the predictions instead of just returning the highest one.
Following the suggestions from #Daniel Darabos:
I tried to use the LogisticRegression function from ml instead of mllib
Unfortunately it doesn't support the multi-class logistic regression but only the binary one.
I took a look at PredictedPoint
and modified it so that it prints all the probabilities for each class. Here it is what it looks like:
def predictPointForMulticlass(featurizedVector:Vector,weightsArray:Vector,intercept:Double,numClasses:Int,numFeatures:Int) : Seq[(String, Double)] = {
val weightsArraySize = weightsArray.size
val dataWithBiasSize = weightsArraySize / (numClasses - 1)
val withBias = false
var bestClass = 0
var maxMargin = 0.0
var margins = new Array[Double](numClasses - 1)
var temp_marginMap = new HashMap[Int, Double]()
var res = new HashMap[Int, Double]()
(0 until numClasses - 1).foreach { i =>
var margin = 0.0
var index = 0
featurizedVector.toArray.foreach(value => {
if (value != 0.0) {
margin += value * weightsArray((i * dataWithBiasSize) + index)
}
index += 1
}
)
// Intercept is required to be added into margin.
if (withBias) {
margin += weightsArray((i * dataWithBiasSize) + featurizedVector.size)
}
val prob = 1.0 / (1.0 + Math.exp(-margin))
margins(i) = margin
temp_marginMap += (i -> margin)
if(margin > maxMargin) {
maxMargin = margin
bestClass = i + 1
}
}
for ((k,v) <- temp_marginMap){
val calc =probCalc(maxMargin,v)
res += (k -> calc)
}
return res
}
where probCalc() is simply defined as:
def probCalc(maxMargin:Double,margin:Double) :Double ={
val res = 1.0 / (1.0 + Math.exp(-(margin - maxMargin)))
res
}
I'm returning a Hashmap[Int, Double] but that can be changed as you wish.
Hopefully this helps!
Problem definition: consider the "Simpletest" model (from pymc3 examples)which is something similar to the following one:
model = Model()
data = np.random.normal(size=(2, 20))
with model:
x = Normal('x', mu=.5, tau=2. ** -2, shape=(2, 1))
z = Beta('z', alpha=10, beta=5.5)
d = Normal('data', mu=x, tau=.75 ** -2, observed=data)
step = NUTS()
trace = sample(1000, step)
I'd like to change it so that I'll have a fixed model structure but run the sampling several iterations, each time adding a new data point to the previous (observed) dataset. Since the observed data is somehow embedded inside the model definition, the only way I know to do this is to put the whole model definition inside a loop:
model = Model()
# a set of initial data points
data = getInitPoints((2,5))
for i in xrange(m):
with model:
x = Normal('x', mu=.5, tau=2. ** -2, shape=(2, 1))
z = Beta('z', alpha=10, beta=5.5)
d = Normal('data', mu=x, tau=.75 ** -2, observed=data)
step = NUTS()
trace = sample(1000, step)
data = numpy.vstack( (data,getnewPoint( (2,1) ) ) )
#use the samples
This may produce some unnecessary overhead specially if the model is large. To refrain from the overhead of repeatedly defining the same model, I wonder if there is a solution so that the same results could be achieved with something similar to the following idea:
with model:
x = Normal('x', mu=.5, tau=2. ** -2, shape=(2, 1))
z = Beta('z', alpha=10, beta=5.5)
data = getInitPoints()
for i in xrange(m):
# only necessary parts are included in the loop
with model:
d = Normal('data', mu=x, tau=.75 ** -2, observed=data)
step = NUTS()
trace = sample(1000, step)
data = numpy.vstack((data,getnewPoint()))
or even better:
data = getInitPoints()
dataHandle = magicHandle(data)
with model:
x = Normal('x', mu=.5, tau=2. ** -2, shape=(2, 1))
z = Beta('z', alpha=10, beta=5.5)
#
d = Normal('data', mu=x, tau=.75 ** -2, observed=dataHandle)
step = NUTS()
for i in xrange(m):
with model:
trace = sample(1000, step)
#
dataHandle = numpy.vstack((data,getnewPoint()))
It seems that it's not possible right know. But there is an open issue on this topic with possible solutions here : https://github.com/pymc-devs/pymc/issues/10
I have separate x and y arrays and want to connect the dots using a line path. This seems to be about the simplest possible example but I don't quite grok the writing the function. Here is my code:
<!DOCTYPE html>
<meta charset="utf-8">
<body>
<script src = "http://d3js.org/d3.v3.min.js"> </script>
<script>
var margin = {top: 20, right: 20, bottom: 20, left: 20},
width = 600 - margin.left - margin.right,
height = 270 - margin.top - margin.bottom;
var xdata = d3.range(20);
var ydata = [1, 4, 5, 9, 10, 14, 15, 15, 11, 10, 5, 5, 4, 8, 7, 5, 5, 5, 8, 10];
var xscl = d3.scale.linear()
.domain(d3.extent(xdata))
.range([0, width])
var yscl = d3.scale.linear()
.domain(d3.extent(ydata))
.range([height, 0])
var slice = d3.svg.line()
.x(function(d) { return xscl(xdata[d]);})
.y(function(d) { return yscl(ydata[d]);})
var svg = d3.select("body")
.append("svg")
.attr("width", width + margin.left + margin.right)
.attr("height", height + margin.top + margin.bottom)
svg.append("path")
.attr("class", "line")
.attr("d", slice)
</script>
</body>
But it returns an error Uncaught TypeError: Cannot read property 'length' of undefined, so clearly the function returned by d3.svg.line() doesn't have the right form. What's wrong? I pray not a typo!
I know it has been more than a year, but I had to deal with this problem also.
Storing path data (x, y) in 2 separate arrays is much more memory efficient than the 2D array d3.svg.line expects. For a very large number of points, the accepted answer is also inefficient by looping through all elements to create the 2D array.
The solution I found without adding any loops is write a wrapper function for d3.svg.line as follows:
var line = function(x, y){
return d3.svg.line()
.x(function(d,i) { return x[i]; })
.y(function(d,i) { return y[i]; })
(Array(x.length));
}
and then set the path attributes:
svg.append("path")
.attr("d", line(x_array, y_array))
See the updated fiddle here
Based on Elijah's spot on remark about d3.svg.line, I think it is hard to go about this without putting the array as expected by this function. So:
var xy = [];
for(var i=0;i<xdata.length;i++){
xy.push({x:xdata[i],y:ydata[i]});
}
I made other changes regarding .domain and the slice function per se. Here is a FIDDLE with the results of my effort.
d3.svg.line can only take one data source. However, you can feed it your two data sources by putting them into an object:
newData = {x: xdata, y: ydata};
var slice = d3.svg.line()
.x(function(d,i) { return xscl(d.xdata[i]);})
.y(function(d,i) { return yscl(d.ydata[i]);})
Then point your line function at newData and you should be set:
svg.append("path")
.attr("class", "line")
.attr("d", slice(newData))
Typically, though, you're better off building an array of coordinate pairs, since that's what it's expecting.