I'm having quite a hard time in figuring out where the error should be in this simple Rjags model:
dt=
[1] 0.000000e+00 0.000000e+00 0.000000e+00 0.000000e+00 0.000000e+00 2.304157e-06 1.014568e-05
[8] 1.798720e-05 2.582872e-05 3.367025e-05 4.151177e-05 4.935329e-05 5.719481e-05 6.503634e-05
[15] 7.287786e-05 8.071938e-05 8.856090e-05 9.640242e-05 1.042439e-04 1.120855e-04 1.200225e-04
[22] 1.274917e-04 1.332419e-04 1.426773e-04 1.682728e-04 2.018543e-04 2.302676e-04 2.810248e-04
[29] 3.408895e-04 3.942806e-04 4.818033e-04 5.682188e-04 6.298504e-04 7.320266e-04 9.604787e-04
[36] 1.315336e-03 1.695445e-03 1.945434e-03 2.047065e-03 2.192386e-03 2.404184e-03 2.629989e-03
[43] 2.931693e-03 3.414607e-03 4.085636e-03 5.009995e-03 5.897174e-03 6.532098e-03 7.205807e-03
[50] 8.125645e-03 9.097041e-03 9.857559e-03 1.021277e-02 1.026859e-02 1.053671e-02 1.125948e-02
[57] 1.207536e-02 1.250217e-02 1.297049e-02 1.372405e-02 1.463063e-02 1.552144e-02 1.604347e-02
[64] 1.628257e-02 1.701820e-02 1.843134e-02 2.045894e-02 2.294199e-02 2.517307e-02 2.666305e-02
[71] 2.762321e-02 2.890740e-02 3.095278e-02 3.290344e-02 3.429784e-02 3.502680e-02 3.525143e-02
[78] 3.534536e-02 3.508330e-02 3.431497e-02 3.302332e-02 3.136818e-02 2.979437e-02 2.850674e-02
[85] 2.713331e-02 2.539821e-02 2.301579e-02 2.002319e-02 1.702893e-02 1.447840e-02 1.217091e-02
[92] 1.006765e-02 8.119336e-03 6.326128e-03 4.835775e-03 3.611105e-03 2.514688e-03 1.618690e-03
[99] 1.006864e-03 6.681004e-04 4.693963e-04 3.256632e-04
data = list('dt')
library(rjags);library(foreign);library(coda)
cat('model{
for(k in 1:102){ dt[k] ~ dgamma(a, 0.5)}
a=ax+ay
X ~ dgamma(ax, 0.5)
Y ~ dgamma(ay, 0.5)
ax ~ dgamma(3, 1/10)
ay ~ dgamma(3, 1/10)
}',
file='model1.bug')
inits1 = list('ax'=15, 'ay'=15)
jags_mod = jags.model('model1.bug', data=data, inits=inits1, n.chains=1, n.adapt=2000)
update(jags_mod, n.iter=2000)
posts=coda.samples(model=jags_mod, variable.names=c('ax','ay','a', 'X', 'Y'), n.iter=1000000, thin=2000)
The main idea should be that variable dt is a Gamma, sum of two Gamma random variables with the same rate parameter.
I have made sure that there can't be negative values for the parameters, by assigning also Gamma priors on the two scale parameters, so I don't understand why Rjags is complaining here.
I would like to infer on both parameters ax, ay (and secondarily on a).
Thanks for all the help you will provide!
Best, Emanuele
The gamma distribution is strictly positive, so cannot be zero (or negative). The first 5 observations of your data are therefore not consistent with your model, hence the error.
Possible solutions are to model this as censoring, a mixture model with a mass at zero, or possibly to simply add a constant to your data, but without knowing what your data represents it isn't possible to say which is best.
Related
I have a binary classification problem for detecting AO/Non-AO images, using PyTorch for this purpose.
First, I load the data using the ImageFolder utility.
The Dataset class to label mapping in dataset.class_to_idx is {0: 'AO', 1: 'Non-AO'}.
So, my 'positive class' AO, is assigned a label 0, and my 'negative class' Non-AO is assigned a label 1.
Then I train and validate the model without any issues.
When I come to testing, I need to calculate some metrics on the test data.
Here is where I am confused.
[Method A]
fpr, tpr, thresholds = roc_curve(y_true, y_score)
roc_auc = auc(fpr, tpr)
[Method B]
# because 0 is my actual 'positive' class for this problem
fpr, tpr, thresholds = roc_curve(y_true, y_score, pos_label=0)
roc_auc = auc(fpr, tpr)
Now, this second curve is basically the mirror of the first one along the diagonal, right?
And I think, that it can't be the correct curve, because I checked the accuracy of the model by directly comparing y_true and y_pred to get the following accuracies.
Apart from this, here is what my confusion matrix looks like.
So, my first question is, am I doing something wrong? What is the significance of the curve from Method B? Can I say that Method A gives me the correct ROC curve for my classification task? If not, then how do I proceed for getting the correct curve?
What does true positive or true negative or any of the other terms signify for my confusion matrix? Does the matrix consider 0 : AO as negative and 1 : Non-AO as positive (I think so, yes) or the vice versa?
If 0 is indeed being considered as negative, when I actually want 0 to be considered as positive, how can I make changes to reflect so in the matrix (because I am using the matrix later to calculate other matrics like specificity, sensitivity, etc) ?
I trained my model in keras for binary classification. I used Resnet preformer on ImageNet and I got 95% of accuracy. In my dataset, I have 9004 images for training divided into the two class and 2250 images for test divided into the two class. But the confusion matrix give me
4502 0
4502 0
can some one help me to know what is the meaning of this resul?
The interpretation for your result is the following (please note that for simplicity the indices start from 1 and not 0):
For calculating the number of correct predictions on your test dataset, one needs to sum the main diagonal of the matrix.
For the first class (class 1), all your predictions are correct.
This can be inferred from your confusion matrix, as the element on the position [1,1] (first_row,first_col) is 4502. Since you have 0 the element on the position [1,2], it means that all the predictions for class 1 are correct.
However, for the second class, which has on the position [2,2] the value 0, this means that none of your predictions for this class are correct.
Practically, we can easily verify that 4502 is on the position [2,1].
Notes:
You may have calculated the accuracy/confusion matrix on the wrong dataset. According to your description, 4502 * 2 = 9004, which means that the confusion matrix that you are giving here is for the training set not for the test set.
Whenever you see a number on the confusion matrix that does not belong to the main diagonal, that means that you have a case of FP(false positive) or FN(false negative).
I want to fit poission distribution on my data points and want to decide based on chisquare test that should I accept or reject this proposed distribution. I only used 10 observations. Here is my code
#Fitting function:
def Poisson_fit(x,a):
return (a*np.exp(-x))
#Code
hist, bins= np.histogram(x, bins=10, density=True)
print("hist: ",hist)
#hist: [5.62657158e-01, 5.14254073e-01, 2.03161280e-01, 5.84898068e-02,
1.35995217e-02,2.67094169e-03,4.39345778e-04,6.59603327e-05,1.01518320e-05,
1.06301906e-06]
XX = np.arange(len(hist))
print("XX: ",XX)
#XX: [0 1 2 3 4 5 6 7 8 9]
plt.scatter(XX, hist, marker='.',color='red')
popt, pcov = optimize.curve_fit(Poisson_fit, XX, hist)
plt.plot(x_data, Poisson_fit(x_data,*popt), linestyle='--',color='red',
label='Fit')
print("hist: ",hist)
plt.xlabel('s')
plt.ylabel('P(s)')
#Chisquare test:
f_obs =hist
#f_obs: [5.62657158e-01, 5.14254073e-01, 2.03161280e-01, 5.84898068e-02,
1.35995217e-02, 2.67094169e-03, 4.39345778e-04, 6.59603327e-05,
1.01518320e-05, 1.06301906e-06]
f_exp= Poisson_fit(XX,*popt)
f_exp: [6.76613820e-01, 2.48912314e-01, 9.15697229e-02, 3.36866185e-02,
1.23926144e-02, 4.55898806e-03, 1.67715798e-03, 6.16991940e-04,
2.26978650e-04, 8.35007789e-05]
chi,p_value=chisquare(f_obs,f_exp)
print("chi: ",chi)
print("p_value: ",p_value)
chi: 0.4588956658201067
p_value: 0.9999789643475111`
I am using 10 observations so degree of freedom would be 9. For this degree of freedom I can't find my p-value and chi value on Chi-square distribution table. Is there anything wrong in my code?Or my input values are too small that test fails? if P-value >0.05 distribution is accepted. Although p-value is large 0.999 but for this I can't find chisquare value 0.4588 on table. I think there is something wrong in my code. How to fix this error?
Is this returned chi value is the critical value of tails? How to check proposed hypothesis?
I'm doing a network meta-analysis including several clinical trials. The response is binomial. Each trial contains several treatments.
When I do a random effects model, the output from JAGS and WinBUGS is similar. When I do a fixed effects model, the DIC and pD components are way out, though the posteriors of the parameters I'm interested in are similar.
I've got similar models that have Gaussian response, not binomial, and JAGS and WinBUGS are in agreement.
The BUGS/JAGS code for the fixed effects model is lifted from page 61 of this and appears below. However, the same code runs and produces similar posteriors using WinBUGS and JAGS, it's just DIC and pD that differ markedly. So I don't think this code is the problem.
for(i in 1:ns){ # Loop over studies
mu[i] ~ dnorm(0, .0001)
# Vague priors for all trial baselines
for (k in 1:na[i]) { # Loop over arms
r[i, k] ~ dbin(p[i, k], n[i, k])
# binomial likelihood
logit(p[i, k]) <- mu[i] + d[t[i, k]] - d[t[i, 1]]
# model for linear predictor
rhat[i, k] <- p[i, k] * n[i, k]
# expected value of the numerators
dev[i, k] <-
2 * (r[i, k] * (log(r[i, k]) - log(rhat[i, k])) +
(n[i, k] - r[i, k]) * (log(n[i, k] - r[i, k]) +
- log(n[i, k] - rhat[i, k]) ))
# Deviance contribution
}
resdev[i] <- sum(dev[i, 1:na[i]])
# summed residual deviance contribution for this trial
}
totresdev <- sum(resdev[])
# Total Residual Deviance
d[1] <- 0
# treatment effect is zero for reference treatment
for (k in 2:nt){
d[k] ~ dnorm(0, .0001)
} # vague priors for treatment effects
I found an old post describing a known issue, but that's too old for me to think it's the same problem.
Are there any known issues with JAGS reporting the wrong DIC and pD? (Searching for "JAGS bugs" isn't all that helpful.)
I'd be grateful of any pointers.
There are a number of different ways to calculate pD, and the method used by JAGS differs to that used by WinBUGS. See the Details section of the following help file:
?rjags::dic
Specifically:
DIC (Spiegelhalter et al 2002) is calculated by adding the “effective number of parameters” (pD) to the expected deviance. The definition of pD used by dic.samples is the one proposed by Plummer (2002) and requires two or more parallel chains in the model.
The details are in the (lengthy, but well worth reading) discussion of the following paper:
Spiegelhalter, D., N. Best, B. Carlin, and A. van der Linde (2002), Bayesian measures of model complexity and fit (with discussion). Journal of the Royal Statistical Society Series B 64, 583-639.
In general: all methods to calculate pD (of which I am aware) are approximations, and if two such methods disagree then it is probably because the assumptions behind for one (or both) approximation(s) are not being met.
Matt
I am trying to implement Expectation Maximization algorithm(Gaussian Mixture Model) on a data set data=[[x,y],...]. I am using mv_norm.pdf(data, mean,cov) function to calculate cluster responsibilities. But after calculating new values of covariance (cov matrix) after 6-7 iterations, cov matrix is becoming singular i.e determinant of cov is 0 (very small value) and hence it is giving errors
ValueError: the input matrix must be positive semidefinite
and
raise np.linalg.LinAlgError('singular matrix')
Can someone suggest any solution for this?
#E-step: Compute cluster responsibilities, given cluster parameters
def calculate_cluster_responsibility(data,centroids,cov_m):
pdfmain=[[] for i in range(0,len(data))]
for i in range(0,len(data)):
sum1=0
pdfeach=[[] for m in range(0,len(centroids))]
pdfeach[0]=1/3.*mv_norm.pdf(data[i], mean=centroids[0],cov=[[cov_m[0][0][0],cov_m[0][0][1]],[cov_m[0][1][0],cov_m[0][1][1]]])
pdfeach[1]=1/3.*mv_norm.pdf(data[i], mean=centroids[1],cov=[[cov_m[1][0][0],cov_m[1][0][1]],[cov_m[1][1][0],cov_m[0][1][1]]])
pdfeach[2]=1/3.*mv_norm.pdf(data[i], mean=centroids[2],cov=[[cov_m[2][0][0],cov_m[2][0][1]],[cov_m[2][1][0],cov_m[2][1][1]]])
sum1+=pdfeach[0]+pdfeach[1]+pdfeach[2]
pdfeach[:] = [x / sum1 for x in pdfeach]
pdfmain[i]=pdfeach
global old_pdfmain
if old_pdfmain==pdfmain:
return
old_pdfmain=copy.deepcopy(pdfmain)
softcounts=[sum(i) for i in zip(*pdfmain)]
calculate_cluster_weights(data,centroids,pdfmain,soft counts)
Initially, I've passed [[3,0],[0,3]] for each cluster covariance since expected number of clusters is 3.
Can someone suggest any solution for this?
The problem is your data lies in some manifold of dimension strictly smaller than the input data. In other words for example your data lies on a circle, while you have 3 dimensional data. As a consequence when your method tries to estimate 3 dimensional ellipsoid (covariance matrix) that fits your data - it fails since the optimal one is a 2 dimensional ellipse (third dimension is 0).
How to fix it? You will need some regularization of your covariance estimator. There are many possible solutions, all in M step, not E step, the problem is with computing covariance:
Simple solution, instead of doing something like cov = np.cov(X) add some regularizing term, like cov = np.cov(X) + eps * np.identity(X.shape[1]) with small eps
Use nicer estimator like LedoitWolf estimator from scikit-learn.
Initially, I've passed [[3,0],[0,3]] for each cluster covariance since expected number of clusters is 3.
This makes no sense, covariance matrix values has nothing to do with amount of clusters. You can initialize it with anything more or less resonable.