I am attempting to overlay a netcdf4 raster containing Aerosol Height data over Hawaii specifically. A sample file is available here. My variables of interest are latitude, longitude, time and aerosol height. Here is some reproducible data.
s1 <- data.frame(as.vector(lon), as.vector(lat), as.vector(ah))
s1
# as.vector.lon. as.vector.lat. as.vector.ah.
#1 -127.45199 -79.15431 NA
#2 -126.99632 -79.16919 NA
#3 -126.54577 -79.18321 NA
#4 -126.10027 -79.19641 NA
#5 -125.65974 -79.20880 NA
#6 -125.22412 -79.22042 NA
#7 -124.79333 -79.23129 NA
crsLatLon <- "+proj=longlat +datum=WGS84"
ex <- extent(c(-180,180,-90,90))
#empty raster with 0.1 degree resolution
pmraster <- raster(ncol=360*10, nrow=180*10, crs=crsLatLon,ext=ex)
#fills the empty raster with values from dataframe, s1
pmraster <- rasterize(s1[,1:2], pmraster, s1[,3], fun=mean, na.rm=T)
show(pmraster)
#class : RasterLayer
#dimensions : 1800, 3600, 6480000 (nrow, ncol, ncell)
#resolution : 0.1, 0.1 (x, y)
#extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
#crs : +proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0
#source : r_tmp_2020-06-26_114048_16840_75885.grd
#names : layer
#values : 0.1314196, 9424.118 (min, max)
#specifies region over Hawaii
exHI <- extent(c(-180,-140,10,30))
levelplot(crop(pmraster,exHI))
#Error: $ operator is invalid for atomic vectors
#In addition: Warning messages:
#1: In min(x) : no non-missing arguments to min; returning Inf
#2: In max(x) : no non-missing arguments to max; returning -Inf
#3: In min(x) : no non-missing arguments to min; returning Inf
#4: In max(x) : no non-missing arguments to max; returning -Inf
Can anyone help explain why I am getting this error message and how I may proceed to produce the desired raster image? Thank you in advance!
Here is a minimal and reproducible example:
library(raster)
library(lattice)
f <- system.file("external/test.grd", package="raster")
r <- raster(f)
levelplot(r)
#Error in UseMethod("levelplot") :
# no applicable method for 'levelplot' applied to an object of class "c('RasterLayer', #'Raster', 'BasicRaster')"
The lattice package methods do not know what a Raster* is. Hence you cannot use levelplot like this. The good news is that the rasterVis package implements a lattice method for Raster* objects; so all you need to do is
library(rasterVis)
levelplot(r)
And see ?rasterVis for many more ways to use levelplot
ssplot is also built on levelplot
spplot(r)
After trying this on different files, it became clear to me that there was no data available over the specific region I was isolating, resulting in my confusion.
Related
I have two points pattern (ppp) objects p1 and p2. There are X and Y points in p1 and p2 respectively. I have fitted a ppm model (with location coordinates as independent variables) in p1 and then used it to predict "intensity" for each of the Y points in p2.
Now I want to get the probability for event occurrence at that point/zone in p2. How can I use the predicted intensities for this purpose?
Can I do this using Spatstat?
Are there any other alternative.
The intensity is the expected number of points per unit area. In small areas (such as pixels) you can just multiply the intensity by the pixel area to get the probability of presence of a point in the pixel.
fit <- ppm(p1, .......)
inten <- predict(fit)
pixarea <- with(inten, xstep * ystep)
prob <- inten * pixarea
This rule is accurate provided the prob values are smaller than about 0.4.
In a larger region W, the expected number of points is the integral of the intensity function over that region:
EW <- integrate(inten, domain=W)
The result EW is a numeric value, the expected total number of points in W. To get the probability of at least one point,
P <- 1- exp(-EW)
You can also compute prediction intervals for the number of points, using predict.ppm with argument interval="prediction".
Your question, objective and current method are not very clear to me. It
would be beneficial, if you could provide code and graphics, that explains
more clearly what you have done, and what you are trying to obtain. If you
cannot share your data you can use e.g. the built-in dataset chorley as an
example (or simply simulate artificial data):
library(spatstat)
plot(chorley, cols = c(rgb(0,0,0,1), rgb(.8,0,0,.2)))
X <- split(chorley)
X1 <- X$lung
X2 <- X$larynx
mod <- ppm(X1 ~ polynom(x, y, 2))
inten <- predict(mod)
summary(inten)
#> real-valued pixel image
#> 128 x 128 pixel array (ny, nx)
#> enclosing rectangle: [343.45, 366.45] x [410.41, 431.79] km
#> dimensions of each pixel: 0.18 x 0.1670312 km
#> Image is defined on a subset of the rectangular grid
#> Subset area = 315.291058349571 square km
#> Subset area fraction = 0.641
#> Pixel values (inside window):
#> range = [0.002812544, 11.11172]
#> integral = 978.5737
#> mean = 3.103715
plot(inten)
Predicted intensities at the 58 locations in X2
intenX2 <- predict.ppm(mod, locations = X2)
summary(intenX2)
#> Min. 1st Qu. Median Mean 3rd Qu. Max.
#> 0.1372 4.0025 6.0544 6.1012 8.6977 11.0375
These predicted intensities intenX2[i] say that in a small neighbourhood
around each point X2[i] the estimated number of points from X1 is Poisson
distributed with mean intenX2[i] times the area of the small neighbourhood.
So in fact you have estimated a model where in any small area you have a
probability distribution for any number of points happening in that area. If
you want the distribution in a bigger region you just have to integrate the
intensity over that region.
To get a better answer you have to provide more details about your problem.
Created on 2018-12-12 by the reprex package (v0.2.1)
I fitted a survival model using an inverse weibull distribution in flexsurvreg:
if (require("actuar")){
invweibull <- list(name="invweibull",
pars=c("shape","scale"),
location="scale",
transforms=c(log, log),
inv.transforms=c(exp, exp),
inits=function(t){ c(1, median(t)) })
invweibull <- flexsurvreg(formula = kpnsurv~iaas, data = kpnrs2,
dist=invweibull)
invweibull
}
And I got the following output:
Call:
flexsurvreg(formula = kpnsurv ~ iaas, data = kpnrs2, dist = invweibull)
Estimates:
data. mean. est L95% U95% se exp(est) L95% U95%
shape NA 0.4870 0.4002 0.5927 0.0488 NA NA NA
scale NA 62.6297 36.6327 107.0758 17.1371 NA NA NA
iaas 0.4470 -0.6764 -1.2138 -0.1391 0.2742 0.5084 0.2971 0.8701
N = 302, Events: 54, Censored: 248
Total time at risk: 4279
Log-likelihood = -286.7507, df = 3
AIC = 579.5015
How can I get the p-value of the covariate estimate (in this case iaas)? Thank you for your help.
Just in case this is still useful to anyone, this worked for me. First extract the matrix of coefficient information from the model:
invweibull.res <- invweibull$res
Then divide the estimated coefficients by their standard errors to calculate the Wald statistics, which have asymptotic standard normal distributions:
invweibull.wald <- invweibull.res[,1]/invweibull.res[,4]
Finally, get the p-values:
invweibull.p <- 2*pnorm(-abs(invweibull.wald))
I am trying to apply Benedict Escoto's method from the paper "Bayesian Claim Severity with Mixed Distributions," published in Variance. I seem to be running into a JAGS simulation problem. When I run the code, JAGS gives me the following error:
Error in node ones[1] Node inconsistent with parents
The data is two-fold. First, there is a list of ground-up insurance claims. Information on their age in years, deductibles (truncation) and whether they have been capped (censoring, True or False) is provided. Second, there is a list of prior means and corresponding probability weights for a mixed exponential distribution.
One thing I noticed is that it works for one set of data of priors for mixed exponential distribution, but fails for another.
It works with:
Mean Weight
50 0.3
100 0.25
500 0.25
1500 0.1
5000 0.07
20000 0.03
But it fails with:
Mean Weight
3 0.72
14 0.19
42 0.05
138 0.02
503 0.01
1501 0.01
So it may have to do with parameter requirements for mixed exponential.
Data packet for JAGS is assembled as below
jags.data <- list(claims=(claim.df$x[!claim.df$capped]
- claim.df$truncation[!claim.df$capped]),
capped.claims=(claim.df$x[claim.df$capped]
- claim.df$truncation[claim.df$capped]),
alpha=alpha,
means=actual.means,
ones=rep(1, length(claim.df$[claim.df$capped])),
ages=claim.df$age[!claim.df$capped],
capped.ages=claim.df$age[claim.df$capped],
trend.shape=trend.shape,
trend.rate=1/trend.scale)
Notice that object "ones" is given values of 1 for each capped claim.
The initial values are supplied as below:
jags.init <- list(means=list(weights=prior.weights),
equal=list(weights=rep(1/m,m)))
Some miscellaneous values are provided as follows:
m <- length(actual.means)
alpha0 <- 20
alpha <- prior.weights * alpha0
trend.prior.mu <- .05
trend.prior.sigma <- .01
trend.scale <- trend.prior.sigma^2 / (1+trend.prior.mu)
trend.shape <- (1+trend.prior.mu)/trend.scale
The JAGS model is coded as below:
model <- "model {
weights ~ ddirch(alpha)
trend.factor ~ dgamma(trend.shape, trend.rate)
for (i in 1:length(claims)) {
buckets[i] ~ dcat(weights)
mu[i] <- means[buckets[i]] / trend.factor^ages[i]
claims[i] ~ dexp(1/mu[i])
}
for (i in 1:length(capped.claims)) {
capped.buckets[i] ~ dcat(weights)
capped.mu[i] <- means[capped.buckets[i]]/trend.factor^capped.ages[i]
prob.capped[i] <- exp(-capped.claims[i]/capped.mu[i])
ones[i] ~ dbern(prob.capped[i])
}
}"
Dirichlet, Categorical and Gamma distributions are used for priors. Ones is Bernoulli distributed to characterize claims as capped or uncapped.
Finally, the model is run in JAGS with the following:
model.out <- autorun.jags(model, data=jags.data, inits=jags.init,
monitor=c("weights","trend.factor"),
startburnin=1000, startsample=5000,
n.chains=n.chains, interactive=FALSE, thin=thin.factor)
Would anyone have an idea what goes wrong? Thanks
I'm currently trying to impute the missing data through Gaussian mixture model.
My reference paper is from here:
http://mlg.eng.cam.ac.uk/zoubin/papers/nips93.pdf
I currently focus on bivariate dataset with 2 Gaussian components.
This is the code to define the weight for each Gaussian component:
myData = faithful[,1:2]; # the data matrix
for (i in (1:N)) {
prob1 = pi1*dmvnorm(na.exclude(myData[,1:2]),m1,Sigma1); # probabilities of sample points under model 1
prob2 = pi2*dmvnorm(na.exclude(myData[,1:2]),m2,Sigma2); # same for model 2
Z<-rbinom(no,1,prob1/(prob1 + prob2 )) # Z is latent variable as to assign each data point to the particular component
pi1<-rbeta(1,sum(Z)+1/2,no-sum(Z)+1/2)
if (pi1>1/2) {
pi1<-1-pi1
Z<-1-Z
}
}
This is my code to define the missing values:
> whichMissXY<-myData[ which(is.na(myData$waiting)),1:2]
> whichMissXY
eruptions waiting
11 1.833 NA
12 3.917 NA
13 4.200 NA
14 1.750 NA
15 4.700 NA
16 2.167 NA
17 1.750 NA
18 4.800 NA
19 1.600 NA
20 4.250 NA
My constraint is, how to impute the missing data in "waiting" variable based on particular component.
This code is my first attempt to impute the missing data using conditional mean imputation. I know, it is definitely in the wrong way. The outcome would not lie to the particular component and produce outlier.
miss.B2 <- which(is.na(myData$waiting))
for (i in miss.B2) {
myData[i, "waiting"] <- m1[2] + ((rho * sqrt(Sigma1[2,2]/Sigma1[1,1])) * (myData[i, "eruptions"] - m1[1] ) + rnorm(1,0,Sigma1[2,2]))
#print(miss.B[i,])
}
I would appreciate if someone could give any advice on how to improve the imputation technique that could work with latent/hidden variable through Gaussian mixture model.
Thank you in advance
This is a solution for one type of covariance structure.
devtools::install_github("alexwhitworth/emclustr")
library(emclustr)
data(faithful)
set.seed(23414L)
ff <- apply(faithful, 2, function(j) {
na_idx <- sample.int(length(j), 50, replace=F)
j[na_idx] <- NA
return(j)
})
ff2 <- em_clust_mvn_miss(ff, nclust=2)
# hmm... seems I don't return the imputed values.
# note to self to update the code
plot(faithful, col= ff2$mix_est)
And the parameter outputs
$it
[1] 27
$clust_prop
[1] 0.3955708 0.6044292
$clust_params
$clust_params[[1]]
$clust_params[[1]]$mu
[1] 2.146797 54.833431
$clust_params[[1]]$sigma
[1] 13.41944
$clust_params[[2]]
$clust_params[[2]]$mu
[1] 4.317408 80.398192
$clust_params[[2]]$sigma
[1] 13.71741
I'm trying to reconstruct 3D points from 2D image correspondences. My camera is calibrated. The test images are of a checkered cube and correspondences are hand picked. Radial distortion is removed. After triangulation the construction seems to be wrong however. The X and Y values seem to be correct, but the Z values are about the same and do not differentiate along the cube. The 3D points look like as if the points were flattened along the Z-axis.
What is going wrong in the Z values? Do the points need to be normalized or changed from image coordinates at any point, say before the fundamental matrix is computed? (If this is too vague I can explain my general process or elaborate on parts)
Update
Given:
x1 = P1 * X and x2 = P2 * X
x1, x2 being the first and second image points and X being the 3d point.
However, I have found that x1 is not close to the actual hand picked value but x2 is in fact close.
How I compute projection matrices:
P1 = [eye(3), zeros(3,1)];
P2 = K * [R, t];
Update II
Calibration results after optimization (with uncertainties)
% Focal Length: fc = [ 699.13458 701.11196 ] ± [ 1.05092 1.08272 ]
% Principal point: cc = [ 393.51797 304.05914 ] ± [ 1.61832 1.27604 ]
% Skew: alpha_c = [ 0.00180 ] ± [ 0.00042 ] => angle of pixel axes = 89.89661 ± 0.02379 degrees
% Distortion: kc = [ 0.05867 -0.28214 0.00131 0.00244 0.35651 ] ± [ 0.01228 0.09805 0.00060 0.00083 0.22340 ]
% Pixel error: err = [ 0.19975 0.23023 ]
%
% Note: The numerical errors are approximately three times the standard
% deviations (for reference).
-
K =
699.1346 1.2584 393.5180
0 701.1120 304.0591
0 0 1.0000
E =
0.3692 -0.8351 -4.0017
0.3881 -1.6743 -6.5774
4.5508 6.3663 0.2764
R =
-0.9852 0.0712 -0.1561
-0.0967 -0.9820 0.1624
0.1417 -0.1751 -0.9743
t =
0.7942
-0.5761
0.1935
P1 =
1 0 0 0
0 1 0 0
0 0 1 0
P2 =
-633.1409 -20.3941 -492.3047 630.6410
-24.6964 -741.7198 -182.3506 -345.0670
0.1417 -0.1751 -0.9743 0.1935
C1 =
0
0
0
1
C2 =
0.6993
-0.5883
0.4060
1.0000
% new points using cpselect
%x1
input_points =
422.7500 260.2500
384.2500 238.7500
339.7500 211.7500
298.7500 186.7500
452.7500 236.2500
412.2500 214.2500
368.7500 191.2500
329.7500 165.2500
482.7500 210.2500
443.2500 189.2500
402.2500 166.2500
362.7500 143.2500
510.7500 186.7500
466.7500 165.7500
425.7500 144.2500
392.2500 125.7500
403.2500 369.7500
367.7500 345.2500
330.2500 319.7500
296.2500 297.7500
406.7500 341.2500
365.7500 316.2500
331.2500 293.2500
295.2500 270.2500
414.2500 306.7500
370.2500 281.2500
333.2500 257.7500
296.7500 232.7500
434.7500 341.2500
441.7500 312.7500
446.2500 282.2500
462.7500 311.2500
466.7500 286.2500
475.2500 252.2500
481.7500 292.7500
490.2500 262.7500
498.2500 232.7500
%x2
base_points =
393.2500 311.7500
358.7500 282.7500
319.7500 249.2500
284.2500 216.2500
431.7500 285.2500
395.7500 256.2500
356.7500 223.7500
320.2500 194.2500
474.7500 254.7500
437.7500 226.2500
398.7500 197.2500
362.7500 168.7500
511.2500 227.7500
471.2500 196.7500
432.7500 169.7500
400.2500 145.7500
388.2500 404.2500
357.2500 373.2500
326.7500 343.2500
297.2500 318.7500
387.7500 381.7500
356.2500 351.7500
323.2500 321.7500
291.7500 292.7500
390.7500 352.7500
357.2500 323.2500
320.2500 291.2500
287.2500 258.7500
427.7500 376.7500
429.7500 351.7500
431.7500 324.2500
462.7500 345.7500
463.7500 325.2500
470.7500 295.2500
491.7500 325.2500
497.7500 298.2500
504.7500 270.2500
Update III
See answer for corrections. Answers computed above were using the wrong variables/values.
** Note all reference are to Multiple View Geometry in Computer Vision by Hartley and Zisserman.
OK, so there were a couple bugs:
When computing the essential matrix (p. 257-259) the author mentions the correct R,t pair from the set of four R,t (Result 9.19) is the one where the 3D points lay in front of both cameras (Fig. 9.12, a) but doesn't mention how one computes this. By chance I was re-reading chapter 6 and discovered that 6.2.3 (p.162) discusses depth of points and Result 6.1 is the equation needed to be applied to get the correct R and t.
In my implementation of the optimal triangulation method (Algorithm 12.1 (p.318)) in step 2 I had T2^-1' * F * T1^-1 where I needed to have (T2^-1)' * F * T1^-1. The former translates the -1.I wanted, and in the latter, to translate the inverted the T2 matrix (foiled again by MATLAB!).
Finally, I wasn't computing P1 correctly, it should have been P1 = K * [eye(3),zeros(3,1)];. I forgot to multiple by the calibration matrix K.
Hope this helps future passerby's !
It may be that your points are in a degenerate configuration. Try to add a couple of points from the scene that don't belong to the cube and see how it goes.
More information required:
What is t? The baseline might be too small for parallax.
What is the disparity between x1 and x2?
Are you confident about the accuracy of the calibration (I'm assuming you used the Stereo part of the Bouguet Toolbox)?
When you say the correspondences are hand-picked, do you mean you selected the corresponding points on the image or did you use an interest point detector on the two images are then set the correspondences?
I'm sure we can resolve this problem :)