Need assistance regarding the Principal components to be displayed in pyLDAvis. It shows PC1 and PC2 by default, however I am interested in exploring other components as well.
I referred this documentation but the specified option plot_opts={'xlab': 'PC1', 'ylab': 'PC2'} only changes plot labels.
I am using pyLDAvis through gensim:
pyLDAvis.enable_notebook()
vis = pyLDAvis.gensim.prepare(lda_model, corpus, id2word)
You can not change the dimensions in this library. However, you could do a multidimensional scaling on your own. In this package, they use "Dimension reduction via Jensen-Shannon Divergence & Principal Coordinate Analysis (aka Classical Multidimensional Scaling)".
Just replicate this method and set dimensions to more the amount you want.
You can do it with scikit-learn: https://scikit-learn.org/stable/auto_examples/manifold/plot_mds.html
Related
I understand that in order to create a color image, three channel information of input data must be maintained inside the network. However, data must be flattened to pass through the linear layer. If so, can GAN consisting of only FC layer generate only black and white images?
Your fully connected network can generate whatever you want. Even three channel outputs. However, the question is: does it make sense to do so? Flattened your input will inherently lose all kinds of spatial and feature consistency that is naturally available when represented as an RGB map.
Remember that an RGB image can be thought of as 3-element features describing each spatial location of a 2D image. In other words, each of the three channels gives additional information about a given pixel, considering these channels as separate entities is a loss of information.
I have a dataset with 10 variables and I am looking to reduce it to a single "score". I understand the basics of PCA, it uses the covariance matrix of the variables to create 10 eigenvectors and 10 eigenvalues. Normally what is done is people multiply the eigenvectors by the normalized data to generate principal components, they pick some arbitrary number of principal components, and throw them into a regression and get the fitted value. In other words, the coefficients multiplied by the principal components allow for a data reduction to a single variable.
My question is whether I need to do the regression step (I do not have a dependent variable). Instead of doing the regression, could I use the eigenvalues as my coefficients? In other words, could I take the inner product of the vector of eigenvalues and the principal component to create a single variable?
I've never seen it used this way (and as far as I'm aware no one has even asked this question), but it seems intuitive to me. Am I missing something or is this legit and I just haven't looked in the right places? Thanks!
Do you have a target variable?
Because there are two things here,
principal component regression and
principal components analysis.
If you have a target variable (a variable you want to predict or explain) the regression if not the second.
PC Regression --> you need a target variable
PC Analysis --> you do not need any regression step.
I think you are in the second case. So you just want to reduce your 10 variables into 1 that summarize everything.
Perhaps and interesting tutorial in Python: https://jakevdp.github.io/PythonDataScienceHandbook/05.09-principal-component-analysis.html
Two different institutions are using the same dataset for spatial interpolation using Ordinary Kriging. However, the resulting maps shows deviances.What are the potential causes of differences in maps?
Some options which come to mind would be:
Variogram fit parameter differences
Neighborhood radius limit
Interpolated grid density (if interpolating into a grid)
Also, you may have better luck on gis.stackexchange.com, and I’d recommend you include a minimal publishable example and whatever other details you can find (what softwares/libraries in which company, etc).
I'm working on a 3D reconstruction system and want to generate a triangular mesh from the registered point cloud data using Python 3. My objects are not convex, so the marching cubes algorithm seems to be the solution.
I prefer to use an existing implementation of such method, so I tried scikit-image and Open3d but both the APIs do not accept raw point clouds as input (note that I'm not expert of those libraries). My attempts to convert my data failed and I'm running out of ideas since the documentation does not clarify the input format of the functions.
These are my desired snippets where pcd_to_volume is what I need.
scikit-image
import numpy as np
from skimage.measure import marching_cubes_lewiner
N = 10000
pcd = np.random.rand(N,3)
def pcd_to_volume(pcd, voxel_size):
#TODO
volume = pcd_to_volume(pcd, voxel_size=0.05)
verts, faces, normals, values = marching_cubes_lewiner(volume, 0)
open3d
import numpy as np
import open3d
N = 10000
pcd = np.random.rand(N,3)
def pcd_to_volume(pcd, voxel_size):
#TODO
volume = pcd_to_volume(pcd, voxel_size=0.05)
mesh = volume.extract_triangle_mesh()
I'm not able to find a way to properly write the pcd_to_volume function. I do not prefer a library over the other, so both the solutions are fine to me.
Do you have any suggestions to properly convert my data? A point cloud is a Nx3 matrix where dtype=float.
Do you know another implementation [of the marching cube algorithm] that works on raw point cloud data? I would prefer libraries like scikit and open3d, but I will also take into account github projects.
Do you know another implementation [of the marching cube algorithm] that works on raw point cloud data?
Hoppe's paper Surface reconstruction from unorganized points might contain the information you needed and it's open sourced.
And latest Open3D seems to be containing surface reconstruction algorithms like alphaShape, ballPivoting and PoissonReconstruction.
From what I know, marching cubes is usually used for extracting a polygonal mesh of an isosurface from a three-dimensional discrete scalar field (that's what you mean by volume). The algorithm does not work on raw point cloud data.
Hoppe's algorithm works by first generating a signed distance function field (a SDF volume), and then passing it to marching cubes. This can be seen as an implementation to you pcd_to_volume and it's not the only way!
If the raw point cloud is all you have, then the situation is a little bit constrained. As you might see, the Poisson reconstruction and Screened Poisson reconstruction algorithm both implement pcd_to_volume in their own way (they are highly related). However, they needs additional point normal information, and the normals have to be consistently oriented. (For consistent orientation you can read this question).
While some Delaunay based algorithm (they do not use marching cubes) like alphaShape and this may not need point normals as input, for surfaces with complex topology, it's hard to get a satisfactory result due to orientation problem. And the graph cuts method can use visibility information to solve that.
Having said that, if your data comes from depth images, you will usually have visibility information. And you can use TSDF to build a good surface mesh. Open3D have already implemented that.
I'm trying to graph data similarly to this, except using an actual 3rd dimension instead of color. Vis.js would have been exactly what I want, except it doesn't seem to support labels for individual points. Where can I get interactive, preferably in-browser, 3D graph visualization for this type of data?
The Graph3d of vis.js doesn't support setting labels for individual data points, but you can set tooltips, see: http://visjs.org/examples/graph3d/11_tooltips.html. Maybe that's a workable solution for you? If not, you could open a feature request for this: https://github.com/almende/vis/issues