Can someone please explain to me the difference in the OSI Model between the 802.3 and 802.11 protocols?
As far as I understand they differ only in the MAC and PHY Layer. Is this accurate?
Are the remaining layers identical?
Yes, the IEEE 802.3 and IEEE 802.11 are standards that address the MAC and PHY layers. Anything above those layers remain agnostic from them (MAC & PHY layers).
Related
Did anyone try quantizing model with Nvidia TensorRT, some layers are not supported such as Mish/Softplus functions. Does anyone have a solution on how to use those with quantized models?
I am new to ML and Pytorch and I have the following problem:
I am looking for a Fully Convolutional Network architecture in Pytorch, so that the input would be an RGB image (HxWxC or 480x640x3) and the output would be a single channel image (HxW or 480x640). In other words, I am looking for a network that will preserve the resolution of the input (HxW), and will loose the channel dimension. All of the networks that I've came across (ResNet, Densenet, ...) end with a fully connected layer (without any upsampling or deconvolution). This is problematic for two reasons:
I am restricted with the choice of the input size (HxWxC).
It has nothing to do with the output that I expect to get (a single channel image HxW).
What am I missing? Why is there even a FC layer? Why is there no up-sampling, or some deconvolution layers after feature extraction? Is there any build-in torchvision.model that might suit my requirements? Where can I find such pytorch architecture? As I said, I am new in this field so I don't really like the idea of building such a network from scratch.
Thanks.
You probably came across the networks that are used in classification. So they end up with a pooling and a fully connected layer to produce a fixed number of categorical output.
Have a look at Unet
https://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/
Note: the original unet implementation use a lot of tricks.
You can simply downsample and then upsample symmetrically to do the work.
Your kind of task belongs to dense classification tasks, e.g. segmentation. In those tasks, we use fully convolution nets (see here for the original paper). In the FCNs you don't have any fully-connected layers, because when applying fully-connected layers you lose spatial information which you need for the dense prediction. Also have a look at the U-Net paper. All state-of-the art architectures use some kind of encoder-decoder architecture extended for example with a pyramid pooling module.
There are some implementations in the pytorch model zoo here. Search also Github for pytorch implementations for other networks.
I am looking for an advice for as fast as possible implementation of a convolution algorithm for CNN inference but not a training.
This convolution neural networks modeled as alexnet, mobilenet, resnet etc.. will run on embedded ARM device (A72, A53, A35) and possibly on embedded GPU as well.
I understand there is various implementation outthere and NN frameworks which have various implementations such as direct convolution, unrolling based convolution (im2col), FFT based or Winograd but mine primary focus is to execute CNN under performance constrain of embedded device.
If anybody has experience and can recommend convolution implementation for CPU and parallel implementation as well, point to research paper or open source implementation I would very appreciate it.
If it is still actual. I found a small framework to inference of pre-trained neural network on CPU. It uses Simd Library to accelerate its work. The library has very fast (single thread) implementation of Convolution, Pooling, Relu and many other network layers for CPU (x86 and ARM). CNN convolution includes Winograd's method.
I need to use Support Vector Machine (SVM) classifier with a Dynamic Time Warping (DTW) kernel for some audio processing task. All the tools I know (Weka, LibSVM, SKLearn) have SVMs with standard kernels (linear, poly, RBF) only. Where can I find a SVM tool/library with DTW kernel?
Does encoder must have the same number of layers as the decoder, in Variational Autoencoder? I got a little better result with the encoder (Dense): 54-10-5-3 and Decoder (Dense): 3-5-10-25-35-45-54
You asked an excellent question and the answer is no.
Since Aldream has already pointed a few works that use asymmetric architecture, I would like to point out the theoretical work that is related to your question.
I believe the best explanation is from ICLR 2017 conference paper: "Variational Lossy Autoencoder", Xi Chen and et al.
Section 2 is the must-read section. The author links variational autoencoder to bit-back coding and shows that the average code length (which is equivalent to variational lowerbound) always has extra code length from using approximate posterior distribution. When the decoder of VAE (which is log-datalikelihood p(x|z)) can model the data without the need of q(z|x), VAE will ignore the latent variable!
It means that if we have a very strong decoder (e.g. the same number of layers as the encoder), then there is a risk that VAE might completely ignore learning q(z|x) by simply set q(z|x) to the prior p(z), hence, keeps the KL loss to 0.
For more details, please read the paper above. The connection of VAE to bit-back coding is elegant in my opinion.
No, it is not uncommon to have asymmetric architectures, e.g. [1, 2, 3, etc.].
Tang, Shuai, et al. "Exploring Asymmetric Encoder-Decoder Structure for Context-based Sentence Representation Learning." arXiv preprint arXiv:1710.10380 (2017). pdf
LiNalisnick, Eric, and Padhraic Smyth. "Stick-breaking variational autoencoders." International Conference on Learning Representations (ICLR). 2017. pdf
Nash, Charlie, and Chris KI Williams. "The shape variational autoencoder: A deep generative model of part‐segmented 3D objects." Computer Graphics Forum. Vol. 36. No. 5. 2017. pdf