Loading MNIST Dataset into Octave - io

I am currently looking to dip my toes into deep learning after a few weeks reading some books and doing some more basic machine learning code. I found the MNIST digit database here http://yann.lecun.com/exdb/mnist/ and am currently trying to determine how to actually use the data.
The data appears to be saved in the IDX3 format, of which I am completely unfamiliar.
I have the training and test data sets saved as text files, but that seems to be fairly useless. For some reason, when I try to load them into Octave using the fopen command, the result is simply '-1'
Does anyone know of the correct way to load this data into Octave? Any help would be greatly appreciated.

Does this code work in Octave?
https://github.com/davidstutz/matlab-mnist-two-layer-perceptron/blob/master/loadMNISTImages.m
Note that is fopen returns -1, then maybe the file path is not correct.

Related

find an altered image from the original image data set

Here is my problem:
I must match two images. One image from the project folder and this folder have over 20.000 images. The other one is from a camera.
What I have done?
I can compare images with basic OpenCV example codes that I found in the documentation. OpenCV Doc I can also compare and find an image by using the hash of my image data set. It is so fast and it is only suitable for 2 exact images. One for query the other one is the target. But they are the same exact image.
So, I need something as reliable as feature matching and as fast as hash methods. But I can't use machine learning or anything on that level. It should be basic. Plus, I'm new to these stuff. So, my term project is on risk.
Example scenario:
If I ever take a picture of an image in my image data set from my computer's screen. This would change many features of the original image. In the case of defining what's in that image, a human won't struggle much but a comparison algorithm will struggle. Such a case leaves lot's of basic comparison algorithm out of the game. But, a machine-learning algorithm could solve the problem but it's forbidden to use in my project.
Needs:
It must be fast.
It must be accurate.
It must be easy to understand.
Any help is okay. A piece of code, maybe an article or a tutorial. Even an advice or a topic title might be really helpful to me.
Once saw this camera model identification challenge on kaggle. This notebook discusses about noise pattern changes with changing devices. May be you should look in to this and other notebooks in that challenge. Thanks!

How to read/change/write ISO-8859-1 files for non coders

I am not a developer so please bear with me with some basic questions.
There is a website where you can fill in a client data form on the site and then export it as a ISO-8859-1 coded file. You can also import ISO-8859-1 coded files there.
I on the other hand have client data in excel in CSV format.
What I would like to do is convert my CSV file in to the ISO encoded format so I can upload my client data files on the site without having to manually type in the form.
I have basic understanding of CSV files, but as I said im not a developer. First I thought I could open the ISO file in excel, just change the data in right columns, save and upload the file but it seems some of the data is indeed coded and not readable.
Is there some relatively easy way to learn how to do this (meaning for a non-coder putting a few hours of learning behind it)?
If yes, roughly where should I start looking, is there some kind of encoding programs I need to download, does Excel offer some kind of funcionality for this?
If no, meaning I would basically need to learn to program, where should I look to find a person who can do this? I guess it's very easy for someone who knows what they are doing? Any specific skills the person needs to have or is it basic programming question regardless of languages?
Thankful for any help, Vilho

Python OpenCV 3.x use threshold in LBHP algorithm

I wrote a Face detection script with the LBPH algorithm (in Python) cv2.face.createLBPHFaceRecognizer().
My problem is any other person that the algorithm is not trained on, returns me my number. (If it is me it returns 1 but if it's an other person it does the same). So I want to know what I can do, I read something about threshold but I don`t know how to use it and I read about a bug Link to bug. But I don't know how to rebuild the stuff. So I want to know what you recommend me, threshold or rebuilding, or anything else.
So I had a wrong indentation in my code. I returned the number for training with the python return command and so it stoped looping and only trained one number and image.

CNTK handwriting recognition

Has anyone ever programmed using CNTK for reading hand-filled documents? I tried OCRs and they dont do handwriting recognition at all (next to nothing). Thinking of using CNTK for the same. I searched and found that not many have tried such a thing. Any advice on libraries or any pointers anyone?
Here a basic OCR example using CNTK:
https://github.com/Microsoft/CNTK/blob/master/Tutorials/CNTK_103B_MNIST_FeedForwardNetwork.ipynb
However, in order to use the model in a real application you will need a way to segment the handwritting.

3d cloud viewer?

I have a file which contains a bunch of points with their x,y, z locations.I am looking for a simple viewer where in I can load this point data and view it .Rotation is essential for me to check the depth of the cloud being generated .Can someone point out a light weight viewer with minimum installation overhead for this?
I have used MeshLab and it worked well for me. IIRC it uses your average Windows installer.
You could also try CyArk viewer (a Java applet), or Leica Cyclone -- I haven't used either one.
Of course if your data format is not a standard one, they may not be able to read it.
R+ is an open source statistical program that I have used for this exact purpose. It can be accomplished in only a few lines of code.
First add the rgl and plotrix libraries.
Enter the following code:
pcd <- read.table(file.choose(),sep="",skip=10)
names(pcd) <- c("x","y","z")
plot3d(pcd$x,pcd$y,pcd$z,col=color.scale(pcd$z,c(0,1,1),c(1,1,0),c(1,0,1)))
Where pcd is the type of file (if I remember correctly), the first ten lines are skipped as they are a header (skip=10) and sep"" represents the delimiter used in the file. This last line of code plots the points and sets the color based upon depth.
I vote for Paraview. I am shocked that no one mentioned it before I did.
No matter which OS you use, Windows, Mac OS or Linux you can use it without any (big) problems. (You know software always has bugs)
Meshlab is good too. In fact you can convert file format easily to make sure they can be used in different software, if you can learn Python.
I do believe someones has already done it. Eg. .off (Meshlab format) to .vtk(Paraview format), like this one.
Update1:
Most visualization softwares are user-friendly, so maybe the problem you have is mainly about how to convert the source data you have to the specified format which can be used in these viewers. It may be helpful if there is an example of data you have.
matlab is a quite good tool to visualize your pcl, expecially for further analysis.

Resources