I am working on machine learning task in Matlab (using Neural networks toolbox of Matlab). Thus I need to run computations that are very demanding in terms of RAM and processor time. My computer is not sufficient for that task but I have an access to Linux server that is powerful enough.
I am looking for the way how to run computational tasks from Matlab GUI that is running on my own computer on that server. Ideally it should work given these condition:
My computer is running Windows 7, the server Red Hat Linux
I can connect to the server only using SSH protocol (currently I'm using putty for that task)
The server does not have Matlab installed. I can ask server admin to install it but that may take some time, it would be better if that is not needed. There is Java virtual machine (and development kit) installed on the server.
I prefer to use Matlab GUI on my computer (not Matlab command line)
How can I do that? If you need more information about the settings or if you know other solutions please leave a comment. I'll be happy to respond.
If you do not have matlab on the server, the only way is to:
modify the code such that it outputs a file
compile your code using mcc (toolbox required)
send it to your server (do you have a shared disk?, otherwise use pscp http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html )
execute it (using putty from command line)
get back the output file (same than sending)
We are all doing that in my lab (except that we are on ubuntu)
As hinted at in the comments, Parallel Computing Toolbox + MATLAB Distributed Computing Server could address this, and if you're actively updating the code, it may be simpler than the mcc approach. However, you do have to install the MDCS piece on the server and get things set up. If you're not using any MEX files, the difference in architecture should be irrelevant. If you are using MEX files, you need to ensure that you can compile them for the server architecture and make them available there.
You can run MATLAB jobs on remote Linux server from your local PC without the MDCS setup however for large scale and multiple users the MCDS is highly recommended. You can use plink to automated the putty command line execution from MATLAB.
We are executing our Simulink model simulations with multiple iterations on a more capable remote Linux server. You will require the following:
1) MATLAB installation on both client and server machine with parallel computing toolbox
2) Mapped Linux server shared network drive (for data I/O)
3) Plink for automated Putty
4) Command line script for silent MATLAB execution on the remote machine
MATLAB code example:
plinkFile = which('plink.exe');
command = [plinkFile ' -ssh ' [ServerUrl] ' -l ' [Username] ...
' -pw ' [Password] ' matlab -nodesktop -nosplash ' ...
'-r "' [RemoteScriptName] ...
'\(\''' InputDataSharedDriveFilePath] '\''\);" &'];
[status,cmdout] = system(command);
Related
Is there any software working like this?
Runs as a standalone program. No install is needed. Thus, can be used as an Ansible module.
After running the program in a remote Linux machine, I can open up a web browser, then open a web page provided by the program. The program provides features similar to file explorer, IDE-level code editor, debugger, etc. In terms of debugger, there is already similar one; gdbgui.
There is another way such as Gnome, KDE or X11. However, these requires much packages to be installed. I don't want they be installed, because my Linux machines are kept to be small and secure.
You might consider having some terminal emulator running inside a browser. Such things exist, e.g. libonion has oterm as an example application. Then you can do all the things that a command line interface thru a unix shell provides (of course, you won't be able to run GUI applications, e.g. X11 clients such as GTK or Qt applications).
You could also consider some webmin like stuff.
Notice that you don't need to have a desktop environment on a remote Linux machine. Most of them (e.g. internet servers) have only command line interface.
Learn more about X11: you could have an X11 server on your laptop (e.g. under Windows if so needed) and run remotely X11 clients (that is GUI applications) with ssh -X on your remote Linux system.
However, these requires much packages to be installed. I don't want they be installed, because my Linux machines are kept to be small and secure.
I don't understand that requirement. On my VPS, running in some OVH datacenter, I do have X11 client applications (notably emacs). I don't believe that lowers the security of my system, and the disk space consumption for X11 applications and libraries is small enough these days. And of course I use standard commands (like cp(1), mv(1), rm(1), grep(1), find(1), less(1), file(1), sed(1) ....) to manage files. Any graphical file manager is useless (and I never use them, while using Unix since 1986)
You really should learn how to use the command line on Linux. It is incredibly powerful.
I want to run a script on a remote machine without using ssh.. Is it possible?
If yes then what all available options are there in linux?
Here are few details regarding the same.
1) On my machine i have a script, say a.sh, which performs installation of few packages. (such as lsof, ntp, vim etc).
2) I need to copy this script to the remote machine, and i want to execute it over there. (So that those packages get installated on to those machines.
3) what i am trying to achieve is if i have to do these packages installation on n no of machines, then with this mechanism i can automate this part.
Is there any preexisting infrastructure in linux which can help me doing this without ssh? (ssh to these virtual machines/remote counterparts is not possible)
Cheers,
Placid/
Ubuntu one has this form of remote package management availible , although on a personal level I simply prefer to scp / ssh exec scripts like this when deploying a change set to multiple machines .
I've recently been working with the NAO. We're trying to connect the NavChip to it and do some experiments related to robot navigation. The NAO uses a modified 2.6 linux kernel on it's geode system. I've managed to make my NavChip work on it (needed to compile the linux cp210x kernel module etcetera). I can therefore run a C program that came with the NavChip and collect data from it. However, the data can only be logged on the local file system. I'd like to stream this data over the network to a windows machine, since all the processing is MATLAB based. Would anyone have any suggestions on how I can send this data from the NAO to a windows machine?
The NAO's system is pretty limited. It has ssh, and some common utilities like cat etc., but nothing advanced.
I'm not sure I understand the problem properly but I think you've answered your own question you mentioned ssh is installed so why not just scp the file? Using some ssh client on the windows box to remotely connect and download relevent log file.
If you really do need to push the file from the remote host to local machine (rather then connect to remote host and download to local) then netcat should work see here: http://www.g-loaded.eu/2006/11/06/netcat-a-couple-of-useful-examples/
Other wise just write your own socket program in C and pipe the file accross (should be pretty trivial).
I need to develop applications for Linux but I don't want to either program on Linux (I already have all my "precious" tools setted up) or test it on Windows using some kind of POSIX for Windows and hoping that if runs well on that runs well on real Linux.
What's the better choice? Preferentially I want to use Eclipse IDE for compile/run/debug and run my programs on a Linux distribution that is running on a local VM or remote.
Right now I'm using a similar approach, but for Java Web Service testing on a remote server. Perhaps that's the way to go?
Edit:
In order to beter explain what I want, here is the steps that I want to follow:
Program in C, for POSIX compliant systems, using Eclipse on Windows
Make small tests on Windows, perhaps using Cygwin (this is not mandatory it's just to be quicker)
From my Windows Eclipse, I want to run/debug my application on a real Linux environment (could be a VM or a remote machine) and, preferentially, redirect the application stdout to my computer. The Linux machine only exists in order to garantee that everything runs ok, no need of even open it.
One thing that I didn't mentioned: all of the applications are command line, no need for GUI, just input from a shell and read the output.
First, install Linux in a virtual PC like VirtualBox or VirtualPC or something from vmware.
Then configure Eclipse for remote development. That allows you to run tools (like the debugger, the compiler suite, etc) on Linux from your Windows desktop inside of Eclipse. You edit the files just like you're used to, you debug as if the app was running local on Windows, etc. Eclipse will do the plumbing.
Remote server is the way to go. But most people have a powerful enough machine to run a 32-bit Linux distro in Virtualbox which is better than a real remote server because you have full control of setup and config.
But install cygwin including GCC and use that to run initial compile (and maybe unit tests) locally. Also, do use Valgrind on your Linux VM to help you produce cleaner code.
I have recently learned, that in MATLAB, the "!" mark runs the code in a command line and follwed by the "&" will pull up the command window, FOR EXAMPLE:
!RUN_FILE.bat&
It was a great day when I discovered this, or should I say when someone told me this. Anyways I was wondering if there was a way to run this batch file on a linux machine that I am connecting to remotley with a Windows desktop? (I am a newbie when it come to anything Linux, so if it doesn't make sense let me know and please dumb down you answers if you don't mind) haha
THANKS,
ME
The Linux shell doesn't understand batch files. That's not to say that the batch file can't be re-written to work in the Linux shell. Post some more information about the Linux shell you are using and your batch script and perhaps we can help you port it to Linux.
On a second read of your question, I interpreted it a different way. Are you running Matlab on a Linux machine that you are remotely connected to, or are you running Matlab on your local Windows machine and you want to send the ! command to the remote Linux box?
If the latter is true, then you should probably be able to write a batch file (to run from Matlab) that will launch the remote script on the Linux box. How are you connected to the Linux machine? VNC? SSH?
Windows and Linux use different shell scripting languages, so you couldn't run a .bat directly on Linux, but could create a bash script to perform the same actions.
You might want to see this bat2bash online converter (I prefer re-writing though):
http://www.filegod.netfirms.com/Bat2Bash.html