My situation here is that I need to send my program to my university's cluster which runs on Linux. To save file on my com locally using Windows, I simply use
fid = fopen('exp.txt', 'w');
fprintf(fid, '%6.2f %12.8f\n', y);
fclose(fid);
What is the Linux equivalent of this command if I want to save to a directory on the cluster for example /home/MyFolder/filename.txt? Do I use
mkdir('/home/MyFolder/')
Furthermore how do I specify a directory in Linux? I have no experience using Linux at all.
If you just start MatLab while you're in your desired directory (cd /path/to/dir/ to change current directory), your existing code will still work.
If you need a different working directory (e.g. to find scripts and other data files), fopen('/path/to/exp.txt', 'w'); should help.
Related
My book states:
Every program that runs on your computer has a current working directory, or cwd. Any filenames or paths that do not begin with the root folder are assumed to be under the current working directory
As I am on OSX, my root folder is /. When I type in os.getcwd() in my Python shell, I get /Users/apple/Documents. Why am I getting the Documents folder in my cwd? Is it saying that Python is using Documents folder? Isn't there any path heading to Python that begins with / (the root folder)? Also, does every program have a different cwd?
Every process has a current directory. When a process starts, it simply inherits the current directory from its parent process; and it's not, for example, set to the directory which contains the program you are running.
For a more detailed explanation, read on.
When disks became large enough that you did not want all your files in the same place, operating system vendors came up with a way to structure files in directories. So instead of saving everything in the same directory (or "folder" as beginners are now taught to call it) you could create new collections and other new collections inside of those (except in some early implementations directories could not contain other directories!)
Fundamentally, a directory is just a peculiar type of file, whose contents is a collection of other files, which can also include other directories.
On a primitive operating system, that was where the story ended. If you wanted to print a file called term_paper.txt which was in the directory spring_semester which in turn was in the directory 2021 which was in the directory studies in the directory mine, you would have to say
print mine/studies/2021/spring_semester/term_paper.txt
(except the command was probably something more arcane than print, and the directory separator might have been something crazy like square brackets and colons, or something;
lpr [mine:studies:2021:spring_semester]term_paper.txt
but this is unimportant for this exposition) and if you wanted to copy the file, you would have to spell out the whole enchilada twice:
copy mine/studies/2021/spring_semester/term_paper.txt mine/studies/2021/spring_semester/term_paper.backup
Then came the concept of a current working directory. What if you could say "from now on, until I say otherwise, all the files I am talking about will be in this particular directory". Thus was the cd command born (except on old systems like VMS it was called something clunkier, like SET DEFAULT).
cd mine/studies/2021/spring_semester
print term_paper.txt
copy term_paper.txt term_paper.backup
That's really all there is to it. When you cd (or, in Python, os.chdir()), you change your current working directory. It stays until you log out (or otherwise exit this process), or until you cd to a different working directory, or switch to a different process or window where you are running a separate command which has its own current working directory. Just like you can have your file browser (Explorer or Finder or Nautilus or whatever it's called) open with multiple windows in different directories, you can have multiple terminals open, and each one runs a shell which has its own independent current working directory.
So when you type pwd into a terminal (or cwd or whatever the command is called in your command language) the result will pretty much depend on what you happened to do in that window or process before, and probably depends on how you created that window or process. On many Unix-like systems, when you create a new terminal window with an associated shell process, it is originally opened in your home directory (/home/you on many Unix systems, /Users/you on a Mac, something more or less like C:\Users\you on recent Windows) though probably your terminal can be configured to open somewhere else (commonly Desktop or Documents inside your home directory on some ostensibly "modern" and "friendly" systems).
Many beginners have a vague and incomplete mental model of what happens when you run a program. Many will incessantly cd into whichever directory contains their script or program, and be genuinely scared and confused when you tell them that you don't have to. If frobozz is in /home/you/bin then you don't have to
cd /home/you/bin
./frobozz
because you can simply run it directly with
/home/you/bin/frobozz
and similarly if ls is in /bin you most definitely don't
cd /bin
./ls
just to get a directory listing.
Furthermore, like the ls (or on Windows, dir) example should readily convince you, any program you run will look in your current directory for files. Not the directory the program or script was saved in. Because if that were the case, ls could only produce a listing of the directory it's in (/bin) -- there is nothing special about the directory listing program, or the copy program, or the word processor program; they all, by design, look in the current working directory (though again, some GUI programs will start with e.g. your Documents directory as their current working directory, by design, at least if you don't tell them otherwise).
Many beginners write scripts which demand that the input and output files are in a particular directory inside a particular user's home directory, but this is just poor design; a well-written program will simply look in the current working directory for its input files unless instructed otherwise, and write output to the current directory (or perhaps create a new directory in the current directory for its output if it consists of multiple files).
Python, then, is no different from any other programs. If your current working directory is /Users/you/Documents when you run python then that directory is what os.getcwd() inside your Python script or interpreter will produce (unless you separately os.chdir() to a different directory during runtime; but again, this is probably unnecessary, and often a sign that a script was written by a beginner). And if your Python script accepts a file name parameter, it probably should simply get the operating system to open whatever the user passed in, which means relative file names are relative to the invoking user's current working directory.
python /home/you/bin/script.py file.txt
should simply open(sys.argv[1]) and fail with an error if file.txt does not exist in the current directory. Let's say that again; it doesn't look in /home/you/bin for file.txt -- unless of course that is also the current working directory of you, the invoking user, in which case of course you could simply write
python script.py file.txt
On a related note, many beginners needlessly try something like
with open(os.path.join(os.getcwd(), "input.txt")) as data:
...
which needlessly calls os.getcwd(). Why is it needless? If you have been following along, you know the answer already: the operating system will look for relative file names (like here, input.txt) in the current working directory anyway. So all you need is
with open("input.txt") as data:
...
One final remark. On Unix-like systems, all files are ultimately inside the root directory / which contains a number of other directories (and usually regular users are not allowed to write anything there, and system administrators with the privilege to do it typically don't want to). Every relative file name can be turned into an absolute file name by tracing the path from the root directory to the current directory. So if the file we want to access is in /home/you/Documents/file.txt it means that home is in the root directory, and contains you, which contains Documents, which contains file.txt. If your current working directory were /home you could refer to the same file by the relative path you/Documents/file.txt; and if your current directory was /home/you, the relative path to it would be Documents/file.txt (and if your current directory was /home/you/Music you could say ../Documents/file.txt but let's not take this example any further now).
Windows has a slightly different arrangement, with a number of drives with single-letter identifiers, each with its own root directory; so the root of the C: drive is C:\ and the root of the D: drive is D:\ etc. (and the directory separator is a backslash instead of a slash, although you can use a slash instead pretty much everywhere, which is often a good idea for preserving your sanity).
Your python interpreter location is based off of how you launched it, as well as subsequent actions taken after launching it like use of the os module to navigate your file system. Merely starting the interpreter will place you in the directory of your python installation (not the same on different operating systems). On the other hand, if you start by editing or running a file within a specific directory, your location will be the folder of the file you were editing. If you need to run the interpreter in a certain directory and you are using idle for example, it is easiest to start by creating a python file there one way or another and when you edit it you can start a shell with Run > Python Shell which will already be in that directory. If you are using the command line interpreter, navigate to the folder where you want to run your interpreter before running the python/python3/py command. If you need to navigate manually, you can of course use the following which has already been mentioned:
import os
os.chdir('full_path_to_your_directory')
This has nothing to do with osx in particular, it's more of a concept shared by all unix-based systems, and I believe Windows as well. os.getcwd() is the equivalent of the bash pwd command - it simply returns the full path of the current location in which you are in. In other words:
alex#suse:~> cd /
alex#suse:/> python
Python 2.7.12 (default, Jul 01 2016, 15:34:22) [GCC] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getcwd()
'/'
It depends from where you started the python shell/script.
Python is usually (except if you are working with virtual environments) accessible from any of your directory. You can check the variables in your path and Python should be available. So the directory you get when you ask Python is the one in which you started Python. Change directory in your shell before starting Python and you will see you will it.
os.getcwd() has nothing to do with OSX in particular. It simply returns the directory/location of the source-file. If my source-file is on my desktop it would return C:\Users\Dave\Desktop\ or let say the source-file is saved on an external storage device it could return something like G:\Programs\. It is the same for both unix-based and Windows systems.
Right now I am creating a text file, and then writing som text to it with the command (in python 3):
userFile = open("users\\"+userName+".txt","w")
This creates the file in the folder named users, but when I run the program on a linux system, it instead creates, in the root folder, a file named users\userName.txt
How is the path definition different for python 3 in linux?
Windows has drives (C:, D:, X: etc) and backslashes or double backslashes, e.g.
C:\Users\JohnSmith is the same as C:\\Users\\JohnSmith
On Linux, there are no drives (per se) and forward slashes, e.g.
/home/name
The best way to get a feel for paths is by using os. Try typing this into your python terminal print(os.path.abspath('.'))
It's not different in python 3 in linux it's different in linux. Generally speaking *nix file paths use / as a directory separator, where as windows uses \ (for what ever reason).
In python 3 you can use the pathlib.Path to abstract your code from the OS. So you can do something like
open(Path(f"~/{username}.txt"), "w")
The tilde ~ refers to a user's home directory. Python will figure out which file system the code is running on and do the right thing to map directory separators. You could also do
open(Path(f"/users/{username}.txt"), "w")
to address a specific user directory, the / refers to the root of the file system and should work on Linux and Windows (although I haven't tested that).
https://docs.python.org/3/library/pathlib.html?highlight=pathlib%20path#module-pathlib
I have this report.tex file which I convert to pdf as follows:
latex report.tex
dvips report.dvi -o report.ps
ps2pdfwr report.ps report.pdf
These commands are placed in a file called linux_build
This works great. However, this works on my Linux machine and most of my work I do on my Mac
I checked out MacTex which is enormous (> 4GB) so I decided to process the latex file remotely (so working on my Mac, and building on my Linux computer). Like this:
ssh latex#192.168.2.8 "cd build && ./linux_build" < report.tex
scp latex#192.168.2.8:build/report.pdf .
All this works, except for one thing:
latex report.tex
That command simply looks on disk, not for some input stream/pipe of whatever. It is at this point I'm not sure anymore how to fix this. Any suggestions ?
scp report.tex latex#192.168.2.8:/home/Jeanluca/build
ssh latex#192.168.2.8 "cd build && ./linux_build" < report.tex
scp latex#192.168.2.8:build/report.pdf
Try sending your tex file over first. To the full path for the build directory.
I know your frustration. It grew so big that I put together a small script which efficiently runs LaTeX on a remote server.
It prestarts LaTeX on the remote end with the preamble of the previous run such that LaTeX has already loaded the required packages and is ready to go when changed contents arrives, and starts streaming the resulting PDF file immediately (before LaTeX has finished writing to it). Also dependencies (images, included source files etc.) are transparently handled.
Maybe you too find it useful: https://github.com/iblech/sshlatex
No installation is necessary. Execution is simple:
$ sshlatex ssh.example.org foo.tex
If latex supports reading from standard input then you can do it that way.
Possibly just latex or maybe latex - or possibly latex /dev/stdin to fake it as a local file.
If that doesn't work then you need to transfer the file over first.
I have a Tcl script that works on Windows. When I run it and pass in file names when prompted, usually with a relative directory path. On Windows the script will take absolute OR relative directory paths ... it works, no problems at all.
On Linux it will NOT work in the same way. It will only work if I spell out the full file path, e.g. /home/gxuser/input/test.txt ... It will NOT take relative directory paths like ./input/test.txt
By "work" I mean it should OPEN the file ... It is there and the permissions are fine.
On Linux the program fails with the following error:
couldn't open "./input/test.txt": no such file or directory
while executing
"open $upload r"
The offending line in the code is:
set infile [open $upload r]
What am I doing wrong? I presume, I have overlooked some nuance of the Tcl language, and assumed it should recognize relative paths.
Relevant information:
% puts $tcl_version
8.4
% info patchlevel
8.4.19
% uname -a
Linux gxengine 2.6.32-60-generic #122-Ubuntu SMP
It is possible that a function you call is changing the current working directory. There is no way to tell which one that could be without inspecting the code, but you can check if this is the case by displaying [pwd] before calling open.
Another way things could go wrong is by the script being started in different ways from Unix and Windows - e.g. by shortcut on Windows and from the command line off the PATH on Unix.
Where you have filenames passed in by the user and you can't be sure that code you call won't cd under your feet, you're advised to process the filenames into absolute filenames as soon as possible in your script, possibly even before any package require or source calls.
This is actually easy to do.
set absoluteFilename [file normalize $userProvidedFilename]
As long as this is done before the first cd (or chdir()/SetCurrentDirectory() at the C/C++ level) future changes of directory will not break the path. The script must be prepared to work with absolute filenames, but that's easily done.
You can turn a relative path into an absolute path with respect to any arbitrary directory with file join:
set filename [file join [file normalize $arbitraryLocation] $relativeFilename]
(The file join is smarter than just inserting a directory separator.)
Here is my problem,
In Windows I am making a zip file in which there is a text .sh file which is supposed to be executed in Linux.
The user on the other end opens the zip file in Linux and tries to execute the .sh file but the execute permission is gone. So the user has to do it manually ( like explained here:add execute permission.
How can I in Windows make the .sh executable and add it to a zip file so that when the zip file opens in linux the .sh file still retains its execute permission ( so that user doesn't have to do it manually)
As far as I know the permission system in Linux is set up in such a way to prevent exactly what you are trying to accomplish.
I think the best you can do is to give your Linux user a custom unzip one-liner to run on the prompt:
unzip zip_name.zip && chmod +x script_name.sh
If there are multiple scripts that you need to give execute permission to, write a grant_perms.sh as follows:
#!/bin/bash
# file: grant_perms.sh
chmod +x script_1.sh
chmod +x script_2.sh
...
chmod +x script_n.sh
(You can put the scripts all on one line for chmod, but I found separate lines easier to work with in vim and with shell script commands.)
And now your unzip one-liner becomes:
unzip zip_name.zip && source grant_perms.sh
Note that since you are using source to run grant_perms.sh, it doesn't need execute permission
The ZIP file format does allow to store the permission bits, but Windows programs normally ignore it.
The zip utility on Cygwin however does preserve the x bit, just like it does on Linux.
If you do not want to use Cygwin, you can take a source code and tweak it so that all *.sh files get the executable bit set.
Or write a script like explained here
This is possible using the Info-Zip open-source Zip utilities. If unzip is run with the -X parameter, it will attempt to preserve the original permissions. If the source filesystem was NTFS and the destination is a Unix one, it will attempt to translate from one to the other. I do not have a Windows system available right now to test the translation, so you will have to experiment with which group needs to be awarded execute permissions. It'll be something like "Users" or "Any user"
Use my windows command line utility zip_exec.zip to set the executable flag for linux/unix and mac (tested on files created with Windows Explorer and 7zip). The cpp source is also available. I searched the internet a lot before making my own utility. It can be modified to set any file attribute.
This is not possible. Linux permissions and windows permissions do not translate. They are machine specific. It would be a security hole to allow permissions to be set on files before they even arrive on the target system.