os.Mkdir and os.MkdirAll permissions - linux

I'm trying to create a log file at the start of my program.
I need to check if a /log directory exists if it doesn't create the directory then move on to creating the log file.
Well I tried to use os.Mkdir (as well as os.MkdirAll), but no matter what value I put into the second parameter I get a locked out folder with no permissions. What value should this be in order to get a read / write for user folder? I thought it would be 0x700 but it doesn't seem to work.
Thanks!

You can use octal notation directly:
os.Mkdir("dirname", 0700)
Permission Bits
+-----+---+--------------------------+
| rwx | 7 | Read, write and execute |
| rw- | 6 | Read, write |
| r-x | 5 | Read, and execute |
| r-- | 4 | Read, |
| -wx | 3 | Write and execute |
| -w- | 2 | Write |
| --x | 1 | Execute |
| --- | 0 | no permissions |
+------------------------------------+
+------------+------+-------+
| Permission | Octal| Field |
+------------+------+-------+
| rwx------ | 0700 | User |
| ---rwx--- | 0070 | Group |
| ------rwx | 0007 | Other |
+------------+------+-------+
A Unix Permission Primer
Common Permission Usages
0755 Commonly used on web servers. The owner can read, write, execute. Everyone else can read and execute but not modify the file.
0777 Everyone can read write and execute. On a web server, it is not advisable to use ‘777’ permission for your files and folders, as it allows anyone to add malicious code to your server.
0644 Only the owner can read and write. Everyone else can only read. No one can execute the file.
0655 Only the owner can read and write, but not execute the file. Everyone else can read and execute, but cannot modify the file.
www.maketecheasier.com/file-permissions-what-does-chmod-777-means/
Directory Permissions on Linux
When applying permissions to directories on Linux, the permission bits have different meanings than on regular files. (source)
Read bit The user can read the file names contained in the directory.
Write bit The user can {add,rename,delete} files names IF the execute bit is set too.
Execute bit The user can enter the directory and access the files inside.
https://unix.stackexchange.com/a/21252
Permissions Calculator
A handy permissions calculator.

#Daniel's statement in his answer is not really correct, and also it talks about a decimal number and then uses an octal one, as #SashaCrofter correctly pointed out in his comment.
In reality, it doesn't matter what form your permission value is in as long as it represents sensible Unix permissions.
Since permission bits on POSIX file systems come in triples of bits — three bits for owner, group and others access, plus three bits of modifiers (such as sticky bits), — it's customary to use octal numbers to represent permissions as each digit in an octal number represents a three-bit value.
Hence, when you use 0700 in Go code, the leading 0 is stripped and is only there to tell the parser it sees an octal number literal, and the following three letters stand for the owner, group and others permissions, in this order. Should you, say, want to also set the group sticky bit as well as making the file system object group-readable and executable, you'd specify 02750 and so on.
Note that the actual permissions the file system object acquires is further modulated by the active umask of the process which creates the object.
To get more grip on these topics, it's best to read the chmod manual pages and general literature on Unix-like operating systems.

You can reset the umask to 0. I would call this as the first thing in my main file
syscall.Umask(0)
Example
_ = os.MkdirAll("/tmp/dirs/1", 0664)
syscall.Umask(0)
_ = os.MkdirAll("/tmp/dirs/2", 0664)
Result
/tmp/dirs$ stat -c '%A %a %n' *
drw-r--r-- 644 1
drw-rw-r-- 664 2

Besides the other answers, remember that on Unix and Linux style operating systems, all programs run with a umask setting. The umask, which in many cases defaults to 022 or sometimes 002, is the set of permissions that the system will automatically remove from file and directory creation requests.
What this means is that most programs–there are several exceptions to this rule—should use mode 0666 for creating files and mode 0777 for creating directories. The user's configuration, recorded in the running process, says which of these permissions to take away. If the user's setting is 022, and we create a file with mode 0666, the actual setting we get is rw-r--r--: read and write for the user, read-only for the group, and read-only for others.
If a user wishes to extend writability to their group, they need only set their umask to 2: now they take away write permission for others, but leave it for their group. New files are now created with mode rw-rw-r--. The program does not change: it still uses 0666 for its mode. But the files are created with mode 0664.
Similarly, if you call os.Mkdir or os.MkdirAll with 0777, the umask will take away the unwanted permissions, leaving you with the right permissions.
But I mentioned that there are exceptions. These include programs that make copies of sensitive information meant only for the user: these should generally use mode 0700 for directories and 0600 for files. They may include long-running servers that act as a system user rather than any one individual ... although those servers could be run with a correct umask, in which case, 0777 or 0666 is fine.
You must apply some judgment here. Programs that are especially security-conscious, such as ssh or similar, may wish to use limited permissions, and may even want to check (with os.Lstat or similar) that permissions are appropriately tight on important directories.
(Note that the umask does not apply to os.Chmod calls. Here you choose the mode directly.)

One way to make sure that you're setting the kind of permissions you want, without figuring out the complex calculations in octal, is to use the very convenient FileMode constants in package os:
https://golang.org/pkg/os/#FileMode
I usually use os.ModePerm (which is actually coded as 0777) for fully permissive directories, such as those required for caches or temporary files, but your mileage may vary. To set the additional bits (sticky, etc.), which, as #kostix has noted, has to deal with the issue of octal representation of flags in Go, you can always use something like:
if err := os.MkdirAll("my/tmp/dir", os.ModeSticky|os.ModePerm); err != nil {
... handle error ...
}
Go playground
As always, it's worth mentioning again that these permissions are 'filtered' by whatever umask has been set.

Related

How to buil an app like google pdf viewer? [duplicate]

So the idea is to make an encryption software which will work only on .txt files and apply some encryption functions on it and generate a new file. To avoid the hassle of user having to drag-and-drop the file, I have decided to make an option similar to my anti-virus here.
I want to learn how to make these for various OS, irrespective of the architecture :)
What are these menus called? I mean the proper name so next time I can refer to them in a more articulate way
How to make these?
My initial understanding:
What I think it will do is: pass the file as an argument to the main() method and then leave the rest of the processing to me :)
Probably not exactly the answer you were hoping for, but it seems that this is a rather complicated matter. Anyway, I'll share what I know about it and it will hopefully prove enough to (at least) get you started.
Unfortunately, the easiest way to create a context menu using Java is editing the Registry. I'll try to summarize the milestones of the overall requirements and steps to achieve our objective.
<UPDATE>
See at the end of the post for links to sample code and a working demo.
</UPDATE>
What needs to be done
We need to edit the Registry adding an additional entry (for our java-app) in the context menus of the file-types we are interested in (e.g. .txt, .doc, .docx).
We need to determine which entries in Registry to edit, because our targeted file-extensions might be associated with another 'Class' (I couldn't test it on XP, but on Windows 7/8 this seems to be the case). E.g. instead of editing ...\Classes\.txt we might need to edit ...\Classes\txtfile, which the .txt Class is associated with.
We need to specify the path to the installed jre (unless we can be sure that the directory containing javaw.exe is in the PATH variable).
We need to insert the proper keys, values and data under the proper Registry nodes.
We need a java-app packaged as a .JAR file, with a main method expecting a String array containing one value that corresponds to the path of the file we need to process (well, that's the easy part - just stating the obvious).
All this is easier said than done (or is it the other way around ?), so let's see what it takes to get each one done.
First of all, there are some assumption we'll be making for the rest of this post (for the sake of simplicity/clarity/brevity and the like).
Assumptions
We assume that the target file-category is .TXT files - the same steps could be applied for every file-category.
If we want the changes (i.e. context-menus) to affect all users, we need to edit Registry keys under HKCR\ (e.g. HKCR\txtfile), which requires administrative priviledges.
For the sake of simplicity, we assume that only current user's settings need to be changed, thus we will have to edit keys under HKCU\Software\Classes (e.g. HKCU\Software\Classes\txtfile), which does not require administrative priviledges.
If one chooses to go for system-wide changes, the following modifications are necessary:
In all REG ADD/DELETE commands, replace HKCU\Software\Classes\... with HKCR\... (do not replace it in REG QUERY commands).
Have your application run with administrative priviledges. Two options here (that I am aware of):
Elevate your running instance's priviledges (can be more complicated with latest windows versions, due to UAC). There are plenty of resources online and here in SO; this one seems promising (but I haven't tested it myself).
Ask the user to explicitely run your app "As administrator" (using right-click -> "Run as administrator" etc).
We assume that only simple context-menu entries are needed (as opposed to a context-submenu with more entries).
After some (rather shallow) research, I have come to believe that adding a submenu in older versions of Windows (XP, Vista), would require more complex stuff (ContextMenuHandlers etc). Adding a submenu in Windows 7 or newer is considerably more easy. I described the process in the relevant part of this answer (working demo provided ;)).
That said, let's move on to...
Getting things done
You can achieve editing the Registry by issuing commands of the form REG Operation [Parameter List], with operations involving ADD, DELETE, QUERY (more on that later).
In order to execute the necessary commands, we can use a ProcessBuilder instance. E.g.
String[] cmd = {"REG", "QUERY", "HKCR\\.txt", "/ve"};
new ProcessBuilder(cmd).start();
// Executes: REG QUERY HKCR\.txt /ve
Of course, we will probably want to capture and further process the command's return value, which can be done via the respective Process' getInputStream() method. But that falls into scope "implementation details"...
"Normally" we would have to edit the .txt file-class, unless it is associated with another file-class. We can test this, using the following command:
// This checks the "Default" value of key 'HKCR\.txt'
REG QUERY HKCR\.txt /ve
// Possible output:
(Default) REG_SZ txtfile
All we need, is parse the above output and find out, if the default value is empty or contains a class name. In this example we can see the associated class is txtfile, so we need to edit node HKCU\Software\Classes\txtfile.
Specifying the jre path (more precisely the path to javaw.exe) falls outside the scope of this answer, but there should be plenty of ways to do it (I don't know of one I would 100% trust though).
I'll just list a few off the top of my head:
Looking for environment-variable 'JAVA_HOME' (System.getenv("java.home");).
Looking in the Registry for a value like HKLM\Software\JavaSoft\Java Runtime Environment\<CurrentVersion>\JavaHome.
Looking in predifined locations (e.g. C:\Program Files[ (x86)]\Java\).
Prompting the user to point it out in a JFileChooser (not very good for the non-experienced user).
Using a program like Launch4J to wrap your .JAR into a .EXE (which eliminates the need of determining the path to 'javaw.exe' yourself).
Latest versions of Java (1.7+ ?) put a copy of javaw.exe (and other utilities) on the path, so it might be worth checking that as well.
3. So, after collecting all necessary data, comes the main part: Inserting the required values into Registry. After compliting this step, our HKCU\Software\Classes\txtfile-node should look like this:
HKCU
|_____Software
|_____Classes
|_____txtfile
|_____Shell
|_____MyCoolContextMenu: [Default] -> [Display name for my menu-entry]
|_____Command: [Default] -> [<MY_COMMAND>]*
*: in this context, a '%1' denotes the file that was right-clicked.
Based on how you addressed step (1.2), the command could look like this:
"C:\Path\To\javaw.exe" -jar "C:\Path\To\YourApp.jar" "%1"
Note that javaw.exe is usually in ...\jre\bin\ (but not always only there - recently I've been finding it in C:\Windows\System32\ as well).
Still being in step (1.3), the commands we need to execute, in order to achieve the above structure, look as follows:
REG ADD HKCU\Software\Classes\txtfile\Shell\MyCoolContextMenu /ve /t REG_SZ /d "Click for pure coolness" /f
REG ADD HKCU\Software\Classes\txtfile\Shell\MyCoolContextMenu\Command /ve /t REG_SZ /d "\"C:\Path\To\javaw.exe\" -jar \"C:\Path\To\Demo.jar\" \"%%1\" /f"
// Short explanation:
REG ADD <Path\To\Key> /ve /t REG_SZ /d "<MY_COMMAND>" /f
\_____/ \___________/ \_/ \_______/ \_______________/ \_/
__________|_______ | | |___ | |
|Edit the Registry | | _______|________ | _______|_______ |
|adding a key/value| | |Create a no-name| | |Set the data | |
-------------------- | |(default) value | | |for this value.| |
| ------------------ | |Here: a command| |
_______________|______________ | |to be executed.| |
|Edit this key | | ----------------- |
|(creates the key plus | ____|_________ _________|_____
| any missing parent key-nodes)| |of type REG_SZ| |No confirmation|
-------------------------------- |(string) | -----------------
----------------
Implementation Considerations:
It is probably a good idea to check if our target class (e.g. txtfile), does already have a context-menu entry named "MyCoolContextMenu", or else we might be overriding an existing entry (which will not make our user very happy).
Since the data part of the value (the part that comes after /d and before /f) needs to be enclosed in "", keep in mind that you can escape " inside the string as \".
You also need to escape the %1 so that it is stored in the Registry value as-is (escape it like: %%1).
It is a good idea to provide your user with an option to "un-register" your context-menu entry.
The un-registering can be achieved by means of the command:
REG DELETE HKCU\Software\Classes\txtfile\Shell\MyCoolContextMenu /f
Omitting the /f at the end of the commands may prompt the "user" (in this case your app) for confirmation, in which case you need to use the Process' getOutputStream() method to output "Yes" in order for the operation to be completed.
We can avoid that unnecessary interaction, using the force flag (/f).
Almost, there !
Finding ourselves at step (2), we should by now have the following:
A context-menu entry registered for our files in category txtfile (note that it is not restricted to .TXT files, but applies to all files pertained by the system as "txtfiles").
Upon clicking that entry, our java-app should be run and its main() method passed a String array containing the path to the right-clicked .TXT file.
From there, our app can take over and do its magic :)
That's (almost) all, folks !
Sorry, for the long post. I hope it turns out to be of use to someone.
I'll try to add some demo-code soon (no promises though ;)).
UPDATE
The demo is ready !
I created a tiny demo-project.
Here is the source code.
Here is a ready-to-go JARred App.

file command generating 'invalid argument'

I have a perl script that traverses a set of directories and when it hits one of them it blows up with an Invalid Argument and I want to be able to programmatically skip it. I thought I could start by finding out the file type with the file command but it too blows up like this:
$ file /sys/devices/virtual/net/br-ex/speed
/sys/devices/virtual/net/br-ex/speed: ERROR: cannot read `/sys/devices/virtual/net/br-ex/speed' (Invalid argument)
If I print out the mode of the file with the perl or python stat function it tells me 33060 but I'm not sure what all the bits mean and I'm hoping a particular one would tell me not to try to look inside. Any suggestions?
To understand the stats number you got, you need to convert the number to octal (in python oct(...)).
Then you'll see that 33060 interprets to 100444. You're interested only in the last three digits (444). The first digit is file owner permissions, the second is group and the third is everyone else.
You can look at each of the numbers (in your case all are 4) as 3 binary bits in this order:
read-write-execute.
Since in your case owner, group & other has 4, it is translated (for all of them) to 100 (in binary) which means that only the read bit is on for all three - meaning that all three can only read the file.
As far as file permissions go, you should have been successful reading /sys/devices/virtual/net/br-ex/speed.
There are two reasons for the read to fail:
- Either speed is a directory, (directories require execute permissions to read inside).
- Or it's a special file - which can be tested using the -f flag in perl or bash, or using os.path.isfile(...) in python.
Anyhow, you can use the following links to filter files & directories according to their permissions in the 3 languages you mentioned:
ways to test permissions in perl.
ways to test permissions in python.
ways to test permissions in bash.
Not related to this particular case, but I hit the same error when I ran it on a malicious ELF (Linux executable) file. In that case it was because the program headers of the ELF was intentionally corrupted. Looking at the source code for file command, this is clear as it checks the ELF headers and bails out with the same error in case the headers are corrupted:
/*
* Loop through all the program headers.
*/
for ( ; num; num--) {
if (pread(fd, xph_addr, xph_sizeof, off) <
CAST(ssize_t, xph_sizeof)) {
file_badread(ms);
return -1;
}
TLDR; The file command checks not only the magic bytes, but it also performs other checks to validate a file type.

How many combinations can chmod in linux have?

I know there's 0,1,5,6 and 7 for each category of user
e.g. 755, 644, 600 etc
how many combinations can we have?
also, there's this u+755... what is this about really?
There are 4 user-manipulable permissions locations (special permissions [setuid, setgid, sticky], user owner, group owner, other), and each can have one of 8 values.
8 ** 4 = 4096
The 3 numbers denote owner/group/world
Each of them has read/write/execute bits. You are setting these when it comes to the 3 numbers.
So, 000 becomes
owner group world
rwx rwx rwx
000 000 000
777 (which allows everyone to read, write and execute the file) becomes
owner group world
rwx rwx rwx
111 111 111
644 allows owner to read/write. Group and world to only read.
owner group world
rwx rwx rwx
110 100 100
And so on..
In total, there are 8 bits, which can be turned on/off, giving you 8^3. In addition, there are the special modes Sticky bit, SUID and SGID, and their various combinations, further giving you 8 possibilities.
the unix privileges on directories is categorised by [User][Group][Other] each of these have the options of [Read][Write][Execute]
so the form is [User]{RWX},[Group]{RWX},[Other]{RWX}
so the permissions are [U]RWX [G]RWX [O]RWX so if you are familiar with the binary representation of decimal numbers you can understand that
755 would be decoded to [U]111 [G]101 [O]101 so this means [U]RWX [G]R-X [O]R-X
600 would be decoded to [U]110 [G]000 [O]000 so this means [U]RW- [G]--- [O]---
644 would be decoded to [U]110 [G]100 [O]100 so this means [U]RW- [G]R-- [O]R--
and about the available combinations are 2^9= 512 available permissions
where 2 is the number of available choices (0,1) that could be placed in one of the 9 available places
[U]123 [G]456 [O]789 in the general form.
you can check this link for further details and tutorial.
Every file and directory has their certial set of permissions on User (who owns the files), Group (Group is to which that user belongs and all the users belongs to this group will fall in this category) and Others ( rest of the users and group present inside the system).
Now each of these categories (User/Group/Others) can have combination of Read, Write or Execute permissions.
Read - 4
Write -2
Execute - 1
So if a file has [read,write, execute] permissions for User and [read,execute] permission for Group and Others both then it is denoted as
chmod 755 filename
also, chmod u+rwx, g+rx, o+rx filename
Now if you just want to give USER (the first category) permissions on the file as [read,write] then it will be like :-
chmod u+rw filename.
I don't think the command (chmod u+755 filename) you have written is correct.

is there any POSIX way through fstat() to check whether a file is a symbolic link or not?

Is there any POSIX way through fstat(2) to check whether a file is a symbolic link or not?
There is flag O_NOFOLLOW in open(2) which can check it, however, it's not POSIX.
There is S_ISLNK in fstat(2), which is said in man fstat:
The S_ISLNK() and S_ISSOCK() macros are not in POSIX.1-1996,
but both are present in POSIX.1-2001; the former is from SVID
4, the latter from SUSv2.
and the compile will fail on my machine.
Also, there is another S_IFLNK in lstat(2), however, it won't work with fstat(2) (which will follow the link to the file referred to).
No.
It's not true that fstat follows symlinks. Instead, open follows symlinks. Once you get to fstat, it is too late and the information is gone.
Tell us why you need to know, and we can help with that problem. (Open another question.)
How files work:
Here is some pseudo-C / shell code:
system("echo 'Hello, World!' >A.txt");
system("ln A.txt B.txt");
system("ln -s A.txt C.txt");
fdes = open("C.txt");
unlink("A.txt");
unlink("C.txt");
data = read(fdes);
write(stdout, data);
The result: your program prints "Hello, world!". The state of the world looks like this:
+--Application--+ +--Kernel--+ +-------Disk-------+
| | | | | |
| fdes --------------> file ---------> inode #973 <-------+
| | | | | "Hello World!" | |
+---------------+ +----------+ | | |
| directory ---------+
| "B.txt" |
| |
+------------------+
As far as the kernel is concerned, the file open is "inode #973". The data structure in kernel memory has some additional information such as the current position, but it does NOT know the path. The kernel is not expected to know that information.
If you asked the kernel what the path is, it could say "you have B.txt" open. But you never opened "B.txt", you opened "C.txt" which was a symlink to "A.txt", and both "A.txt" and "C.txt" have been deleted (they were just names to begin with).
Simple analogy:
You get a phone call from an old friend. He asks, "Did I look up your number in the phone directory, did I memorize it, or did I have to ask someone for your number?"
You have no way of knowing the answer. All you know is who is on the other end of the line. Just like an open file doesn't store information about what it was named (hard link or symbolic link), it just has information about permissions and data.
The solution: Just use lstat (yes, there is a race condition). If you didn't open the file yourself (e.g., you get it from a parent process or you get it over a socket), then it's more or less impossible to know if it was opened through a symbolic link.

shared memory (ipc)in linux

Shared memory means one process will create a memory portion oher process can acess.
My question is: how will the other process know the created shared memory shmid (i.e shmid=shmget(key-t ,size , permission);)?
To simplify: The process that creates the shared memory defines the name and permissions (process of the logged in user).
Basically you create a file:
file_descriptor = shm_open("/some.shared.memory",
(O_CREAT | O_RDWR),
(S_IREAD | S_IWRITE))
Or if it exists you can open it:
file_descriptor = shm_open("/some.shared.memory",
(O_CREAT | O_EXCL | O_RDWR),
(S_IREAD | S_IWRITE))
Thus, the name "/some.shared.memory" is your way to find the shared segment. (Note the flags)
Have a look here http://www.kernel.org/doc/man-pages/online/pages/man7/shm_overview.7.html
Do a Google search on "shm_open example" and you'll find lots of examples.
The process that creates the file will determine the permissions (owner+group) See the chown command and fchmod functions which you can call to set the permissions. Also have a look at umask.
They need to use the same way to derive the key_t; usually you use the ftok function where both sides use the same file name and id value, so they end up with the same key.

Resources