Is there a data: URI-like construct for paths on Linux? - linux

If you "open" an URI like data:text/html,<p>test</p>, the opened "file" contains <p>test</p>.
Is there a corresponding approach to apply this principle to Linux paths?
Example:
I want a path to a "virtual file" that "contains" example-data, ideally without actually creating this file.
So I'm basically looking for something you can replace some_special_path_results_in with in /some_very_special_path_results_in/example-data so that the opened "file" just "contains" example-data.

You can use process substitution in bash.
some_command <(printf '%s' '<p>test</p>')

I want a path to a "virtual file" that "contains" example-data, ideally without actually creating this file.
Maybe you should consider using tmpfs.
On Linux, creating a file is a very common and basic operation.
Why can't you create some "temporary" file? or some FUSE filesystem?
You could technically write your kernel module providing a new file system.
Be aware that files are mostly inode(7)-s (see stat(2)). They do have some meta data (but no MIME types). And any process can try to open(2) a file (sometimes, two processes are opening or accessing the same file). See path_resolution(7) and credentials(7).
Maybe you want pipe(7), fifo(7) or unix(7) sockets.
Read also Advanced Linux Programming, syscalls(2), a good textbook on operating systems, and see the KernelNewbies and Linux From Scratch and Linux BootPrompt websites
Technically Linux is open source: you are allowed to download, study and improve and recompile its source code. See this for the kernel code (it is free software), GNU libc, GCC, etc....
PS. Take into account legal software licensing considerations. Ask your lawyer to explain you the GPL licenses.

Related

Read folders as files

After doing some research and finding nothing, I wanted to ask:
Is it possible to read folders as files?
What i mean by this is, is it possible to read a folder as an object, like one can read a text file? (example below)
with open("sometext.txt", "r") as data:
print(data)
fo = open("sometext.txt", "w")
information = "somestring"
fo.write(information)
fo.close()
Thanks in advance for any help.
Yes, on some operating systems, but probably not in the way you intend. There is a library directory (disclaimer: I have never used this, I found it while researching your question) that binds the low-level system call opendir and readdir. These are C functions that interact with the Unix kernel to manipulate directories.
This is possible because in Unix, directories are a special type of file that contain pointers to the contents of that directory. This may be true of other operating systems, but I wouldn't count on there being existing tools for interacting with the low-level components of those systems with Python.
As the people who commented above have pointed out, there are very few (though not none) scenarios that would require you to get this close to the machinery of the operating system. It's likely that whatever you're trying to do can be done using either the os module or some other part of the standard library.

How to create a "fake filesystem" that forwards system calls to my program?

I would like to write a tool that can be used to mount archives such as tar, tgz, zip, 7z, etc. to some directory for as long as it's running, such that I can then open it with whatever file manager I want.
To do this, I would somehow need to make a fake filesystem that forwards system calls such as opening and reading files to my program. How would I did this? Would I have to make my own filesystem driver, or does a library for this already exist?
FUSE is what you're looking for, in principle. One implementation of the archive-mounting you're looking for is/was archive mount, but I am unsure how well it is maintained.
You want to write a FUSE filesystem. It is explained in this article and there is an example in python: https://thepythoncorner.com/dev/writing-a-fuse-filesystem-in-python/
No need to call this "fake filesystem", because for linux everything can be a file, so you write just a "filesystem".
Your question is very general. I will assume you want to write your program using Python because it is what I would do.
Chosen the language, now we need to look for examples of somebody doing something similar. Github would do the trick:
To mount wikipedia: wikifs
To mount Google Music: GmusicFS
N.B. Wikifs is least than experimental, but the script is short and servers perfectly as example.
For python the most popular library is fuse-python.
Happy hacking

File systems with support to directory hard-linking

Does anybody know one? preferrably with linux implementation?
alternatively, does anybody know how much effort would it take to add it in any open-source implementation? (i mean: maybe it's enough to change an if statement, maybe i have to go carefully trhough the whole fs implementation adding tests; do you have that notion? ).
thanks....
HFS+ allows directory hardlinks in OSX 10.5. Only TimeMachine can create them since OSX 10.6, and HFS+ does some sanity checking that they do not introduce cycles.
However, Linux will not read them. Besides filesystems, this could be enforced at the VFS layer. Even if there are no cycles, some userspace tools rely on having no directory hard links (eg, a GNU find optimisation that lets it skip many directories; it can be disabled with -noleaf ).
Technically nothing keeps you from opening /dev/sda with a hex editor and creating one. However everything else in your system will fall apart if you do.
The best explanation i could find is this quote from jta:
User-added hardlinks to directories
are forbidden because they break the
directed acyclic graph structure of
the filesystem (which is an ASSERT in
Unixiana, roughly), and because they
confuse the hell out of
file-tree-walkers (a term Multicians
will recognize at sight, but Unix
geeks can probably figure out without
problems too.

Pseudo filesystems on *nix

I need some opinions pointers on creating pseudo-filesystems for linux/*nix systems.
Firstly when I say pseudo-filesystem I mean something like /proc where the structure within does not represent actual files on disks or such but the state of the kernel. I would like to try something similar as an interface to an application.
As an example you could say, mount a ftp url to your filesystem and your browser app could then allow you to interact with the remote system doing ls et al on it and translating the standard filesystem requests into ftp ones.
So the first question is: how does one go about doing that? I have read a bit about it and it looks like you need to implement a new kernel module. If possible I would like to avoid that - my thinking being that someone may have already provided a tool for doing this sort of thing and provided the module to assist already.
My second question is: does anyone have a good list of examples of applications/services/whatever using this sort of technique to provide a filesystem based interface.
Lastly if anyone has any opinions on why this might be a good/bad idea to do such a thing on a generic level I would like to hear it.
A userspace filesystem via fuse would probably be your best way to go.
Regarding the next part of your question (which applications use this method), there is the window manager wmii, it uses the 9p filesystem via v9fs, which is a port of 9p to Linux. There are many examples on plan9, most notably acme. I suggested fuse because it seems more actively developed and mainstream in the Linux world, but plan9 is pretty much the reference for this approach as far as I know.

Licensing and using the Linux kernel [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 7 years ago.
Improve this question
I would like to write my own OS, and would like to temporarily jump over the complicated task of writing the kernel and come back to it later by using the Linux kernel in the mean time. However, I would like to provide the OS as closed source for now. What license is the Linux kernel under and is it possible to use it for release with a closed source OS?
Edit: I am not interested in closing the source of the Linux kernel, I would still provide that as open sourced. I am wondering if I could use a closed source OS with an open source kernel.
Further edit: By OS, I mean the system that runs on top of the kernel and is used to launch other programs. I certainly did not mean to include the kernel in the closed source statement.
You can of course write whatever closed-source OS over the Linux kernel that you like provided you are compatible with the licensing of components you link against.
Of course that's likely to include the gnu C library (or some other C library). You may also need some command line utilities which will probably be GPL to do things such as filesystem maintenance, network setup etc. But provided you leave those as their own standalone programs, it should not be a problem.
Anything that you link into the kernel itself (e.g. custom modules, patches) should be released as open source GPL to comply with the kernel's licence.
The Linux kernel is released under the GPLv2 and you can use it as part of a closed-source OS but you have to keep the kernel and all modifications released GPLv2.
Edit: Btw, you may want to use something like OpenSolaris instead. It's much easier to work with, in my opinion (obviously very subjective), and you can keep modifications closed-source if you so choose, so long as you follow the terms of the CDDL.
I think you're going to have to be more specific about what you mean by 'OS'. It's by no means a clear concept. Some would say that the kernel is all of the OS. Others would say that the shell and core utilities such as 'ls' are part of the OS. Others would go as far as to say that standard applications such as Notepad are part of the OS.
IANAL, but I don't believe there's anything to stop you from bundling the Linux kernel with a load of closed-source programs of your own. Take care not to use any GPL library code however (LGPL is OK).
I do question your motives.
It's GPL version 2 and you may certainly not close its source.
You must keep the source open, and any works derived from the code, however, if you use the Kernel, write your own application stack on top of that (pretty much ALL the GNU stuff) then you don't have to open that up.
The GPL says that "derived" works... so if you're writing new code, instead of expanind on, then that's fine. In fact, you could even, for example, use the GNU toolchain, the Linux Kernel, and then have your own system on top of that (or just a DE) that is closed source.
It's when you modify/derive from something that you have to keep it open!
Linux has the GPL (v2) as its licence, which means you have to open source any derivative works.
You may want to use BSD, its license is a lot les restrictive in what you can do with derived works
If the filesystem you use is to be linked into the kernel itself, and if you plan to distribute it to others, the GPL pretty unambiguously requires that the filesystem be GPL'ed as well.
That being said: one way to legally interface Linux with a GPL-incompatible filesystem is via FUSE (filesystem in userspace). This has been used, for example, to run the GPL-incompatible ZFS filesystem on top of Linux. Running a filesystem in userspace does, however, carry a performance penalty that may be significant.
It is GPL. Short answer -- no.
You can always keep any extensions (modules) and/or applications you write closed source, but the kernel itself will need to remain open source.
There's a not-so-obvious aspect of the GPLv2 that you can exploit while testing the system: you only need to release source code to those who have access to the system. The GPLv2 states that you need to give full access to the source code to anyone with access to the binary/compiled distribution of the program. So, if you are only going to use the software inside of the company that is paying to develop it, you don't need to distribute the source code to the rest of the world, but just them.
Generally I would say that you're allowed to do such a thing, as long as you provide the source for the kernel, but there's one point where I'm unsure:
On a normal Linux system between the (GPL) kernel and a non-GPL compatible application, there is always the GNU libc, which is LGPL and thus allows derived works that are non-free. Now, if you have a non-free libc, that might be considered a derived work, since you are directly calling the kernel, and also using kernel headers.
As many others have said before, you might be better off using a *BSD.
If you're serious in developing a new operating system and want a working kernel to start with I suggest that you look into the FreeBSD kernel. It has a much more relaxed license than Linux, I think you might find it worthwhile.
Just my 2 cents...
I agree with MarkR but nobody has stated the obvious to you. If you are serious, you need to consult a lawyer with expertise in this area.

Resources