I use scons cache functionality (--cache=nolinked --cache-dir=<path> options).
I want it to create hard links from the cache to the build tree. Is that supported by scons?
Not currently.
You can see the code here:
https://github.com/SCons/scons/blob/master/src/engine/SCons/CacheDir.py#L80
Is your cachedir on the same filesystem as your build area? If not, then hardlinks won't work.
What's the goal here? Reduced disk usage? Or something else?
Related
I'm trying to force a build internal pre-processor used for built-sources to not rely on shared libraries installed in my host machine without having to uninstall them.
Although there is a LD_PRELOAD environment variable which forces the loader (ld-linux) to fetch the specified shared libraries before anything else, I'd like to do quite the opposite, forcing the loader not to fetch the specified libraries during the setup process (kind of LD_NEVERLOAD variable).
Is there some way to do so without breaking my entire system (aka, removing such libraries)?
PS: I've renamed my system libraries to test this specific use case, but this is definitely not an elegant way of doing so.
Reading the manual pages ld(1) and ld.so(8) you might try playing with LD_LIBRARY_PATH, LD_RUNPATH and options in both manuals which are related to "rpath".
I am thinking of writing a system life saver application for ubuntu, which can restore system to an earlier state. This could be much useful in situations of system break.
User can create restore point before and then use them to restore their system.
This would be used for packages initially and then later on for restoring previous versions of files,somewhat like system restore functionality in microsoft windows.
Here is the idea page Idea page
I have gone through some ideas of implementing it like that which is done in windows, by keeping information about the files in the filesystem, the filesystem is intelligent enough to be used for this feature. But we don't have such file system available in linux, one such file system is brtfs but using this will lead to users creating partitions, which will be cumbersome. So I am thinking of a "copy-on-write and save-on-delete" approach. When a restore point is created I will create a new directory for backup like "backup#1" in the restore folder created by application earlier and then create hard links for the files needed to be restored. Now if any file is deleted from its original location I would have its hard link with me which can be used to restore the file, when needed. But this approach doesn't work on modification. For modification I am thinking of creating hooks in the file system (using redirfs ) which will call my attached callbacks which will check for the modifications in various parts of the files. I will keep these all changes in the database and then reverse the changes as soon as a restore is needed.
Please suggest me some efficient approaches for doing this.
Thanks
Like the comments suggested, the LVM snapshot ability provides a good basis for such an undertaking. It would work on a per-partition level and saves only sectors changed in comparison with the current system state. The LVM howto gives a good overview.
You'll have to set up the system from the very start with LVM, though, and leave sufficient space for snapshots.
Gentoo has a feature in portage, that prevents and logs writes outside of the build and packaging directories.
Checkinstall is able to monitor writes, and package up all the generated files after completion.
Autotools have the DESTDIR macro that enables you to usually direct most of the filesystem activity to an alternate location.
How can I do this myself with the
safety of the Gentoo sandboxing
method?
Can I use SELinux, rlimit, or
some other resource limiting API?
What APIs are available do this from
C, Python?
Update0
The mechanism used will not require root privileges or any involved/persistent system modification. This rules out creating users and using chroot().
Please link to the documentation for APIs that you mention, for some reason they're exceptionally difficult to find.
Update1
This is to prevent accidents. I'm not worried about malicious code, only the poorly written variety.
The way Debian handles this sort of problem is to not run the installation code as root in the first place. Package build scripts are run as a normal user, and install scripts are run using fakeroot - this LD_PRELOAD library redirects permission-checking calls to make it look like the installer is actually running as root, so the resulting file ownership and permissions are right (ie, if you run /usr/bin/install from within the fakeroot environment, further stats from within the environment show proper root ownership), but in fact the installer is run as an ordinary user.
Builds are also, in some cases (primarily for development), done in chroots using eg pbuilder - this is likely easier on a binary distribution however, as each build using pbuilder reinstalls all dependencies beyond the base system, acting as a test that all necessary dependencies are specified (this is the primary reason for using a chroot; not for protection against accidental installs)
One approach is to virtualize a process, similar to how wine does it, and reinterpret file paths. That's rather heavy duty to implement though.
A more elegant approach is to use the chroot() system call which sets a subtree of the filesystem as a process's root directory. Create a virtual subtree, including /bin, /tmp, /usr, /etc as you want the process to see them, call chroot with the virtual tree, then exec the target executable. I can't recall if it is possible to have symbolic links within the tree reference files outside, but I don't think so. But certainly everything needed could be copied into the sandbox, and then when it is done, check for changes against the originals.
Maybe get the sandbox safety with regular user permissions? So the process running the show has specific access to specific directories.
chroot would be an option but I can't figure out how to track these tries to write outside the root.
Another idea would be along the lines of intercepting system calls. I don't know much about this but strace is a start, try running a program through it and check if you see something you like.
edit:
is using kernel modules an option? because you could replace the write system call with your own so you could prevent whatever you needed and also log it.
It sounds a bit like what you are describing is containers. Once you've got the container infrastructure set up, it's pretty cheap to create containers, and they're quite secure.
There are two methods to do this. One is to use LD_PRELOAD to hook library calls that result in syscalls, such as those in libc, and call dlsym/dlopen. This will not allow you to directly hook syscalls.
The second method, which allows hooking syscalls, is to run your executable under ptrace, which provides options to stop and examine syscalls when they occur. This can be set up programmatically to sandbox calls to restricted areas of the filesystem, among other things.
LD_PRELOAD can not intercept syscalls, but only libcalls?
Dynamic linker tricks: Using LD_PRELOAD to cheat, inject features and investigate programs
Write Yourself an Strace in 70 Lines of Code
I'm currently designing a Linux-based system. Users of the system will be allowed to download contents, i.e. programs, from the Internet. The contents will be distributed in zip packages given special extension names, e.g. .cpk instead of .zip, and with zero compression.
I want to give users the same experience found in iOS and Android, in which contents are distributed in contained packages and run from there.
My question is that can I make my Linux system to run programs from inside the packages without unzipping them? If not, then is there another approach to what I'm after in Linux?
Please note that I don't want to extract contents into a temp folder and delete them after execution because that might take longtime, specially for large contents. That will also double the storage space requirements for running the contents.
Thank you in advance.
klik (at least in the klik2 CMG format) used an zISO image, which can be mounted by the kernel or by a FUSE client, rather than a zip. You could use other filesystem types that are supported by the kernel or via FUSE. Maybe fuse-zip is worth a shot?
You could also modify the loader to read directly out of the bundle. For example, Android's Dalvik VM can load dex files directly from apk bundles, which are effectively zip files. (Native code on Android, however, still needs to be unpacked first, and does take more time and space. Modifying the native loader is… tricky.)
I have one doubt. I am doing a project related to system restore concept in Linux. There i am planning to perform application wise rollback in case of failure. Is there any way to figure out what are all the files used by an application in the system?
Ok. I will make it a little clear. For instance consider the firefox application. When it is installed many files are written from the .deb file to folders like /etc, /usr, /opt etc. In windows all the files are installed in one folder under program files while in linux its not. So is there any way to figure out the files that belong to a software?
Thanks.
Well this can cover several things.
If you mean, which files are provided by the installation of your application ? Then the answer is, use decent package management, provide your software as an rpm/deb/... whatever package, and unstallation will take care of the rest.
If you mean, which libraries are being referenced by our application ? Then you can use ldd this will tell your which dynamic libraries are used when executing this application.
If you mean, which files is my application actively using ? Then take a look at the output of lsof (lsof = list open files) (or alternatively ls /proc//fd/), this will show all file descriptors open by your application (files, sockets, pipes, tty's, ...)
Or you could use all of the above.
One thing you can't track (unless you log this yourself) is which files have been created by your application during its lifetime.
To determine all the files installed along with the app depends on the package manager. All the ones I've dealt with (apt, pacman) have had this capability.
To determine all the files currently open by an application, use lsof.
Well, that depends ...
Most Linux system have some kind of packet management software, like aptitude in debian and ubuntu. There, you have information about what belongs to a packet. You might be able to use that information. That does not cover files created during runtime of apps though.
If you are using an RPM based distro
# rpm -Uvh --repackage pkg-1-1.i386.rpm
will repackage the old files and upgrade in a transaction so you can later rollback if something went wrong. To rollback to yesterday's state for example
# rpm -Uvh --rollback yesterday
See this article for other examples.