We're trying to use oprofile to track down performance problems on a server cluster. However, the servers in question have a read-only file system, where /var/tmp is the only writeable directory.
OProfile wants to create two directories whenever it runs: /root/.oprofile and /var/lib/oprofile, but it can't, because the filesystem is read-only. I can use the --session-dir command line option to make it write its logs to elsewhere than /var/lib, but I can't find any such option to make it use some other directory than /root/.oprofile.
The filesystem is read-only because it is on nonwriteable media, not because of permissions -- ie, not even superuser can write to those directories. We can cook a new ROM image of the filesystem (which is how we installed oprofile, obviously), but there is no way for a runtime program to write to /root, whether it is superuser or not.
I tried creating a symlink in the ROM that points /root/.oprofile -> /var/tmp/oprofile, but apparently oprofile doesn't see this symlink as a directory, and fails when run:
redacted#redacted:~$ sudo opcontrol --no-vmlinux --start --session-dir=/var/tmp/oprofile/foo
mkdir: cannot create directory `/root/.oprofile': File exists
Couldn't mkdir -p /root/.oprofile
We must run our profilers on this particular system, because the performance issues we're trying to investigate don't manifest if we build and run the app on a development server. We can't just run our tests on a programmer's workstation and profile the app there, because the problem doesn't happen there.
Is there some way to configure oprofile so that it doesn't use /root ?
I guess it should be as simple as overriding the HOME environment variable:
HOME=/tmp/fakehome sudo -E opcontrol --no-vmlinux --start --session-dir=/var/tmp/oprofile/foo
If that doesn't work out, you could have a look at
unionfs
aufs
to create a writable overlay. You might even just mount tmpfs on /root,or something simple like that.
It turns out that this directory is hardcoded into the opcontrol bash script:
# location for daemon setup information
SETUP_DIR="/root/.oprofile"
SETUP_FILE="$SETUP_DIR/daemonrc"
Editing those lines seemed to get it working, more or less.
Related
Can not login to system after start server:
openpam_check_desc_owner_perms() : /etc/pam.d/login insecure perms
Started system in single-user mod. There are all system dirs (/boot , /lib ...)
nobody:nogroup
ownership.
Ofc, i cannot just chown system dirs. So, how can i restore ownership to root?
Firstly, you should use a live boot downloaded from official repository. I don't think rescue mode (in this case) can save you.
In live the live environment, just mount your impacted filesystems somewhere:
mount /dev/${your_partition} /mnt
Now, you can use mtree to set good rights on all default files and directory. This command will re-create your tree with good rights. Before executing it, you can run it to check state of your filesystem.
# check before act
mtree -f /etc/mtree/BSD.root.dist -p /mnt
# you can now apply change
mtree -u -f /etc/mtree/BSD.root.dist -p /mnt
You can find more mtree files in /etc/mtree:
BSD.debug.dist
BSD.include.dist
BSD.sendmail.dist
BSD.usr.dist
BSD.var.dist
After doing that, you can now run mergemaster. mergemaster will warn when some files aren't well configured with good rights:
mergemaster -iD /mnt
If you have still issue, you can download or fetch FreeBSD source from SVN, extract them, and reinstall your configuration files manually (do a backup of your configuration before doing that and please read official FreeBSD documentation about using source).
cd /usr/src
svnlite https://svnweb.freebsd.org/base/releng/${your_freebsd_release} .
cd /usr/src/etc
make install DESTDIR=/mnt
You can now reboot your computer or server without live CD.
I have a dynamic linker which is /lib64/libc.so.6
I stupidly renamed it to /lib64/libc.so.6.old and now NO commands work.
I cannot do ls or mv to rename it back.
I can run ldconfig but it says permission denied and I cannot run sudo or su - What on earth can I do to fix this? I am running Oracle Linux redhat 6.7
LD_PRELOAD=/lib64/libc.so.6.old mv /lib64/libc.so.6.old /lib64/libc.so.6
Start from a recovery/install iso and rename the file back.
If you can't reboot or don't have physical access to the machine you could try to install a compiled version of BusyBox https://busybox.net/FAQ.html#getting_started and use its su and mv commands. Since BusyBox is statically linked it should work without libc.so.
Go to single user mode, mount the file system with rw, since you know the location of the renamed file move /lib64/libc.so.6.old /lib64/libc.so.6
I would also propose a workaround with a mount point as already mentioned by #wildplasser.
You can make majority of command line tools working again if you have a mounted directory on your broken host. If you are lucky to have one then all you need is to upload the libc-x.yz.so (which you can take from another host of from Internet) on the share, rename it there to libc.so.6 and add the mounted directory to LD_LIBRARY_PATH.
If the version x.yz is the same as for the one which you thoughtlessly moved then the commands like ls, cp etc. will work again in the console where you set LD_LIBRARY_PATH. You should not logout from this console, because you won't be able to login again.
! Be aware that setuid command line tools won't work (see https://askubuntu.com/a/1029363/832810). Unfortunately "sudo" is one of them, this is why you won't be able to put back easily your long-suffering .so (unless you have a root# console). However it gives you a possibility to save all data and finish all actions before you do some hard restore.
If you managed to do the above-mentioned trick and you have enough time you can try to build a statically linked version of "sudo" as suggested on https://askubuntu.com/a/1030475/832810 (probably even build on another host and copy through NFS) and move the .so back using it.
I have an Ubuntu 14.04 LTS server running a few different programs under Supervisor. Many of the programs need to store sockets and other named pipes on the filesystem, and /run seems like the ideal choice for these types of files. Unfortunately, /run is tmpfs and removed on every reboot, and root privileges are needed to (re)create the directories that each program can write to.
I need a way to create a few subdirectories in /run and set the owner/mode to something that each program can work with, and do so on each reboot before Supervisor tries to start them. It does not look like Supervisor supports a mechanism to run pre-start commands before it starts a program.
Most other answers for this type of question suggest doing it in the init script, but that belongs to Supervisor's package and I do not want to mess with it (or have to maintain it when it changes upstream).
If this machine had Systemd it seems like I could use /etc/tmpfiles.d, but it does not.
The best idea I came up with was to use a separate Upstart pre-start script for each program that only creates the directories without actually launching any processes. Something like:
/etc/init/myapp1.conf
start on runlevel [2345]
pre-start script
mkdir -p -m 0755 /var/run/myapp1
chown app1user: /var/run/myapp1
end script
...without any exec line. I'm not 100% sure this is valid or sane, but it appears to work. Are there cleaner ways to do something like this?
Do you run your apps under supervisor under a specific user? Because by default applications are run with root as owner.
What I would do is a simple script which does the following:
Checks if the required files/folders are created.
Sets the owner if necessary.
Then starts your application
Put this script into your supervisor config instead of directly starting your application. Make sure that it is run with root (remove user from the config or set user=root).
This way you can always make sure that your environment is set up and your directories exist. So if you clear the tempfs for some reasons, your scripts will still run without reboot.
If you NEED to run your applications under a specific user, you can do the following:
Move the first 2 points into a separate setup script (as you would do now using your solution).
Create another script which calls your setup script with sudo and starts your application
Add your custom user and script to the sudo file so that your user can call that script as root without a password prompt. (Be aware: this is a security risk, if someone gets access to your server. Make sure that your setup script is NOT writable)
If I start a new distro (e.g. LFS):
How can I change the directory structure?
What should I expect after it's ready? (probably can't install most of the packages without modification, right?
But, before you down vote: I've been asked to make a new distro for a specific project which they need (actually, want) a new directory structure with a few changes, for example remove the var and bin directories, but without halting the system. The application of this distro is so limited, so i think it shouldn't be a big deal as they need just a few packages to be installed.
These are few pointers that come to my mind and definitely it is not complete:
Your PATH should be updated in the startup scripts like ~/.bashrc, /etc/profile.d, and so on to reflect the updated directories.
Configuration files tend to use /var quite often. (/var/log, /var/tmp) You'd need to modify all these location references.
Basically your kernel is going to start /sbin/init which is going to start the initialization at /etc/rc.d or equivalent. If you start tracing all the scripts and services invoked in these startup scripts, I believe you should be able to capture all the places you'd need to modify the path names.
you can create a folder called system and the move all the files into the /system folder. and after that create symlinks, so the system can still be used. example:
su -i
cd /
mkdir system
mv /usr /system/usr
ln -s /system/usr /usr
I just did it........it broke my system XD (i think it was because i moved all the files into /system , including /boot witch is used by GRUB)
The chance of breaking the system is huge if you don't have background knowledge. Example if you move /bin to /system/bin, then you won't be able to create the symlink afterwards because the ln command (which is a program) is located in the /bin folder so moving it will end up in an error.
also, check out gobolinux.org which is based around what i just did. The entire system has been reorganized and to maintain compatibility, they have created symlinks so that they don't have to reprogram an app when porting applications.
I'm sure this has been asked before but I have no clue what to search for
I am trying to create a custom Linux image (for the Raspberry Pi) - I am currently manipulating the filesystem of the .img but I've discovered it's not as simple as dropping in the binary :( if only...
What is the accepted way to "pre-install" a package on a disk image where you can only manipulate the filesystem and ideally not run it first? Am I best to boot up, install, and then create the image from that, or is there a way of doing it beforehand in the same way you can change configuration settings etc?
Usually, when I have to change something in a disk image, I do the following:
sudo mount --bind /proc /mnt/disk_image/proc
sudo mount --bind /sys /mnt/disk_image/sys
sudo mount --bind /dev /mnt/disk_image/dev
These action are needed as this folder are create during boot process, mounting them in your system image will emulate a full boot. Then, you can chroot on it safely:
sudo chroot /mnt/disk_image
You're now able to issue commands in the chroot environment:
sudo apt-get install chromium
Of course, change /mnt/disk_image to the path where you have mounted your filesystem. apt-get will only works on Debian based system, change it according to your distribution.
You could find problem connecting to the internet and it can be cause by DNS configuration. The best thing you can do, is to copy your /etc/resolv.conf file in the remote filesystem as this file is usually changed by dhcp and it's empty on chroot environment.
This is the only solution that gives you full access to the command line of the system you're trying to modify.
This is an untested idea:
The dpkg tool, which can install .deb packages, has a --root option which can set a different filesystem than the local / path.
From the man page:
--instdir=dir
Change default installation directory which refers to the
directory where packages are to be installed. instdir is
also the directory passed to chroot(2) before running
package’s installation scripts, which means that the
scripts see instdir as a root directory. (Defaults to /)
--root=dir
Changing root changes instdir to dir and admindir to
dir/var/lib/dpkg.
If you mount your image and pass its mountpoint as --root, it should work.
There are things like the Ubuntu Customization Kit which allow you to create your own version of the distro with your own packages.
Crunchbang even has a utility like this, which is the distro I have personally selected for experimenting with my Pi.