I am migrating a server configuration to supervisord (from init.d files).
There are a few instances of varish running. I remember when I started using varnish I had ulimit problems so there is the following lines in the init.d/varnish scripts
ulimit -n ${NFILES:-131072}
ulimit -l ${MEMLOCK:-82000}
I am configuring supervisord to run the /usr/sbin/varnishd program with arguments.
How do you configure the ulimit settings via supervisord? Do I just wrap the varnishd program in a script?
If you really need to go this route it would be preferred to set the ulimit at a system level (see limits.conf). But varnish performs best with a malloc backend rather than a file backend, so if you have the memory resources, this would solve your ulimit problem and improve your performance. The documentation is here, but basically you have something like "-s malloc,1G" in your /etc/default/varnish:
DAEMON_OPTS="-a :80 \
-T localhost:1234 \
-f /etc/varnish/default.vcl \
-s malloc,1G"
Related
I am currently tracking a weird issue we are experiencing using dockerd 17.10.0-ce on an Alpine Linux 3.7 host. It seems for all the containers on this host, the process tree initiated as the entrypoint/command of the Docker image is NOT visible within the container itself. In comparison, on an Ubuntu host, the same image will have the process tree visible as PID 1.
Here is an example.
Run a container with an explicit known entrypoint/command:
% docker run -d --name testcontainer --rm busybox /bin/sh -c 'sleep 1000000'
Verify the processes are seen by dockerd properly:
% docker top testcontainer
PID USER TIME COMMAND
6729 root 0:00 /bin/sh -c sleep 1000000
6750 root 0:00 sleep 1000000
Now, start a shell inside that container and check the process list:
% docker exec -t -i testcontainer /bin/sh
/ # ps -ef
PID USER TIME COMMAND
6 root 0:00 /bin/sh
12 root 0:00 ps -ef
As can be observed, our entrypoint command (/bin/sh -c 'sleep 1000000') is not visible inside the container itself. Even running top will yield the same results.
Is there something I am missing here? On an Ubuntu host with the same docker engine version, the results are as I would expect. Could this be related to Alpine's hardened kernel causing an issue with how the container PID space is separated?
Any help appreciated for areas to investigate.
-b
It seems this problem is related to grsecurity module which the Alpine kernel implements. In this specific case, the GRKERNSEC_CHROOT_FINDTASK kernel setting is used to limit what processes can do outside of the chroot environment. This is controlled by the kernel.grsecurity.chroot_findtask sysctl variable.
From the grsecurity docs:
kernel.grsecurity.chroot_findtask
If you say Y here, processes inside a chroot will not be able to kill,
send signals with fcntl, ptrace, capget, getpgid, setpgid, getsid, or
view any process outside of the chroot. If the sysctl option is
enabled, a sysctl option with name "chroot_findtask" is created.
The only workaround I have found for now is to disable this flag as well as the chroot_deny_mknod and chroot_deny_chmod flags in order to get the same behaviour as with a non-grsecurity kernel.
kernel.grsecurity.chroot_deny_mknod=0
kernel.grsecurity.chroot_deny_chmod=0
kernel.grsecurity.chroot_findtask=0
Of course this is less than ideal since it bypasses and disables security features of the system but might be a valid workaround for a development environment.
I am new to writing script and not sure whether I am correct in writing such script for tcpdump to collect pcap info.
tcpdump -s 0 port ftp or ssh or http or https -i eth0 -w mycap.pcap
#run the tcpdump and store all the info in mycap.pcap
sudo kill -2 #for exit purpose
This enables me to run tcpdump which is good, however, I wish to stop this (due to the space for mycap.pcap meet the max capacity of 3GB per file) automatically via the same script and run again but this time round, I will store it in another file (eg. mycap1.pcap)
Then the cycle goes again until I stop the process by pressing crtl+c
Can this be done?
You don't need to write a script for that.
tcpdump -C <filesize> -s 0 port ftp or ssh or http or https -i eth0 -w mycap.pcap
Have a look at the man-page for tcpdump.
I have following scenario. previously I had only one server running the application and it had housekeeper script running to archive log files. Now we have implemented HA ( high availability) and, I want to know in this case, how the script should identify which server is running currently and then to run the script accordingly, HA is implemented in active/passive mode. There are 2 physical servers to support HA, if one is down , other one will be active.
Use this Shell ecript for high availability on passive server.
#!/bin/bash
crumb_id=$(curl -s 'http://locathost:8080/crumbIssuer/api/xml?xpath=contact(//crumbRequestField,":",//crumb)' -u admin:admin)
curl -s -XPOST 'http://locathost:8080/reload' -u admin:admin -H "$crumb_id"
~
I've seeing occasional segmentation faults in glibc on several different Fedora Core 9 Hudson Slaves. I've attempted to configure each slave to generate core files and place them in /corefiles, but have had no luck.
Here is what I've done on each linux slave:
1) Create a corefile storage location
sudo install -m 1777 -d /corefiles
2) Directed the corefiles to the storage location by adding the following to /etc/sysctl.conf
kernel.core_pattern = /corefiles/core.%e-PID:%p-%t-signal_%s-%h
3) Enabled unlimited corefiles for all users by adding the following to /etc/profile
ulimit -c unlimited
Is there some additional Linux magic required or do I need to do something to the Hudson slave or JVM?
Thanks for the help
Did you reboot or run "sysctl -p" (as root) after editing /etc/sysctl.conf ?
Also, if i remember correctly, ulimit values are per user and calling ulimit wont survive a boot. You should add this to /etc/security/limits.conf:
* soft core unlimited
Or call ulimit in the script that starts hudson if you don't wont everyone to produce coredumps.
I figured this out :-).
The issue is Hudson invokes the bash shell as a non-interactive shell, which will bypass the ulimit setting in /etc/profile. The solution is to add the BASH_ENV environmental variable tothe Hudson slaves and set the value to a file with ulimit -c unlimited set.
I have the following script:
cat > /tmp/script.sh <<EndOfScript
#!/bin/sh
ulimit -n 8192
run_app
EndOfScript
which runs smoothly locally, it is always ok. But if I try to run it remotely through ssh:
scp /tmp/script.sh user#host:/tmp/script.sh
ssh user#host "chmod 755 /tmp/script.sh; /tmp/script.sh"
I got the error:
ulimit: open files: cannot modify limit: Operation not permitted
I also tried the following command:
ssh user#host "ulimit -n 8192"
same error.
It looks like that ssh remote command execution is enforcing a 1024 hard limit on nofile limit, but I can not find out how to modify this default value. I tried to modify /etc/security/limits.conf and restart sshd, still the same error.
Instead of using the workaround of /etc/initscript (and do not make a typo in that file.. :), if you just want sshd to honor the settings you made in /etc/security/limits.conf, you should make sure you have UsePAM yes in /etc/ssh/sshd_config, and /etc/pam.d/sshd lists session required pam_limits.so (or otherwise includes another file that does so).
That should be all there is to it.
In older versions od openssh (<3.6 something) there was also a problem with UsePrivilegeSeparation that prevented limits being honored, but it was fixed in newer versions.
Fiannly figured out the answer: add the following to /etc/initscript
ulimit -c unlimited
ulimit -HSn 65535
# Execute the program.
eval exec "$4"
ulimit requires superuser privileges to run.
I would suggest you to ask the server administrator to modify that value for you on the server you are trying to run the script on.
He/She can do that by modifying /etc/secutiry/limits.conf on Linux. Here is an example that might help:
* soft nofile 8192
* hard nofile 8192
After that, you don't need to restart sshd. Just logout and login again.
I would suggest you to ask the same question in ServerFault though. You'll get better server-side related answers there.
Check the start up scripts (/etc/profile, ~/.??*) for a call to ulimit. IIRC, once a limit has been imposed, it can't be widened anymore.