Limit CPU time of process group - linux

Question

Is there a way to limit the absolute CPU time (in CPU seconds) spend in a process group?
ulimit -t 10; ./my-process looks like a good option but if my-process forks then each process in the process group gets its own limit. The whole process group can use an arbitrary amount of time by forking every 9 seconds.
The accepted answer on a similar question is to use cgroups but doesn't explain how. However, there are other answers (Limit total CPU usage with cgroups) saying that this is not possible in cgroups and only relative cpu usage can be limited (for example, 0.2 seconds out of every 1 second).
Liran Funaro suggested using a long period for cpu.cfs_period_us (https://stackoverflow.com/a/43660834/892961) but the parameter for the quota can be at most 1 second. So even with a long period I don't see how to set a CPU time limit of 10 seconds or an hour.
If ulimit and cgroups cannot do this, is there another way?

Answer 1

you can do it with cgroups. Do as root:
# Create cgroup
cgcreate -g cpu:/limited
# set shares (cpu limit)
cgset -r cpu.shares=256 limited
# run your program
cgexec -g cpu:limited /my/hungry/program
Alternatively you can use the cpulimit program which can freeze your code periodically. cgroups is the most advanced method though.
to set fixed cpu share :
cgcreate -g cpu:/fixedlimit
# allow fix 25% cpu usage (1 cpu)
cgset -r cpu.cfs_quota_us=25000,cpu.cfs_period_us=100000 fixedlimit
cgexec -g cpu:fixedlimit /my/hungry/program
It turned out, the goal is to limit runtime to certain seconds while measuring it. After setting the desired cgroup limits (in order to get a fair sandbox) you can achieve this goal by running:
((time -p timeout 20 cgexec -g cpu:fixedlimit /program/to/test ) 2>&1) | grep user
After 20 seconds the program will be stopped no matter what, and we can parse for user time (or system or real time) to evaluate it's performance.

Answer 2

This not directly answer the question but refers to the discussion on the actual need of the OP.
If your competition ignores everything except CPU time, it may be fundamentally flawed. One can simply, for example, cache results in the primary storage device. Since you do not count storage access time, it may have the least CPU cycles, but the worse actual performance.
A perfect crime would be to simply send the data via the Internet to another computer, which calculate the task then return the answer. This would finish the task with what appear to be zero cycles.
You actually want to measure "real" time and give this process the highest priority in your system (or actually running it secludedly).
When checking students' homework, we simply used an unrealistic time limit (e.g., 5 minutes for what should be a 10 seconds program), then killing the process if it has not finished in time and failing this submission.
If you want to pick a winner, then simply re-run the best competitors multiple times to ensure the validity of their results.

Answer 3

I found a solution that works for me. It is still far from perfect (read the caveats before using it). I'm somewhat new to bash scripting so any comments about this are welcome.
#!/bin/bash
#
# This script tries to limit the CPU time of a process group similar to
# ulimit but counting the time spent in spawned processes against the
# limit. It works by creating a temporary cgroup to run the process in
# and checking on the used CPU time of that process group. Instead of
# polling in regular intervals, the monitoring process assumes that no
# time is lost to I/O (i.e., wall clock time = CPU time) and checks in
# after the time limit. It then updates its assumption by comparing the
# actual CPU usage to the time limit and waiting again. This is repeated
# until the CPU usage exceeds its limit or the monitored process
# terminates. Once the main process terminates, all remaining processes
# in the temporary cgroup are killed.
#
# NOTE: this script still has some major limitations.
# 1) The monitored process can exceed the limit by up to one second
# since every iteration of the monitoring process takes at least that
# long. It can exceed the limit by an additional second by ignoring
# the SIGXCPU signal sent when hitting the (soft) limit but this is
# configurable below.
# 2) It assumes there is only one CPU core. On a system with n cores
# waiting for t seconds gives the process n*t seconds on the CPU.
# This could be fixed by figuring out how many CPUs the process is
# allowed to use (using the cpuset cgroup) and dividing the remaining
# time by that. Since sleep has a resolution of 1 second, this would
# still introduce an error of up to n seconds.
set -e
if [ "$#" -lt 2 ]; then
echo "Usage: $(basename "$0") TIME_LIMIT_IN_S COMMAND [ ARG ... ]"
exit 1
fi
TIME_LIMIT=$1
shift
# To simulate a hard time limit, set KILL_WAIT to 0. If KILL_WAIT is
# non-zero, TIME_LIMIT is the soft limit and TIME_LIMIT + KILL_WAIT is
# the hard limit.
KILL_WAIT=1
# Update as necessary. The script needs permissions to create cgroups
# in the cpuacct hierarchy in a subgroup "timelimit". To create it use:
# sudo cgcreate -a $USER -t $USER -g cpuacct:timelimit
CGROUPS_ROOT=/sys/fs/cgroup
LOCAL_CPUACCT_GROUP=timelimit/timelimited_$$
LOCAL_CGROUP_TASKS=$CGROUPS_ROOT/cpuacct/$LOCAL_CPUACCT_GROUP/tasks
kill_monitored_cgroup() {
SIGNAL=$1
kill -$SIGNAL $(cat $LOCAL_CGROUP_TASKS) 2> /dev/null
}
get_cpu_usage() {
cgget -nv -r cpuacct.usage $LOCAL_CPUACCT_GROUP
}
# Create a cgroup to measure the CPU time of the monitored process.
cgcreate -a $USER -t $USER -g cpuacct:$LOCAL_CPUACCT_GROUP
# Start the monitored process. In case it fails, we still have to clean
# up, so we disable exiting on errors.
set +e
(
set -e
# In case the process doesn't fork a ulimit is more exact. If the
# process forks, the ulimit still applies to each child process.
ulimit -t $(($TIME_LIMIT + $KILL_WAIT))
ulimit -S -t $TIME_LIMIT
cgexec -g cpuacct:$LOCAL_CPUACCT_GROUP --sticky $#
)&
MONITORED_PID=$!
# Start the monitoring process
(
REMAINING_TIME=$TIME_LIMIT
while [ "$REMAINING_TIME" -gt "0" ]; do
# Wait $REMAINING_TIME seconds for the monitored process to
# terminate. On a single CPU the CPU time cannot exceed the
# wall clock time. It might be less, though. In that case, we
# will go through the loop again.
sleep $REMAINING_TIME
CPU_USAGE=$(get_cpu_usage)
REMAINING_TIME=$(($TIME_LIMIT - $CPU_USAGE / 1000000000))
done
# Time limit exceeded. Kill the monitored cgroup.
if [ "$KILL_WAIT" -gt "0" ]; then
kill_monitored_cgroup XCPU
sleep $KILL_WAIT
fi
kill_monitored_cgroup KILL
)&
MONITOR_PID=$!
# Wait for the monitored job to exit (either on its own or because it
# was killed by the monitor).
wait $MONITORED_PID
EXIT_CODE=$?
# Kill all remaining tasks in the monitored cgroup and the monitor.
kill_monitored_cgroup KILL
kill -KILL $MONITOR_PID 2> /dev/null
wait $MONITOR_PID 2>/dev/null
# Report actual CPU usage.
set -e
CPU_USAGE=$(get_cpu_usage)
echo "Total CPU usage: $(($CPU_USAGE / 1000000))ms"
# Clean up and exit with the return code of the monitored process.
cgdelete cpuacct:$LOCAL_CPUACCT_GROUP
exit $EXIT_CODE