Does anyone know how to change the process name in top?
top - 05:02:47 up 182 days, 10:38, 1 user, load average: 14.53, 13.11, 11.95
Tasks: 4 total, 2 running, 2 sleeping, 0 stopped, 0 zombie
Cpu(s): 57.2%us, 14.8%sy, 0.0%ni, 26.2%id, 1.3%wa, 0.0%hi, 0.5%si, 0.0%st
Mem: 24736852k total, 22519688k used, 2217164k free, 132268k buffers
Swap: 8386552k total, 741900k used, 7644652k free, 12416224k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
6230 user 20 0 47540 6856 1164 R 41.5 0.0 0:03.10 perl
6430 user 20 0 14900 1156 936 R 0.3 0.0 0:00.02 top
6227 user 20 0 47276 7552 2088 S 0.0 0.0 0:00.07 perl
14577 user 20 0 11588 1808 1340 S 0.0 0.0 0:00.46 bash
I have figured out how to change the top -c name
$0 = 'new name.';
However this doesn't accomplish my goal.
I found a non standard module, and it looks very promising,
However I can't use any non standard modules.
http://metacpan.org/pod/Sys::Prctl
# instead of "perl helloworld.pl"
$0 = "helloworld"
prctl_name("helloworld");
I was hoping someone had some input, or knowledge on changing the title/name of a process.
I feel I have gone through perlvar pretty thoroughly however I may have missed a simple $^0. Hoping it's that simple.
Edit
#user2783897, not sure why I didn't think of that, here is the basic example I made.
sub prctl_name {
my $TASK_COMM_LEN = 16;
my $SYS_prctl = 157;
my $SYS_PR_SET_NAME = 15;
my $SYS_PR_GET_NAME = 16;
my ($str) = #_;
if(defined $str) {
my $rv = prctl($SYS_PR_SET_NAME, $str);
if($rv == 0) {
return 1;
} else {
return;
}
} else {
$str = "\x00" x ($TASK_COMM_LEN + 1); # allocate $str
my $ptr = unpack( 'L', pack( 'P', $str ) );
my $rv = prctl($SYS_PR_GET_NAME, $ptr);
if($rv == 0) {
return substr($str, 0, index($str, "\x00"));
} else {
return;
}
}
}
sub prctl {
my $SYS_prctl = 157;
my ($option, $arg2, $arg3, $arg4, $arg5) = #_;
syscall($SYS_prctl, $option,
($arg2 or 0), ($arg3 or 0), ($arg4 or 0), ($arg5 or 0));
}
Why don't you copy Sys/Prctl.pm code inside your own ? It's only a few dozens of lines.
Furthermore, most of the code is dedicated to finding on which kind of kernel the process is running, to select the appropriate SYS_prctl parameter. If you know on what kind of kernel you're running, you can cut the code to its bare bone.
Related
I'm creating a node program to return the output of linux top command, is working fine the only issue is that the name of command is cutted, instead the full command name like /usr/local/libexec/netdata/plugins.d/apps.plugin 1 returns /usr/local+
My code
const topparser=require("topparser")
const spawn = require('child_process').spawn
let proc=null
let startTime=0
exports.start=function(pid_limit,callback){
startTime=new Date().getTime()
proc = spawn('top', ['-c','-b',"-d","3"])
console.log("started process, pid: "+proc.pid)
let top_data=""
proc.stdout.on('data', function (data) {
console.log('stdout: ' + data);
})
proc.on('close', function (code) {
console.log('child process exited with code ' + code);
});
}//start
exports.stop=function(){
console.log("stoped process...")
if(proc){proc.kill('SIGINT')}// SIGHUP -linux ,SIGINT -windows
}//stop
The results
14861 root 20 0 0 0 0 S 0.0 0.0 0:00.00 [kworker/1+
14864 root 20 0 0 0 0 S 0.0 0.0 0:00.02 [kworker/0+
15120 root 39 19 102488 3344 2656 S 0.0 0.1 0:00.09 /usr/bin/m+
16904 root 20 0 0 0 0 S 0.0 0.0 0:00.00 [kworker/0+
19031 root 20 0 0 0 0 S 0.0 0.0 0:00.00 [kworker/u+
21500 root 20 0 0 0 0 Z 0.0 0.0 0:00.00 [dsc] <def+
22571 root 20 0 0 0 0 S 0.0 0.0 0:00.00 [kworker/0+
Any way to fix it?
Best regards
From a top manpage:
In Batch mode, when used without an argument top will format output using the COLUMNS= and LINES=
environment variables, if set. Otherwise, width will be fixed at the maximum 512 columns. With an
argument, output width can be decreased or increased (up to 512) but the number of rows is considā
ered unlimited.
Add '-w', '512' to the arguments.
Since you work with node, you can query netdata running on localhost for this.
Example:
http://london.my-netdata.io/api/v1/data?chart=apps.cpu&after=-1&options=ms
For localhost netdata:
http://localhost:19999/api/v1/data?chart=apps.cpu&after=-1&options=ms
You can also get systemd services:
http://london.my-netdata.io/api/v1/data?chart=services.cpu&after=-1&options=ms
If you are not planning to update the screen per second, you can instruct netdata to return the average of a longer duration:
http://london.my-netdata.io/api/v1/data?chart=apps.cpu&after=-5&points=1&group=average&options=ms
The above returns the average of the last 5 seconds.
Finally, you get the latest values all the metrics netdata monitors, with this:
http://london.my-netdata.io/api/v1/allmetrics?format=json
For completeness, netdata can export all the metrics in BASH format for shell scripts. Check this: https://github.com/firehol/netdata/wiki/receiving-netdata-metrics-from-shell-scripts
I have an homework assignment, we need to add some features to Linux kernel, and we're working on red hat 2.4.18.
I looked in sched.c, function set_user_nice:
void set_user_nice(task_t *p, long nice)
{
unsigned long flags;
prio_array_t *array;
runqueue_t *rq;
if (TASK_NICE(p) == nice || nice < -20 || nice > 19)
return;
/*
* We have to be careful, if called from sys_setpriority(),
* the task might be in the middle of scheduling on another CPU.
*/
rq = task_rq_lock(p, &flags);
if (rt_task(p)) {
p->static_prio = NICE_TO_PRIO(nice);
goto out_unlock;
}
array = p->array;
if (array)
dequeue_task(p, array);
p->static_prio = NICE_TO_PRIO(nice);
p->prio = NICE_TO_PRIO(nice);
if (array) {
enqueue_task(p, array);
/*
* If the task is running and lowered its priority,
* or increased its priority then reschedule its CPU:
*/
if ((NICE_TO_PRIO(nice) < p->static_prio) || (p == rq->curr))
resched_task(rq->curr);
}
out_unlock:
task_rq_unlock(rq, &flags);
}
I don't understand what exactly the code checks in the last if statement,
because few lines above it, we have this line:
p->static_prio = NICE_TO_PRIO(nice);
and then, in the if statement we check:
(NICE_TO_PRIO(nice) < p->static_prio)
Am I missing something?
Thanks
OK, so I looked for this function in a newer source code, and this function implemented in kernel/sched/core.c.
The part I was talking about:
old_prio = p->prio;
3585 p->prio = effective_prio(p);
3586 delta = p->prio - old_prio;
3587
3588 if (queued) {
3589 enqueue_task(rq, p, ENQUEUE_RESTORE);
3590 /*
3591 * If the task increased its priority or is running and
3592 * lowered its priority, then reschedule its CPU:
3593 */
3594 if (delta < 0 || (delta > 0 && task_running(rq, p)))
3595 resched_curr(rq);
3596 }
3597 out_unlock:
So it does seem like now the diff between the old and the new priority calculated properly.
Is 2.4.18 the kernel version? I'm looking at that source and don't see set_user_nice in sched.c.
Anyways, I think that they're handling a race condition there. It's possible between the time they set the new process priority, the process itself has changed it. So they're checking if that's the case and re-scheduling the task if so.
I am doing server-side rendering with wkhtmltoimage and each run is locking up at "88% loading" for 1-2 minutes. I decided to debug what was happening through strace but in a completely bizarre twist the program did NOT lock up. I've found this to be completely repeatable and consistent. Why on earth would strace make a program faster when by all rights the program should be slower ?!
Run without strace:
user#server:~/public_html/shapes$ time wkhtmltoimage --disable-smart-width --width 970 --format jpg '[THE URL]' '[THE PATH].jpg'
Loading page (1/2)
Rendering (2/2)
Done
real 1m45.724s
user 1m42.887s
sys 0m0.623s
Run WITH strace:
user#server:~/public_html/shapes$ time strace wkhtmltoimage --disable-smart-width --width 970 --format jpg '[THE URL]' '[THE PATH].jpg'
execve("/usr/local/bin/wkhtmltoimage", ["wkhtmltoimage", "--disable-smart-width", "--width", "970", "--format", "jpg", "[THE URL]"..., "[THE PATH]"...], [/* 21 vars */]) = 0
brk(0) = 0x311a000
...
exit_group(0) = ?
+++ exited with 0 +++
real 0m6.526s
user 0m0.582s
sys 0m0.377s
Server is private so I've redacted the URL and PATH but otherwise the output is correct. Also both runs create the correct output and I've cleared up temporary files to ensure it's not a caching issue. I've done these runs 10 times each to ensure it's not a random artifact but it happens consistently, THEREFORE the only logical conclusion is that strace is somehow changing the behaviour of wkhtmltoimage and I was really hoping somebody could tell me what. If I knew why strace makes the program not lockup I could probably find a solution.
Here is the hung process:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
772 sigapp 20 0 1485600 45760 14524 R 99.8 2.2 0:57.02 wkhtmltoimage
As root I can attach to the hung image using strace -p $(pidof wkhtmltoimage) and the result is:
gettimeofday({1436692542, 446246}, NULL) = 0
gettimeofday({1436692542, 556958}, NULL) = 0
gettimeofday({1436692542, 557161}, NULL) = 0
gettimeofday({1436692542, 659238}, NULL) = 0
gettimeofday({1436692542, 771381}, NULL) = 0
gettimeofday({1436692542, 771686}, NULL) = 0
gettimeofday({1436692542, 875783}, NULL) = 0
gettimeofday({1436692542, 987490}, NULL) = 0
gettimeofday({1436692542, 987781}, NULL) = 0
gettimeofday({1436692543, 84764}, NULL) = 0
...
System: Linux server 3.13.0-30-generic #55-Ubuntu SMP Fri Jul 4 21:40:53 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux (Running on 2 CPU Xen Guest)
Software: wkhtmltoimage 0.12.2.1 (with patched qt) <- installed from official website via .deb file
I've read this but not sure if relevant: Hung processes resume if attached to strace
UPDATE:
Tried with another program webkit2pdf and seeing lockups as well. This time strace output is different. Both programs use webkit.:
root#server:~# strace -p `pidof webkit2pdf`
Process 4699 attached
restart_syscall(<... resuming interrupted call ...>
On a successful run of strace wkhtmltoimage ... I can see similar calls but not NEARLY as many preceeded by an mmap:
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x7f9dbbbc1000
gettimeofday({1436696839, 361942}, NULL) = 0
gettimeofday({1436696839, 362254}, NULL) = 0
gettimeofday({1436696839, 362505}, NULL) = 0
gettimeofday({1436696839, 362787}, NULL) = 0
gettimeofday({1436696839, 363172}, NULL) = 0
gettimeofday({1436696839, 363568}, NULL) = 0
gettimeofday({1436696839, 363913}, NULL) = 0
mmap(NULL, 28672, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x7f9db5000000
gettimeofday({1436696839, 364701}, NULL) = 0
mmap(NULL, 28672, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x7f9db4ff9000
gettimeofday({1436696839, 365612}, NULL) = 0
gettimeofday({1436696839, 365956}, NULL) = 0
So per pvg's comment below this is most probably the locking mechanism for setAttribute working correctly under strace vs. not working normally.
what 'man top' said is: RES = CODE + DATA
q: RES -- Resident size (kb)
The non-swapped physical memory a task has used.
RES = CODE + DATA.
r: CODE -- Code size (kb)
The amount of physical memory devoted to executable code, also known as the 'text resident set' size or TRS.
s: DATA -- Data+Stack size (kb)
The amount of physical memory devoted to other than executable code, also known as the 'data >resident set' size or DRS.
what when i run 'top -p 4258',i get the following:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ CODE DATA COMMAND
258 root 16 0 3160 1796 1328 S 0.0 0.3 0:00.10 476 416 bash
1796 != 476+416
why?
ps:
linux distribution:
linux-iguu:~ # lsb_release -a
LSB Version: core-2.0-noarch:core-3.0-noarch:core-2.0-ia32:core-3.0-ia32:desktop-3.1-ia32:desktop-3.1-noarch:graphics-2.0-ia32:graphics-2.0-noarch:graphics-3.1-ia32:graphics-3.1-noarch
Distributor ID: SUSE LINUX
Description: SUSE Linux Enterprise Server 9 (i586)
Release: 9
Codename: n/a
kernel version:
linux-iguu:~ # uname -a
Linux linux-iguu 2.6.16.60-0.21-default #1 Tue May 6 12:41:02 UTC 2008 i686 i686 i386 GNU/Linux
I'll explain this with the help of an example of what happens when a program allocates and uses memory. Specifically, this program:
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <string.h>
int main(){
int *data, size, count, i;
printf( "fyi: your ints are %d bytes large\n", sizeof(int) );
printf( "Enter number of ints to malloc: " );
scanf( "%d", &size );
data = malloc( sizeof(int) * size );
if( !data ){
perror( "failed to malloc" );
exit( EXIT_FAILURE );
}
printf( "Enter number of ints to initialize: " );
scanf( "%d", &count );
for( i = 0; i < count; i++ ){
data[i] = 1337;
}
printf( "I'm going to hang out here until you hit <enter>" );
while( getchar() != '\n' );
while( getchar() != '\n' );
exit( EXIT_SUCCESS );
}
This is a simple program that asks you how many integers to allocate, allocates them, asks how many of those integers to initialize, and then initializes them. For a run where I allocate 1250000 integers and initialize 500000 of them:
$ ./a.out
fyi: your ints are 4 bytes large
Enter number of ints to malloc: 1250000
Enter number of ints to initialize: 500000
Top reports the following information:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ SWAP CODE DATA COMMAND
<program start>
11129 xxxxxxx 16 0 3628 408 336 S 0 0.0 0:00.00 3220 4 124 a.out
<allocate 1250000 ints>
11129 xxxxxxx 16 0 8512 476 392 S 0 0.0 0:00.00 8036 4 5008 a.out
<initialize 500000 ints>
11129 xxxxxxx 15 0 8512 2432 396 S 0 0.0 0:00.00 6080 4 5008 a.out
The relevant information is:
DATA CODE RES VIRT
before allocation: 124 4 408 3628
after 5MB allocation: 5008 4 476 8512
after 2MB initialization: 5008 4 2432 8512
After I malloc'd 5MB of data, both VIRT and DATA increased by ~5MB, but RES did not. RES did increase after I touched 2MB of the integers I allocated, but DATA and VIRT stayed the same.
VIRT is the total amount of virtual memory used by the process, including what is shared and what is over-committed. DATA is the amount of virtual memory used that isn't shared and that isn't code-text. I.e., it is the virtual stack and heap of the process. RES is not virtual: it is a measurment of how much memory the process is actually using at that specific time.
So in your case, the large inequality CODE+DATA < RES is likely the shared libraries included by the process. In my example (and yours), SHR+CODE+DATA is a closer approximation to RES.
Hope this helps.
There's a lot of hand-waving and voodoo associated with top and ps. There are many articles (rants?) online about the descrepancies. E.g., this and this.
This explanation is terrific to resolve my some queries. Thanks!
And meanwhile, trying to add something got during my understanding of linux memory management knowledge. If any misunderstand, please correct me!
Modern OS process concepts are based on virtual memory. Virtual memory system includes the RAM+SWAP;
So I think most of the memory concepts related with processes refer to the virtual memory, except that there are some supplement notes.
Any virtual address(page) allocated to a process is in below state:
a) allocated, but no mapping to any physical memory(something like COW)
b) allocated, already mapped to physical memory
c) allocated, already mapped to swapped memory.
The fields ouput of top command:
a) VIRT -- it refers to all virtual memory that the process have the right
to access, no matter it is already mapped to physical memory or swapped
memory, or even has no any mapping.
b) RES -- it refers to the virtual address already mapped to physical address and it still in RAM.
c) SWAP -- refers to the virtual address already mapped to physical address and it is swapped into SWAP space.
d) SHR -- it refers to the shared memory available to a process(VM?)
e) CODE + DATA -- CODE could be in a state of 2.b/2.c, and DATA could be in any of 3 state 2.a/2.b/3.c, and 3.b/3.c also have a fields name called "USED".
4) So the calculation maybe look like:
a) VIRT(VM) = RES(VM in memory) + SWAP(VM in swap) + VM unmapped(DATA, SHR?).
b) USED = RES + SWAP
c) SWAP = CODE(vm in memory) + DATA(vm in memory) + SHR(vm in memory?)
d) RES = CODE(vm in memory) + DATA(vm in memory) + SHR(vm in memory?)
At least DATA segment still have a "DATA(VM unmapped)", this could be observed from above malloc example. That's a little different from the manpage of top command which says "DATA: The amount of physical memory devoted to other than executable code, also known as the Data Resident Set size or DRS". Thanks again.
So amount of (CODE + DATA + SHR) usually larger than RES, because at least DATA(vm unmapped) actually calculated in "DATA", not like the manpge claiming.
Regards,
Linux uses COW to keep memory usage low after a fork, but the way Perl 5 variables work in perl seems to defeat this optimization. For instance, for the variable:
my $s = "1";
perl is really storing:
SV = PV(0x100801068) at 0x1008272e8
REFCNT = 1
FLAGS = (POK,pPOK)
PV = 0x100201d50 "1"\0
CUR = 1
LEN = 16
When you use that string in a numeric context, it modifies the C struct representing the data:
SV = PVIV(0x100821610) at 0x1008272e8
REFCNT = 1
FLAGS = (IOK,POK,pIOK,pPOK)
IV = 1
PV = 0x100201d50 "1"\0
CUR = 1
LEN = 16
The string pointer itself did not change (it is still 0x100201d50), but now it is in a different C struct (a PVIV instead of a PV). I did not modify the value at all, but suddenly I am paying a COW cost. Is there any way to lock the perl representation of a Perl 5 variable so that this time saving (perl doesn't have to convert "0" to 0 a second time) hack doesn't hurt my memory usage?
Note, the representations above were generated from this code:
perl -MDevel::Peek -e '$s = "1"; Dump $s; $s + 0; Dump $s'
The only solution I have found so far, is to make sure I force perl to do all of the conversions I expect in the parent process. And you can see from the code below, even that only helps a little.
Results:
Useless use of addition (+) in void context at z.pl line 34.
Useless use of addition (+) in void context at z.pl line 45.
Useless use of addition (+) in void context at z.pl line 51.
before eating memory
used memory: 71
after eating memory
used memory: 119
after 100 forks that don't reference variable
used memory: 144
after children are reaped
used memory: 93
after 100 forks that touch the variables metadata
used memory: 707
after children are reaped
used memory: 93
after parent has updated the metadata
used memory: 109
after 100 forks that touch the variables metadata
used memory: 443
after children are reaped
used memory: 109
Code:
#!/usr/bin/perl
use strict;
use warnings;
use Parallel::ForkManager;
sub print_mem {
print #_, "used memory: ", `free -m` =~ m{cache:\s+([0-9]+)}s, "\n";
}
print_mem("before eating memory\n");
my #big = ("1") x (1_024 * 1024);
my $pm = Parallel::ForkManager->new(100);
print_mem("after eating memory\n");
for (1 .. 100) {
next if $pm->start;
sleep 2;
$pm->finish;
}
print_mem("after 100 forks that don't reference variable\n");
$pm->wait_all_children;
print_mem("after children are reaped\n");
for (1 .. 100) {
next if $pm->start;
$_ + 0 for #big; #force an update to the metadata
sleep 2;
$pm->finish;
}
print_mem("after 100 forks that touch the variables metadata\n");
$pm->wait_all_children;
print_mem("after children are reaped\n");
$_ + 0 for #big; #force an update to the metadata
print_mem("after parent has updated the metadata\n");
for (1 .. 100) {
next if $pm->start;
$_ + 0 for #big; #force an update to the metadata
sleep 2;
$pm->finish;
}
print_mem("after 100 forks that touch the variables metadata\n");
$pm->wait_all_children;
print_mem("after children are reaped\n");
Anyway if you avoid COW on start and during run you should not forgot END phase of lifetime. In shutdown there are two GC phases when in first there are ref counts updates so it can kill you in nice way. You can in solve it ugly:
END { kill 9, $$ }
This goes without saying, but COW doesn't happen on a per-struct basis, but on a memory page basis. So it's enough that one thing in an entire memory page be modified like this for you to pay the copying cost.
On Linux you can query the page size like this:
getconf PAGESIZE
On my system that's 4096 bytes. You can fit a lot of Perl scalar structs in that space. If one of those things gets modified Linux will have to copy the entire thing.
This is why using memory arenas is a good idea in general. You should separate your mutable and immutable data so that you won't have to pay COW costs for immutable data just because it happened to reside in the same memory page as mutable data.