Running this little script shows me that there is 12 opened descriptors when I run it. I know that 0,1,2 are stdin,stdout,sterr but what about the other 9 and why they're all opened and empty? Also, fd 9 seems to be a directory and I have no idea how to read it since fs.read returns Error: EISDIR, illegal operation on a directory.
fs = require('fs')
for(var i = 0; i < 1000; i++)
{
console.log(i,fs.fstatSync(i).size)
}
You can look at /proc/<pid of node process>/fd/. (If you are on linux)
0 -> /dev/pts/16
1 -> /dev/pts/16
2 -> /dev/pts/16
3 -> pipe:[6567721]
4 -> pipe:[6567721]
5 -> anon_inode:[eventpoll]
6 -> pipe:[6567722]
7 -> pipe:[6567722]
8 -> anon_inode:[eventfd]
9 -> /
So the directory of fd 9 is the root directory and the others are various forms of special file descriptors. I assume the two pairs of pipes are used by node internally to communicate between threads, as no other process is using them (checked with lsof | grep 656772)
Of course this does not tell you why, but it may be a first pointer.
how to read it since fs.read returns Error: EISDIR, illegal operation on a directory.
Directories should be read using fs.readdir
http://nodejs.org/api/fs.html#fs_fs_readdir_path_callback
Related
I just read some Go code that does something along the following lines:
type someType struct {
...
...
rpipe io.ReadCloser
wpipe io.WriteCloser
}
var inst someType
inst.rpipe, inst.wpipe, _ := os.Pipe()
cmd := exec.Command("some_binary", args...)
cmd.Stdout = inst.wpipe
cmd.Stderr = inst.wpipe
if err := cmd.Start(); err != nil {
....
}
inst.wpipe.Close()
inst.wpipe = nil
some_binary is a long running process.
Why is inst.wpipe closed and set to nil? What would happen if its not closed? Is it common/necessary to close inst.wpipe?
Is dup2(pipe_fd[1], 1) the C analogue of cmd.Stdout = inst.wpipe; inst.wpipe.Close()?
That code is typical of a program that wants to read output generated by some other program. The os.Pipe() function returns a connected pair of os.File entities (or, on error—which should not be simply ignored—doesn't) where a write on the second (w or wpipe) entity shows up as readable bytes on the first (r / rpipe) entity. But—this is the key to half the answer to your first question—how will a reader know when all writers are finished writing?
For a reader to get an EOF indication, all writers that have or had access to the write side of the pipe must call the close operation. By passing the write side of the pipe to a program that we start with cmd.Start(), we allow that command to access the write side of the pipe. When that command closes that pipe, one of the entities with access has closed the pipe. But another entity with access hasn't closed it yet: we have write access.
To see an EOF, then, we must close off access to our wpipe, with wpipe.Close(). So that answer the first half of:
Why is inst.wpipe closed and set to nil?
The set-to-nil part may or may not have any function; you should inspect the rest of the code to find out if it does.
Is dup2(pipe_fd[1], 1) the C analogue of cmd.Stdout = inst.wpipe; inst.wpipe.Close()?
Not precisely. The dup2 level is down in the POSIX OS area, while cmd.Stdout is at a higher (OS-independent) level. The POSIX implementation of cmd.Start() will wind up calling dup2 (or something equivalent) like this after calling fork (or something equivalent). The POSIX equivalent of inst.wipe.Close() is close(wfd) where wfd is the POSIX file number in wpipe.
In C code that doesn't have any higher level wrapping around it, we'd have something like:
int fds[2];
if (pipe(fds) < 0) ... handle error case ...
pid = fork();
switch (pid) {
case -1: ... handle error ...
case 0: /* child */
if (dup2(fds[1], 1) < 0 || dup2(fds[1], 2) < 0) ... handle error ...
if (execve(prog, args, env) < 0) ... handle error ...
/* NOTREACHED */
default: /* parent */
if (close(fds[1]) < 0) ... handle error ...
... read from fds[0] ...
}
(although if we're careful enough to check for an error from close, we probably should be careful enough to check whether the pipe system call gave us back descriptors 0 and 1, or 1 and 2, or 2 and 3, here—though perhaps we handle this earlier by making sure that 0, 1, and 2 are at least open to /dev/null).
I'm playing a bit with NativeCall to get familiar with that side of Perl6. Of course, I'm trying to load libstatgrab first (what else?).
So I start with easiest part - the host information. Since no cluster support yet, it's just one result - no worries for complication.
The code:
#!/usr/bin/env perl6
use v6;
use NativeCall;
enum sg_error (
SG_ERROR_NONE => 0,
SG_ERROR_INVALID_ARGUMENT => 1,
...
);
class sg_error_details is repr('CStruct') {
has int32 $.error;
has int32 $.errno_value;
has Str $.error_arg;
};
sub sg_init(int32 $ignore_errors) returns int32 is native('statgrab') { * };
enum sg_host_state (
sg_unknown_configuration => 0,
sg_physical_host => 1,
sg_virtual_machine => 2,
sg_paravirtual_machine => 3,
sg_hardware_virtualized => 4
);
class sg_host_info is repr('CStruct') {
has Str $.os_name;
has Str $.os_release;
has Str $.os_version;
has Str $.platform;
has Str $.hostname;
has uint32 $.bitwidth;
has int32 $.host_state;
has uint32 $.ncpus;
has uint32 $.maxcpus;
has uint64 $.uptime;
has uint64 $.systime;
};
sub sg_get_host_info(size_t is rw) returns Pointer is native('statgrab') is symbol('sg_get_host_info_r') { * };
sub sg_free_host_info(Pointer) is native('statgrab') is symbol('sg_free_stats_buf') { * };
sub MAIN() {
my int32 $ignore_errors = 0;
my $error = sg_init($ignore_errors);
if $error != SG_ERROR_NONE {
say "Maeh: $error";
exit 1;
}
my size_t $num_host_infos = 0;
my $res = sg_get_host_info($num_host_infos);
if $num_host_infos > 0 {
my $host_info = nativecast(sg_host_info, $res);
with $host_info {
say "You're using ", $_.os_name, " on ", $_.hostname;
}
}
sg_free_host_info($res);
}
Starting it (dumb) results in loading library error:
$ perl6 statgrab.p6
Cannot locate native library 'libstatgrab.dylib': dlopen(libstatgrab.dylib, 1): image not found
in method setup at /Users/sno/rakudo/share/perl6/sources/24DD121B5B4774C04A7084827BFAD92199756E03 (NativeCall) line 283
in method CALL-ME at /Users/sno/rakudo/share/perl6/sources/24DD121B5B4774C04A7084827BFAD92199756E03 (NativeCall) line 570
in sub MAIN at statgrab.p6 line 95
in block <unit> at statgrab.p6 line 93
Okay - giving it some search path:
$ LD_LIBRARY_PATH=/opt/pkg/lib:$LD_LIBRARY_PATH perl6 statgrab.p6
Cannot locate native library 'libstatgrab.dylib': dlopen(libstatgrab.dylib, 1): image not found
Same picture when using DYLD_LIBRARY_PATH - which is supported by dlopen(3) on Darwin, too.
But changing in the directory works:
$ (cd /opt/pkg/lib && perl6 /data/Projects/OSS/p6-Unix-Statgrab/statgrab.p6 )
You're using Darwin on ernie.[...]
Is there a lack of search path passthrough in the way how moarvm is called?
doug$ perl6 -v
This is Rakudo Star version 2018.10 built on MoarVM version 2018.10
implementing Perl 6.c.
On a fairly recent Rakudo Star on MacOS High Sierra, the script worked "out of the box" for me:
edited script to remove '...'.
Script failed to load the library (really missing!)
brew install libstatgrab
Script ran successfully:
vader:learning doug$ perl6 nativecall_mac_Sno.pl6
You're using Darwin on Vader.local
Homebrew installed the library as follows:
$ v /usr/local/lib
total 11904
-rw-r--r-- 1 doug admin 6080828 Sep 23 12:40 libmoar.dylib
lrwxr-xr-x 1 doug admin 51 Mar 23 11:02 libstatgrab.10.dylib# -> ../Cellar/libstatgrab/0.91/lib/libstatgrab.10.dylib
lrwxr-xr-x 1 doug admin 44 Mar 23 11:02 libstatgrab.a# -> ../Cellar/libstatgrab/0.91/lib/libstatgrab.a
lrwxr-xr-x 1 doug admin 48 Mar 23 11:02 libstatgrab.dylib# -> ../Cellar/libstatgrab/0.91/lib/libstatgrab.dylib
drwxr-xr-x 3 doug admin 102 Mar 23 11:02 pkgconfig/
For me, the perl6 executable is indeed a shell script, but it worked (there was no need to pass any extra LD_LIBRARY_PATH=...).
doug$ file `which perl6`
/Applications/Rakudo/bin/perl6: POSIX shell script text executable, ASCII text, with very long lines
doug$ set | grep LIBRARY
doug$
I have also had issues with my nativecall scripts finding the library, but have always solved them by fixing the library install and/or supplying 'LD_LIBRARY_PATH'.
Sorry this experience was Less Than Awesome for you
I have a problem with a piece of Linux code (written in C++) that does something like this:
Create a new directory with mkdir("xyz",0755) which succeeds (return code is 0).
Tries to open/create a new file in the justly created directory.
Fails because the permissions on the new directory is actually 0600 and NOT 0755 as requested.
The code looks like this and checks that the path prefix "/tmp/slim" exists before coming to this part:
int somefunc(const string& path)
{
if ( mkdir( path.c_str(), 0755 ) == 0 ) {
// (*) if (!access( path.c_str(), F_OK | R_OK | W_OK | X_OK ) == 0 )
// (*) chmod( path.c_str(), 0755);
string pidinfo = to_string( getpid() ) + "\n";
string pidinfofile = path + "/pid";
int fd = open( pidinfofile.c_str(), O_RDWR | O_CREAT, S_IWUSR | S_IRUSR );
if ( fd == -1 )
return 0;
ssize_t written = write( fd, text.c_str(), text.size() );
// ... do more stuff
}
}
As this snippet of a strace (no lines missing/redacted) shows, the openat() fails even though the mkdir() returned 0.
13661 16:32:22.068465 mkdir("/tmp/slim/testsock", 0755) = 0
13661 16:32:22.068720 getpid() = 13661
13661 16:32:22.068829 openat(AT_FDCWD, "/tmp/slim/testsock/pid", O_RDWR|O_CREAT, 0600) = -1 EACCES (Permission denied)
The result of running getfacl looks like this:
[localhost]$ getfacl /tmp/slim/
getfacl: Removing leading '/' from absolute path names
# file: tmp/slim/
# owner: stk
# group: stk
user::rwx
group::r-x
other::r-x
How can the the mkdir() return 0 but create a directory with permissions that differ from the specified? It's not a umask thing, I've tried setting umask to 0 before creating the directory without any effect. If the two commented lines marked with (*) is enabled/uncommented, things work as they should - but I don't like that sort of symptom treatment that skirts the real problem. There's got to be a reasonable explanation for this seemingly weird behavior.
Part of the story is that this works in an application with multiple threads. Each thread performs the code above (which is a smallish, thread safe function), and most threads succeed, but there's always 1 or 2 (out of 5-10) that fails as described.
Well, as it turned out (and quite expectedly, right?) it was a threading problem alright, as #RobertHarvey also hinted at. But I was also a bit right :-) when I wrote in a comment "there is some hidden, shared state somewhere". Well, the process umask is a shared, but perhaps not exactly hidden, state/variable. Here's what went wrong:
One or more threads was executing the code above, happily creating directories.
Simultaneously another thread was fiddling with the umask to ensure the right permission of a Unix socket it was creating, temporarily setting umask at 0177.
Although the umask fiddling was ever so short, Murphy's Law dictated that the directories sometimes were created exactly while the umask was 0177, also forcing the directories to get permission mask 0600 rather than 0755.
Lesson (re-)learned: Watch out for hidden shared state/variables when using more than 1 thread.
I needed help regarding the subprocess module. This question might sound repeated, and I have seen a number of articles related to it in a number of ways. But even so I am unable to solve my problem. It goes as follows:
I have a C program 2.c it's contents are as follows:
#include<stdio.h>
int main()
{
int a;
scanf("%d",&a);
while(1)
{
if(a==0) //Specific case for the first input
{
printf("%d\n",(a+1));
break;
}
scanf("%d",&a);
printf("%d\n",a);
}
return 0;
}
I need to write a python script which first compiles the code using subprocess.call() and then opens two process using Popen to execute the respective C-program. Now the output of the first process must be the input of the second and vice versa. So essentially, if my initial input was 0, then the first process outputs 2, which is taken by second process. It in turn outputs 3 and so on infinitely.
The below script is what I had in mind, but it is flawed. If someone can help me I would very much appreciate it.
from subprocess import *
call(["gcc","2.c"])
a = Popen(["./a.out"],stdin=PIPE,stdout=PIPE) #Initiating Process
a.stdin.write('0')
temp = a.communicate()[0]
print temp
b = Popen(["./a.out"],stdin=PIPE,stdout=PIPE) #The 2 processes in question
c = Popen(["./a.out"],stdin=PIPE,stdout=PIPE)
while True:
b.stdin.write(str(temp))
temp = b.communicate()[0]
print temp
c.stdin.write(str(temp))
temp = c.communicate()[0]
print temp
a.wait()
b.wait()
c.wait()
If you want the output of the first command a to go as the input of the second command b and in turn b's output is a's input—in a circle like a snake eating its tail— then you can't use .communicate() in a loop: .communicate() doesn't return until the process is dead and all the output is consumed.
One solution is to use a named pipe (if open() doesn't block in this case on your system):
#!/usr/bin/env python3
import os
from subprocess import Popen, PIPE
path = 'fifo'
os.mkfifo(path) # create named pipe
try:
with open(path, 'r+b', 0) as pipe, \
Popen(['./a.out'], stdin=PIPE, stdout=pipe) as b, \
Popen(['./a.out'], stdout=b.stdin, stdin=pipe) as a:
pipe.write(b'10\n') # kick-start it
finally:
os.remove(path) # clean up
It emulates a < fifo | b > fifo shell command from #alexander barakin answer.
Here's more complex solution that funnels the data via the python parent process:
#!/usr/bin/env python3
import shutil
from subprocess import Popen, PIPE
with Popen(['./a.out'], stdin=PIPE, stdout=PIPE, bufsize=0) as b, \
Popen(['./a.out'], stdout=b.stdin, stdin=PIPE, bufsize=0) as a:
a.stdin.write(b'10\n') # kick-start it
shutil.copyfileobj(b.stdout, a.stdin) # copy b's stdout to a' stdin
This code connects a's output to b's input using redirection via OS pipe (as a | b shell command does).
To complete the circle, b's output is copied to a's input in the parent Python code using shutil.copyfileobj().
This code may have buffering issues: there are multiple buffers in between the processes: C stdio buffers, buffers in Python file objects wrapping the pipes (controlled by bufsize).
bufsize=0 turns off the buffering on the Python side and the data is copied as soon as it is available. Beware, bufsize=0 may lead to partial writes—you might need to inline copyfileobj() and call write() again until all read data is written.
Call setvbuf(stdout, (char *) NULL, _IOLBF, 0), to make the stdout line-buffered inside your C program:
#include <stdio.h>
int main(void)
{
int a;
setvbuf(stdout, (char *) NULL, _IOLBF, 0); /* make line buffered stdout */
do {
scanf("%d",&a);
printf("%d\n",a-1);
fprintf(stderr, "%d\n",a); /* for debugging */
} while(a > 0);
return 0;
}
Output
10
9
8
7
6
5
4
3
2
1
0
-1
The output is the same.
Due to the way the C child program is written and executed, you might also need to catch and ignore BrokenPipeError exception at the end on a.stdin.write() and/or a.stdin.close() (a process may be already dead while there is uncopied data from b).
Problem is here
while True:
b.stdin.write(str(temp))
temp = b.communicate()[0]
print temp
c.stdin.write(str(temp))
temp = c.communicate()[0]
print temp
Once communicate has returned, it does noting more. You have to run the process again. Plus you don't need 2 processes open at the same time.
Plus the init phase is not different from the running phase, except that you provide the input data.
what you could do to simplify and make it work:
from subprocess import *
call(["gcc","2.c"])
temp = str(0)
while True:
b = Popen(["./a.out"],stdin=PIPE,stdout=PIPE) #The 2 processes in question
b.stdin.write(temp)
temp = b.communicate()[0]
print temp
b.wait()
Else, to see 2 processes running in parallel, proving that you can do that, just fix your loop as follows (by moving the Popen calls in the loop):
while True:
b = Popen(["./a.out"],stdin=PIPE,stdout=PIPE) #The 2 processes in question
c = Popen(["./a.out"],stdin=PIPE,stdout=PIPE)
b.stdin.write(str(temp))
temp = b.communicate()[0]
print temp
c.stdin.write(str(temp))
temp = c.communicate()[0]
print temp
better yet. b output feeds c input:
while True:
b = Popen(["./a.out"],stdin=PIPE,stdout=PIPE) #The 2 processes in question
c = Popen(["./a.out"],stdin=b.stdout,stdout=PIPE)
b.stdin.write(str(temp))
temp = c.communicate()[0]
print temp
I am working with an I2C-Device under Linux and tried to use the device interface like described under folowing Link.
So if we assume following code:
char outbuf[SIZE] = { 'e', 'b' };
struct i2c_rdwr_ioctl_data msgset;
struct i2c_msg msg[1];
msg[0].addr = 0x53; // access address 0x53
msg[0].flags = 0; // 0 means write
msg[0].len = SIZE; // size is already set to two
msg[0].buf = outbuf
msgset.msgs = msg;
msgset.nmsgs = 1;
ioctl( file, I2C_RDWR, &msgset ); // fille is already assigned, etc.
we would write one message containing two bytes to address 0x53!?
Or we could say,
S Addr Wr [A] Data [A] Data [A] P
in the way like its done here.
But when i look at my scope, i get something like this:
or a litle more detailed:
But this is not what we want and not what the specification says,
furthermore we get
S Addr Wr [A] Data P S Addr Wr [A] Data P
Does anyone know this behavior or could describe it to me?
I tried all types of calls IOCTL, SMBUS, write_block_data.
Everytime there is a new Start Condition between data-bytes and the address is also repeated!
Am I getting something wrong?
Thanks for your time and best Regards!
Befedo
I found the misalignment...
my hardware was set up like this:
Notebook -> DP/VGA -> I2C-Slave
I used an Notebook which only got an DisplayPort output, converted this via an DP-to-VGA adapter and used the I2C-Interface where a simple Slave was attached.
And it looks like the DP-to-VGA adapter only could serve with Bytewide-Access to the I2C-Bus, so I set up a 'new' Laptop which has an VGA-Interface integrated and used it directly...
Which lead to an perfectly aligned transfer, like it was expected.