python3 - easiest way to share variable from parent to child using scripts - python-3.x

Question

Using script parent.py I would like to set variable parvar and execute child.py and have parvar printed. I am having a hard time wrapping my head around the easiest way to accomplish this. It seems like I could use os.fork() as data present in the parent are presented to the child, but I cannot get it to work. Reading through examples of using multiprocessing I cannot find examples that show sharing data across two different scripts like this.
This is what I have so far:
parent.py
#!/usr/bin/env python3
import subprocess, os
parvar = 'parent var'
pid = os.fork()
if pid == 0:
print('child pid is running')
subprocess.call(['python3', 'child.py'])
exit()
child.py
#!/usr/bin/env python3
childvar = 'child var'
print('this is child var: ', childvar)
print(parvar)
Which returns a NameError:
$ ./parent.py
child pid is running
$ this is child var: child var
Traceback (most recent call last):
File "child.py", line 4, in <module>
print(parvar)
NameError: name 'parvar' is not defined
I think I understand why that is not working. The subprocess call replaces the existing process - spawning a new one. Because that was not forked, whatever I made available to my child PID is now inaccessible to that 3rd process.
Can someone help me with a simple example of getting the above to work?

Answer 1

This is not an answer (not yet at least) but figured I would post this here since it seems to move things in the right direction.
Using the mmap example from this page:
https://blog.schmichael.com/2011/05/15/sharing-python-data-between-processes-using-mmap/
I have (I believe correctly) re-factored it for Python 3 (3.6.8):
a.py
#!/usr/bin/env python3
import ctypes
import mmap
import os
import struct
def main():
# Create new empty file to back memory map on disk
fd = os.open('/tmp/mmaptest', os.O_CREAT | os.O_TRUNC | os.O_RDWR)
# Zero out the file to insure it's the right size
assert os.write(fd, b'\x00' * mmap.PAGESIZE) == mmap.PAGESIZE
# Create the mmap instace with the following params:
# fd: File descriptor which backs the mapping or -1 for anonymous mapping
# length: Must in multiples of PAGESIZE (usually 4 KB)
# flags: MAP_SHARED means other processes can share this mmap
# prot: PROT_WRITE means this process can write to this mmap
buf = mmap.mmap(fd, mmap.PAGESIZE, mmap.MAP_SHARED, mmap.PROT_WRITE)
# Now create an int in the memory mapping
i = ctypes.c_int.from_buffer(buf)
# Set a value
i.value = 10
# And manipulate it for kicks
i.value += 1
assert i.value == 11
# Before we create a new value, we need to find the offset of the next free
# memory address within the mmap
offset = struct.calcsize(i._type_)
# The offset should be uninitialized ('\x00')
assert buf[offset] == 0
# Now ceate a string containing 'foo' by first creating a c_char array
s_type = ctypes.c_char * len('foo')
# Now create the ctypes instance
s = s_type.from_buffer(buf, offset)
# And finally set it
s.raw = b'foo'
print('First 10 bytes of memory mapping: %r' % buf[:10])
input('Now run b.py and press ENTER')
print
print('Changing i')
i.value *= i.value
print('Changing s')
s.raw = b'bar'
new_i = input('Enter a new value for i: ')
i.value = int(new_i)
if __name__ == '__main__':
main()
b.py
#!/usr/bin/env python3
import mmap
import os
import struct
import time
def main():
# Open the file for reading
fd = os.open('/tmp/mmaptest', os.O_RDONLY)
# Memory map the file
buf = mmap.mmap(fd, mmap.PAGESIZE, mmap.MAP_SHARED, mmap.PROT_READ)
i = None
s = None
while 1:
new_i, = struct.unpack('i', buf[:4])
new_s, = struct.unpack('3s', buf[4:7])
if i != new_i or s != new_s:
print('i: %s => %d' % (i, new_i))
print('s: %s => %s' % (s, new_s))
print('Press Ctrl-C to exit')
i = new_i
s = new_s
time.sleep(1)
if __name__ == '__main__':
main()
Execution Example
terminal 1
$ ./a.py
First 10 bytes of memory mapping: b'\x0b\x00\x00\x00foo\x00\x00\x00'
Now run b.py and press ENTER
Changing i
Changing s
Enter a new value for i: 87
terminal 2
$ ./b.py
i: None => 11
s: None => b'foo'
Press Ctrl-C to exit
i: 11 => 121
s: b'foo' => b'bar'
Press Ctrl-C to exit
i: 121 => 87
s: b'bar' => b'bar'
Press Ctrl-C to exit