If we return value created by SWIG_NewObjectPtr without flags, it will have reference count of 1 and would never be garbage collected. In order to avoid it I have to use %newobject in .i file.
However, there are quite a few swig primitives that could be used to make the value returned into python. I suspect that some of them should be also annotated as %newobject.
Is there a list or should blindly annotate all functions returning anything into python?
Is there a way to apply %newobject globally or per-file?
For SWIG_NewPointerObj, Python will track the reference if needed. Set the own flag. If that return value isn't assigned to a variable its reference count would be decremented. %newobject isn't intended for Python objects.
%newobject is used to indicate that a function returns an C/C++ allocated object that isn't reference counted, such as return new char[50]; or return (char*)malloc(1000);. In the previous example SWIG will create a Python string by copying the data, but doesn't know to free the returned pointer without %newobject. You may also need to write a %typemap(newfree) if SWIG doesn't already have one for the type.
References:
11.3.3 Using %newobject to release memory
14.2 Object ownership and %newobject
Related
I have fallowing problem:
In Ray software, it is needed to remove all references to an object, to release memory that is used by the object. So, I need to:
def some_func():
# results is very complex object with array, lists, dictionaries
results = ray.get(hash_for_given_job)
result_copy = copy.deepcopy(results)
del results
return results_copy
Nevertheless, it causes that the object will occupy twice of memory at some point in time. Therefore increasing RAM memory usage. How to remove a reference, and return an object, without copying it ?
I strongly suspect you don't actually need to deep copy results. Objects in the ray object store are immutable. When you call ray.get() you're already creating a copy of the object, thus if you do something like
results = ray.get(hash_for_given_job)
results_copy = ray.get(hash_for_given_job)
results.a = "hello"
assert results_copy.a != results_copy.b
This would hold true regardless of whether results and results_copy are in the same function, different tasks, different machines, etc.
Is it possible to get all object/variable references in current running application with count? I don't have objects/variable names.
I tried:
locals()
globals()
gc.get_objects()
The function sys.gettotalrefcount() is available if running the debug build of Python (python_d.exe). It's useful when writing Python C extensions to verify if a function is leaking references.
As I understand it, there are three ways for a WebAssembly instance to return values to the external caller:
returning a value directly as the result of a method call (is this only a single value of the basic datatypes {i32, i64, f32, f64}?)
calling an imported function with arguments, this function can store the arguments or access the memory while it is called and save the contents somewhere else
writing to a memory and have the external environment access it after execution
Are there more? What are the details?
Your assessment is roughly correct. Functions can either return a WebAssembly value (i32, i64, f32, f64) directly or return data indirectly via modifying elements in the global environment such as Memory, Table and Globals. If you are talking about returns data to a caller outside of the WebAssembly module than any such elemennts would themselves need to to be exported from he module. i.e. you can't use the WebAssembly memory to indirectly return data unless he memory itself is also exports.
With the multi-value proposal you will also be able to directly return more than one value: https://github.com/WebAssembly/multi-value. Likewise the GC proposal will allow you to directly return complex structures. But neither of those is available today.
Your list is almost complete- the only other mechanism I can think of is global variables. These can also be exported to allow access by the host environment so could conceivably be used as a mechanism for returning results.
Yes, you are correct in your statement that you can only return a single numeric value from exported functions
I need to store a object describing the memory details of a memory block allocated by sbrk(), at the beggining of the memory block itself.
for example:
metaData det();
void* alloc = sbrk(sizeof(det)+50000);
//a code piece to locate det at the beggining of alocate.
I am not allowed to use placement new, And not allowed to allocate memory using new/malloc etc.
I know that simply assigning it to the memory block would cause undefined behaviour.
I was thinking about using memcpy (i think that can cause problems as det is not dynamicly allocated).
Could assigning a pointer to the object at the beginning work (only if theres no other choise), or memcpy?
thanks.
I am not allowed to use placement new
Placement new is the only way to place an object in an existing block of memory.
[intro.object] An object is created by a definition, by a new-expression, when implicitly changing the active member of a union, or when a temporary object is created
You cannot make a definition to refer to an existing memory region, so that's out.
There's no union, so that's also out.
A temporary object cannot be created in an existing block of memory, so that's also out.
The only remaining way is with a new expression, and of those, only placement new can refer to an existing block of memory.
So you are out of luck as far as the C++ standard goes.
However the same problem exists with malloc. There are tons of code out there that use malloc without bothering to use placement new. Such code just casts the result of malloc to the target type and proceeds from there. This method works in practice, and there is no sign of it being ever broken.
metaData *det = static_cast<metaData *>(alloc);
On an unrelated note, metaData det(); declares a function, and sizeof is not applicable to functions.
I have an array with [ManuallyDrop<Box<T>>] which is filled lazily. To realize this, I "initialize" the array with ManuallyDrop::new(mem::uninitialized()).
Is this well-defined behavior as long as I only call ManuallyDrop::drop() on initialized elements?
Provided that you do not read from uninitialized memory or create pointers to it, then this should not be UB.
You will need to do some careful bookkeeping to disallow access to uninitialized items, and only drop initialized ones. Adding a new item where there is uninitialized memory needs to be done with ptr::write(), to avoid an invalid drop on the underlying memory. But if you overwrite an existing valid value, then you should not use ptr::write because you need that value to be correctly dropped.