I'm trying to get rid of object pinned in shared memory using ray.put.
Here is code sample:
import ray
<create obj>
for ...:
obj_id = ray.put(obj)
<do stuff with obj_id on ray Actors using ray.get(obj_id)>
del obj_id
After this is finished, I look at ray dashboard and see that all obj_id are still in ray shared memory with reference type LOCAL_REFERENCE.
Official docs do not elaborate on whether there is any way of explicitly controlling object lifetime. As far as I understood, it basically suggests to wait until all memory is used, and then rely on ray to clean things up.
Question: how do I explicitly purge object from ray shared memory?
Note: I'm using Jupyter, can it be the case that this object is still alive due to this fact?
The function ray.internal.internal_api.free() performs this function. I can't find any documentation on the Ray docs for this function but it has a good docstring you can find here which I've copy-pasted below.
Free a list of IDs from the in-process and plasma object stores.
This function is a low-level API which should be used in restricted
scenarios.
If local_only is false, the request will be send to all object stores.
This method will not return any value to indicate whether the deletion is
successful or not. This function is an instruction to the object store. If
some of the objects are in use, the object stores will delete them later
when the ref count is down to 0.
Examples:
>>> x_id = f.remote()
>>> ray.get(x_id) # wait for x to be created first
>>> free([x_id]) # unpin & delete x globally
Args:
object_refs (List[ObjectRef]): List of object refs to delete.
local_only (bool): Whether only deleting the list of objects in local
object store or all object stores.
Related
I have fallowing problem:
In Ray software, it is needed to remove all references to an object, to release memory that is used by the object. So, I need to:
def some_func():
# results is very complex object with array, lists, dictionaries
results = ray.get(hash_for_given_job)
result_copy = copy.deepcopy(results)
del results
return results_copy
Nevertheless, it causes that the object will occupy twice of memory at some point in time. Therefore increasing RAM memory usage. How to remove a reference, and return an object, without copying it ?
I strongly suspect you don't actually need to deep copy results. Objects in the ray object store are immutable. When you call ray.get() you're already creating a copy of the object, thus if you do something like
results = ray.get(hash_for_given_job)
results_copy = ray.get(hash_for_given_job)
results.a = "hello"
assert results_copy.a != results_copy.b
This would hold true regardless of whether results and results_copy are in the same function, different tasks, different machines, etc.
I have an object:
class Flow:
this object belongs to list of objects flowList = [Flow1, Flow2, Flow3]
If I do:
del flowList[0]
Do I need to destroy as well Flow1 object? I want my script to process thousands of flows and be memory efficient. Processed flow should die after processing as it won't be needed anymore.
No, this is not required, and there are no reliable ways that I know of of explicitly freeing certain memory anyways.
Python uses Garbage Collection, which means it automatically disposes of objects when they are no longer needed. Once you're done with an object, you can simply leave it. If the only reference to Flow1 was the one inside the list, del flowList[0] is sufficient to allow it to be freed. If there's another reference outside of the list, you can do del Other_Flow1 to delete the other reference to allow it to be freed.
If you have something like this:
huge_list = [1, 2, 3, . . .]
use_huge_list(huge_list)
You could add something like this after the call to use_huge_list to help free huge_list more readily:
del huge_list
This deletes the reference huge_list to the [1, 2, 3, . . .] object. If that's the only reference to the object, it will allow it to be freed. I'd only do this though if memory is a huge concern, and huge_list will remain in scope for an extended period of time, but is never used (although, that may be a smell that suggests that you need to refactor anyway).
I would not use del like that constantly though. It's unnecessary to delete labels in 99% of cases. Don't overthink it. Unless you've profiled and know for sure that objects staying in memory are causing you problems, don't worry about it.
I've read the https://learn.microsoft.com/en-us/windows/desktop/direct3d11/overviews-direct3d-11-render-multi-thread-intro
And it states that I can make calls to ID3D11Device from multiple threads (unless D3D11_CREATE_DEVICE_SINGLETHREADED was used), but calls to ID3D11DeviceContext have to be surrounded with a critical section.
I haven't found any information about releasing resources, using their 'Release' method, for resources such as textures, render targets, vertex/index buffers, shaders.
ID3D11Texture2D, ID3D11Texture3D, ID3D11ShaderResourceView, ID3D11RenderTargetView, ID3D11DepthStencilView
ID3D11Buffer.
ID3D11VertexShader, ID3D11HullShader, ID3D11DomainShader, ID3D11PixelShader.
1) Can I call 'Release' for those resources at any time from any thread without using critical sections while they ARE NOT in use by the render thread's ID3D11DeviceContext?
2) Can I call 'Release' for those resources from other threads even while they ARE in use by ID3D11DeviceContext in the render thread?
Or do I need to surround the Release calls with the same critical section used for accessing ID3D11DeviceContext?
Generally the internal implementation of COM reference counts is done in a thread-safe manner (atomic increments/decrements), so it's safe to call AddRef and Release from multiple threads.
Of course, if the refcount goes to 0 then you have an object destruction so it's important that if you have multiple threads using the same resource, it has the appropriate number of reference counts to keep it live. In Direct3D, object destruction is typically deferred destruction so the actual object cleanup may not happen for a few frames, but you should still keep a non-zero refcount if anyone is referencing it.
Direct3D 11 uses the same rules as Direct3D 10. It uses 'weak references' for the pipeline set methods, so just having a resource set on the device context is not sufficient to increase it's reference count. IOW: if you have two threads both rendering with the same resource, then each thread must hold a reference count on the object to keep it 'live' whether or not it's 'actively set' on a device context at any given moment.
It works this way to avoid the overhead of constantly increment/decrementing reference counts every rendering frame. In Direct3D 9 this was happening thousands of times a frame or more.
Also, if the ID3D11Device reaches a zero ref-count, it and all it's child objects are released regardless of the individual device-child reference counts.
See Microsoft Docs.
The best answer is to use a smart-pointer like Microsoft::WRL::ComPtr and have each thread using a given resource have it's own ComPtr pointing to that resource. That way the only real special-case you'll have is when doing device tear-down (such as responding to a DXGI_ERROR_DEVICE_REMOVED or doing a 'clean exit').
Java docs state following regarding synchronization of constructor:
Note that constructors cannot be synchronized — using the synchronized keyword with a constructor is a syntax error. Synchronizing constructors doesn't make sense, because only the thread that creates an object should have access to it while it is being constructed.
Warning: When constructing an object that will be shared between
threads, be very careful that a reference to the object does not
"leak" prematurely. For example, suppose you want to maintain a List
called instances containing every instance of class. You might be
tempted to add the following line to your constructor:
instances.add(this); But then other threads can use instances to
access the object before construction of the object is complete.
I am not able to understand this whole block. First it states that only the thread that creates an object has access to constructor. Then it warns of premature leak which may cause issues if other threads access the object before construction is complete. Are not these two things in contradiction. If only the creating thread can access the constructor then how can other threads prematurely access the object as it can only be accessed once contructor has run fully?
Any input would be of great help.
Imagine two threads that both have access to a global List (called "instances") holding instances of the class in question. Thread 1 continuously cycles through the list and does something with each instance. Thread 2 goes its own merry way, and occasionally constructs a new instance of the class. If the class would add itself to the List in its constructor (using instances.add(this)) Thread 1 would immediately get access to the instance and could do things with it before it is fully constructed, resulting in unpredictable behavior.
There may be a misunderstanding of the word "should". You wrote: "First it states that only the thread that creates an object has access to constructor. " However, the Java docs say: "only the thread that creates an object should have access to it while it is being constructed", which means that you should take care that only one thread has access to the object while it is being constructed.
A portable thread specific storage reference/identity mechanism, of which boost/thread/tss.hpp is an instance, needs a way to generate a unique keys for itself. This key is unique in the scope of a thread, and is subsequently used to retrieve the object it references. This mechanism is used in code written in a thread neutral manner.
Since boost is a portable example of this concept, how specifically does such a mechanism work ?
Boost thread is portable to the pthread threading library (for unix) and the windows win32 low-level-API's. The library allows a reference to be created which is unique in each thread of execution. The global C API errno is presented as an example of this concept in Boost's documentation.
Ignore If you Want -- it's just a trace through the source code finding the function of interest
The crux of the matter begins in [boost]/boost/thread/tss.hpp with the get function of thread_specific_ptr and the reset function -- i.e., the aquisition and the destruction, respectively, of the object referenced. Note: the data object is not placed in the reference of thread_specific_ptr's ctor, or destroyed by the dtor. The get and reset function call set_tss_data and get_tss_data. Focusing just on the setting aspect of the functionality, the important function call, get_current_thread_data, indirects via the cpp file [boost]/libs/thread/src/[libname]/thread.cpp via a chain of function calls. In get_current_thread_data there is a function call create_current_thread_tls_key and this is the function that will create a unique identifier for the thread_specific_ptr object.
create_current_thread_tls_key calls TlsAlloc() on win32 (link) and pthread_key_create for pthread (link). These calls assure that upon initialization of the ptr, the ptr receives a unique identifier usable in an API-specific manner to retrieve the object's data. The specific threading API uses the thread-id (context specific and resolved by the library itself) and the object identifier to return the object specific to the context of a certain thread.