The following is C++/CLI code gets compiled into a DLL, and is called by a C# application:
void Foo(String^ strManaged) {
marshal_context^ context = gcnew marshal_context();
FooUnmanaged(context->marshal_as<const char*>(strManaged));
}
FooUnmanaged() reads the const char*, runs some processing which takes about a second, and then reads the const char* again, for example:
void FooUnmanaged(const char* str) {
// 1
Log(str);
// Process things unrelated to 'str'
// ...
// 2
Log(str);
}
On occasions, the contents of str changes between the first and the second read inside FooUnmanaged(), as if that memory had been reused for some other purposes. This happens regardless of the processing done in FooUnmanaged(), as long as it takes a noticeable amount of time (I guess, long enough that the GC has a chance to trigger).
This does not happen if Foo is written either this way
void Foo(String^ strManaged) {
marshal_context^ context = gcnew marshal_context();
FooUnmanaged(context->marshal_as<const char*>(strManaged));
delete context; // addded
}
or that way
void Foo(String^ strManaged) {
marshal_context context; // created on the stack
FooUnmanaged(context.marshal_as<const char*>(strManaged));
}
Is the original code incorrect? Why does it not correctly reserve the memory of the const char* for the lifetime of context? Or can the lifetime of context be shorter than I think it is (the scope of Foo())?
Answer by #HansPassant:
Yes, that's a lifetime issue. .NET uses an aggressive collector, it has no idea that the native code relies on the context. The first snippet requires GC::KeepAlive(context); at the end. The last snippet is how it was meant to be used, stack semantics emulates RAII, the auto-generated Dispose() call keeps it alive in similar fashion. And avoids the temporary memory leak. If FooUnmanaged() stores the passed pointer then you can't use marshal_context.
This is confirmed by this article:
The lifetime of [local variables] can depend on the way the program was built. In debug builds, a local variable lasts for as long as the method is on the stack. In release builds, the JIT is able to look at the program structure to work out the last point within the execution that a variable can be used by the method and will discard it when it is no longer required.
Related
I've found some evil bug in my user-level threads library.
My scheduler is actually a singleton class that initializes a signal timer this way:
sigAlarm_ is a member field of the scheduler, and its of type struct sigaction.
This is the related part of the scheduler initialization:
sigAlarm_.sa_handler = timerHandlerGlobal; // Assign the first field of sigAlarm (sa_handler) as needed, others zeroed
if (sigaction(SIGVTALRM, &sigAlarm_, nullptr) != 0) { uthreadSystemError("sigaction"); }
Now, this timerHandlerGlobal is a static function, and not a member function of the scheduler, as C++ doesn't permit passing function members this way.
Now, when I terminate the main thread of the library (which actually runs the scheduler), I'm invoking std::exit(1) which cleans the resources up.
When I'm running my tests with ASan (Address Sanitizer), in some executions, it gets into the timerHandlerGlobal while the scheduler is already nullptr!
Now, I've been already two days on that, inspecting what's the cause.
Now I see that if I'm adding this ugly condition, no problem appears with ASAN:
void timerHandlerGlobal(int signo)
{
if (scheduler_manager)
{
scheduler_manager->timerHandler(signo);
}
}
But, why is after std::exit(1) invoked by the scheduler, the sigaction.sa_handler (which is timerHandlerGlobal), is still running?
Please tell me you know why it is, I just want to omit this awful condition.
I'm writing a function that needs to use mutable static variables to work (a weird implementation of a message loop). In order to allow only one writer to access those variables at any given time, I need to make the access to this function exclusive (to the first thread that accesses it).
Using AtomicBool
My first guess was to use an AtomicBool:
use std::sync::atomic::{Ordering, AtomicBool};
static FLAG: AtomicBool = AtomicBool::new(false);
fn my_exclusive_function() {
if FLAG.load(Ordering::SeqCst) {
panic!("Haha, too late!")
}
FLAG.store(true, Ordering::SeqCst);
/* Do stuff */
}
This code has an obvious flaw: if two threads happen to read FLAG at the same time, they would both think that this is ok for them to continue.
Using Mutex<()>
Then I thought about using a Mutex<()> for its lock.
use std::sync::Mutex;
static FLAG: Mutex<()> = Mutex::new(());
fn my_exclusive_function() {
let _lock = FLAG.try_lock() {
Ok(lock) => lock,
Err(_) => panic!("Haha, too late!"),
};
/* Do stuff */
// `_lock` gets dropped here, another thread can now access the function
}
There are two problems here:
Mutex::new is not a const function which makes me unable to initialize the Mutex. I could use a library like lazy_static here but if I can avoid an additional dependency, it would be great.
Even if I use lazy_static, a user-defined function (the handler inside of the message loop actually) could panic and poison the mutex. Here again, I could use a thread-party library like parking_lot but that would be yet anther additional dependency.
I'm willing to use those if there is no alternatives but if I can avoid it, that is better.
What would the right way to do this look like? Is it just a matter of Ordering?
Related question that did not help me
How to ensure a portion of code is run by just one thread at a time? Answers are specifically focused on C# and .NET and involve features I don't have in Rust.
Shouldn't finalize() execute immediately when gc() is called? The order of output result is a little unconvincing.
class Test
{
int x = 100;
int y = 115;
protected void finalize()
{System.out.println("Resource Deallocation is completed");}
}
class DelObj
{
public static void main(String arg[])
{
Test t1 = new Test();
System.out.println("Values are "+t1.x+", "+t1.y+"\nObject refered by t1 is at location: "+t1);
t1 = null; // dereferencing
System.gc(); // explicitly calling
Test t2= new Test();
System.out.println("Values are "+t2.x+", "+t2.y+"\nObject refered by t2 is at location: "+t2);
}
}
Got the execution result of finalize() after a new object is created, referred by t2:
D:\JavaEx>java DelObj
Values are 100, 115
Object refered by t1 is at location: Test#6bbc4459
Values are 100, 115
Object refered by t2 is at location: Test#2a9931f5
Resource Deallocation is completed
Calling System.gc() only provides a hint to the JVM, but does not guaranty that an actual garbage collection will happen.
However, the bigger problem with your expectation is that garbage collection is not the same as finalization.
Referring to the Java 6 documentation, System.gc() states:
Runs the garbage collector.
Calling the gc method suggests that the Java Virtual Machine expend effort toward recycling unused objects in order to make the memory they currently occupy available for quick reuse. …
Compare to System.runFinalization():
Runs the finalization methods of any objects pending finalization.
Calling this method suggests that the Java Virtual Machine expend effort toward running the finalize methods of objects that have been found to be discarded but whose finalize methods have not yet been run. …
So there can be “pending finalization”, resp. “objects that have been found to be discarded but whose finalize methods have not yet been run”.
Unfortunately, Java 6’s documentation of finalize() starts with the misleading sentence:
Called by the garbage collector on an object when garbage collection determines that there are no more references to the object.
whereas garbage collection and finalization are two different things, hence, the finalize() method is not called by the garbage collector. But note that the subsequent documentation says:
The Java programming language does not guarantee which thread will invoke the finalize method for any given object.
So when you say “The order of output result is a little unconvincing”, recall that we’re talking about multi-threading here, so in absence of additional synchronization, the order is outside your control.
The Java Language Specification even says:
The Java programming language does not specify how soon a finalizer will be invoked, except to say that it will happen before the storage for the object is reused.
and later on
The Java programming language imposes no ordering on finalize method calls. Finalizers may be called in any order, or even concurrently.
In practice, the garbage collector will only enqueue objects needing finalization, while one or more finalizer threads poll the queue and execute the finalize() methods. When all finalizer threads are busy executing particular finalize() methods, the queue of objects needing finalization may grow arbitrary long.
Note that modern JVMs contain an optimization for those classes not having a dedicated finalize() method, i.e. inherit the method from Object or just have an empty method. Instances of these classes, the majority of all objects, skip this finalization step and their space gets reclaimed immediately.
So if you added a finalize() method just to find out when the object gets garbage collected, it’s the very presence of that finalize() method which slows down the process of the memory reclamation.
So better refer to the JDK 11 version of finalize():
Deprecated.
The finalization mechanism is inherently problematic. Finalization can lead to performance issues, deadlocks, and hangs. Errors in finalizers can lead to resource leaks; there is no way to cancel finalization if it is no longer necessary; and no ordering is specified among calls to finalize methods of different objects. Furthermore, there are no guarantees regarding the timing of finalization. The finalize method might be called on a finalizable object only after an indefinite delay, if at all. Classes whose instances hold non-heap resources should provide a method to enable explicit release of those resources, and they should also implement AutoCloseable if appropriate. The Cleaner and PhantomReference provide more flexible and efficient ways to release resources when an object becomes unreachable.
So when your object does not contain a non-memory resource, hence, doesn’t actually need finalization, you can use
class Test
{
int x = 100;
int y = 115;
}
class DelObj
{
public static void main(String[] arg)
{
Test t1 = new Test();
System.out.println("Values are "+t1.x+", "+t1.y+"\nObject refered by t1 is at location: "+t1);
WeakReference<Test> ref = new WeakReference<Test>(t1);
t1 = null; // dereferencing
System.gc(); // explicitly calling
if(ref.get() == null) System.out.println("Object deallocation is completed");
else System.out.println("Not collected");
Test t2= new Test();
System.out.println("Values are "+t2.x+", "+t2.y+"\nObject refered by t2 is at location: "+t2);
}
}
The System.gc() call still is only a hint, but you will find your object being collected afterwards in most practical cases. Note that the hash code printed for the objects, like with Test#67f1fba0 has nothing to do with memory locations; that’s a tenacious myth. The patterns behind object memory addresses is often unsuitable for hashing, further most modern JVMs can move objects to different memory locations during their lifetime, whereas the identity hash code is guaranteed to stay the same.
I have a multithreaded app that has to read some data often, and occasionally that data is updated. I have problems with writing by using unique_lock and problems with reading by using upgrade_lock
There is examples of my problems:
void unlock(){
test.stream = 0;
test.mtx.unlock();
}
void lock_mtx(int i){
boost::unique_lock<boost::shared_mutex> lock(test.mtx);
test.stream = i;
boost::this_thread::sleep_for(boost::chrono::milliseconds(10000));
unlock();
boost::this_thread::sleep_for(boost::chrono::milliseconds(1000));
}
When I destruct lock , mutex is already unlocked by this thread, and sometimes it is locked by another thread, but destructor make it free again. After destruction of lock (in the first thread) third thread take mutex and I have two writers at the one moment
void lock_mtx(int i){
boost::upgrade_lock<boost::shared_mutex> lock(test.mtx);
read_from_locked();
boost::this_thread::sleep_for(boost::chrono::milliseconds(5000));
boost::upgrade_to_unique_lock<boost::shared_mutex> uniqueLock(lock);
write_to_locked();
boost::this_thread::sleep_for(boost::chrono::milliseconds(10000));
}
The second problem, is that when some thread takes upgrade_lock, other threads can't read shared objects
Both of problems occur in MS VisualStudio 2013 and Windows8 x64
void lock_mtx(int i)
{
{
boost::unique_lock<boost::shared_mutex> lock(test.mtx);
test.stream = i;
boost::this_thread::sleep_for(boost::chrono::milliseconds(10000));
}
boost::this_thread::sleep_for(boost::chrono::milliseconds(1000));
}
The purpose of unique_lock and lock_guard is to automatically unlock the mutex when it goes out of scope. This pattern is known as RAII.
Rule of thumb: Never manually call lock()/unlock() on your (Basic|Shared)Lockable objects. It's an antipattern because
It's extremely hard to get right (think of exception safety)
It usually indicates a code smell (locks being held across different method calls). If you even need this, consider making the RAII lock guard (lock_guard or unique_lock) a member of the containing class, or return the unique_lock so the caller has the option to explicitly adopt the lock, or to just let it be automatically released by the guard.
I am reading the concept of language ch 5 and find that
static's disadvantage : subprograms cannot share same storage.
stack-dynamic's advantage : without recursion, it is not without merit subprograms can share the same memory space for theirs locals.
and I think that due to static bound from begin to terminal, all subprogram should can see it and use
it
like the code I test
#include<iostream>
static int test = 0;
void func1(){ cout << test++ << endl;}
void func2(){ cout << test++ << endl;}
int main(){
func1();
func2();
}
and stack-dynamic allocation each time execute the function, like being push to stack(LIFO), so they
are in different momory space.
I don't know where is the error in my thought?
Thx in advance.
You program runs in a dedicated memory space.
Static variables have global scope.
Given that, I suppose "Subprograms cannot share same storage" means that static variable is instantiated only once and it's the same object during the entire lifetime of the program.
This has several consequences:
If your independent functions need storage as part of their execution, they shouldn't address the same static variables because they will affect the other functions using the same variables.
If your functions can run in parallel (e.g. on several processors) and they address same static variables, these variable become a shared resource, meaning it must be protected against parallel access (which can corrupt the data and result in incorrect behavior).
Dynamically allocated/stack variables use the same memory space in which the program runs, but they can be instantiated many times, and the instances are independent of each other. E.g., if stack variable is defined inside a function, it's allocated upon function entry and released upon function exit. If the function is entered again, new variable will be created. It's same memory space, but different stack frame in this space.