I am finding it difficult to write something with more or less common UI at least for Mac. My application has to have tray icon and be able to show system notifications
The issue is the goroutines themselves. Any call to UI frameworks on Mac requires that the call is made from main thread, or at least in a thread-safe manner.
The issue arise when I am already running UI (well, for GUI application that is a must, no?) and try to show notification. The reason for this seems to be that systray package Init function has to be locked to main thread using runtime.LockOsThread and never releases it. Then if I try to show notification which also requires runtime.LockOsThread it causes following error:
2016-01-11 22:56:27.973 main[30162:4094392] *** Assertion failure in +[NSUndoManager _endTopLevelGroupings], /Library/Caches/com.apple.xbs/Sources/Foundation/Foundation-1256.1/Misc.subproj/NSUndoManager.m:359
2016-01-11 22:56:27.974 main[30162:4094392] +[NSUndoManager(NSInternal) _endTopLevelGroupings] is only safe to invoke on the main thread.
2016-01-11 22:56:27.977 main[30162:4094392] (
0 CoreFoundation 0x00007fff8d42bae2 __exceptionPreprocess + 178
1 libobjc.A.dylib 0x00007fff8bb03f7e objc_exception_throw + 48
2 CoreFoundation 0x00007fff8d42b8ba +[NSException raise:format:arguments:] + 106
3 Foundation 0x00007fff8cb4c88c -[NSAssertionHandler handleFailureInMethod:object:file:lineNumber:description:] + 198
4 Foundation 0x00007fff8cad24c1 +[NSUndoManager(NSPrivate) _endTopLevelGroupings] + 170
5 AppKit 0x00007fff8514206a -[NSApplication run] + 844
6 main 0x0000000004166200 nativeLoop + 128
7 main 0x0000000004165bca _cgo_8c6479959095_Cfunc_nativeLoop + 26
8 main 0x000000000405a590 runtime.asmcgocall + 112
)
2016-01-11 22:56:27.977 main[30162:4094392] *** Assertion failure in +[NSUndoManager _endTopLevelGroupings], /Library/Caches/com.apple.xbs/Sources/Foundation/Foundation-1256.1/Misc.subproj/NSUndoManager.m:359
2016-01-11 22:56:27.978 main[30162:4094392] An uncaught exception was raised
2016-01-11 22:56:27.978 main[30162:4094392] +[NSUndoManager(NSInternal) _endTopLevelGroupings] is only safe to invoke on the main thread.
Is there a workaround that? All I could think of so far is to put UI and Notifications into separate binaries and make them to communicate with main over some sort of IPC. But I may be missing something.
Since there is not enough traction on this question I've decided to post my own solution I found while trying to workaround this issue. This won't be marked as answer yet since someone else may provide better solution.
I have moved one of the UI processes (namely part that uses systray) into another binary and call it using cmd := exec.Command(...) and cmd.Start() then I pipe stdin and stdout and communicate with this child process through those.
The example code can be found on Github. Warning there is an error in this gist where after child exits main process will start burning through CPU cycles. Feel free to fix it yourself.
The reason why I did not want to go through with RPC is because this will become slightly too complex to what I want to achieve and does not provide easy way to do two way communication.
It looks like the two libraries that you are using both correctly use runtime.LockOSThread to make main-thread-only API calls; unfortunately, to use more than one such library, you'll have to do something fancier than the example code that either provides. You'll need to write your own main thread / main.Main-invoked message loop that handles calls to multiple MTO APIs.
runtime.LockOSThread is part of the solution to operating with APIs such as this; the golang wiki has a page about how to use it to interact with "call from main thread only" APIs.
An extremely short description of how your program should change:
You'll want to use runtime.LockOSThread in main.init to make sure that the main thread is running main.Main; main.Main should be refactored into two parts:
starts a goroutine or goroutines that run what previously was in main.Main;
enters a message loop receiving messages to take certain main-thread actions on one or more channels
Related
I use FireDAC mostly from background threads with occasional use from the main GUI thread.
According to the docs, there are 4 possible execution modes, from which only 2 seem to be appropriate in my case:
amBlocking The calling thread and GUI are blocked until an action is finished.
This one is the default and seem to be working fine. Although I'm not happy that it says "and GUI are blocked" - I don't need to block the GUI while background thread does the job. This sort of defeats the purpose of the threads. It can also lead to non-obvious deadlocks.
amNonBlocking The calling thread is blocked until an action is finished. The GUI is not blocked.
This mode seems more appropriate, as it blocks only the calling thread, and this appears to be what I need. But in practice, it's not. From time to time the call fails with exception
EFDException [FireDAC][Phys][SQLite]-326. Cannot perform the action, because the previous action is in progress.
All calls to FireDAC across all threads are serialized with a critical section - according to docs this is enough for thread safe operation of FireDAC. All datasets/queries are closed before critical section is released.
The main question is, what execution mode to choose?
Sub-question is - is it a bug in FireDAC and amNonBlocking is broken?
In case anyone has the same problem, the solution is as follows, before any operation with a database, call:
if GetCurrentThreadId = MainThreadID then
FDConn.ResourceOptions.CmdExecMode := amBlocking
else
FDConn.ResourceOptions.CmdExecMode := amNonBlocking;
I have a multithreaded program that needs to run many executables at once and wait for their results.
I use [nstask waitUntilExit] in an NSOperationQueue that runs it on non-main thread (running NSTask on the main thread is completely out of the question).
My program randomly crashes or runs into assertion failures, and the crash stacks always point to the runloop run by waitUntilExit, which executes various callbacks and handlers, including—IMHO incorrectly—KVO and bindings updating the UI, which causes them to run on non-main thread (It's probably the problem described by Mike Ash)
How can I safely use waitUntilExit?
Is it a problem of waitUntilExit being essentially unusable, or do I need to do something special (apart from explicitly scheduling my callbacks on the main thread) when using KVO and IB bindings to prevent them from being handled on a wrong thread running waitUntilExit?
As Mike Ash points out, you just can't call waitUntilExit on a random runloop. It's convenient, but it doesn't work. You have to include "doesn't work" in your computation of "is this actually convenient?"
You can, however, use terminationHandler in 10.7+. It does not pump the runloop, so shouldn't create this problem. You can recreate waitUntilExit with something along these lines (untested; probably doesn't compile):
dispatch_group group = dispatch_group_create();
dispatch_group_enter(group);
task.terminationHandler = ^{ dispatch_group_leave(group); };
[task launch];
dispatch_group_wait(group, DISPATCH_TIME_FOREVER);
// If not using ARC:
dispatch_release(group);
Hard to say without general context of what are you doing...
In general you can't update interface from the non main threads. So if you observe some KVO notifications of NSTasks in non main thread and update UI then you are wrong.
In that case you can fix situation by simple
-[NSObject performSelectorOnMainThread:];
or similar when you want to update UI.
But as for me more grace solution:
write separated NSOperationQueue with maxConcurentOperationsCount = 1 (so FIFO queue) and write subclass of NSOperation which will execute NSTask and update UI through delegate methods. In that way you will control amount of executing tasks in application. (or you may stop all of them or else)
But high level solution for your problem I think will be writing privileged helper tool. Using this approach you will get 2 main benefits: your NSTask's will be executes in separated process and you will have root privilegies for executing your tasks.
I hope my answer covers your problem.
I am writing a little program,the program execute 4 threads which will execute funcA(),I use for event objects and WaitForMultipleObject() to make the main thread block until 4 threads is over their task,it is easy to kown I will use setEvent() in funA(),I am wonder what should I do if SetEvent() is execute failed,the main will always block?Is thier any way to avoid this?
P.S:I only know SetEvent will failed when the HANDLE is invalid,is their any more reason for its failed?
In C++ you can throw a C++ exception to intentionally crash the process.
In C you can crash the process nicely using the Windows API function RaiseException to raise a Windows structured exception.
Apple's multithreading docs don't list NSIndexPath as threadsafe or not! As an immutable class, I'd generally expect it to be threadsafe.
Previously, I'm sure the documentation used to state that NSIndexPath instances were shared and globally unique. That seems to have disappeared now though, leading me to suspect that design was revised for iOS5 / Mac OS X 10.7.
I'm seeing quite a lot of crash reports from customers on Mac OS X 10.6 (Snow Leopard) which appear to be crashing trying to access an index path. Thus I wonder: are the actual instances thread safe, but that the logic for pulling them out of the shared cache isn't? Does anybody have any insight?
Here's an example stack trace BTW:
Dispatch queue: com.apple.root.default-priority
0 libobjc.A.dylib 0x96513f29 _cache_getImp + 9
1 libobjc.A.dylib 0x965158f0 class_respondsToSelector + 59
2 com.apple.CoreFoundation 0x948bcb49 ___forwarding___ + 761
3 com.apple.CoreFoundation 0x948bc7d2 _CF_forwarding_prep_0 + 50
4 com.apple.Foundation 0x994b10c5 -[NSIndexPath compare:] + 93
5 com.apple.Foundation 0x99415686 _NSCompareObject + 76
6 com.apple.CoreFoundation 0x948af61c __CFSimpleMergeSort + 236
7 com.apple.CoreFoundation 0x948af576 __CFSimpleMergeSort + 70
8 com.apple.CoreFoundation 0x948af38c CFSortIndexes + 252
9 com.apple.CoreFoundation 0x948fe80d CFMergeSortArray + 125
10 com.apple.Foundation 0x994153d3 _sortedObjectsUsingDescriptors + 639
11 com.apple.Foundation 0x994150d8 -[NSArray(NSKeyValueSorting) sortedArrayUsingDescriptors:] + 566
To me, that is an NSIndexPath instance trying to compare itself to a deallocated instance.
So far the best answer I have is as I suspect:
As of OS X 10.7 and iOS 5, NSIndexPath is thread safe. Prior to that, instances are thread safe because they are immutable, but the shared retrieval of existing instances is not.
To my method which returns index paths on-demand, I did this:
- (NSIndexPath *)indexPath;
{
NSIndexPath *result = … // create the path as appropriate
return [[result retain] autorelease];
}
Since implementing that last line of code, we've had no more crash reports from index paths.
The index paths are created by -indexPathByAddingIndex: or +indexPathWithIndex:.
The results I am seeing make me pretty certain that (prior to 10.7/iOS5) these methods are returning an existing NSIndexPath instance. That instance is not retained by the current thread in any way though, so the thread which first created the instance (main in our case) is releasing the path (likely through popping the autorelease pool) and leaving our worker thread with a dangling pointer, which crashes when used, as seen in the question.
It's all a bit terrifying, because if my analysis is correct, the retain/autorelease dance I've added is simply replacing one race condition with another, less-likely one.
Prior to 10.7/iOS5, I can think of only one true workaround: Limit all creation of index paths to the main thread. That could be rather slow if such code gets called a lot, so could be improved — at the cost of memory — by maintaining some kind of instance cache of your own for background threads to use. If the cache is retaining a path, then you know it won't be deallocated by the main thread.
Apple don't specifically list NSIndexPath as thread safe, but they do say that immutable classes are generally safe and mutable ones generally aren't. Since NSIndexPath is immutable it's safe to assume it's thread safe.
But "thread safe" doesn't mean that it can't cause crashes by being released by one thread before you use it on another though. Thread safe generally just means that its mutator methods contain locking to prevent glitches due to two threads setting properties concurrently (which is why classes without mutator methods are generally thread safe, although lazy getters and shared instances can also cause problems).
It sounds like your bug is more likely due to using an autorelease pool or some other mechanism that causes your object to be released at a time outside your control. You should probably ensure that any concurrently accessed objects are stored in properties of long-lived classes so that you can control their lifespan.
Creating an autoreleased object and accessing it from another thread after you've removed all strong references to it is a dangerous racing game that is likely to cause hard-to-trace crashes regardless of whether the object in question is "thread safe".
We have an application that is undergoing performance testing. Today, I decided to take a dump of w3wp & load it in windbg to see what is going on underneath the covers. Imagine my surprise when I ran !threads and saw that there are 640 background threads, almost all of which seem to say the following:
OS Thread Id: 0x1c38 (651)
Child-SP RetAddr Call Site
0000000023a9d290 000007ff002320e2 Microsoft.Practices.EnterpriseLibrary.Caching.ProducerConsumerQueue.WaitUntilInterrupted()
0000000023a9d2d0 000007ff00231f7e Microsoft.Practices.EnterpriseLibrary.Caching.ProducerConsumerQueue.Dequeue()
0000000023a9d330 000007fef727c978 Microsoft.Practices.EnterpriseLibrary.Caching.BackgroundScheduler.QueueReader()
0000000023a9d380 000007fef9001552 System.Threading.ExecutionContext.runTryCode(System.Object)
0000000023a9dc30 000007fef72f95fd System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
0000000023a9dc80 000007fef9001552 System.Threading.ThreadHelper.ThreadStart()
If i had to give a guess, I'm thinkign that one of these threads are getting spawned for each run of our app - we have 2 app servers, 20 concurrent users, and ran the test approximately 30 times...it's in the neighborhood.
Is this 'expected behavior', or perhaps have we implemented something improperly? The test ran hours ago, so i would have expected any timeouts to have occurred already.
Edit: Thank you all for your replies. It has been requested that more detail be shown about the callstack - here is the output of !mk from sosex.dll.
ESP RetAddr
00:U 0000000023a9cb38 00000000775f72ca ntdll!ZwWaitForMultipleObjects+0xa
01:U 0000000023a9cb40 00000000773cbc03 kernel32!WaitForMultipleObjectsEx+0x10b
02:U 0000000023a9cc50 000007fef8f5f595 mscorwks!WaitForMultipleObjectsEx_SO_TOLERANT+0xc1
03:U 0000000023a9ccf0 000007fef8f59f49 mscorwks!Thread::DoAppropriateAptStateWait+0x41
04:U 0000000023a9cd50 000007fef8e55b99 mscorwks!Thread::DoAppropriateWaitWorker+0x191
05:U 0000000023a9ce50 000007fef8e2efe8 mscorwks!Thread::DoAppropriateWait+0x5c
06:U 0000000023a9cec0 000007fef8f0dc7a mscorwks!CLREvent::WaitEx+0xbe
07:U 0000000023a9cf70 000007fef8fba72e mscorwks!Thread::Block+0x1e
08:U 0000000023a9cfa0 000007fef8e1996d mscorwks!SyncBlock::Wait+0x195
09:U 0000000023a9d0c0 000007fef9463d3f mscorwks!ObjectNative::WaitTimeout+0x12f
0a:M 0000000023a9d290 000007ff002321b3 *** ERROR: Module load completed but symbols could not be loaded for Microsoft.Practices.EnterpriseLibrary.Caching.DLL
Microsoft.Practices.EnterpriseLibrary.Caching.ProducerConsumerQueue.WaitUntilInterrupted()(+0x0 IL)(+0x11 Native)
0b:M 0000000023a9d2d0 000007ff002320e2 Microsoft.Practices.EnterpriseLibrary.Caching.ProducerConsumerQueue.Dequeue()(+0xf IL)(+0x18 Native)
0c:M 0000000023a9d330 000007ff00231f7e Microsoft.Practices.EnterpriseLibrary.Caching.BackgroundScheduler.QueueReader()(+0x9 IL)(+0x12 Native)
0d:M 0000000023a9d380 000007fef727c978 System.Threading.ExecutionContext.runTryCode(System.Object)(+0x18 IL)(+0x106 Native)
0e:U 0000000023a9d440 000007fef9001552 mscorwks!CallDescrWorker+0x82
0f:U 0000000023a9d490 000007fef8e9e5e3 mscorwks!CallDescrWorkerWithHandler+0xd3
10:U 0000000023a9d530 000007fef8eac83f mscorwks!MethodDesc::CallDescr+0x24f
11:U 0000000023a9d790 000007fef8f0cbd2 mscorwks!ExecuteCodeWithGuaranteedCleanupHelper+0x12a
12:U 0000000023a9da20 000007fef945e572 mscorwks!ReflectionInvocation::ExecuteCodeWithGuaranteedCleanup+0x172
13:M 0000000023a9dc30 000007fef7261722 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)(+0x60 IL)(+0x51 Native)
14:M 0000000023a9dc80 000007fef72f95fd System.Threading.ThreadHelper.ThreadStart()(+0x8 IL)(+0x2a Native)
15:U 0000000023a9dcd0 000007fef9001552 mscorwks!CallDescrWorker+0x82
16:U 0000000023a9dd20 000007fef8e9e5e3 mscorwks!CallDescrWorkerWithHandler+0xd3
17:U 0000000023a9ddc0 000007fef8eac83f mscorwks!MethodDesc::CallDescr+0x24f
18:U 0000000023a9e010 000007fef8f9ae8d mscorwks!ThreadNative::KickOffThread_Worker+0x191
19:U 0000000023a9e330 000007fef8f59374 mscorwks!TypeHandle::GetParent+0x5c
1a:U 0000000023a9e380 000007fef8e52045 mscorwks!SVR::gc_heap::make_heap_segment+0x155
1b:U 0000000023a9e450 000007fef8f66139 mscorwks!ZapStubPrecode::GetType+0x39
1c:U 0000000023a9e490 000007fef8e1c985 mscorwks!ILCodeStream::GetToken+0x25
1d:U 0000000023a9e4c0 000007fef8f594e1 mscorwks!Thread::DoADCallBack+0x145
1e:U 0000000023a9e630 000007fef8f59399 mscorwks!TypeHandle::GetParent+0x81
1f:U 0000000023a9e680 000007fef8e52045 mscorwks!SVR::gc_heap::make_heap_segment+0x155
20:U 0000000023a9e750 000007fef8f66139 mscorwks!ZapStubPrecode::GetType+0x39
21:U 0000000023a9e790 000007fef8e20e15 mscorwks!ThreadNative::KickOffThread+0x401
22:U 0000000023a9e7f0 000007fef8e20ae7 mscorwks!ThreadNative::KickOffThread+0xd3
23:U 0000000023a9e8d0 000007fef8f814fc mscorwks!Thread::intermediateThreadProc+0x78
24:U 0000000023a9f7a0 00000000773cbe3d kernel32!BaseThreadInitThunk+0xd
25:U 0000000023a9f7d0 00000000775d6a51 ntdll!RtlUserThreadStart+0x1d
Yes, the caching block has some - issues - with regard to the scavenger threads in older versions of Entlib, particularly if things are coming in faster than the scavenging settings let them come out.
This was completely rewritten in Entlib 5, so that now you'll never have more than two threads sitting in the caching block, regardless of the load, and usually it'll only be one.
Unfortunately there's no easy tweak to change the behavior in earlier versions. The best you can do is change the cache settings so that each scavenge will clean out more items at a time so not as many scavenge requests need to get scheduled.
640 threads is very bad for performance. If they are all waiting for something, then I'd say it's a fair bet that you have a deadlock and they will never exit. If they are all running (not waiting)... well, with 600+ threads on a 2 or 4 core processor none of them will get enough time slices to run very far! ;>
If your app is set up with a main thread that waits on the thread handles to find out when the threads exit, and the background threads get caught up in a loop or in a wait state and never exit the thread proc, then the process and all of its threads will never exit.
Check your thread code to make sure that every threadproc has a clear path to exit the threadproc. It's bad form to write an infinite loop in a background thread on the assumption that the thread will be forcibly terminated when the process shuts down.
If the background thread code spins in a loop waiting for an event handle to signal, make sure that you have some way to signal that event so that the thread can perform a normal orderly exit. Otherwise, you need to write the background thread to wait on multiple events and unblock when any one of the events signals. One of those events can be the activity that the background thread is primarily interested in and the other can be a shutdown event.
From the names of things in the stack dump you posted, it would appear that the thread is waiting for something to appear in the ProducerConsumerQueue. Investigate how that queue object is supposed to be shut down, probably on the producer side, and whether shutting down the queue will automatically release all consumers that are waiting on that queue.
My guess is that either the queue is not being shut down correctly or shutting it down does not implicitly release the consumers that are waiting on it. If the latter case, you may need to pump a terminate message through the queue to wake up all the consumers waiting on that queue and tell them to break out of their wait loop and exit.
You have an major issue. Every Thread occupies 1MB of stack and there is significant cost paid for Context Switching every thread in and out. Especially it becomes worst with managed code because every time GC has to run , it would have walk the threads stack to look for roots and when these threads are paged to the disk the cost to read from the disk is expensive,which adds up Perf issue.
Creating threads are Bad unless you know what you are doing? Jeffery Richter has written in detail about this.
To solve the above issue I would look what these threads are blocked on and also put a break-point on Thread Create (example sxe ct within windbg)
And later rearchitect from avoid creating threads , instead use the thread pool.
It would have been nice to some callstacks of these threads.
In Microsoft Enterprise Library 4.1, the BackgroundScheduler class creates a new thread each time an object is instantiated. It will be fixed in version 5.0. I do not know enough of this Microsoft Library to advise you how to avoid that behavior, but you may try the beta version: http://entlib.codeplex.com/wikipage?title=EntLib5%20Beta2