All these APIs (DbgHelp, DbgEng, DIA SDK) can be used to recover function name/source file location from function pointers, so are usable to decrypt stack traces.
However, every DbgHelp function has the following remark:
All DbgHelp functions, such as this one, are single threaded. Therefore, calls from more than one thread to this function will likely result in unexpected behavior or memory corruption. To avoid this, you must synchronize all concurrent calls from more than one thread to this function.
As for DbgEng, there aren't such claims, however it seems that it is based on DbgHelp, so cannot be thread-safe: even if DbgEng guards every call to DbgHelp, there may be calls to DbgHelp not from DbgEng, but directly from the user's code.
I'm afraid if DIA SDK is also implemented using DbgHelp.
So is there any robust symbols API that can be used in a library designed for a multithreaded application?
DIA SDK is assumed to be a thread safe solution.
There's an issue in Boost.Stacktrace repo that asks for switching to DIA SDK. Unsafety of DbgHelp and DbgEng is mentioned there.
Also there's an option to roll own implementation; a possible starting point is here: https://github.com/microsoft/microsoft-pdb
Related
I am working on a custom VCL-only date edit component. Am planning on using the System.SysUtils.FormatDateTime function to convert a TDate to a string. There are two versions of FormatDateTime--one is thread-safe and the other is not. Since the VCL is not thread-safe, should I prefer the thread-safe version or is the non thread-safe version okay to use?
tl;dr use threadsafe version
If you use the non threadsafe version then you constrain any consumer of your component not to use that same non threadsafe version in a thread.
That's not an unreasonable constraint by any measure. Using the non threadsafe version is only really viable in a program that never does so away from the main thread. So a program would have to be breaking the rules in the first place for your component to be caught up in the fallout.
Having said that, a component author should as a principle avoid making any assumptions about the consuming program. So best practice is to call the threadsafe version. Then there can be no debate. Your program cannot be involved in any thread safety issue with these locale global variables.
So long the caller will be the main thread, it doesn't matter if you choose a non thread safe variant of the function. Which in this case seems to be (if you are not creating inside of your component a worker thread from which you were calling that function, and you adhere to the rule that you won't use your control inside any worker thread, you'll be safe with it).
But there's more to consider. If you have recent Delphi version and keep UpdateFormatSettings property enabled, a globally declared format settings variable used in non thread safe overload of the FormatDateTime function will get updated when the user modifies their local settings on their system. I can't say anything about a control notification (so you could update the output) because I'm having only D2009 by hand right now and these changes has been added later.
Actually I am using visual C++ to try to bind lua functions as callbacks for socket events(in another thread). I initialize the lua stuff in one thread and the socket is in another thread, so every time the socket sends/receives a message, it will call the lua function and the lua function determines what it should do according to the 'tag' within the message.
So my questions are:
Since I pass the same Lua state to lua functions, is that safe? Doesn't it need some kinda protection? The lua functions are called from another thead so I guess they might be called simultaneously.
If it is not safe, what's the solution for this case?
It is not safe to call back asynchronously into a Lua state.
There are many approaches to dealing with this. The most popular involve some kind of polling.
A recent generic synchronization library is DarkSideSync
A popular Lua binding to libev is lua-ev
This SO answer recommends Lua Lanes with LuaSocket.
It is not safe to call function within one Lua state simultaneously in multiple threads.
I was dealing with the same problem, since in my application all basics such as communication are handled by C++ and all the business logic is implemented in Lua. What I do is create a pool of Lua states that are all created and initialised on an incremental basis (once there's not enough states, create one and initialise with common functions / objects). It works like this:
Once a connection thread needs to call a Lua function, it checks out an instance of Lua state, initialises specific globals (I call it a thread / connection context) in a separate (proxy) global table that prevents polluting the original global, but is indexed by the original global
Call a Lua function
Check the Lua state back in to the pool, where it is restored to the "ready" state (dispose of the proxy global table)
I think this approach would be well suited for your case as well. The pool checks each state (on an interval basis) when it was last checked out. When the time difference is big enough, it destroys the state to preserve resources and adjust the number of active states to current server load. The state that is checked out is the most recently used among the available states.
There are some things you need to consider when implementing such a pool:
Each state needs to be populated with the same variables and global functions, which increases memory consumption.
Implementing an upper limit for state count in the pool
Ensuring all the globals in each state are in a consistent state, if they happen to change (here I would recommend prepopulating only static globals, while populating dynamic ones when checking out a state)
Dynamic loading of functions. In my case there are many thousands of functions / procedures that can be called in Lua. Having them constantly loaded in all states would be a huge waste. So instead I keep them byte code compiled on the C++ side and have them loaded when needed. It turns out not to impact performance that much in my case, but your mileage may vary. One thing to keep in mind is to load them only once. Say you invoke a script that needs to call another dynamically loaded function in a loop. Then you should load the function as a local once before the loop. Doing it otherwise would be a huge performance hit.
Of course this is just one idea, but one that turned out to be best suited for me.
It's not safe, as the others mentioned
Depends on your usecase
Simplest solution is using a global lock using the lua_lock and lua_unlock macros. That would use a single Lua state, locked by a single mutex. For a low number of callbacks it might suffice, but for higher traffic it probably won't due to the overhead incurred.
Once you need better performance, the Lua state pool as mentioned by W.B. is a nice way to handle this. Trickiest part here I find synchronizing the global data across the multiple states.
DarkSideSync, mentioned by Doug, is useful in cases where the main application loop resides on the Lua side. I specifically wrote it for that purpose. In your case this doesn't seem a fit. Having said that; depending on your needs, you might consider changing your application so the main loop does reside on the Lua side. If you only handle sockets, then you can use LuaSocket and no synchronization is required at all. But obviously that depends on what else the application does.
I was checking documentation on Asio, SSL part to be precise and run into ssl::stream
And the Thread safe statement:
Shared objects: Unsafe. The application must also ensure that all asynchronous operations are performed within the same implicit or explicit strand.
Now I understand that OpenSSL BIO doesn't alow concurrent SSL_Read and SSL_Write operations, but nowhere I could find which functions specificaly to use with strand.
Because using strand.post(async_read) / strand.post(async_write) shouldn't (in my knowledge) ensure that concurrent SSL_Read and SSL_Write won't happen, because even though these async functions won't execute at once, they just return right away and thus don't read any data.
Also using strand on handler functions also doesn't seem like any good (to me), since when these handlers are called data is already read and copyed into user-defined buffer.
Thank you
Ok after more googling I've found what I was looking for.
Solution was in example HTTP Server 3, which uses strand on completition handlers.
Even though I still don't understand how it helps, it does prevent concurrent SSL operations and thus is not crashing my program.
I'm using 3.9.7 cURL library to download files from the internet, so I created a dynamic bibioteca of viculo. dll written in C using VC + + 6.0 the problem is that when either I call my function from within my vb6 application window locks and unlocks only after you have downloaded the file how do I solve this problem?
The problem is that when you call the function from your DLL, it "blocks" your app's execution until it gets finished. Basically, execution goes from the piece of code that makes the function call, to the code inside of the function call, and then only comes back to the next line after the function call after the code inside of the function has finished running. In fact, that's how all function calls work. You can see this for yourself by single-stepping through your code in the VB 6 development environment.
You don't normally notice this because the code inside of a function being called doesn't take very long to execute before control is returned to the caller. But in this case, since the function you're calling from the DLL is doing a lot of processing, it takes a while to execute, so it "blocks" the execution of your application's code for quite a while.
This is a good general explanation for the reason why your application window appears to be frozen. A bit more technically, it's because the message pump that is responsible for processing user interaction with on-screen elements is not running (it's part of your code that has been temporarily suspended until the function that you called finishes processing). This is a bit more difficult for a VB programmer to appreciate, since none of this nitty-gritty stuff is exposed in the world of VB. It's all happening behind the scenes, just like it is in a C program, but you don't normally have to deal with any of it. Occasionally, though, the abstraction leaks, and the nitty-gritty rears its ugly head. This is one of those cases.
The correct solution to this general problem, as others have hinted at, is to run lengthy operations on a background thread. This leaves your main thread (right now, the only one you have, the one your application is running on) free to continue processing user input, while the other thread can process the data and return that processed data to the main thread when it is finished. Of course, computers can't actually do more than one thing at a time, but the magic of the operating system rapidly switching between one task and another means that you can simulate this. The mechanism for doing so involves threads.
The catch comes in the fact that the VB 6 environment does not have any type of support for creating multiple threads. You only get one thread, and that's the main thread that your application runs on. If you freeze execution of that one, even temporarily, your application freezes—as you've already found out.
However, if you're already writing a C++ DLL, there's no reason you can't create multiple threads in a VB 6 app. You just have to handle everything yourself as if you were using another lower-level language like C++. Run the C++ code on a background thread, and only return its results to the main thread when it is completely finished. In the mean time, your main thread is free.
This is still quite a bit of work, though, especially if you're inexperienced when it comes to Win32 programming and the issues surrounding multiple threads. It might be easier to find a different library that supports asynchronous function calls out-of-the-box. Antagony suggests using VB's AsyncRead method. That is probably a good option; as Karl Peterson says in the linked article, it keeps everything in pure VB 6 code, which can be a real time saver as well as a boon to future maintenance programmers. The only problem is that you'll still have to process the data somehow once you obtain it. And if that's slow, you're right back where you started from…
Check out this article, which demonstrates how to asynchronously transfer large files using a little-known method in user controls.
In WinForms, pretty much all your UI is thread-specific. You have to use [STAThread] so that the common dialogs will work, and you can't (safely) access a UI element from any thread other than the one that created it. From what I've heard, that's because that's just how Windows works -- window handles are thread-specific.
In WPF, these same restrictions were kept, because ultimately it's still building on top of the same Windows API, still window handles (though mostly just for top-level windows), etc. In fact, WPF even made things more restrictive, because you can't even access things like bitmaps across threads.
Now along comes WinRT, a whole new way of accessing Windows -- a fresh, clean slate. Are we still stuck with the same old threading restrictions (specifically: only being able to manipulate a UI control from the thread that created it), or have they opened this up?
I would expect it to be the same model - but much easier to use, at least from C# and VB, with the new async handling which lets you write a synchronous-looking method which just uses "await" when it needs to wait for a long-running task to complete before proceeding.
Given the emphasis on making asynchronous code easier to write, it would be surprising for MS to forsake the efficiency of requiring single-threaded access to the UI at the same time.
The threading model is identical. There is still a notion of single threaded and multi-threaded apartments (STA/MTA), it must be initialized by a call to RoInitialize. Which behaves very much like CoInitialize in name, argument and error returns. The user interface thread is single threaded, confirmed at 36:00 in this video.
The HTML/CSS UI model is inherently single threaded (until the advent of web workers recently, JS didn't support threads). Xaml is also single threaded (because it's really hard for developers to write code to a multithreaded GUI).
The underlying threading model does have some key differences. When your application starts, an ASTA (Application STA) is created to run your UI code as I showed in the talk. This ASTA does not allow reentrancy - you will not receive unrelated calls while making an outgoing call. This is a significant difference from STAs.
You are allowed to create async workitems - see the Windows.System.Threadpool namespace. These workitem threads are automatically initialized to MTA. As Larry mentioned, webworkers are the JS equivalent concept.
Your UI components are thread affined. See the Windows.UI.Core.CoreDispatcher class for information on how to execute code on the UI thread. You can check out the threading sample for some example code to update the UI from an async operation.
Things are different in pretty important ways.
While it's true the underlying threading model is the same, your question is generally related to how logical concurrency works with UI, and with respect to this what developers see in Windows 8 will be new.
As you mention most dialogs previously blocked. For Metro apps many UI components do not block all. Remember the talk of WinRT being asynchronous? It applies to UI components also.
For example this .NET 4 code will not necessarily kill your harddrive because the UI call blocks on Show (C# example):
bool formatHardDrive = true;
if (MessageBox.Show("Format your harddrive?") == NO)
formatHardDrive = false;
if (formatHardDrive == true)
Format();
With Windows 8 Metro many UI components like Windows.UI.Popups.MessageDialog, are by default Asynchronous so the Show call would immediately (logically) fall through to the next line of code before the user input is retrieved.
Of course there is an elegant solution to this based on the await/promise design patterns (Javascript example):
var md = Windows.UI.Popups.MessageDialog("Hello World!");
md.showAsync().then(function (command) {
console.log("pressed: " + command.label); });
The point is that while the threading model doesn't change, when most people mention UI and threading they are thinking about logical concurrency and how it affects the programming model.
Overall I think the asynchronous paradigm shift is a positive thing. It requires a bit of a shift in perspective, but it's consistent with the way other platforms are evolving on both the client and server sides.