I'm trying to reverse-engineer an undocumented protocol. A server is sending UDP updates to a client, and I have the client application in OllyDbg. Within Olly, I see that function recvfrom in WS2_32.dll is used to get the data I'm interested in (verified by wireshark). I'm wondering how I can see where this data goes with the application. The thread that calls recvfrom appears to be simply looping and storing the data in the heap, and as it loops it overwrites it with new data. This leads me to believe that a separate thread is actually parsing the data, as shown below:
My ultimate goal is to follow the data as it flows within the client to see how it is parsed, and eventually document the protocol.
To sum up: my question is, how would you go about determining which thread is involved in handling data located in a known heap memory location?
I figured out how I can do this. 1) Break as soon as the thread that calls recvfrom() stores that received data in the heap. 2) In OllyDbg, you can right-click in the heap and set a breakpoint on memory access (or write). The thread that it breaks on is the thread that is accessing the memory that was previously created from the thread with recvfrom().
Related
I want to implement ring buffer for classic Producer--Consumer interaction. In the future both P and C will be implemented as permanent threads running during data processing task, and GUI will be the third thread only for displaying actual data and coordinate starts and stops of data processing by user interaction. C can be quite slow to be able to fully process all incoming data, but only a bit and for a short periods of time. So I want to just allocate ring buffer of several P's MTUs in size, but in any case, if C will be too slow to process existing data it's okay to loose old data in favor of new one (overwrite policy).
I've read QSemaphore example in Qt help and realized that by usage of semaphore's acquires and releases I can only implement discard policy, because acquiring of specified chunk in queue will block until there are no free space.
Are there any ways of implementing overwrite policy together with QSemaphore or I just need to go and implement another approach?
I've came to this solution. If we should push portion of the src data to the ring buffer at any costs (it's ok to drop possible newly incoming data) we should use acquire() in Producer part - that would provide us discard policy. In case we need overwrite policy we should use tryAcquire() in Producer - thus at the very first possible moment of time only the newest data will be pushed to the ring buffer
I figured it would be a lot easier if I drew a picture of my problem. Here it is:
Everything that is black in the diagram is part of the old design. Everything that is blue is part of the new design. Basically, I need to add a new thread (Worker Thread C) that will handle most of the work that Worker Thread B used to do. Worker Thread A is listening for real time updates from an external application. When he receives an update, he posts a message to Worker Thread B. Worker Thread B will set its copy of the new data (he still needs it in the new design) and then notify the GUI Thread as well as Worker Thread C that new data has arrived.
The user will send a request from the GUI to the new thread (Worker Thread C). Worker Thread C will process the request using the last received copy of the data that originally came from Worker Thread A. So my question is: Will Worker Thread C always be using the latest copy of the data when processing a request with this new design? What if Worker Thread B is too slow to update and then the user submits a request from the GUI? Thanks!
If I'm not mistaken, worker A is conceptually different than workers B and C, right? It rather looks like B and C handle user requests in the background in order to not block the UI. So, there could be a whole list of these background workers that perform UI operations or even none, while there will always be a worker A that pulls/receives updates.
Now, what I would do is that the worker A sends new data to the UI. The UI then uses this data in the next request. When it starts one of the workers like B or C, it just passes the data along with the other info that tells the thread what to do.
Note that you need to take care that you don't modify the data in different threads. The easiest way is to always copy the data when passing it between different parts, but that is often too expensive. Another easy way is to make the data constant. In worker A, you use a unique_ptr<Data> to accumulate the update and then send that data as a shared_ptr<Data const> to the UI thread. From that point on, this data is immutable (the compiler makes sure that you don't change it by accident) so it can be shared between threads without any further lock.
When creating a worker for a background operation, you pass in the shared_ptr<Data const>. If it needs to modify that data, it would first have to copy it, but usually that isn't something that can't be avoided.
Notes:
The basic idea is that you have either shared and immutable data or exclusive-owned and mutable data.
The data received from thread A is stored in the UI here, but conceptually it is part of the model in an MVC design. There, you only keep a reference to the last update, the earlier ones can be discarded. The worker thread still using the data won't notice, because the data is refcounted using shared_ptr.
At some point, I would consider aborting the background workers. Computing anything based on old data is not necessary, so it could be worthwhile to not waste time on it but to restart based on recent data.
I'm assuming that the channels between the threads (message queues) are synchronized. If they are already synchronized, that is all that you need.
If you're using C++98, you will need auto_ptr instead of unique_ptr and Boost's shared_ptr.
I am wondering what is the best stable way to handle multiple connections at the same time?
I am using vb6 and currently using winsock api's no Winsock control. I tried that before and its not multi threaded too.
At the moment it's only a single thread which is not efficient when the thread is busy sending data the other connector is delayed. Until the thread be free.
I am using WSAAsyncSelect non-blocking socket.
So since VB6 isn't stable at multithreading. I am thinking of using an ASM
DLL and then call it from vb6 that will handle the connections. But what is the best way? create a thread for each connection then terminate the thread after recv? or keep the connection open all time until the other part closes it?
Because the server running the client is not that good specifications. So more threads consumes more resources.
i have not much knowledge about what is better in performance so please share your opinions.
Also how to be sure that all data have been sent from send function on a non-blocking sockets?
should loop through send and count each time how many bytes sent? or just call it once? i have noticed if i send large data that can not be processed at 1 time the window that i specified at call to WSAAsyncSelect to handle the network events gets called again so there is more data to be sent but how to be sure that this is belongs to this partial send? or recv?
Note: Max connections can be connected at same time is about 100.
Here is an example of problem i am having while sending a pic over network size (5 kb)
sometimes it is all received with 1 recv call while sometimes its being split into pieces
If Bytes = PicSize Then
MsgBox "All data are sent 1 time"
Else
MsgBox "there is more data left"
While Bytes <> PicSize
bytesRecieved = recv(s, Buffer(Bytes), UBound(Buffer), 0)
If bytesRecieved > 0 Then
Bytes = Bytes + bytesRecieved
End If
DoEvents
Wend
End If
The return value of recv is always WSAEWOULDBLOCK so i am getting inside an infinite loop.
Any suggestions?
You've ask more than one question, which makes it hard to answer. Whether using async winsock directly or using the WinSock control, it is important to realize that when you think you are "busy sending data" all you are doing is passing the data to the network stack. This happens quickly and your code continues on. That data will, hopefully, eventually, make it to the destination. This part does not happen as quickly, but your code has moved on to process the next task.
I have four threads in a C++/CLI GUI I'm developing:
Collects raw data
The GUI itself
A background processing thread which takes chunks of raw data and produces useful information
Acts as a controller which joins the other three threads
I've got the raw data collector working and posting results to the controller, but the next step is to store all of those results so that the GUI and background processor have access to them.
New raw data is fed in one result at a time at regular (frequent) intervals. The GUI will access each new item as it arrives (the controller announces new data and the GUI then accesses the shared buffer). The data processor will periodically read a chunk of the buffer (a seconds worth for example) and produce a new result. So effectively, there's one producer and two consumers which need access.
I've hunted around, but none of the CLI-supplied stuff sounds all that useful, so I'm considering rolling my own. A shared circular buffer which allows write-locks for the collector and read locks for the gui and data processor. This will allow multiple threads to read the data as long as those sections of the buffer are not being written to.
So my question is: Are there any simple solutions in the .net libraries which could achieve this? Am I mad for considering rolling my own? Is there a better way of doing this?
Is it possible to rephrase the problem so that:
The Collector collects a new data point ...
... which it passes to the Controller.
The Controller fires a GUI "NewDataPointEvent" ...
... and stores the data point in an array.
If the array is full (or otherwise ready for processing), the Controller sends the array to the Processor ...
... and starts a new array.
If the values passed between threads are not modified after they are shared, this might save you from needing the custom thread-safe collection class, and reduce the amount of locking required.
Is
OutputDebugString(PAnsiChar(''));
thread safe?
I/we have been using it in threads for debugging, and it never occurred to me if I should be doing it a different way.
(Delphi 7)
Well, not that it isn't true, it is, but just so that you don't have to just take Lieven word for it:
Passing of data between the
application and the debugger is done
via a 4kbyte chunk of shared memory,
with a Mutex and two Event objects
protecting access to it. These are the
four kernel objects involved.
Understanding Win32 OutputDebugString is an excellent article on the matter.
Don't worry, it is.
When OutputDebugString() is called by an application, it takes these
steps. Note that a failure at any point abandons the whole thing and
treats the debugging request as a no-op (the string isn't sent
anywhere).
Open DBWinMutex and wait until we have exclusive access to it.
Map the DBWIN_BUFFER segment into memory: if it's not found,
there is no debugger running so the entire request is ignored.
Open the DBWIN_BUFFER_READY and DBWIN_DATA_READY events. As with
the shared memory segment, missing objects mean that no debugger is
available.
Wait for the DBWIN_BUFFER_READY event to be signaled: this says
that the memory buffer is no longer in use. Most of the time, this
event will be signaled immediately when it's examined, but it won't
wait longer than 10 seconds for the buffer to become ready (a timeout
abandons the request).
Copy up to about 4kbytes of data to the memory buffer, and store
the current process ID there as well. Always put a NUL byte at the end
of the string.
Tell the debugger that the buffer is ready by setting the
DBWIN_DATA_READY event. The debugger takes it from there.
Release the mutex
Close the Event and Section objects, though we keep the handle to
the mutex around for later.
I've had trouble once, though, with strings in an ISAPI DLL. For some odd reason the IsMultiThread boolean defined in System.pas was not set!
It was causing weird AccessViolations, once the thread was running more than one thread... A simple "IsMultiThread:=true;" in a unit initialization fixed it.