I have a strange issue with multi-threading. I want to print a table view and therefore start a new thread which runs with a progress bar. Eventually this thread dies with memory errors for which I'm seeking the cause. Right now I got
malloc: * error for object 0x10000078c: Invalid signature for
pointer dequeued from free list
set a breakpoint in malloc_error_break to debug CaLister(27054,0x7fff73ea3300) malloc: error for object
0x60800043bcc0: Invalid pointer dequeued from free list
* set a breakpoint in malloc_error_break to debug
But most of the times (still it happens rarely!) it just stops with no specific (but definitely memory related) error. When I look from the debugger my data are nil. But they have not been touched since they are still being available for display in my table view.
Now the question: is there any precaution I need to take to access data being allocated in the main thread so I can safely access them from the detached thread?
Edit:
My print thread is dispatched like this (stripped code):
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0)) { self.performPrint (true) }
func performPrint (async:Bool) {
//First get the shared print info object so we know page sizes. The shared print info object acts like a global variable.
let sharedPrintInfo = NSPrintInfo.sharedPrintInfo()
let printObject = PSPrint ()
//Allocate a new instance of NSView into the variable printPageView
var frame = NSRect(x: 0, y: 0, width: sharedPrintInfo.paperSize.width-sharedPrintInfo.leftMargin-sharedPrintInfo.rightMargin, height: sharedPrintInfo.paperSize.height-sharedPrintInfo.topMargin-sharedPrintInfo.bottomMargin)
let basePrintPageView = PSPrintView(frame: frame)
var printPageView:PSPrintView
for pageNo in 1...paperDimensions.pages {
paperDimensions.pageNo = pageNo
printPageView = basePrintPageView.clone(pageNo)
//Set the option for the printView for what it should draw.
paperDimensions.pageNo = pageNo
//Finally append the view to the PSPrint Object.
printObject.printViews.append(printPageView)
}
dispatch_async(dispatch_get_main_queue()) {
printObject.printTheViews() //print all the views, each view being a 'page'.
}
}
Within
printPageView = basePrintPageView.clone(pageNo)
the access to my table view (where I get the data to be printed) returns nil sometimes.
Edit2: I just noticed that it's not the background thread which crashed, but the main thread :-/ Scratching my head even more but likely I have to close this question.
Related
Some questions about KeSetSystemAffinityThread function, since MSDN is quite laconic.
NOTE: I can't use the more complete KeSetSystemAffinityThreadEx because I must still support Windows XP.
1) How can I restore the previous affinity? The function does not return the old value, how can I obtain it?
2) Is it true that passing 0 to the function restores the default system affinity? I have found such assertion in some forums, but I can't find it in official MS documentation.
3) Is the new thread's system affinity mask maintained after a return to user mode, or is it restored to the default each time the thread enters in system mode?
4) What happens if previous system affinity mask is not restored?
(I'd rather post four different questions, but they seem to me too interdependent)
Use the undocumented KeRevertToUserAffinityThread(void) in WinXP. A quick search yields little information about the API but I found an implementation of the same function in ReastOS :
ReactOS KeRevertToUserAffinityThread
It is rather simple so I copy & paste it here:
VOID NTAPI KeRevertToUserAffinityThread ( VOID )
{
KIRQL OldIrql;
PKPRCB Prcb;
PKTHREAD NextThread, CurrentThread = KeGetCurrentThread();
ASSERT_IRQL_LESS_OR_EQUAL(DISPATCH_LEVEL);
ASSERT(CurrentThread->SystemAffinityActive != FALSE);
/* Lock the Dispatcher Database */
OldIrql = KiAcquireDispatcherLock();
/* Set the user affinity and processor and disable system affinity */
CurrentThread->Affinity = CurrentThread->UserAffinity;
CurrentThread->IdealProcessor = CurrentThread->UserIdealProcessor;
CurrentThread->SystemAffinityActive = FALSE;
/* Get the current PRCB and check if it doesn't match this affinity */
Prcb = KeGetCurrentPrcb();
if (!(Prcb->SetMember & CurrentThread->Affinity))
{
/* Lock the PRCB */
KiAcquirePrcbLock(Prcb);
/* Check if there's no next thread scheduled */
if (!Prcb->NextThread)
{
/* Select a new thread and set it on standby */
NextThread = KiSelectNextThread(Prcb);
NextThread->State = Standby;
Prcb->NextThread = NextThread;
}
/* Release the PRCB lock */
KiReleasePrcbLock(Prcb);
}
/* Unlock dispatcher database */
KiReleaseDispatcherLock(OldIrql);
}
Note the function takes no argument and just restore the affinity from some elements in the currrent KTHREAD struct. I guess this answer your question 1 & 2. Just call this function with no argument. I have done a test in 32bit WinXP and confirmed this. Question 4 is simple, your thread will continue to run using the processor affinity your've set.
I have no idea to your question 3. But most likely a switch between user and kernel mode has no effect on the current processor affinity in effect since this is something stored in the KTHREAD struct.
I have many doubts related with Cairo and GTK# (that runs on .NET and Mono). I'm developing a GTK# application for MS Windows and Linux. I'm using GTK# 2.12 over .NET right now while I'm working on the application.
I've created a custom widget that uses Cairo.ImageSurface and Cairo.Context objects. As far as I know, I'm calling the Dispose method of every ImageSurface object and every Context object I create inside the widget code.
The widget responds to the "MouseOver" event, redrawing some parts of its DrawingArea.
The (first) problem:
almost every redrawing operation increases a little bit the amount of used memory. When the amount of used memory has increased 3 or 4 Kbytes the Monodevelop tracelog panel shows me the following message:
Cairo.Surface is leaking, programmer is missing a call to Dispose
Set MONO_CAIRO_DEBUG_DISPOSE to track allocation traces
The code that redraws a part of the widget is something like:
// SRGB is a custom struct, not from Gdk nor Cairo
void paintSingleBlock(SRGB color, int i)
{
using (Cairo.Context g = CairoHelper.Create (GdkWindow)) {
paintSingleBlock (g, color, i);
// We do this to avoid memory leaks. Cairo does not work well with the GC.
g.GetTarget().Dispose ();
g.Dispose ();
}
}
void paintSingleBlock(Cairo.Context g, SRGB color, int i)
{
var scale = Math.Pow (10.0, TimeScale);
g.Save();
g.Rectangle (x(i), y(i), w(i), h(i));
g.ClosePath ();
g.Restore ();
// We don't directly use stb.Color because in some cases we need more flexibility
g.SetSourceRGB (color.R, color.G, color.B);
g.LineWidth = 0;
g.Fill ();
}
The (second) problem: Ok, Monodevelop tells me that I should set MONO_CAIRO_DEBUG_DISPOSE to "track allocation traces" (In order to find the leak, I suppose)... but I don't know how to set this environment variable (I'm in Windows). I've tried using bash and executing something like:
MONO_CAIRO_DEBUG_DISPOSE=1 ./LightCreator.exe
But nothing appears in stderr nor stdout... (neither the messages that appear in the Monodevelop's applicationt trace panel). I also don't know how to get the debugging messages that see inside Monodevelop but without Monodevelop.
There's anyone with experience debugging GTK# or Cairo# memory leaks?
Thanks in advance.
Just wanted to throw my 2c here as I was fighting a similar leak problem in Cairo with surfaces. What I noticed is that if I create a Surface object the ReferenceCount property becomes 1 and if I attach this surface to a Context if becomes not 2 but 3. After disposing the Context the ReferenceCount comes back but to 2.
So I used some reflection to call the native methods in Cairo to decrease the ReferenceCount when I really want to Dispose a surface. I use this code:
public static void HardDisposeSurface (this Surface surface)
{
var handle = surface.Handle;
long refCount = surface.ReferenceCount;
surface.Dispose ();
refCount--;
if (refCount <= 0)
return;
var asm = typeof (Surface).Assembly;
var nativeMethods = asm.GetType ("Cairo.NativeMethods");
var surfaceDestroy = nativeMethods.GetMethod ("cairo_surface_destroy", BindingFlags.Static | BindingFlags.NonPublic);
for (long i = refCount; i > 0; i--)
surfaceDestroy.Invoke (null, new object [] { handle });
}
After using it I still have some leaks, but they seem to be related to other parts of Cairo and not with the surfaces.
I have found that a context created with CairoHelper.Create() will have a reference count of two.
A call to dispose reduces the reference count by one. Thus the context is never freed and keeps its target alive, too.
The native objects have manual reference counting, but the Gtk# wrappers want to keep a native object alive as long as there is a C# instance referencing it.
If a native object is created for a C# wrapper instance it does not need to increment the reference count because the wrapper instance 'owns' the native object and the reference count has the correct value of one. But if a wrapper instance is created for an already existing native object the reference count of the native object needs to be manually incremented to keep the object alive.
This is decided by a bool parameter when a wrapper instance is created.
Looking at the code for CairoHelper.Create() will show something like this
public static Cairo.Context Create(Gdk.Window window) {
IntPtr raw_ret = gdk_cairo_create(window == null ? IntPtr.Zero : window.Handle);
Cairo.Context ret = new Cairo.Context (raw_ret, false);
return ret;
}
Even though the native context was just created 'owned' will be false and the C# context will increment the reference count.
There is no fixed version right now, it can only be corrected by fixing the source and building Gtk# yourself.
CairoHelper is an auto-generated file, to change the parameter to true this attribute must be included in gdk/Gdk.metadata.
<attr path="/api/namespace/class[#cname='GdkCairo_']/method[#name='Create']/return-type" name="owned">true</attr>
Everything to build Gtk# can be found here.
https://github.com/mono/gtk-sharp
I have created two threads in DigitalMicrograph and they are executed as soon as the script is executed.
I want something different.
Let's imagine two buttons for the threads (start and stop thread).
How can I add code to activate the threads only when I push buttons ?
It would be very helpful if you had a code exemple for me.
There are a couple of things to consider:
You can not allocate new objects from within the UIframe object. (To be more precise: From a method invoked by a UI action. You can allocate f.e. in the constructor or with an Init() method at start.) Therefore you allocate them beforehand and then let the UIframe object know about them.
You often want the UIframe object to be aware of the thread object, but also the thread-object to be aware of the UIframe object. (So that the UI can change if something in the thread object wants it to.)
Having objects as member variables of objects is a bit dangerous, because those objects can only be released once the 'keeping' object gets released to. If two objects hold each-other as member variables, you're in a dead-locked situation! For this reason, it is save to use weak referencing: Keep only the objectID numbers as member variables and look objects up on need.
The following code-example should give you a starting point. It consists of 2 classes and a main call. The code is split in this answer, just copy & paste it into a single script file for testing.
First the thread object:
class myThread:Thread
{
number linkedDLG_ID
number externalBreak
myThread( object self )
{
result( self.ScriptObjectGetID() + " created.\n" )
}
~myThread( object self )
{
result( self.ScriptObjectGetID() + " destroyed.\n" )
}
void SetLinkedDialogID( object self, number ID ) { linkedDLG_ID = ID; }
void InterruptAtNextChance( object self ) { externalBreak = 1; }
void RunThread( object self )
{
number maxLoop = 30
object callDLG = GetScriptObjectFromID( linkedDLG_ID )
externalBreak = 0
for( number i=0; i<maxLoop; i++ )
{
sleep( 0.1 )
Result( i + "\n" )
if ( callDLG.ScriptObjectIsValid() )
{
callDLG.DLGSetProgress( "progress", (i+1)/maxLoop )
callDLG.ValidateView()
}
if ( externalBreak )
break;
}
// Cleanup at end of thread
if ( callDLG.ScriptObjectIsValid() )
{
callDLG.DLGSetProgress( "progress", 0 )
callDLG.LookUpElement( "DBevel" ).DLGValue( 0 )
callDLG.ValidateView( )
}
}
}
Any threading class is derived from the class Thread.
The class has two member variables. One will hold the ID of the UI-object, the other is a simple Boolean to allow 'outside' calls to stop a running thread.
The first two methods are the constructor and the destructor. They are not really needed in this example, but it is good practice to put them in during script development, because they will indicate in the results-window when an object of that class gets created and when it gets destroyed. That helps tracking memory leaks and dead-lock situations.
The next two methods allow 'outside' calls to set the two member variables.
The RunThread method is the heart of any Thread class. It has to be of exactly this signature because it overrides the according method of the parent class Thread from which we derive our class MyThread. The RunThread method gets launched into a separate background thread, when the method StartThread() is called. ( StartThread() is a method of the class Thread. )
The actual code in RunThread is in two parts:
An 'action-loop' doing anything you want but allowing a quick-exit if the Boolean variable changes value. This is how external calls can interrupt. This is discussed a bit further down.
A 'clean-up' part where the object can influence the UI object, discussed below as well.
Next is the UI class:
class myDialog:UIframe
{
object callThread
myDialog( object self )
{
result( self.ScriptObjectGetID() + " created.\n" )
}
~myDialog( object self )
{
result( self.ScriptObjectGetID() + " destroyed.\n")
}
TagGroup CreateDLG( object self )
{
image i := IntegerImage( "", 1, 0, 25, 25)
i = 0; i[ 2 , 2 , 23 , 23 ] = 1;
image onImage, offImage
onImage = RGB( 0*i , 200*i , 0*i )
offImage = RGB( 200*i , 0*i , 0*i )
TagGroup tgItems, tg, button, label, progress
tg = DLGCreateDialog("Dialog",tgItems)
button = DLGCreateDualStateBevelButton( "DBevel", onImage, offImage, "StartPressed" )
progress = DLGCreateProgressBar( "progress" ).DLGfill( "X" )
label = DLGCreateLabel( "start/stop" )
tgItems.DLGAddElement( DLGGroupItems( button , label ).DLGTableLayout( 2 , 1 , 0 ) )
tgItems.DLGAddElement( progress )
return tg
}
object Init(object self, number callThreadID )
{
// Assign thread-object via weak-reference
callThread = GetScriptObjectFromID( callThreadID )
if ( !callThread.ScriptObjectIsvalid() )
Throw( "Invalid thread object passed in! Object of given ID not found." )
// Pass weak-reference to thread object
callThread.SetLinkedDialogID( self.ScriptObjectGetID() )
return self.super.init( self.CreateDLG() )
}
void StartPressed( object self )
{
number active = self.LookupElement( "DBevel" ).DLGGetValue()
if ( active )
callThread.StartThread()
else
callThread.InterruptAtNextChance()
}
}
Any dialog (UI) class is derived from the class UIframe.
This class has only one member variable: An object, which will be the thread-object.
Again there are a constructor/destructor method for easier debugging.
The CreateDLG method builds the tagGroup describing the dialog. I will not go into details here, but essentially it creates the following dialog when displayed:
The Init() method initializes the object. The Init() method of the base class UIframe requires a descriptive TagGroup and returns the UI object itself. We call on this in the last line of our extended Init() method, and use our class-method to create the tagGroup:
return self.super.init( self.CreateDLG() )
The code before is what links our thread-object to the UI-object. We pass in a number, which is the object-ID of our thread-object. We now get the according object from memory and assign it to our local member variable. (NB: The variable now holds the object itself, not a copy or clone of it! )
callThread = GetScriptObjectFromID( callThreadID )
Right away, we check if the lookup was successful and actually returned a valid object. If not, we stop our script with a thrown exception. From now on, the UI-object 'contains' the thread-object and can use it.
Now comes the back-link. Now that the UI object has been allocated, it also has an object-ID. We feed this number into our thread-object.
callThread.SetLinkedDialogID( self.ScriptObjectGetID() )
From now on, the thread object is nicely linked to the UI-object. Looking back to the myThread class, you will notice that we use the same trick of looking up and locally storing the object in the RunThread() method.
StartPressed() is the method linked to our dialog button. This button is a bevel button, so we query its state, which is the state after the bevel-button changed, and act accordingly. We either launch the RunThread() method of our thread object as a background-thread, or invoke the according 'interrupt' method, which simply sets the Boolean variable
Finally the main script:
void StartScript()
{
object threadObject = alloc( myThread )
object dlgObject = alloc( myDialog ).Init( threadObject.ScriptObjectGetID() )
dlgObject.display( "Dialog" )
}
StartScript()
Not a lot going on here. We first create the threadObject of the myThread class, and then the dialog UI object.
We initialize the dialog object with the ID of the existing threadObject, and then display it as a modeless dialog.
Some points to notice:
Whenever you use object variables in DigitalMicrograph scripting, you should put them into a structure block. This ensures that the objects get out-of-scope and deleted, when the structure block is left. Object variables defined and allocated in the main script are not destructed at the end of the script. For this reason, we have encapsulated the main script in a method itself.
We have used two different methods of linking in this example:
Direct: The myDialog class really keeps the thread-object itself as a member variable. Although we initialized it with the ID only, we immediately linked the object to a member variable.
Weak reference: The myThread class only holds the object-ID of the dialog-object as a member variable.
Why have we done this? If the myThread class would keep the dialog-object as a member, then the two objects would hold each-other in a dead-lock situation. Neither can be destructed because of the other. But why have we not used the same for the myDialog class? Because we want to display the dialog as a modeless dialog in a background thread itself!
Think of the main-script:
We create the thread-object
We create the dialog-object
We display the dialog-object (But we don't stop script execution here!)
The script ends
But when the script ends, the object variables threadObject and dlgObject go out of scope! They will be immediately destructed, unless something keeps them in memory. The dlgObject stays in memory, because we displayed it as modeless dialog. It will be released, when the according dialog window is closed. But what keeps the threadObject? Nothing! Once the RunThread() method has finished, it would be released and subsequently destructed. However, because it is a member of the dlgObject it will not be destructed.
can anybody answer, i should change sprites image and i do it using my function
-(void) openKeyWithSprite:(id) sender withSpriteName:(NSString*)spriteName
can this lead to leaks of memory or it's ok?
in init
_spriteBonus=[CCSprite spriteWithFile:#"monstr_1_1.png"];
in schedule
-(void) openKeyWithSprite:(id) sender withSpriteName:(NSString*)spriteName
{
CCTexture2D* tex = [[CCTextureCache sharedTextureCache] addImage:spriteName];
[_spriteBonus setTexture: tex];
}
The code you've shown here should be fine and won't leak any memory. setTexture releases the old texture reference and retains the new one.
However, assuming you're not using ARC: [CCSprite spriteWithFile:#"monstr_1_1.png"]; is in the autorelease pool, so make sure you add it to a parent CCNode (before exiting whatever function you're creating it in) in order to keep it retained in memory.
If you want to create frame animation, use CCAnimate action. For example,
id yourAnimation = [CCAnimation animationWithFrames: arrayOfFrames];
id animateAction = [CCAnimate actionWithDuration: animationDuration
animation: yourAnimation
restoreOriginalFrame: YES];
[yourSprite runAction: animateAction];
if you need to repeat your animation, use CCRepeat of CCRepeatForever action, for example
id resultAction = [CCRepeatForever actionWithAction: animateAction];
[yourSprite runAction: resultAction];
I am writing a COM Server which have a plenty of Interfaces and methods. And most of the methods have the BSTR as the parameters and as local parameters used for the return. A snippet looks like
Update 5:
The real code. This fetches from bunch of Data based on a specific condition the DB to populate an array of Object.
STDMETHODIMP CApplication::GetAllAddressByName(BSTR bstrParamName, VARIANT *vAdddresses)
{
AFX_MANAGE_STATE(AfxGetStaticModuleState())
//check the Database server connection
COleSafeArray saAddress;
HRESULT hr;
// Prepare the SQL Strings dan Query the DB
long lRecCount = table.GetRecordCount();
if (lRecCount > 0)
{
//create one dimension safe array for putting details
saAddress.CreateOneDim(VT_DISPATCH,lRecCount);
IAddress *pIAddress = NULL;
//retrieve details
for(long iRet = table.MoveFirst(),iCount=0; !iRet; iRet = table.MoveNext(),iCount++)
{
CComObject<CAddress> *pAddress;
hr = CComObject<CAddress>::CreateInstance(&pAddress);
if (SUCCEEDED(hr))
{
BSTR bstrStreet = ::SysAllocString(table.m_pRecordData->Street);
pAddress->put_StreetName(bstrStreet);
BSTR bstrCity = ::SysAllocString(table.m_pRecordData->City);
pAddress->put_CityName(bstrCity);
}
hr = pAddress->QueryInterface(IID_IAddress, (void**)&pIAddress);
if(SUCCEEDED(hr))
{
saAddress.PutElement(&iCount,pIAddress);
}
}
*vAdddresses=saAddress.Detach();
}
table.Close();
return S_OK;
}
STDMETHODIMP CAddress::put_CityName(BSTR bstrCityName)
{
AFX_MANAGE_STATE(AfxGetStaticModuleState())
// m_sCityName is of CComBSTR Type
m_sCityName.Empty();//free the old string
m_sCityName = ::SysAllocString(bstrCityName);//create the memory for the new string
return S_OK;
}
The problem lies in the Memory Freeing part. The code works very fine in any Win XP machines, but when comes to WIN2K8 R2 and WIN7 the code crashes and pointing to the ::SysFreeString() as the culprit. The MSDN is not adequate to the solution.
Can anyone please help in finding the right solution?
Thanks a lot in advance :)
Update 1:
I have tried using the CComBSTR as per the suggestion in the place of raw BSTR, initialized using direct CString's and excluded the SysFreeString(). But for my trouble, on getting out of scope the system is calling the SysFreeString() which again causes the crash :(
Update 2:
With the same CComBSTR i tried to allocate using the SysAllocString() , the problem remains same :(
Update 3:
I am tired of all the options and in peace I am having only question in mind
Is it necessary to free the BSTR through SysFreeString() which was
allocated using SysAllocString()/string.AllocSysString()?
Update 4:
I missed to provide the information about the crash. When I tried to debug the COM server crashed with a error saying
"Possible Heap Corruption"
. Please help me out of here.. :(
// Now All Things are packed in to the Object
obj.Name = bstrName;
obj.Name2 = bstrname2;
I don't quite understand what do you mean by saying that things are packed since you're just copying pointers to the strings, and at the moment when you call SysFreeString obj.Name and obj.Name2 will point to an invalid block of memory. Although this code is not safe, it looks like if the source of your problem is class CFoo. You should show us more details of your code
I suggest you to use a CComBSTR class which will take a responsibility for releasing the memory.
UPDATE
#include <atlbase.h>
using namespace ATL;
...
{
CComBSTR bstrname(_T("Some Name"));
CComBSTR bstrname2(_T("Another Name"));
// Here one may work with these variables if needed
...
// Copy the local values to the Obj's member Variable
bstrname.Copy(&obj.Name);
bstrname2.Copy(&obj.Name2);
}
UPDATE2
First of all one should free bstrCity and bstrStreetName with SysFreeString or use CComBSTR instead within this block:
if (SUCCEEDED(hr))
{
BSTR bstrStreet = ::SysAllocString(table.m_pRecordData->Street);
pAddress->put_StreetName(bstrStreet);
BSTR bstrCity = ::SysAllocString(table.m_pRecordData->City);
pAddress->put_CityName(bstrCity);
// SysFreeString(bstrStreet)
// SysFreeString(bstrCity)
}
Consider to amplify the loop's condition !iRet with iCount < lRecCount.
for(...; !iRet /* && (iCount < lRecCount) */; ...)
Also, here:
m_sCityName = ::SysAllocString(bstrCityName);
you allocate memory but never release it since internally CComBSTR& operator = (OLESTR ..) allocates a new storage itself. One should rewrite is as follows:
m_sCityName = bstrCityName;
Everything else, looks good for me
UPDATE3
Well, Heap corruption is often a consequence of writing some values outside of the allocated memory block. Say you allocate an array of length 5 and put some value to the 6th position
Finally I have found the real reason for the Heap Corruption that happened in the code.
The put_StreetName/put_CityName of the IAddress/CAddress is designed in a following way.
STDMETHODIMP CAddress::put_CityName(BSTR bstrCityName)
{
AFX_MANAGE_STATE(AfxGetStaticModuleState())
m_sCityName.Empty();
TrimBSTR(bstrCityName);
m_sCityName = ::SysAllocString(bstrCityName);
return S_OK;
}
BSTR CAddress::TrimBSTR(BSTR bstrString)
{
CString sTmpStr(bstrString);
sTmpStr.TrimLeft();
sTmpStr.TrimRight();
SysReAllocString(&bstrString,sTmpStr); // The Devilish Line
}
The Devilish Line of code is the real culprit that caused the Memory to go hell.
What caused the trouble?
In this line of code, the BSTR string passed as a parameter is from another application and the real memory is in another realm. So the system is trying to reallocate teh string. Either succeed or not, the same is tried to cleared off from the memory in original application/realm, thus causing a crash.
What still unsolved?
Why the same piece of code doesn't crashed a single time in Win XP and
older systems? :(
Thanks for all who took their time to answer and solve my problem :)