Parse query stalls main thread for up to a minute - multithreading

Question

I'm using the Parse package from the asset store to save userdata for my Unity3D mobile game (iOS/Android). Now I've got the problem that the parse queries I make stall my unity mainthread for a long time. I was under the impression that the 'Task' concept of Parse was there to avoid that but somehow it doesn't seem to work. The stalling happens in the Editor as well, only a few seconds compared to up to a minute on mobile. The query.FindAsync().ContinueWith(t => {myContinuationBlock}); method returns immediately and subsequent calls get executed. The Unity3d main thread stalls a few seconds after the query is executed and immediately BEFORE the code in {myContinuationBlock} is executed.
I made a video of the problem. In the video you can also see that only Unity is being stalled, the (native) iAds on the bottom are not affected:
https://www.youtube.com/watch?v=XWaCGk9Hbus (the stall is from 0:07 - 1:07)
Here is my query code (note, this code runs through without problem. The stall happens later, right before the ContinueWith code gets executed):
public static void RetrieveHighScores()
{
DebugLog.Log ("start of RetrieveHighScores() method");
try
{
ParseQuery<ParseUser> query = ParseUser.Query
.WhereGreaterThan (_highScoreKey, 1)
.WhereGreaterThan("updatedAt", DateTime.Now - TimeSpan.FromDays(1))
.OrderByDescending(_score24HoursKey)
.Limit (1000);
DebugLog.Log ("query.FindAsync()");
query.FindAsync().ContinueWith(t =>
{
try
{
if (t.IsFaulted || t.IsCanceled)
{
DebugLog.Log("Retrieving 24 Hours High Scores Failed");
}
else
{
DebugLog.Log("Retrieving 24 Hours High Scores successful! ");
IEnumerable<ParseUser> results = t.Result;
foreach(ParseUser user in results)
{
//doing something with user..
}
DebugLog.Log ("24 Hours High Scores processed. "+_24HourHighScoreList.Count.ToString()+" entries.");
}
}
catch(System.Exception e)
{
DebugLog.Log("Failed to retrieve 24 Hours High Scores. Reason: " + e.Message);
}
});
DebugLog.Log ("FindAsync() returned");
}
catch (System.Exception e)
{
DebugLog.Log("Failed to retrieve 24 Hours High Scores. Reason: " + e.Message);
}
try
{
ParseQuery<ParseUser>query = ParseUser.Query
.WhereGreaterThan (_highScoreKey, 1)
.OrderByDescending(_highScoreKey)
.Limit (1000);
DebugLog.Log ("query.FindAsync()");
query.FindAsync().ContinueWith(t =>
{
DebugLog.Log("Retrieving Alltime High Scores successful! ");
IEnumerable<ParseUser> results = t.Result;
foreach(ParseUser user in results)
{
//doing something with user...
}
DebugLog.Log ("Alltime High Scores processed. "+_allTimeHighScoreList.Count.ToString()+" entries.");
});
DebugLog.Log ("FindAsync() returned");
}
catch (System.Exception e)
{
DebugLog.Log("Failed to retrieve alltime Highscores. Reason: " + e.Message);
}
DebugLog.Log ("end of RetrieveHighScores() method");
}
So, the very next thing I see in the console output after the stall is
"Retrieving Alltime High Scores successful! "
Now, I know that i'm querying 1000 objects here and yes there may be better ways for implementing highscores, but I don't understand why this code is stalling my Unity3D mainthread? Why does it stall sometimes for up to a minute and sometimes it's not noticable at all?
This is a serious problem as my game is released already. It started surfacing only once the user database grew bigger and now I need a quick fix for it.
The stalling does not happen if I don't call the RetrieveHighScores() function so it must be something happening after the parse code received the data from the server and passes it to ContinueWith code.
if I click pause in XCode during the stall, I see the following:
Thread 1, Queue : com.apple.main-thread
#0 0x017eaeb0 in GC_mark_from ()
#1 0x017eb520 in GC_mark_some ()
#2 0x017e5d1c in GC_stopped_mark ()
#3 0x017e6228 in GC_try_to_collect_inner ()
#4 0x017e64f0 in GC_collect_or_expand ()
#5 0x017e6a38 in GC_allocobj ()
#6 0x017e957c in GC_generic_malloc_inner ()
#7 0x017e965c in GC_generic_malloc ()
#8 0x017e9920 in GC_malloc_atomic ()
#9 0x01783814 in mono_array_new_specific ()
#10 0x00c5886c in m_wrapper_managed_to_native_object___icall_wrapper_mono_array_new_specific_intptr_int at /Users/me/myproj/build/device/Libraries/mscorlib.dll.s:187982
#11 0x009ee54c in m_System_Text_RegularExpressions_Interpreter_ResetGroups at /Users/me/myproj/build/device/Libraries/System.dll.s:6462
#12 0x009eb864 in m_System_Text_RegularExpressions_Interpreter_Reset at /Users/me/myproj/build/device/Libraries/System.dll.s:5761
#13 0x009edb6c in m_157 at /Users/me/myproj/build/device/Libraries/System.dll.s:6244
#14 0x009ebc54 in m_155 at /Users/me/myproj/build/device/Libraries/System.dll.s:5813
#15 0x009eb804 in m_System_Text_RegularExpressions_Interpreter_Scan_System_Text_RegularExpressions_Regex_string_int_int at /Users/me/myproj/build/device/Libraries/System.dll.s:5745
#16 0x009f3ce8 in m_System_Text_RegularExpressions_Regex_Match_string_int at /Users/me/myproj/build/device/Libraries/System.dll.s:9106
#17 0x00370204 in m_Parse_Internal_Json_Accept_string_int_System_Text_RegularExpressions_Regex_int__System_Text_RegularExpressions_Match_ at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:3770
#18 0x0036fa80 in m_Parse_Internal_Json_ParseString_string_int_int__object_ at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:3577
#19 0x0036f56c in m_Parse_Internal_Json_ParseMember_string_int_int__object_ at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:3447
#20 0x0036f3b8 in m_Parse_Internal_Json_ParseObject_string_int_int__object_ at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:3413
#21 0x0036f99c in m_Parse_Internal_Json_ParseValue_string_int_int__object_ at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:3553
#22 0x0036f780 in m_Parse_Internal_Json_ParseArray_string_int_int__object_ at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:3500
#23 0x0036f9b8 in m_Parse_Internal_Json_ParseValue_string_int_int__object_ at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:3556
#24 0x0036f5ec in m_Parse_Internal_Json_ParseMember_string_int_int__object_ at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:3457
#25 0x0036f3b8 in m_Parse_Internal_Json_ParseObject_string_int_int__object_ at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:3413
#26 0x0036e920 in m_Parse_Internal_Json_Parse_string at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:3143
#27 0x00383168 in m_Parse_ParseClient_DeserializeJsonString_string at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:13121
#28 0x0038381c in m_Parse_ParseClient__c__DisplayClass8__RequestAsyncb__7_System_Threading_Tasks_Task_1_System_Tuple_2_System_Net_HttpStatusCode_string at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:13363
#29 0x0036de50 in m_Parse_Internal_InternalExtensions__c__DisplayClass1_2__OnSuccessb__0_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:2697
#30 0x0036e04c in m_Parse_Internal_InternalExtensions__c__DisplayClass7_1__OnSuccessb__6_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:2788
#31 0x003a3a4c in m_System_Threading_Tasks_Task__c__DisplayClass3_1__c__DisplayClass5__ContinueWithb__2 ()
#32 0x003a36b8 in m_System_Threading_Tasks_Task___cctorb__23_System_Action at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31346
#33 0x003a39bc in m_System_Threading_Tasks_Task__c__DisplayClass3_1__ContinueWithb__1_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31474
#34 0x003a4140 in m_System_Threading_Tasks_Task_1_RunContinuations at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31766
#35 0x003a4278 in m_System_Threading_Tasks_Task_1_TrySetResult_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31801
#36 0x003a4654 in m_System_Threading_Tasks_TaskCompletionSource_1_TrySetResult_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31950
#37 0x003a4fa0 in m_3d9 at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:32334
#38 0x003a44b0 in m_System_Threading_Tasks_Task_1__c__DisplayClass1__ContinueWithb__0_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31872
#39 0x003a3b3c in m_System_Threading_Tasks_Task__c__DisplayClass8__ContinueWithb__7_System_Threading_Tasks_Task ()
#40 0x00519a84 in m_System_Threading_Tasks_Task__c__DisplayClass3_1__c__DisplayClass5_int__ContinueWithb__2 ()
#41 0x003a36b8 in m_System_Threading_Tasks_Task___cctorb__23_System_Action at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31346
#42 0x004a9548 in m_1ce9 at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:165091
#43 0x003a8dcc in m_System_Threading_Tasks_Task_ContinueWith_int_System_Func_2_System_Threading_Tasks_Task_int_System_Threading_CancellationToken at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:34526
#44 0x003a321c in m_System_Threading_Tasks_Task_ContinueWith_System_Action_1_System_Threading_Tasks_Task_System_Threading_CancellationToken at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31155
#45 0x003a3168 in m_System_Threading_Tasks_Task_ContinueWith_System_Action_1_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31129
#46 0x003a3fdc in m_System_Threading_Tasks_Task_1_ContinueWith_System_Action_1_System_Threading_Tasks_Task_1_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31721
#47 0x003a4ed4 in m_3d8 at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:32304
#48 0x003a44b0 in m_System_Threading_Tasks_Task_1__c__DisplayClass1__ContinueWithb__0_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31872
#49 0x003a3b3c in m_System_Threading_Tasks_Task__c__DisplayClass8__ContinueWithb__7_System_Threading_Tasks_Task ()
#50 0x00519a84 in m_System_Threading_Tasks_Task__c__DisplayClass3_1__c__DisplayClass5_int__ContinueWithb__2 ()
#51 0x003a36b8 in m_System_Threading_Tasks_Task___cctorb__23_System_Action at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31346
#52 0x004a9548 in m_1ce9 at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:165091
#53 0x003a4140 in m_System_Threading_Tasks_Task_1_RunContinuations at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31766
#54 0x003a4278 in m_System_Threading_Tasks_Task_1_TrySetResult_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31801
#55 0x003a4654 in m_System_Threading_Tasks_TaskCompletionSource_1_TrySetResult_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31950
#56 0x003a3a60 in m_System_Threading_Tasks_Task__c__DisplayClass3_1__c__DisplayClass5__ContinueWithb__2 at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31498
#57 0x003a36b8 in m_System_Threading_Tasks_Task___cctorb__23_System_Action at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31346
#58 0x003a39bc in m_System_Threading_Tasks_Task__c__DisplayClass3_1__ContinueWithb__1_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31474
#59 0x003a4140 in m_System_Threading_Tasks_Task_1_RunContinuations at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31766
#60 0x003a4278 in m_System_Threading_Tasks_Task_1_TrySetResult_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31801
#61 0x003a4654 in m_System_Threading_Tasks_TaskCompletionSource_1_TrySetResult_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31950
#62 0x003a4fa0 in m_3d9 at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:32334
#63 0x003a44b0 in m_System_Threading_Tasks_Task_1__c__DisplayClass1__ContinueWithb__0_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31872
#64 0x003a3b3c in m_System_Threading_Tasks_Task__c__DisplayClass8__ContinueWithb__7_System_Threading_Tasks_Task ()
#65 0x00519a84 in m_System_Threading_Tasks_Task__c__DisplayClass3_1__c__DisplayClass5_int__ContinueWithb__2 ()
#66 0x003a36b8 in m_System_Threading_Tasks_Task___cctorb__23_System_Action at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31346
#67 0x004a9548 in m_1ce9 at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:165091
#68 0x003a8dcc in m_System_Threading_Tasks_Task_ContinueWith_int_System_Func_2_System_Threading_Tasks_Task_int_System_Threading_CancellationToken at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:34526
#69 0x003a321c in m_System_Threading_Tasks_Task_ContinueWith_System_Action_1_System_Threading_Tasks_Task_System_Threading_CancellationToken at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31155
#70 0x003a3168 in m_System_Threading_Tasks_Task_ContinueWith_System_Action_1_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31129
#71 0x003a3fdc in m_System_Threading_Tasks_Task_1_ContinueWith_System_Action_1_System_Threading_Tasks_Task_1_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31721
#72 0x003a4ed4 in m_3d8 at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:32304
#73 0x003a44b0 in m_System_Threading_Tasks_Task_1__c__DisplayClass1__ContinueWithb__0_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31872
#74 0x003a3b3c in m_System_Threading_Tasks_Task__c__DisplayClass8__ContinueWithb__7_System_Threading_Tasks_Task ()
#75 0x00519a84 in m_System_Threading_Tasks_Task__c__DisplayClass3_1__c__DisplayClass5_int__ContinueWithb__2 ()
#76 0x003a36b8 in m_System_Threading_Tasks_Task___cctorb__23_System_Action at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31346
#77 0x004a9548 in m_1ce9 at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:165091
#78 0x003a4140 in m_System_Threading_Tasks_Task_1_RunContinuations at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31766
#79 0x003a4278 in m_System_Threading_Tasks_Task_1_TrySetResult_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31801
#80 0x003a4654 in m_System_Threading_Tasks_TaskCompletionSource_1_TrySetResult_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31950
#81 0x003a3a60 in m_System_Threading_Tasks_Task__c__DisplayClass3_1__c__DisplayClass5__ContinueWithb__2 at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31498
#82 0x003a36b8 in m_System_Threading_Tasks_Task___cctorb__23_System_Action at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31346
#83 0x003a39bc in m_System_Threading_Tasks_Task__c__DisplayClass3_1__ContinueWithb__1_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31474
#84 0x003a4140 in m_System_Threading_Tasks_Task_1_RunContinuations at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31766
#85 0x003a4278 in m_System_Threading_Tasks_Task_1_TrySetResult_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31801
#86 0x003a4654 in m_System_Threading_Tasks_TaskCompletionSource_1_TrySetResult_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31950
#87 0x003a0848 in m_Parse_PlatformHooks__c__DisplayClass2f__c__DisplayClass35__RequestAsyncb__2a_UnityEngine_WWW at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:29764
#88 0x003a0334 in m_Parse_PlatformHooks__c__DisplayClass20__RegisterNetworkRequestb__1f at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:29572
#89 0x003a0dbc in m_Parse_PlatformHooks__RunDispatcherd__39_MoveNext at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:29915
#90 0x0126ecec in scripting_method_invoke(ScriptingMethod*, MonoObject*, ScriptingArguments&, MonoException**) at /Applications/buildAgent/work/d63dfc6385190b60/Runtime/Scripting/Backend/Mono/ScriptingBackendApi_Mono.cpp:196
#91 0x01309390 in ScriptingInvocation::Invoke(MonoException**, bool) at /Applications/buildAgent/work/d63dfc6385190b60/Runtime/Scripting/Backend/ScriptingInvocation.cpp:128
#92 0x0130935c in ScriptingInvocation::Invoke(MonoException**) at /Applications/buildAgent/work/d63dfc6385190b60/Runtime/Scripting/Backend/ScriptingInvocation.cpp:113
#93 0x01309308 in bool ScriptingInvocation::Invoke<bool>(MonoException**) at /Applications/buildAgent/work/d63dfc6385190b60/Runtime/Scripting/Backend/ScriptingInvocation.cpp:80
#94 0x012df7e4 in Coroutine::InvokeMoveNext(MonoException**) at /Applications/buildAgent/work/d63dfc6385190b60/Runtime/Mono/Coroutine.cpp:196
#95 0x012df57c in Coroutine::Run() at /Applications/buildAgent/work/d63dfc6385190b60/Runtime/Mono/Coroutine.cpp:221
#96 0x012df544 in Coroutine::ContinueCoroutine(Object*, void*) at /Applications/buildAgent/work/d63dfc6385190b60/Runtime/Mono/Coroutine.cpp:78
#97 0x012593c4 in DelayedCallManager::Update(int) at /Applications/buildAgent/work/d63dfc6385190b60/Runtime/GameCode/CallDelayed.cpp:164
#98 0x012d0630 in PlayerLoop(bool, bool, IHookEvent*) at /Applications/buildAgent/work/d63dfc6385190b60/Runtime/Misc/Player.cpp:1880
#99 0x01117878 in UnityPlayerLoop at /Applications/buildAgent/work/d63dfc6385190b60/PlatformDependent/iPhonePlayer/LibEntryPoint.mm:241
#100 0x00d2ff34 in -[UnityAppController(Rendering) repaint] at /Users/me/myproj/build/device/Classes/UnityAppController+Rendering.mm:55
It seems parse's JSON parsing runs on the mainthread, but why?
What's going on here and how can I avoid having the mainthread getting stalled?
//note: occasionally, the stalling does not happen (the query succeeds nevertheless) - as if Parse would sometimes successfully do the parsing on another thread, most of the time however do it on the mainthread.

Answer 1

I'd suggest that if you hook up the Unity profiler to your device, you're going to find that your Linq query is creating immense amounts of garbage - and it's the garbage collection on your device that is stalling your main thread.
Reimplement your leaderboard functionality the hard way and don't use Linq. Linq saves programmers time - not machines, don't use it in your games.