Hang of worker process php-fpm - linux

I faced with emegrency problem on my server 1 hour ago.
I user php5-fpm cgi + nginx. The following php code leads to cgi worker hung.
$fp = fsockopen("localhost", 6699, $errno, $errstr, 30);
$result = '';
if (!$fp) {
$this->SysLog2(ALERT, "Unable to send command to RmServer. $errstr ($errno)");
} else {
fwrite($fp, $command);
while (!feof($fp)) {
$result .= fgets($fp, 128);
}
fclose($fp);
}
#0 0x00007f58ac392313 in __GI___poll (fds=<optimized out>, nfds=<optimized out>, timeout=<optimized out>) at ../sysdeps/unix/sysv/linux/poll.c:87
#1 0x00000000006682f4 in ?? ()
#2 0x0000000000470d8a in ?? ()
#3 0x000000000065b708 in ?? ()
#4 0x000000000065c3d9 in _php_stream_get_line ()
#5 0x00000000005daefd in zif_fgets ()
#6 0x000000000070f52d in ?? ()
#7 0x00000000006c009b in execute ()
#8 0x000000000069b600 in zend_execute_scripts ()
#9 0x0000000000647c73 in php_execute_script ()
#10 0x000000000042ba55 in ?? ()
#11 0x00007f58ac2cb76d in __libc_start_main (main=0x42ab00, argc=3, ubp_av=0x7fffa0becca8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>,
stack_end=0x7fffa0becc98) at libc-start.c:226
#12 0x000000000042c2b5 in _start ()
[09-Feb-2014 21:30:28.789264] DEBUG: pid 10529, fpm_scoreboard_init_main(), line 40: got clock tick '100'
[09-Feb-2014 21:30:28.789503] DEBUG: pid 10529, fpm_event_init_main(), line 331: event module is epoll and 3 fds have been reserved
[09-Feb-2014 21:30:28.789576] NOTICE: pid 10529, fpm_init(), line 80: fpm is running, pid 10529
[09-Feb-2014 21:30:28.789603] DEBUG: pid 10529, fpm_pctl_heartbeat(), line 460: heartbeat have been set up with a timeout of 3333ms
[09-Feb-2014 21:30:28.789617] DEBUG: pid 10529, fpm_event_loop(), line 360: 1280 bytes have been reserved in SHM
[09-Feb-2014 21:30:28.789624] NOTICE: pid 10529, fpm_event_loop(), line 361: ready to handle connections
[09-Feb-2014 21:30:29.790664] DEBUG: pid 10529, fpm_pctl_perform_idle_server_maintenance(), line 362: [pool www] currently 0 active children, 0 spare children
[09-Feb-2014 21:30:31.947235] DEBUG: pid 10529, fpm_children_make(), line 421: [pool www] child 10530 started
[09-Feb-2014 21:30:31.947278] DEBUG: pid 10529, fpm_pctl_on_socket_accept(), line 536: [pool www] got accept without idle child available .... I forked
[09-Feb-2014 21:30:31.947286] DEBUG: pid 10529, fpm_event_loop(), line 409: event module triggered 1 events
[09-Feb-2014 21:30:32.794231] DEBUG: pid 10529, fpm_pctl_perform_idle_server_maintenance(), line 362: [pool www] currently 1 active children, 0 spare children
[09-Feb-2014 21:30:33.795326] DEBUG: pid 10529, fpm_pctl_perform_idle_server_maintenance(), line 362: [pool www] currently 1 active children, 0 spare children
[09-Feb-2014 21:30:42.126558] WARNING: pid 10529, fpm_request_check_timed_out(), line 271: [pool www] child 10530, script '/proj/manager/ssl-server/RPRCP/queues.php' (request: "GET /RPRCP/queues.php") executing too slow (10.179030 sec), logging
[09-Feb-2014 21:30:42.126626] DEBUG: pid 10529, fpm_got_signal(), line 72: received SIGCHLD
[09-Feb-2014 21:30:42.126640] NOTICE: pid 10529, fpm_children_bury(), line 227: child 10530 stopped for tracing
[09-Feb-2014 21:30:42.126647] NOTICE: pid 10529, fpm_php_trace(), line 142: about to trace 10530
[09-Feb-2014 21:30:42.126773] NOTICE: pid 10529, fpm_php_trace(), line 170: finished trace of 10530
[09-Feb-2014 21:30:42.126787] DEBUG: pid 10529, fpm_event_loop(), line 409: event module triggered 1 events
[09-Feb-2014 21:30:42.802499] DEBUG: pid 10529, fpm_pctl_perform_idle_server_maintenance(), line 362: [pool www] currently 1 active children, 0 spare children
Most likely workers hungs at '_php_stream_get_line'. But why it alive after SIGCHLD?
Also I can't find 'slow' log.. I have the followith statemens in config:
prefix = /var/log
slowlog = /php5-fpm.log.slow
request_slowlog_timeout = 10s
EDIT:
My remote process deadlocked by a mutex. But why php behaviour is so dangerous?

Related

Terraform plan with 1Password provider fails with rpc error unavailable desc transport is closing

After adding some new secrets to Terraform using the 1Password provider, we saw an error without much helpful output.
$ terraform plan
...
Error: rpc error: code = Unavailable desc = transport is closing
Error: rpc error: code = Canceled desc = context canceled
...
Terraform provider:
terraform {
required_providers {
onepassword = {
source = "anasinnyk/onepassword"
version = "~> 1.2.1"
}
}
required_version = "~> 0.13"
}
Terraform yml:
data "onepassword_item_password" "search_cloud_id" {
name = "Azure Elastic Cloud ID"
vault = data.onepassword_vault.vault_name.id
}
data "onepassword_item_password" "search_api_key" {
name = "Azure Elastic Cloud API key"
vault = data.onepassword_vault.vault_name.id
}
resource "kubernetes_secret" "search" {
metadata {
name = "search"
namespace = kubernetes_namespace.production.id
}
data = {
"ELASTICSEARCH_CLOUD_ID" = data.onepassword_item_password.api_search_cloud_id.password
"ELASTICSEARCH_API_KEY" = data.onepassword_item_password.api_search_api_key.password
}
type = "Opaque"
}
We managed to get some useful output by removing one data reference at a time, which lead to the errors printing:
panic: runtime error: invalid memory address or nil pointer dereference
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x147d1bd]
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1:
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: goroutine 194 [running]:
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: github.com/anasinnyk/terraform-provider-1password/onepassword.resourceItemPasswordRead(0x19418a0, 0xc0004ac540, 0xc000096f80, 0x173d040, 0xc0007ac740, 0xc0003bce40, 0xc000119910, 0x100c9b8)
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: github.com/anasinnyk/terraform-provider-1password/onepassword/resource_item_password.go:75 +0x18d
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0xc0004613f0, 0x1941820, 0xc000384300, 0xc000096f80, 0x173d040, 0xc0007ac740, 0x0, 0x0, 0x0)
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: github.com/hashicorp/terraform-plugin-sdk/v2#v2.0.0/helper/schema/resource.go:288 +0x1ec
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).ReadDataApply(0xc0004613f0, 0x1941820, 0xc000384300, 0xc000304b80, 0x173d040, 0xc0007ac740, 0xc0007ac740, 0xc000304b80, 0x0, 0x0)
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: github.com/hashicorp/terraform-plugin-sdk/v2#v2.0.0/helper/schema/resource.go:489 +0xff
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: github.com/hashicorp/terraform-plugin-sdk/v2/internal/helper/plugin.(*GRPCProviderServer).ReadDataSource(0xc00026e6a0, 0x1941820, 0xc000384300, 0xc0003842c0, 0xc00026e6a0, 0xc00026e6b0, 0x185a058)
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: github.com/hashicorp/terraform-plugin-sdk/v2#v2.0.0/internal/helper/plugin/grpc_provider.go:1102 +0x4c5
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: github.com/hashicorp/terraform-plugin-sdk/v2/internal/tfplugin5._Provider_ReadDataSource_Handler.func1(0x1941820, 0xc000384300, 0x17dcd60, 0xc0003842c0, 0xc000384300, 0x1773c80, 0xc0004ac401, 0xc000304640)
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: github.com/hashicorp/terraform-plugin-sdk/v2#v2.0.0/internal/tfplugin5/tfplugin5.pb.go:3348 +0x86
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: github.com/hashicorp/terraform-plugin-sdk/v2/plugin.Serve.func3.1(0x19418e0, 0xc0003d4480, 0x17dcd60, 0xc0003842c0, 0xc000304620, 0xc000304640, 0xc0007c8ba0, 0x11b81c8, 0x17c7a20, 0xc0003d4480)
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: github.com/hashicorp/terraform-plugin-sdk/v2#v2.0.0/plugin/serve.go:76 +0x87
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: github.com/hashicorp/terraform-plugin-sdk/v2/internal/tfplugin5._Provider_ReadDataSource_Handler(0x17fdb60, 0xc00026e6a0, 0x19418e0, 0xc0003d4480, 0xc0004ac4e0, 0xc00000d080, 0x19418e0, 0xc0003d4480, 0xc000010090, 0x90)
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: github.com/hashicorp/terraform-plugin-sdk/v2#v2.0.0/internal/tfplugin5/tfplugin5.pb.go:3350 +0x14b
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: google.golang.org/grpc.(*Server).processUnaryRPC(0xc00027ae00, 0x1949c60, 0xc000103380, 0xc00018e000, 0xc00020acf0, 0x1e49910, 0x0, 0x0, 0x0)
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: google.golang.org/grpc#v1.30.0/server.go:1171 +0x50a
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: google.golang.org/grpc.(*Server).handleStream(0xc00027ae00, 0x1949c60, 0xc000103380, 0xc00018e000, 0x0)
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: google.golang.org/grpc#v1.30.0/server.go:1494 +0xccd
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: google.golang.org/grpc.(*Server).serveStreams.func1.2(0xc0000382e0, 0xc00027ae00, 0x1949c60, 0xc000103380, 0xc00018e000)
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: google.golang.org/grpc#v1.30.0/server.go:834 +0xa1
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: created by google.golang.org/grpc.(*Server).serveStreams.func1
2021-08-27T15:34:29.367+0930 [DEBUG] plugin.terraform-provider-onepassword_v1.2.1: google.golang.org/grpc#v1.30.0/server.go:832 +0x204
2021-08-27T15:34:29.368+0930 [WARN] plugin.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing"
2021/08/27 15:34:29 [ERROR] eval: *terraform.evalReadDataRefresh, err: rpc error: code = Unavailable desc = transport is closing
2021/08/27 15:34:29 [ERROR] eval: *terraform.evalReadDataRefresh, err: rpc error: code = Unavailable desc = transport is closing
2021/08/27 15:34:29 [ERROR] eval: *terraform.evalReadDataRefresh, err: rpc error: code = Unavailable desc = transport is closing
2021/08/27 15:34:29 [ERROR] eval: *terraform.EvalSequence, err: rpc error: code = Unavailable desc = transport is closing
2021-08-27T15:34:29.369+0930 [DEBUG] plugin: plugin process exited: path=.terraform/plugins/registry.terraform.io/anasinnyk/onepassword/1.2.1/darwin_amd64/terraform-provider-onepassword_v1.2.1 pid=17549 error="exit status 2"
2021/08/27 15:34:29 [ERROR] eval: *terraform.EvalSequence, err: rpc error: code = Unavailable desc = transport is closing
2021/08/27 15:34:29 [TRACE] [walkRefresh] Exiting eval tree: data.onepassword_item_password.search_api_key
2021/08/27 15:34:29 [ERROR] eval: *terraform.EvalSequence, err: rpc error: code = Unavailable desc = transport is closing
2021/08/27 15:34:29 [TRACE] vertex "data.onepassword_item_password.search_api_key": visit complete
2021/08/27 15:34:29 [TRACE] vertex "data.onepassword_item_password.search_api_key": dynamic subgraph encountered errors
2021/08/27 15:34:29 [TRACE] vertex "data.onepassword_item_password.search_api_key": visit complete
2021/08/27 15:34:29 [TRACE] vertex "data.onepassword_item_password.search_api_key (expand)": dynamic subgraph encountered errors
2021/08/27 15:34:29 [TRACE] vertex "data.onepassword_item_password.search_api_key (expand)": visit complete
2021/08/27 15:34:29 [TRACE] dag/walk: upstream of "provider[\"registry.terraform.io/hashicorp/kubernetes\"] (close)" errored, so skipping
2021/08/27 15:34:29 [TRACE] dag/walk: upstream of "provider[\"registry.terraform.io/anasinnyk/onepassword\"] (close)" errored, so skipping
2021/08/27 15:34:29 [TRACE] dag/walk: upstream of "root" errored, so skipping
2021-08-27T15:34:29.501+0930 [DEBUG] plugin: plugin exited
2021-08-27T15:34:29.502+0930 [WARN] plugin.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing"
2021-08-27T15:34:29.507+0930 [DEBUG] plugin: plugin process exited: path=.terraform/plugins/registry.terraform.io/hashicorp/kubernetes/1.13.3/darwin_amd64/terraform-provider-kubernetes_v1.13.3_x4 pid=17673
2021-08-27T15:34:29.507+0930 [DEBUG] plugin: plugin exited
!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!
Terraform crashed! This is always indicative of a bug within Terraform.
A crash log has been placed at "crash.log" relative to your current
working directory. It would be immensely helpful if you could please
report the crash with Terraform[1] so that we can fix this.
When reporting bugs, please include your terraform version. That
information is available on the first line of crash.log. You can also
get it by running 'terraform --version' on the command line.
SECURITY WARNING: the "crash.log" file that was created may contain
sensitive information that must be redacted before it is safe to share
on the issue tracker.
[1]: https://github.com/hashicorp/terraform/issues
!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!
This led us to find that one of our team members managed to create two 1Password entries with the same name in the same vault.
After deleting the duplicate entry in 1Password, terraform plan ran without error again.

SOGO: No child available to handle incoming request / pid xxx has been hanging in the same request for x minutes

I am trying to add SOGo to an already working server with Postfix + Dovecot.
The server is Centos 7, 2 core with 3Gb RAM with less than 10 users. Sogo installed from official repo: sogo-2.3.8-1.el7.centos.x86_64
/etc/sysconfig/sogo is set up for 10 workers: PREFORK=10"
Sogo is configured with 10 workers with this configuration:
WOListenQueueSize=10;
WOWatchDogRequestTimeout=60;
SOGoMaximumPingInterval = 354;
SOGoMaximumSyncInterval = 354;
SOGoInternalSyncInterval = 15;
SOGoMaximumSyncWindowSize = 50;
SOGoMaximumSyncResponseSize = 2048;
The problem seems to be whith activesync clients (ms outlook). SOGo processes starts eating all the RAM and sometimes hangs (process can't be killed with signal 15). The problem seems to be related to The log file reports:
Feb 19 13:30:26 sogod [13164]: Sleeping 15 seconds while detecting changes in Ping...
Feb 19 13:30:26 sogod [13163]: Sleeping 15 seconds while detecting changes in Ping...
Feb 19 13:30:26 sogod [13150]: [ERROR] No child available to handle incoming request!
Feb 19 13:30:26 sogod [13155]: Sleeping 15 seconds while detecting changes in Ping...
Feb 19 13:30:27 sogod [13152]: Sleeping 15 seconds while detecting changes in Ping...
Feb 19 13:30:27 sogod [13150]: [WARN] pid 13168 has been hanging in the same request for 3 minutes
Feb 19 13:30:28 sogod [13150]: [ERROR] No child available to handle incoming request!
Feb 19 13:30:28 sogod [13150]: [WARN] pid 13164 has been hanging in the same request for 3 minutes
Feb 19 13:30:29 sogod [13150]: [ERROR] No child available to handle incoming request!
Feb 19 13:30:29 sogod [13150]: [WARN] pid 13163 has been hanging in the same request for 2 minutes
Feb 19 13:30:30 sogod [13168]: Sleeping 15 seconds while detecting changes in Ping...
Feb 19 13:30:30 sogod [13150]: [WARN] pid 13151 has been hanging in the same request for 1 minutes
Feb 19 13:35:03 sogod [13150]: [WARN] pid 13153 has been hanging in the same request for 5 minutes
Feb 19 13:35:04 sogod [13150]: [ERROR] No child available to handle incoming request!
Feb 19 13:35:06 sogod [13150]: [ERROR] No child available to handle incoming request!
Feb 19 13:35:07 sogod [13153]: Sleeping 15 seconds while detecting changes in Ping...
Feb 19 13:35:07 sogod [13150]: [ERROR] No child available to handle incoming request!
Feb 19 13:35:08 sogod [13164]: Sleeping 15 seconds while detecting changes in Ping...
I used gdb to get a trace of one of the hanging processes. The response is this:
#0 0x00007f176ddcc49d in nanosleep () from /lib64/libc.so.6
#1 0x00007f176ddcc334 in sleep () from /lib64/libc.so.6
#2 0x00007f17608e8a99 in -[SOGoActiveSyncDispatcher processPing:inResponse:] () from /usr/lib64/GNUstep/SOGo/ActiveSync.SOGo/./ActiveSync
#3 0x00007f17608eee4b in -[SOGoActiveSyncDispatcher dispatchRequest:inResponse:context:] () from /usr/lib64/GNUstep/SOGo/ActiveSync.SOGo/./ActiveSync
#4 0x00007f1760d50d84 in -[SOGoMicrosoftActiveSyncActions microsoftServerActiveSyncAction] () from /usr/lib64/GNUstep/SOGo/MainUI.SOGo/./MainUI
#5 0x00007f1773e61113 in -[WODirectAction performActionNamed:] () from /lib64/libNGObjWeb.so.4.9
#6 0x00007f1773ee3834 in -[SoActionInvocation callOnObject:withPositionalParametersWhenNotNil:inContext:] () from /lib64/libNGObjWeb.so.4.9
#7 0x00007f1773edee98 in -[SoObjectMethodDispatcher dispatchInContext:] () from /lib64/libNGObjWeb.so.4.9
#8 0x00007f1773ee0f09 in -[SoObjectRequestHandler handleRequest:inContext:session:application:] () from /lib64/libNGObjWeb.so.4.9
#9 0x00007f1773e72753 in -[WORequestHandler handleRequest:] () from /lib64/libNGObjWeb.so.4.9
#10 0x00007f1773e3433c in -[WOCoreApplication dispatchRequest:usingHandler:] () from /lib64/libNGObjWeb.so.4.9
#11 0x00007f1773e3463f in -[WOCoreApplication dispatchRequest:] () from /lib64/libNGObjWeb.so.4.9
#12 0x00007f17751fbb4d in -[SOGo dispatchRequest:] ()
#13 0x00007f1773ed1a85 in -[WOHttpTransaction _run] () from /lib64/libNGObjWeb.so.4.9
#14 0x00007f1773ed1de5 in -[WOHttpTransaction run] () from /lib64/libNGObjWeb.so.4.9
#15 0x00007f1773ecd9e4 in -[WOHttpAdaptor runConnection:] () from /lib64/libNGObjWeb.so.4.9
#16 0x00007f1773ecdc02 in -[WOHttpAdaptor _handleAcceptedConnection:] () from /lib64/libNGObjWeb.so.4.9
#17 0x00007f1773ecdff7 in -[WOHttpAdaptor _handleConnection:] () from /lib64/libNGObjWeb.so.4.9
#18 0x00007f1773ece2c3 in -[WOHttpAdaptor acceptControlMessage:] () from /lib64/libNGObjWeb.so.4.9
#19 0x00007f177261613f in -[NSNotificationCenter _postAndRelease:] () from /lib64/libgnustep-base.so.1.24
#20 0x00007f17732a0e3d in -[NSObject(FileObjectWatcher) receivedEvent:type:extra:forMode:] () from /lib64/libNGExtensions.so.4.9
#21 0x00007f177271ceea in -[GSRunLoopCtxt pollUntil:within:] () from /lib64/libgnustep-base.so.1.24
#22 0x00007f177265d870 in -[NSRunLoop acceptInputForMode:beforeDate:] () from /lib64/libgnustep-base.so.1.24
#23 0x00007f177265dd22 in -[NSRunLoop runMode:beforeDate:] () from /lib64/libgnustep-base.so.1.24
#24 0x00007f1773e33b94 in -[WOCoreApplication run] () from /lib64/libNGObjWeb.so.4.9
#25 0x00007f17751fb1fe in -[SOGo run] ()
#26 0x00007f1773e7bc5e in -[WOWatchDog _runChildWithControlSocket:] () from /lib64/libNGObjWeb.so.4.9
#27 0x00007f1773e7c0f1 in -[WOWatchDog _spawnChild:] () from /lib64/libNGObjWeb.so.4.9
#28 0x00007f1773e7c7d9 in -[WOWatchDog _ensureChildren] () from /lib64/libNGObjWeb.so.4.9
#29 0x00007f1773e7d7f6 in -[WOWatchDog run:argc:argv:] () from /lib64/libNGObjWeb.so.4.9
#30 0x00007f1773e7df21 in WOWatchDogApplicationMain () from /lib64/libNGObjWeb.so.4.9
#31 0x00007f17751fa491 in main ()
Any help please?
vim /etc/sysconfig/sogo
PREFORK=10
USER=sogo
vim /etc/rc.d/init.d/sogod
PREFORK=10

chef 12 configuration error - object not found while running knife cli

Running chef 12 on Ubuntu 14.1. Using self signed certs to setup the server, when I try to run knife commands from my client it fails with following error. Any operation has same error. chef server logs do not have any errors or info during the query.
knife config
[root#ip-10-233-2-40 ~]# cat ~/.chef/knife.rb
log_level :debug
log_location STDOUT
node_name 'admin'
client_key '/root/.chef/admin.pem'
validation_client_name 'dev'
validation_key '/root/.chef/dev-validator.pem'
chef_server_url 'https://chef.example.com/organizations/dev'
syntax_check_cache_path '/root/.chef/syntax_check_cache'
root#ip-10-233-2-177:~/ssl-certs# chef-server-ctl status
run: bookshelf: (pid 1092) 1998s; run: log: (pid 1064) 1998s
run: nginx: (pid 6140) 723s; run: log: (pid 1063) 1998s
run: oc_bifrost: (pid 1077) 1998s; run: log: (pid 1058) 1998s
run: oc_id: (pid 1091) 1998s; run: log: (pid 1061) 1998s
run: opscode-erchef: (pid 1090) 1998s; run: log: (pid 1066) 1998s
run: opscode-expander: (pid 1076) 1998s; run: log: (pid 1060) 1998s
run: opscode-expander-reindexer: (pid 1096) 1998s; run: log: (pid 1059) 1998s
run: opscode-solr4: (pid 1075) 1998s; run: log: (pid 1057) 1998s
run: postgresql: (pid 1085) 1998s; run: log: (pid 1056) 1998s
run: rabbitmq: (pid 1062) 1998s; run: log: (pid 1046) 1998s
run: redis_lb: (pid 6124) 723s; run: log: (pid 1065) 1998s
[root#ip-10-233-2-40 ~]# knife environment create staging
ERROR: The object you are looking for could not be found
/opt/chef/embedded/lib/ruby/2.1.0/net/http/response.rb:325:in `stream_check': undefined method `closed?' for nil:NilClass (NoMethodError)
from /opt/chef/embedded/lib/ruby/2.1.0/net/http/response.rb:199:in `read_body'
from /opt/chef/embedded/lib/ruby/2.1.0/net/http/response.rb:226:in `body'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/knife.rb:499:in `rescue in format_rest_error'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/knife.rb:497:in `format_rest_error'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/knife.rb:459:in `humanize_http_exception'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/knife.rb:418:in `humanize_exception'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/knife.rb:409:in `rescue in run_with_pretty_exceptions'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/knife.rb:400:in `run_with_pretty_exceptions'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/knife.rb:203:in `run'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/application/knife.rb:142:in `run'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/bin/knife:25:in `<top (required)>'
from /usr/bin/knife:54:in `load'
from /usr/bin/knife:54:in `<main>'`enter code here`
Update
[root#ip-10-233-2-40 ~]# knife client list -VV
INFO: Using configuration from /root/.chef/knife.rb
DEBUG: Chef::HTTP calling Chef::HTTP::JSONInput#handle_request
DEBUG: Chef::HTTP calling Chef::HTTP::JSONOutput#handle_request
DEBUG: Chef::HTTP calling Chef::HTTP::CookieManager#handle_request
DEBUG: Chef::HTTP calling Chef::HTTP::Decompressor#handle_request
DEBUG: Chef::HTTP calling Chef::HTTP::Authenticator#handle_request
DEBUG: Signing the request as admin
DEBUG: Chef::HTTP calling Chef::HTTP::RemoteRequestID#handle_request
DEBUG: Using 10.233.0.182:3128 for proxy
DEBUG: Initiating GET to https://chef.example.com/organizations/dev/clients
DEBUG: ---- HTTP Request Header Data: ----
DEBUG: Accept: application/json
DEBUG: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
DEBUG: X-OPS-SIGN: algorithm=sha1;version=1.0;
DEBUG: X-OPS-USERID: admin
DEBUG: X-OPS-TIMESTAMP: 2015-10-21T17:40:17Z
DEBUG: X-OPS-CONTENT-HASH: 2jmj7l5rSw0yVb/vlWAYkK/YBwk=
DEBUG: X-OPS-AUTHORIZATION-1: m/vlWcZBPE7XUN7qhX6t/T9hXTT+2x/JehpOYq6My1ffEID6n+U+Xc+lHWto
DEBUG: X-OPS-AUTHORIZATION-2: Lq4ZEfNT1ltZkkYZ9Ii8EoF3eajUQmb2buwKMWae3yvxrZ5rgllJPf5q4gy3
DEBUG: X-OPS-AUTHORIZATION-3: IEqUUst+KzmoRHCiC1LeYxKXy+oeo45F4Vw4xHlOWgS0piqXfrmXnkrxs8Um
DEBUG: X-OPS-AUTHORIZATION-4: ZDqdLvcQ10WjoW9Wz4F2+fRh/BdRHjwMF80LVPwrtylf+GbdIhmCU3xxVvOq
DEBUG: X-OPS-AUTHORIZATION-5: w1Z2p03UcpRfMZy1pQV59A0Y3yv57Db5n3PJdjD9TlitNK++/HXcqO3IfO2U
DEBUG: X-OPS-AUTHORIZATION-6: 0QbZYZaeGSkJw0ArQDeffnjbpzAhSXhUfbs+in9tRg==
DEBUG: HOST: chef.example.com:443
DEBUG: X-Ops-Server-API-Version: 1
DEBUG: X-REMOTE-REQUEST-ID: 6a00a52a-7eeb-43d6-920d-fffc685c1b2a
DEBUG: ---- End HTTP Request Header Data ----
/opt/chef/embedded/lib/ruby/2.1.0/net/http/response.rb:119:in `error!': 404 "Not Found" (Net::HTTPServerException)
from /opt/chef/embedded/lib/ruby/2.1.0/net/http/response.rb:128:in `value'
from /opt/chef/embedded/lib/ruby/2.1.0/net/http.rb:915:in `connect'
from /opt/chef/embedded/lib/ruby/2.1.0/net/http.rb:863:in `do_start'
from /opt/chef/embedded/lib/ruby/2.1.0/net/http.rb:852:in `start'
from /opt/chef/embedded/lib/ruby/2.1.0/net/http.rb:1375:in `request'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/http/basic_client.rb:65:in `request'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/http.rb:266:in `block in send_http_request'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/http.rb:298:in `block in retrying_http_errors'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/http.rb:296:in `loop'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/http.rb:296:in `retrying_http_errors'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/http.rb:260:in `send_http_request'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/http.rb:143:in `request'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/http.rb:110:in `get'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/api_client_v1.rb:198:in `list'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/knife/client_list.rb:38:in `run'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/knife.rb:405:in `block in run_with_pretty_exceptions'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/local_mode.rb:44:in `with_server_connectivity'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/knife.rb:404:in `run_with_pretty_exceptions'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/knife.rb:203:in `run'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/application/knife.rb:142:in `run'
from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/bin/knife:25:in `<top (required)>'
from /usr/bin/knife:54:in `load'
from /usr/bin/knife:54:in `<main>'
[root#ip-10-233-2-40 ~]# telnet chef.example.com 443
Trying 10.233.2.177...
Connected to chef.example.com.
Escape character is '^]'.

Parse query stalls main thread for up to a minute

I'm using the Parse package from the asset store to save userdata for my Unity3D mobile game (iOS/Android). Now I've got the problem that the parse queries I make stall my unity mainthread for a long time. I was under the impression that the 'Task' concept of Parse was there to avoid that but somehow it doesn't seem to work. The stalling happens in the Editor as well, only a few seconds compared to up to a minute on mobile. The query.FindAsync().ContinueWith(t => {myContinuationBlock}); method returns immediately and subsequent calls get executed. The Unity3d main thread stalls a few seconds after the query is executed and immediately BEFORE the code in {myContinuationBlock} is executed.
I made a video of the problem. In the video you can also see that only Unity is being stalled, the (native) iAds on the bottom are not affected:
https://www.youtube.com/watch?v=XWaCGk9Hbus (the stall is from 0:07 - 1:07)
Here is my query code (note, this code runs through without problem. The stall happens later, right before the ContinueWith code gets executed):
public static void RetrieveHighScores()
{
DebugLog.Log ("start of RetrieveHighScores() method");
try
{
ParseQuery<ParseUser> query = ParseUser.Query
.WhereGreaterThan (_highScoreKey, 1)
.WhereGreaterThan("updatedAt", DateTime.Now - TimeSpan.FromDays(1))
.OrderByDescending(_score24HoursKey)
.Limit (1000);
DebugLog.Log ("query.FindAsync()");
query.FindAsync().ContinueWith(t =>
{
try
{
if (t.IsFaulted || t.IsCanceled)
{
DebugLog.Log("Retrieving 24 Hours High Scores Failed");
}
else
{
DebugLog.Log("Retrieving 24 Hours High Scores successful! ");
IEnumerable<ParseUser> results = t.Result;
foreach(ParseUser user in results)
{
//doing something with user..
}
DebugLog.Log ("24 Hours High Scores processed. "+_24HourHighScoreList.Count.ToString()+" entries.");
}
}
catch(System.Exception e)
{
DebugLog.Log("Failed to retrieve 24 Hours High Scores. Reason: " + e.Message);
}
});
DebugLog.Log ("FindAsync() returned");
}
catch (System.Exception e)
{
DebugLog.Log("Failed to retrieve 24 Hours High Scores. Reason: " + e.Message);
}
try
{
ParseQuery<ParseUser>query = ParseUser.Query
.WhereGreaterThan (_highScoreKey, 1)
.OrderByDescending(_highScoreKey)
.Limit (1000);
DebugLog.Log ("query.FindAsync()");
query.FindAsync().ContinueWith(t =>
{
DebugLog.Log("Retrieving Alltime High Scores successful! ");
IEnumerable<ParseUser> results = t.Result;
foreach(ParseUser user in results)
{
//doing something with user...
}
DebugLog.Log ("Alltime High Scores processed. "+_allTimeHighScoreList.Count.ToString()+" entries.");
});
DebugLog.Log ("FindAsync() returned");
}
catch (System.Exception e)
{
DebugLog.Log("Failed to retrieve alltime Highscores. Reason: " + e.Message);
}
DebugLog.Log ("end of RetrieveHighScores() method");
}
So, the very next thing I see in the console output after the stall is
"Retrieving Alltime High Scores successful! "
Now, I know that i'm querying 1000 objects here and yes there may be better ways for implementing highscores, but I don't understand why this code is stalling my Unity3D mainthread? Why does it stall sometimes for up to a minute and sometimes it's not noticable at all?
This is a serious problem as my game is released already. It started surfacing only once the user database grew bigger and now I need a quick fix for it.
The stalling does not happen if I don't call the RetrieveHighScores() function so it must be something happening after the parse code received the data from the server and passes it to ContinueWith code.
if I click pause in XCode during the stall, I see the following:
Thread 1, Queue : com.apple.main-thread
#0 0x017eaeb0 in GC_mark_from ()
#1 0x017eb520 in GC_mark_some ()
#2 0x017e5d1c in GC_stopped_mark ()
#3 0x017e6228 in GC_try_to_collect_inner ()
#4 0x017e64f0 in GC_collect_or_expand ()
#5 0x017e6a38 in GC_allocobj ()
#6 0x017e957c in GC_generic_malloc_inner ()
#7 0x017e965c in GC_generic_malloc ()
#8 0x017e9920 in GC_malloc_atomic ()
#9 0x01783814 in mono_array_new_specific ()
#10 0x00c5886c in m_wrapper_managed_to_native_object___icall_wrapper_mono_array_new_specific_intptr_int at /Users/me/myproj/build/device/Libraries/mscorlib.dll.s:187982
#11 0x009ee54c in m_System_Text_RegularExpressions_Interpreter_ResetGroups at /Users/me/myproj/build/device/Libraries/System.dll.s:6462
#12 0x009eb864 in m_System_Text_RegularExpressions_Interpreter_Reset at /Users/me/myproj/build/device/Libraries/System.dll.s:5761
#13 0x009edb6c in m_157 at /Users/me/myproj/build/device/Libraries/System.dll.s:6244
#14 0x009ebc54 in m_155 at /Users/me/myproj/build/device/Libraries/System.dll.s:5813
#15 0x009eb804 in m_System_Text_RegularExpressions_Interpreter_Scan_System_Text_RegularExpressions_Regex_string_int_int at /Users/me/myproj/build/device/Libraries/System.dll.s:5745
#16 0x009f3ce8 in m_System_Text_RegularExpressions_Regex_Match_string_int at /Users/me/myproj/build/device/Libraries/System.dll.s:9106
#17 0x00370204 in m_Parse_Internal_Json_Accept_string_int_System_Text_RegularExpressions_Regex_int__System_Text_RegularExpressions_Match_ at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:3770
#18 0x0036fa80 in m_Parse_Internal_Json_ParseString_string_int_int__object_ at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:3577
#19 0x0036f56c in m_Parse_Internal_Json_ParseMember_string_int_int__object_ at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:3447
#20 0x0036f3b8 in m_Parse_Internal_Json_ParseObject_string_int_int__object_ at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:3413
#21 0x0036f99c in m_Parse_Internal_Json_ParseValue_string_int_int__object_ at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:3553
#22 0x0036f780 in m_Parse_Internal_Json_ParseArray_string_int_int__object_ at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:3500
#23 0x0036f9b8 in m_Parse_Internal_Json_ParseValue_string_int_int__object_ at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:3556
#24 0x0036f5ec in m_Parse_Internal_Json_ParseMember_string_int_int__object_ at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:3457
#25 0x0036f3b8 in m_Parse_Internal_Json_ParseObject_string_int_int__object_ at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:3413
#26 0x0036e920 in m_Parse_Internal_Json_Parse_string at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:3143
#27 0x00383168 in m_Parse_ParseClient_DeserializeJsonString_string at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:13121
#28 0x0038381c in m_Parse_ParseClient__c__DisplayClass8__RequestAsyncb__7_System_Threading_Tasks_Task_1_System_Tuple_2_System_Net_HttpStatusCode_string at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:13363
#29 0x0036de50 in m_Parse_Internal_InternalExtensions__c__DisplayClass1_2__OnSuccessb__0_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:2697
#30 0x0036e04c in m_Parse_Internal_InternalExtensions__c__DisplayClass7_1__OnSuccessb__6_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:2788
#31 0x003a3a4c in m_System_Threading_Tasks_Task__c__DisplayClass3_1__c__DisplayClass5__ContinueWithb__2 ()
#32 0x003a36b8 in m_System_Threading_Tasks_Task___cctorb__23_System_Action at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31346
#33 0x003a39bc in m_System_Threading_Tasks_Task__c__DisplayClass3_1__ContinueWithb__1_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31474
#34 0x003a4140 in m_System_Threading_Tasks_Task_1_RunContinuations at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31766
#35 0x003a4278 in m_System_Threading_Tasks_Task_1_TrySetResult_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31801
#36 0x003a4654 in m_System_Threading_Tasks_TaskCompletionSource_1_TrySetResult_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31950
#37 0x003a4fa0 in m_3d9 at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:32334
#38 0x003a44b0 in m_System_Threading_Tasks_Task_1__c__DisplayClass1__ContinueWithb__0_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31872
#39 0x003a3b3c in m_System_Threading_Tasks_Task__c__DisplayClass8__ContinueWithb__7_System_Threading_Tasks_Task ()
#40 0x00519a84 in m_System_Threading_Tasks_Task__c__DisplayClass3_1__c__DisplayClass5_int__ContinueWithb__2 ()
#41 0x003a36b8 in m_System_Threading_Tasks_Task___cctorb__23_System_Action at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31346
#42 0x004a9548 in m_1ce9 at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:165091
#43 0x003a8dcc in m_System_Threading_Tasks_Task_ContinueWith_int_System_Func_2_System_Threading_Tasks_Task_int_System_Threading_CancellationToken at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:34526
#44 0x003a321c in m_System_Threading_Tasks_Task_ContinueWith_System_Action_1_System_Threading_Tasks_Task_System_Threading_CancellationToken at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31155
#45 0x003a3168 in m_System_Threading_Tasks_Task_ContinueWith_System_Action_1_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31129
#46 0x003a3fdc in m_System_Threading_Tasks_Task_1_ContinueWith_System_Action_1_System_Threading_Tasks_Task_1_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31721
#47 0x003a4ed4 in m_3d8 at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:32304
#48 0x003a44b0 in m_System_Threading_Tasks_Task_1__c__DisplayClass1__ContinueWithb__0_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31872
#49 0x003a3b3c in m_System_Threading_Tasks_Task__c__DisplayClass8__ContinueWithb__7_System_Threading_Tasks_Task ()
#50 0x00519a84 in m_System_Threading_Tasks_Task__c__DisplayClass3_1__c__DisplayClass5_int__ContinueWithb__2 ()
#51 0x003a36b8 in m_System_Threading_Tasks_Task___cctorb__23_System_Action at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31346
#52 0x004a9548 in m_1ce9 at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:165091
#53 0x003a4140 in m_System_Threading_Tasks_Task_1_RunContinuations at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31766
#54 0x003a4278 in m_System_Threading_Tasks_Task_1_TrySetResult_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31801
#55 0x003a4654 in m_System_Threading_Tasks_TaskCompletionSource_1_TrySetResult_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31950
#56 0x003a3a60 in m_System_Threading_Tasks_Task__c__DisplayClass3_1__c__DisplayClass5__ContinueWithb__2 at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31498
#57 0x003a36b8 in m_System_Threading_Tasks_Task___cctorb__23_System_Action at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31346
#58 0x003a39bc in m_System_Threading_Tasks_Task__c__DisplayClass3_1__ContinueWithb__1_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31474
#59 0x003a4140 in m_System_Threading_Tasks_Task_1_RunContinuations at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31766
#60 0x003a4278 in m_System_Threading_Tasks_Task_1_TrySetResult_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31801
#61 0x003a4654 in m_System_Threading_Tasks_TaskCompletionSource_1_TrySetResult_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31950
#62 0x003a4fa0 in m_3d9 at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:32334
#63 0x003a44b0 in m_System_Threading_Tasks_Task_1__c__DisplayClass1__ContinueWithb__0_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31872
#64 0x003a3b3c in m_System_Threading_Tasks_Task__c__DisplayClass8__ContinueWithb__7_System_Threading_Tasks_Task ()
#65 0x00519a84 in m_System_Threading_Tasks_Task__c__DisplayClass3_1__c__DisplayClass5_int__ContinueWithb__2 ()
#66 0x003a36b8 in m_System_Threading_Tasks_Task___cctorb__23_System_Action at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31346
#67 0x004a9548 in m_1ce9 at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:165091
#68 0x003a8dcc in m_System_Threading_Tasks_Task_ContinueWith_int_System_Func_2_System_Threading_Tasks_Task_int_System_Threading_CancellationToken at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:34526
#69 0x003a321c in m_System_Threading_Tasks_Task_ContinueWith_System_Action_1_System_Threading_Tasks_Task_System_Threading_CancellationToken at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31155
#70 0x003a3168 in m_System_Threading_Tasks_Task_ContinueWith_System_Action_1_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31129
#71 0x003a3fdc in m_System_Threading_Tasks_Task_1_ContinueWith_System_Action_1_System_Threading_Tasks_Task_1_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31721
#72 0x003a4ed4 in m_3d8 at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:32304
#73 0x003a44b0 in m_System_Threading_Tasks_Task_1__c__DisplayClass1__ContinueWithb__0_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31872
#74 0x003a3b3c in m_System_Threading_Tasks_Task__c__DisplayClass8__ContinueWithb__7_System_Threading_Tasks_Task ()
#75 0x00519a84 in m_System_Threading_Tasks_Task__c__DisplayClass3_1__c__DisplayClass5_int__ContinueWithb__2 ()
#76 0x003a36b8 in m_System_Threading_Tasks_Task___cctorb__23_System_Action at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31346
#77 0x004a9548 in m_1ce9 at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:165091
#78 0x003a4140 in m_System_Threading_Tasks_Task_1_RunContinuations at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31766
#79 0x003a4278 in m_System_Threading_Tasks_Task_1_TrySetResult_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31801
#80 0x003a4654 in m_System_Threading_Tasks_TaskCompletionSource_1_TrySetResult_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31950
#81 0x003a3a60 in m_System_Threading_Tasks_Task__c__DisplayClass3_1__c__DisplayClass5__ContinueWithb__2 at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31498
#82 0x003a36b8 in m_System_Threading_Tasks_Task___cctorb__23_System_Action at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31346
#83 0x003a39bc in m_System_Threading_Tasks_Task__c__DisplayClass3_1__ContinueWithb__1_System_Threading_Tasks_Task at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31474
#84 0x003a4140 in m_System_Threading_Tasks_Task_1_RunContinuations at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31766
#85 0x003a4278 in m_System_Threading_Tasks_Task_1_TrySetResult_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31801
#86 0x003a4654 in m_System_Threading_Tasks_TaskCompletionSource_1_TrySetResult_T at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:31950
#87 0x003a0848 in m_Parse_PlatformHooks__c__DisplayClass2f__c__DisplayClass35__RequestAsyncb__2a_UnityEngine_WWW at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:29764
#88 0x003a0334 in m_Parse_PlatformHooks__c__DisplayClass20__RegisterNetworkRequestb__1f at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:29572
#89 0x003a0dbc in m_Parse_PlatformHooks__RunDispatcherd__39_MoveNext at /Users/me/myproj/build/device/Libraries/Parse.Unity.dll.s:29915
#90 0x0126ecec in scripting_method_invoke(ScriptingMethod*, MonoObject*, ScriptingArguments&, MonoException**) at /Applications/buildAgent/work/d63dfc6385190b60/Runtime/Scripting/Backend/Mono/ScriptingBackendApi_Mono.cpp:196
#91 0x01309390 in ScriptingInvocation::Invoke(MonoException**, bool) at /Applications/buildAgent/work/d63dfc6385190b60/Runtime/Scripting/Backend/ScriptingInvocation.cpp:128
#92 0x0130935c in ScriptingInvocation::Invoke(MonoException**) at /Applications/buildAgent/work/d63dfc6385190b60/Runtime/Scripting/Backend/ScriptingInvocation.cpp:113
#93 0x01309308 in bool ScriptingInvocation::Invoke<bool>(MonoException**) at /Applications/buildAgent/work/d63dfc6385190b60/Runtime/Scripting/Backend/ScriptingInvocation.cpp:80
#94 0x012df7e4 in Coroutine::InvokeMoveNext(MonoException**) at /Applications/buildAgent/work/d63dfc6385190b60/Runtime/Mono/Coroutine.cpp:196
#95 0x012df57c in Coroutine::Run() at /Applications/buildAgent/work/d63dfc6385190b60/Runtime/Mono/Coroutine.cpp:221
#96 0x012df544 in Coroutine::ContinueCoroutine(Object*, void*) at /Applications/buildAgent/work/d63dfc6385190b60/Runtime/Mono/Coroutine.cpp:78
#97 0x012593c4 in DelayedCallManager::Update(int) at /Applications/buildAgent/work/d63dfc6385190b60/Runtime/GameCode/CallDelayed.cpp:164
#98 0x012d0630 in PlayerLoop(bool, bool, IHookEvent*) at /Applications/buildAgent/work/d63dfc6385190b60/Runtime/Misc/Player.cpp:1880
#99 0x01117878 in UnityPlayerLoop at /Applications/buildAgent/work/d63dfc6385190b60/PlatformDependent/iPhonePlayer/LibEntryPoint.mm:241
#100 0x00d2ff34 in -[UnityAppController(Rendering) repaint] at /Users/me/myproj/build/device/Classes/UnityAppController+Rendering.mm:55
It seems parse's JSON parsing runs on the mainthread, but why?
What's going on here and how can I avoid having the mainthread getting stalled?
//note: occasionally, the stalling does not happen (the query succeeds nevertheless) - as if Parse would sometimes successfully do the parsing on another thread, most of the time however do it on the mainthread.
I'd suggest that if you hook up the Unity profiler to your device, you're going to find that your Linq query is creating immense amounts of garbage - and it's the garbage collection on your device that is stalling your main thread.
Reimplement your leaderboard functionality the hard way and don't use Linq. Linq saves programmers time - not machines, don't use it in your games.

Kannel: ERROR: System error 98: Address already in use

I cannot run kannel(an open source sms gateway) run properly. Everytime I try I get this error. ERROR: System error 98: Address already in use
Here is my kannel.conf
group = core
admin-port = 13000
admin-password = 123
status-password = 123
smsbox-port = 13003
wapbox-port = 13004
log-file = "/home/eclipse/kannel_core.log"
log-level = 1
wdp-interface-name = "*"
#group = smsc
#smsc = fake
#port = 1000
group = smsbox
bearerbox-host = 127.0.0.1
sendsms-port = 13013
log-file = "/home/eclipse/kannel_smsbox.log"
log-level = 1
group = wapbox
bearerbox-host = 127.0.0.1
log-file = "/home/eclipse/kannel_wapbox.log"
group = sendsms-user
username = 123
password = 123
user-allow-ip = "127.0.0.1"
group = sms-service
url="http://localhost/sms/send.php?sender=%s&&text=%t"
accept-x-kannel-headers = true
and the log file for the core:
2012-02-23 21:26:36 [9194] [0] INFO: Added logfile `/home/eclipse/kannel_core.log' with level `1'.
2012-02-23 21:26:36 [9194] [0] INFO: HTTP: Opening server at port 13000.
2012-02-23 21:26:36 [9194] [0] ERROR: bind failed
2012-02-23 21:26:36 [9194] [0] ERROR: System error 98: Address already in use
2012-02-23 21:26:36 [9194] [0] INFO: Adding interface *
2012-02-23 21:26:36 [9194] [0] ERROR: Couldn't bind a UDP socket to port 9200
2012-02-23 21:26:36 [9194] [0] ERROR: System error 98: Address already in use
2012-02-23 21:26:36 [9194] [0] ERROR: Couldn't bind a UDP socket to port 9201
2012-02-23 21:26:36 [9194] [0] ERROR: System error 98: Address already in use
2012-02-23 21:26:36 [9194] [0] INFO: ----------------------------------------
2012-02-23 21:26:36 [9194] [0] INFO: Kannel bearerbox II version 1.4.3 starting
2012-02-23 21:26:36 [9194] [7] ERROR: bind failed
2012-02-23 21:26:36 [9194] [7] ERROR: System error 98: Address already in use
2012-02-23 21:26:36 [9194] [7] PANIC: Could not open wapbox port 13004
2012-02-23 21:26:36 [9194] [0] INFO: MAIN: Start-up done, entering mainloop
2012-02-23 21:26:36 [9194] [7] PANIC: /usr/sbin/bearerbox(gw_panic+0xd2) [0x80d1b62]
2012-02-23 21:26:36 [9194] [7] PANIC: /usr/sbin/bearerbox() [0x8055bc0]
2012-02-23 21:26:36 [9194] [7] PANIC: /usr/sbin/bearerbox() [0x80c8bbb]
2012-02-23 21:26:36 [9194] [7] PANIC: /lib/i386-linux-gnu/libpthread.so.0(+0x6d31) [0x7a5d31]
2012-02-23 21:26:36 [9194] [7] PANIC: /lib/i386-linux-gnu/libc.so.6(clone+0x5e) [0xbba0ce]
I have tried to do a lot of changes but none successful. Kindly let me know which point I am missing. Thanks.
Something else is running at ports 13000, 9200, 9201 and 13004 - most likely another instance of kannel.
do sudo netstat -lanp to see what process it is.
Try to stop kannel service /etc/init.d/kannel stop and start it /etc/init.d/kannel start after few seconds.
I found out that sometime if you stop kannel service, it takes some few seconds to shutdown completely.
I had this error occur without the kannel piece but the etc/hosts file had a bad host/ip configured and the DNS at the site was pointing to the wrong server so Oracle listener could not be reached.

Resources