Azure autoscaling based on length of queue - azure

Question

I want to autoscale my azure worker role based on the length of an azure queue. According to all the documentation I've seen this should be very straightforward, using the queueLength operand.
I've implemented the autoscaler, uploaded it to my service and added a bunch of elements to my queue, but the number of instances are not increasing.
What's the best way of troubleshooting? I've remote desktopped to the role and there isn't anything in the event log. Is there a log of autoscaling events/errors I can check?
Edit:
When I run the app in my dev environment, I see that the autscaler has successfully loaded the ServiceInfo.xml. There is a queue entry and a role entry. However, the rules don't appear to have been loaded out of the rules.xml file.
More edit:
When I removed the reactiverules and operands nodes from the rules.xml file, the constraint rule loads successfully. So the problem is in one of those nodes.
My serviceinfo xml file:
<?xml version="1.0" encoding="utf-8"?>
<serviceModel xmlns="http://schemas.microsoft.com/practices/2011/entlib/autoscaling/serviceModel">
<subscriptions>
<subscription name="MySubscription" subscriptionId="blah" certificateThumbprint="‎blah" certificateStoreName="My" certificateStoreLocation="CurrentUser">
<storageAccounts>
<storageAccount alias="targetstorage" connectionString="DefaultEndpointsProtocol=https;AccountName=blah; AccountKey="blah">
<queues>
<queue alias="auditqueue" queueName="auditqueue"/>
</queues>
</storageAccount>
</storageAccounts>
<services>
<service dnsPrefix="blah" slot="Production" scalingMode="Scale">
<roles>
<role alias="ScalingWebRole" roleName="ScalingWebRole" wadStorageAccountName="targetstorage" />
</roles>
</service>
</services>
</subscription>
</subscriptions>
</serviceModel>
My rules xml file:
<constraintRules>
<rule name="Default" enabled="true" rank="1">
<actions>
<range target="ScalingWebRole" min="1" max="10" />
</actions>
</rule>
</constraintRules>
<reactiveRules>
<rule name="Scale up when queue is long" enabled="true">
<actions>
<scale target="ScalingWebRole" by="1" />
</actions>
<when>
<greaterOrEqual operand="QueueLength_Avg" than="5" />
</when>
</rule>
<rule name="Scale down when queue is short" enabled="true">
<actions>
<scale target="ScalingWebRole" by="-1" />
</actions>
<when>
<less operand="QueueLength_Avg" than="5" />
</when>
</rule>
</reactiveRules>
<operands>
<queueLength alias="QueueLength_Avg" aggregate="Average" queue="auditqueue" timespan="00:01:00" />
</operands>
</rules>

Answer 1

Wasabi publishes a lot of diagnostic information, but you have to enable it. In the role where you are hosting the autoscaler, update the app.config to include the following:
<system.diagnostics>
<sources>
<source name="Autoscaling General" switchValue="All">
<listeners>
<add name="AzureDiag" />
</listeners>
</source>
<source name="Autoscaling Updates" switchValue="All">
<listeners>
<add name="AzureDiag" />
</listeners>
</source>
</sources>
<sharedListeners>
<add type="Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener, Microsoft.WindowsAzure.Diagnostics, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"
name="AzureDiag"/>
</sharedListeners>
<trace>
<listeners>
<add type="Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener, Microsoft.WindowsAzure.Diagnostics, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"
name="AzureDiagnostics">
<filter type="" />
</add>
</listeners>
</trace>
</system.diagnostics>
Then, if your role is configured to upload the log entries to the WADs table (Windows Azure Diagnostic table), you will see the entries there.
More info about the log entries that Wasabi generates here.

Answer 2

With the assistance provided by the diagnostics (thanks #Julian Dominguez), it became clear that the first time the autoscaling block was trying to hit the service (to check the queue length), it was failing. This is because the system was looking for the certificate in the CurrentUser store rather than the LocalMachine store.
Once I changed that setting in the configuration, it started to work.