I want to go through the queue data in Azure queue storage in a HTTP trigger function App.
I add the "äzure-storage" Node.js package and using following code:
var azure = require('azure-storage');
But either the code var queueSvc = azure.createQueueService(); or var queueSvc = azure.createQueueService(<storage connection string>); is failure when execute the function. I got below exception
"reason": "PassThrough",
"exception": {
"ClassName": "System.Threading.Tasks.TaskCanceledException",
"Message": "A task was canceled.",
"Data": null,
"InnerException": null,
"HelpURL": null,
"StackTraceString": " at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at AzureFunctions.Code.PassThroughRequestManager.d__2.MoveNext()",
"RemoteStackTraceString": null,
"RemoteStackIndex": 0,
"ExceptionMethod": "8\nThrowForNonSuccess\nmscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089\nSystem.Runtime.CompilerServices.TaskAwaiter\nVoid ThrowForNonSuccess(System.Threading.Tasks.Task)",
"HResult": -2146233029,
"Source": "mscorlib",
"WatsonBuckets": null
}
}
I can't reproduce this issue in a simple HTTP trigger function with Javascript:
Install azure-storage via Kudu debug console (https://<functionname>.scm.azurewebsites.net/DebugConsole):
In the index.js file I have this:
Related
I am calling an http triggered Azure function app in data factory pipeline using ADF function activity. It is executing successfully in debug mode, but when I publish that pipeline and run the same code using data factory triggers I get below error-
{
"errorCode": "3600",
"message": "Object reference not set to an instance of an object.",
"failureType": "UserError",
"target": "AzureFunction"
}
Please let me know if I need to make some additional properties changes or I am missing anything here. Also is there any way I can see what URL is getting generated when I call function app through function activity in ADF.
I have tried calling same function app using web activity in ADF and that is working fine in both debug and trigger mode.
Linked service code for Azure function
{
"name": "linkedservicesAzureFunctions",
"type": "Microsoft.DataFactory/factories/linkedservices",
"properties": {
"typeProperties": {
"functionAppUrl": "https://xyz.azurewebsites.net",
"functionKey": {
"type": "AzureKeyVaultSecret",
"store": {
"type": "LinkedServiceReference",
"referenceName": "linkedservicesKeyVault"
},
"secretName": "functions-host-key-default"
}
},
"type": "AzureFunction"
}
}
There is a known Bug in Azure Data Factory and they are working on that. For now if you are creating the Azure Data Factory using .NET SDK then you'll need to set Headers like this in Azure Function Activity.
new AzureFunctionActivity
{
Name = "CopyFromBlobToSnowFlake",
LinkedServiceName = new LinkedServiceReference(pipelineStructure.AzureFunctionLinkedService),
Method = "POST",
Body = body,
FunctionName = "LoadBlobsIntoSnowFlake",
Headers = new Dictionary<string, string>{ },
DependsOn = new List<ActivityDependency>
{
new ActivityDependency{
Activity = "CopyFromOPSqlServerToBlob",
DependencyConditions= new List<string>{"Succeeded" }
}
}
}
If you are creating Azure Function Activity through UI then just update the description of Activity then Publish and Headers will automatically get Initialized.
I have an Azure Function Core running on a Linux consumption plan running in WestUS. The function seems to be running OK - I had a previous .NET Standard 2.0 version of the same function running OK in Azure. However, when I try to create a subscription to an Event Grid Topic I get the following error:
Deployment has failed with the following error: {"code":"Url validation","message":"The attempt to validate the provided endpoint https://insysfunctiongetweathercore.azurewebsites.net/runtime/webhooks/EventGrid failed. For more details, visit https://aka.ms/esvalidation."}
I am using an EventGridTrigger so should not have to do anything to handle validation - this should happen automatically:
public static async Task Run([EventGridTrigger] EventGridEvent eventGridEvent, ILogger log)
Any ideas on what I might need to do differently when the function is hosted in Linux consumption plan?
EDIT
Here's a screenshot of the Create Event Subscription Form with error:
and the resulting JSON:
{
"name": "InSysWeatherPull",
"properties": {
"topic": "/subscriptions/xxxxxxxxxxx/resourceGroups/InergySystemsWest/providers/Microsoft.EventGrid/Topics/InSysEventGridWest",
"destination": {
"endpointType": "WebHook",
"properties": {
"endpointUrl": "https://insysfunctiongetweathercore.azurewebsites.net/runtime/webhooks/EventGrid?functionName=ProcessWeatherRequest&code=xxxxxxxxxxxx"
}
},
"filter": {
"includedEventTypes": [
"weather-zip-request"
],
"advancedFilters": []
},
"labels": [],
"eventDeliverySchema": "EventGridSchema"
}
}
I get this error when I am posting via postman.
Other methods work perfectly. Just this method, that addresses the user verification and its response is an access token, has error.
I think the length of the string token is the origin of the error.
Post man response :
"StackTrace": " at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)\r\n at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)\r\n at System.Threading.Tasks.Task`1.get_Result()\r\n at FactoryJaApi.Controllers.UserController.<Login>d__14.MoveNext() \r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at
System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()\r\n at FactoryJaApi.Controllers.UserController.d__12.MoveNext()
"TargetSite": {
"Name": "ThrowIfExceptional",
"AssemblyName": "mscorlib, Version=4.0.0.0, Culture=neutral,
"ClassName": "System.Threading.Tasks.Task",
"Signature": "Void ThrowIfExceptional(Boolean)",
"Signature2": "System.Void ThrowIfExceptional(System.Boolean)",
"MemberType": 8,
"GenericArguments": null
},
"ErrorLine": null,
"ExceptionType": "System.AggregateException",
"ExceptionUrl": null,
"InnerException": {
"DateTime": "0001-01-01T00:00:00",
"Message": "An error occurred while sending the request.",
"HelpLink": null,
"Source": null,
"StackTrace": null,
"TargetSite": null,
"ErrorLine": null,
"ExceptionType": null,
"ExceptionUrl": null,
"InnerException": {
"DateTime": "0001-01-01T00:00:00",
"Message": "Unable to connect to the remote server",
"HelpLink": null,
"Source": null,
"StackTrace": " at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, TransportContext& context)\r\n at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar)",
"TargetSite": null,
"ErrorLine": null,
"ExceptionType": null,
"ExceptionUrl": null,
"InnerException": null
}
}
}
}
Thank all of you that your information does not help me and was far from my ask.
I pointed this error just happens on verifying and generating token string method. and other method work perfectly and correct.
Token method has a self-Query on itself . so when your query goes to server Kaarix.work , IIS cannot access to same address from interior. for Solving : just you need to define a redirection local url and redirect your queries that call from inside machine.
See the screenshot if you need the answer.
enter image description here
I'm trying to add a Site to my DirectLine channel in my Azure Bot Registration via the REST APIs. When I perform a GET against the following ARM REST URL, I get a JSON payload with any existing sites listed.
https://management.azure.com/subscriptions/{azure.SubscriptionId}/resourceGroups/{MyResourceGroupName}/providers/Microsoft.BotService/botServices/{MyBotHandle}/channels/DirectLineChannel?api-version=2017-12-01
If I then take the JSON output from the GET call and add a Site to it like this...
{
"id": "/subscriptions/{SubscriptionID}/resourceGroups/{ResourceGroupName}/providers/Microsoft.BotService/botServices/{BotHandle}/channels/DirectLineChannel",
"name": "{BotHandle}/DirectLineChannel",
"type": "Microsoft.BotService/botServices/channels",
"etag": "W/\"8e7678d65f7dbeea8863f0c68af75f606/6/2018 6:31:37 PM\"",
"location": "global",
"properties": {
"properties": {
"DirectLineEmbedCode": null,
"sites": [
{
"siteId": "uXuynMwVIBI",
"siteName": "Default Site",
"key": "uXuynMwVIBI.cwA.ve8.m2IbJ0McgjmxAq8vlZ1ZY1jAFdpdQj14GbgrS4z-qpA",
"key2": "uXuynMwVIBI.cwA.gXE.4aZw5_DIJuLPsQ2zjIYSCMztTxUxNpX4trwfO-6Gb9k",
"isEnabled": true,
"isV1Enabled": true,
"isV3Enabled": true
}
]
},
"etag": "W/\"8e7678d65f7dbeea8863f0c68af75f606/6/2018 6:31:37 PM\"",
"channelName": "DirectLineChannel",
"location": "global",
"provisioningState": "Succeeded"
}
}
... and then perform a PUT against the same initial URL, I get the following error:
An unexpected error occurred. Exception: 'Microsoft.Bot.Schema.APIException: Empty service error response. ID: \r\n at Intercom.Channels.Models.Shared.TimeLimitedConfigHttpClient1.<GetFromResponseAsync>d__20.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Intercom.Channels.Models.Shared.TimeLimitedConfigHttpClient1.d__10.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Intercom.Channels.Models.Shared.ConfigApi1.<PutAsync>d__7.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult()\r\n at Intercom.ResourceProvider.Channels.ArmChannelManager.d__22.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult()\r\n at Intercom.ResourceProvider.DataProviders.ChannelResourceDataProvider.d__5.MoveNext()'.
I noticed this JSON looks like an ARM template as well so I tried adding it to a deployment and I get the exact same error above. It provisions the bot and then I get an InternalServerError when it's provisioning the DirectLineChannel. The Operation details link gives me the above error.
Any thoughts on how to do this? Am I way off?
Thanks!
As shown in my Stack Overflow question Azure webjob not appearing to respect MaxDequeueCount property, I'm troubleshooting a problem where despite my setting of the MaxDequeueCount property to 1, some items are dequeued many times before they're poisoned (in fact some items may never be poisoned at all and just dequeue, fail and are retried and fail endlessly).
The Webjobs SDK handles the retries and poisoning of queue triggered messages automatically and I'm looking for logs that contain details of that handling.
For example, I can see that my function has detected a new queue item by viewing the webjob's log through the SCM at https://myappengine.scm.azurewebsites.net/vfs/data/jobs/continuous/StuffProcessor/job_log.txt (By the way if I've enabled detailed logging to Azure storage on the web app can I get this same info in a Blob?).
[02/22/2017 01:47:22 > ec8d0f: INFO] Executing: 'StuffProcessor.ProcessQueueMessage' - Reason: 'New queue message detected on 'stuff-processor'.'
[02/22/2017 01:47:26 > ec8d0f: INFO] Executed: 'StuffProcessor.ProcessQueueMessage' (Succeeded)
[02/22/2017 01:47:26 > ec8d0f: INFO] Executing: 'StuffProcessor.ProcessQueueMessage' - Reason: 'New queue message detected on 'stuff-processor'.'
I can also get some information on an item's dequeue count by viewing the logs in the azure-jobs-host-archive container once I've enabled detailed logging to Azure storage on the web app:
{
"Type": "FunctionCompleted",
"EndTime": "2017-02-22T00:07:40.8133081+00:00",
"Failure": {
"ExceptionType": "Microsoft.Azure.WebJobs.Host.FunctionInvocationException",
"ExceptionDetails": "Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: ItemProcessor.ProcessQueueMessage ---> MyApp.Exceptions.MySpecialAppExceptionType: Exception of type 'MyApp.Exceptions.MySpecialAppExceptionType' was thrown.
},
"ParameterLogs": {},
"FunctionInstanceId": "1ffac7b0-1290-4343-8ee1-2af0d39ae2c9",
"Function": {
"Id": "MyApp.Processors.ItemProcessor.ProcessQueueMessage",
"FullName": "MyApp.Processors.ItemProcessor.ProcessQueueMessage",
"ShortName": "ItemProcessor.ProcessQueueMessage",
"Parameters": [
{
"Type": "QueueTrigger",
"AccountName": "MyStorageAccount",
"QueueName": "stuff-processor",
"Name": "sourceFeedItemQueueItem"
},
{
"Type": "BindingData",
"Name": "dequeueCount"
},
{
"Type": "ParameterDescriptor",
"Name": "logger"
}
]
},
"Arguments": {
"sourceFeedItemQueueItem": "{\"SourceFeedUpdateID\":437530,\"PodcastFeedID\":\"2d48D2sf2\"}",
"dequeueCount": "96",
"logger": null
},
"Reason": "AutomaticTrigger",
"ReasonDetails": "New queue message detected on 'stuff-processor'.",
"StartTime": "2017-02-22T00:07:40.6017341+00:00",
"OutputBlob": {
"ContainerName": "azure-webjobs-hosts",
"BlobName": "output-logs/1ffd3c7b012c043438ed12af0d39ae2c9.txt"
},
"ParameterLogBlob": {
"ContainerName": "azure-webjobs-hosts",
"BlobName": "output-logs/1cf2c1b012sa0d3438ee12daf0d39ae2c9.params.txt"
},
"LogLevel": "Info",
"HostInstanceId": "d1825bdb-d92a-4657-81a4-36253e01ea5e",
"HostDisplayName": "ItemProcessor",
"SharedQueueName": "azure-webjobs-host-490daea03c70316f8aa2509438afe8ef",
"InstanceQueueName": "azure-webjobs-host-d18252sdbd92a4657d1a436253e01ea5e",
"Heartbeat": {
"SharedContainerName": "azure-webjobs-hosts",
"SharedDirectoryName": "heartbeats/490baea03cfdfd0416f8aa25aqr438afe8ef",
"InstanceBlobName": "zd1825bdbdsdgga465781a436q53e01ea5e",
"ExpirationInSeconds": 45
},
"WebJobRunIdentifier": {
"WebSiteName": "myappengine",
"JobType": "Continuous",
"JobName": "ItemProcessor",
"RunId": ""
}
}
What I can't find though are logs which show detail for a particular queue item where processing fails due to an exception and is placed in the poison queue. I'm looking for these in an attempt to further troubleshoot the apparent ignoring of the MaxDequeueCount property. Is this logged?
UPDATE: I found the post Azure WebJobs with Storage Queue and Exceptions and it contained the following screenshot:
That screenshot shows the standard "New queue message detected..." messages (which I see both on Azure and running locally in the emulator) and it also shows "Message has reached MaxDequeueCount of X...Moving message to queue 'xyz-poison'", which I only see locally in the emulator. Does the Azure-based runtime not show this info for some reason? I never see the poisoning-related messages like this when running locally in the console window or via the webjobs dashboard or when running on Azure.
Are you using Azure Storage version 8.* ? It contains a breaking change.
When a queue message object gets inserted into a message queue in Azure Storage 8, the Message Id of the original queue message object is overwritten with the unique message id of the newly inserted message.
This means that it looses its reference to the poison message and can no longer remove it.
https://github.com/Azure/azure-webjobs-sdk/issues/985
As for logging, 2 options:
Create your own QueueProcessorFactory and QueueProcessor - you can override the method that handles poison messages to produce more debug output. There you can also fix the issue altogether if you'd like to stay on Azure Storage 8.*.
Register a trace monitor. https://github.com/Azure/azure-webjobs-sdk-extensions/wiki/Error-Monitoring#tracemonitor
var traceMonitor = new TraceMonitor()
.Filter(p => true, "Trace Handler")
.Subscribe(TraceHandler.Process);
config.Tracing.Tracers.Add(traceMonitor);
In your previous post, you have the triggered function as follows
public void ProcessQueueMessage([QueueTrigger("azurewejobtestingqueue")] string item, TextWriter logger)
{
if ( item == "exception" )
{
throw new Exception();
}
}
Anything you write to the logger should appear in the logs on the WebJobs dashboard. My suggestions are:
Change 'if (item == "exception")' to 'if (item.ToLower().StartsWith("exception")'. Then append a number to the exception message when testing. Log the text of the queue message to the TextWriter so you can be sure that you're looking at the same message you think you are inspecting.
You can get the message's dequeue count as a parameter to a function. See here. Check out the FailAlways method in the example. Log this to the TextWriter as well.
Try running this locally. All output will be written to the console window. See if you get the same behavior.
I'm really interested in finding what's causing this behavior, so make sure you let us know if you crack it!