Since last week we started using SDK version 3.34.1 (and also tested this with 3.35.0). When we send a PATCH request to a SAP service we get a HTTP 204 No-Content response back from our SAP service (SAP Gateway). When the SDK tries to read that response, it tries to parse the response body which is empty. This leads to the following exception:
2020-12-17 16:13:51.767 ERROR 106363 --- [ut.sap.cases]-0] .s.c.s.d.o.c.r.ODataRequestResultGeneric :
Failed to buffer HTTP response. Unable to buffer HTTP entity.
java.lang.IllegalArgumentException: Wrapped entity may not be null
at org.apache.http.util.Args.notNull(Args.java:54)
at org.apache.http.entity.HttpEntityWrapper.<init>(HttpEntityWrapper.java:59)
at org.apache.http.entity.BufferedHttpEntity.<init>(BufferedHttpEntity.java:59)
at com.sap.cloud.sdk.datamodel.odata.client.request.ODataRequestResultGeneric.lambda$getHttpResponse$4f00ca4e$1(ODataRequestResultGeneric.java:180)
at io.vavr.control.Try.of(Try.java:75)
at com.sap.cloud.sdk.datamodel.odata.client.request.ODataRequestResultGeneric.getHttpResponse(ODataRequestResultGeneric.java:180)
at com.sap.cloud.sdk.datamodel.odata.client.request.ODataHealthyResponseValidator.requireHealthyResponse(ODataHealthyResponseValidator.java:44)
at io.vavr.control.Try.andThenTry(Try.java:250)
at com.sap.cloud.sdk.datamodel.odata.client.request.ODataRequestGeneric.tryExecute(ODataRequestGeneric.java:194)
at com.sap.cloud.sdk.datamodel.odata.client.request.ODataRequestGeneric.tryExecuteWithCsrfToken(ODataRequestGeneric.java:225)
at com.sap.cloud.sdk.datamodel.odata.client.request.ODataRequestUpdate.execute(ODataRequestUpdate.java:136)
at com.sap.cloud.sdk.datamodel.odata.helper.FluentHelperUpdate.executeRequest(FluentHelperUpdate.java:372)
at com.alliander.gvrn.pmd.adapter.out.sap.cases.SapCasesClient.updateCase(SapCasesClient.java:103)
at com.alliander.gvrn.pmd.adapter.out.sap.cases.SapCasesClient.persistOn(SapCasesClient.java:81)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at ....
We use generated typed OData V2 client, which are generated by providing our EDMX files a per the SDK documentation.
Below a code snippet of the function that's calling the service. The matrixCase is a autogenerated object. The OData PATCH is properly handled by the SAP service.
private void updateCase(final ExternalId caseId, final PMDFlow pmdFlow, String jwtToken) {
final HttpDestination sapMatrix = httpDestinationProvider.providePrincipalPropagationDestination(jwtToken);
// "Create matrixCase object with key
MatrixCase matrixCase = MatrixCase.builder()
.psReference(caseId.getValue())
.build();
// Set PmdAppControl explicitly, otherwise the generated client doesn't know which fields are updated.
matrixCase.setPMDAppControl(pmdFlow.getSapNotation());
try {
casesService
.updateMatrixCase(matrixCase)
.executeRequest(sapMatrix);
} catch (ODataException e) {
OdataLogger.logODataException(e);
throw new SapClientException(e);
}
}
We've updated to SDK 3.34.1 due to other issues, however before we used 3.32.0 and I don't remember having this issue in version 3.32.0
Any ideas?
Danny
Yes, your observation is correct that 204 represents a valid answer and is not worth logging an error. Hence, the Cloud SDK team will adjust the log entry level to be less alarming.
Regards,
Tanvi
Related
I've created a simple Azure function which is triggered on a service bus queue.
I'd like to inject a the full message payload into the trigger. Now there are 2 libraries available for working with Azure Service Bus - the older https://www.nuget.org/packages/Microsoft.Azure.ServiceBus/ and the newer https://www.nuget.org/packages/Azure.Messaging.ServiceBus/
If I inject in the message payload using Microsoft.Azure.ServiceBus.Message works, however using the newer Azure.Messaging.ServiceBus.ServiceBusMessage gives a run-time error.
Ideally would like to use the newer library - is there any config change required to make this work?
The error is
Executed 'IncomingMessageProcessingFunction' (Failed, Id=01b3f0c1-fe4d-4d0f-89db-7814a4a0ddbe, Duration=2756ms)
[2021-09-13T10:56:47.944Z] System.Private.CoreLib: Exception while executing function: IncomingMessageProcessingFunction. Microsoft.Azure.WebJobs.Host: Exception binding parameter 'myQueueItem'. Microsoft.Azure.WebJobs.ServiceBus: Binding parameters to complex objects (such as 'ServiceBusMessage') uses Json.NET serialization or XML object serialization.
[2021-09-13T10:56:47.946Z] 1. If ContentType is 'application/json' deserialize as JSON
[2021-09-13T10:56:47.948Z] 2. If ContentType is not 'application/json' attempt to deserialize using Message.GetBody, which will handle cases like XML object serialization
[2021-09-13T10:56:47.949Z] 3. If this deserialization fails, do a final attempt at JSON deserialization to catch cases where the content type might be incorrect
[2021-09-13T10:56:47.951Z] The JSON parser failed: Error converting value 123 to type 'Azure.Messaging.ServiceBus.ServiceBusMessage'. Path '', line 1, position 3.
Works
[FunctionName("IncomingMessageProcessingFunction")]
public static void Run([ServiceBusTrigger("incomingmessageprocessingqueue", Connection = "SERVICEBUS")]Microsoft.Azure.ServiceBus.Message myQueueItem, ILogger log)
{
log.LogInformation($"C# ServiceBus queue trigger function processed message: {myQueueItem.Body}");
I have a Azure function with an HTTP trigger. It produces a 400 response when an input value is not specified:
if (string.IsNullOrEmpty(artistName))
{
return new BadRequestObjectResult("Artist name not specified.");
}
When the function is running locally (func host start) and the API is called incorrectly (I am using curl -X POST ...), the above error message is shown.
When deployed to Azure and called in a similar way - instead of the detailed error message - only Bad Request is returned.
Why?
How can I change this behaviour?
It's a known issue for function runtime 2.0.11888 that HttpTrigger does not return response content properly.
If you want to solve this, go to Application settings, pin your FUNCTIONS_EXTENSION_VERSION to previous runtime 2.0.11857-alpha until new version is released.
See Azure Functions Runtime 2.0.11888 Preview.
Update
This issue has already been fixed from 2.0.11933.
i am encountering a problem i'm not familiar with.
So i'm trying to log exceptions from a test Azure Function but when i throw an exception and return a 400 bad request, application insight registers the log as a successful request.
As i understand it it is probably register the function's successful run but what i don't understand is how i then should log the exception.
So what i have done so far is this.
(I Will be referring to Application Insights as AI from here on out)
I started by created an AI-Resource.
Then i took the instrument key and applied it to the app settings of my function.
After that i installed the AI NUGET to my function bu creating a Projet.json file and then pasting something like this which installed the necessary assemblies and such.
{
"frameworks": {
"net46":{
"dependencies": {
"Microsoft.ApplicationInsights": "2.4.0"
}
}
}
}
after this i initialize the TelemetryClient in the function and try to log and exception in a catch:
Initiation:
string key = TelemetryConfiguration.Active.InstrumentationKey = System.Environment.GetEnvironmentVariable("APPINSIGHTS_INSTRUMENTATIONKEY", EnvironmentVariableTarget.Process);
TelemetryClient telemetry = new TelemetryClient() {
InstrumentationKey = key
};
Catch:
catch (Exception e)
{
Dictionary<string,string> properties = new Dictionary<string,string>();
properties.Add("Function Payload", data.ToString());
properties.Add("Function Exception", e.ToString());
telemetry.TrackException(e, properties);
return req.CreateResponse(HttpStatusCode.BadRequest, e);
}
Test running the function i get:
2018-03-07T14:24:36.171 [Info] Function started (Id=0292b455-314d-4c4c-872a-2b8137a72305)
2018-03-07T14:24:37.092 [Info] Function completed (Success, Id=0292b455-314d-4c4c-872a-2b8137a72305, Duration=931ms)
In Application insights i can can only see bad requests for StatusCode: 500
but 400 Bad requests gets logged as Successful requests.
And also the TrackException functionality doesn't log any of the custom properties...
So what am i missing?
MORE DETAILS ABOUT EXCEPTION LOGGING:
#Mikhail is right that we treat this as a success because the function is a success. We didn't want to use status codes to guess whether there was a successful operation or not, so we look for whether the function threw an exception.
Your exception isn't appearing in that screen because it hasn't been property correlated with this function execution. If you go to App Insights Analytics and query for that ExceptionTelemetry, you should see it.
In order to correlate it with a specific function, you'd need to set the OperationId, which is the same as the function's InvocationId. There is a sample that shows how to do this with Events, Metrics, and Dependencies, but doing it for Exceptions is the same (you can ignore the User.Id assignment): https://learn.microsoft.com/en-us/azure/azure-functions/functions-monitoring#custom-telemetry-in-c-functions
Update: From what you've shown of your function above, you may be able to get away with doing something like:
catch (Exception e)
{
log.Error("Function Payload " + data.ToString());
throw;
}
That would return a 500 (rather than a 400), and Functions would log the trace to Application Insights, and then log the exception as well as the Request failure. If you're not using your TelemetryClient anywhere else, you could remove that from your code.
The server (Azure Function) processed the request without errors, you returned a result from it, so from the point of view of Function App runtime the request was processed successfully. You can see that from the log too:
... Function completed (Success, ...
So, it makes sense that the Function App registers the call as success in Application Insights too. At least, that's how they chose to implement it.
the "for this operation" not showing exceptions implies that the exception that you sent does not have the same operationId as the azure function. operation
id is how application insights "links" related telemetry together.
your "exeption logging" screenshot is not an exception, but a request, so the custom properties logged on your exception won't be there.
if you want your azure function to fail, and show as a failed request, and log an exception, why are you catching the exception and logging it yourself? doesn't catching the exception then cause the azure function to succeed? why not just let the exception trickle out and let the function runtime do that part for you? (doesn't it?)
I have a strange problem. I have a .NET Core App which works fine on local machine and passes unit tests.
Inside the app it basically calls our platform web service:
using( WebClient client = new WebClient() )
{
NetworkCredential creds = new NetworkCredential(_userName, _password);
CredentialCache credCache = new CredentialCache();
credCache.Add(new System.Uri(_baseUrl), "Basic", creds);
client.Credentials = credCache;
var url = _baseUrl + "/api/v1/Pricing/Rates";
client.Headers.Add(HttpRequestHeader.ContentType, "application/json");
var request = JsonConvert.SerializeObject(data);
System.Console.Out.WriteLine(request);
var response = client.UploadString(url, request);
var responseObject = JObject.Parse(response);
var products = responseObject["PricingProducts"].Children();
var result = new Dictionary<string, double>();
foreach( var product in products )
{
result.Add(product.Value<string>("LoanProgramName"),
product.Value<double>("Rate"));
}
return result;
}
When I execute this on local machine using dotnet run, everything works fine. Unit tests work great too. The logs on the App Service don't tell me much except that I am getting a 403 from the platform web service.
ers.RatesController.Get (AlexaRates) with arguments ((null)) - ModelState is Valid
2018-02-24 06:37:44.418 +00:00 [Information] Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker: Executed action AlexaRates.Controllers.RatesController.Get (AlexaRates) in 201.3483ms
2018-02-24 06:37:44.447 +00:00 [Error] Microsoft.AspNetCore.Server.Kestrel: Connection id "0HLBRA4B41EO8", Request id "0HLBRA4B41EO8:00000002": An unhandled exception was thrown by the application.
System.Net.WebException: The remote server returned an error: (403) Forbidden.
at System.Net.HttpWebRequest.GetResponse()
at System.Net.WebClient.GetWebResponse(WebRequest request)
at System.Net.WebClient.DownloadBits(WebRequest request, Stream writeStream)
at System.Net.WebClient.UploadBits(WebRequest request, Stream readStream, Byte[] buffer, Int32 chunkSize, Byte[] header, Byte[] footer)
at System.Net.WebClient.UploadDataInternal(Uri address, String method, Byte[] data, WebRequest& request)
at System.Net.WebClient.UploadString(Uri address, String method, String data)
at Rates.RetrieveLatest() in D:\home\site\repository\AlexaRates\Rates.cs:line 50
at AlexaRates.Controllers.RatesController.Get() in D:\home\site\repository\AlexaRates\Controllers\RatesController.cs:line 22
at lambda_method(Closure , Object , Object[] )
at Microsoft.Exten
Has anyone experienced anything similar? I see a bunch 403 posts, but they are mostly about people calling a REST API hosted on the service not calling out.
The 403 forbidden error usually means the server understood the request but refuses to authorize it.
According to your error message, it seems that the error happens in Rates class and RatesController class, which you haven’t showed for us. You could set a break point to check the code in these classes by using remote debugging.
You say the project is working fine locally, but get error in Azure, so please make sure you have published all your projects and data sources to Azure. Check whether the ‘_baseUrl ‘ is from Azure. And make sure you have started the Azure App Service.
There may be other causes of 403 forbidden error. Such as page cache and logging in of cookie. You could refer to this article to learn how to fix the 403 Forbidden Error.
Cause of 403 Forbidden Errors
403 errors are almost always caused by issues where you're trying to access something that you don't have access to.
My fix was that I realized that our infrastructure guys added a IP restriction on the azure app. That is why the app was bouncing back with a 403.
I removed the IP restrictions on the "Networking" -> "Access Restrictions" page.
After trying to add headers and doing various other things the end result was the same - I was getting a 403 error on calling out to a web service.
The solution was to convert from a Web App to a VM and deploy the application there using the old school setup. The application worked there.
I want to do something when/if an insert operation on Azure Table Storage fails. Assume that I want to return false from the below code when I receive an error. _table is of type CloudTable and the code below works.
public bool InsertEntity(TableEntity entity)
{
var insertOperation = TableOperation.Insert(entity);
var result = _table.Execute(insertOperation);
return (result.HttpStatusCode == (int)System.Net.HttpStatusCode.OK);
}
I get the result 203 when the operation succeeds. But there are other possible results like "200 OK".
How can I write a piece of code that will allow me to understand from the status code that something went wrong?
Using the .NET SDK, any situation that needs to be handled will throw an exception. i.e. Any status code that is not 2xx will cause an exception.
To handle situations where something went wrong, I don't have to manually check the status code of the result for every request. All I have to do is to write exception handling code. Like below:
try
{
var result = _table.Execute(insertOperation);
}
catch (Exception)
{
Log("Something went wrong in table operation.");
}
From this page:
REST API operations for Azure storage services return standard HTTP
status codes, as defined in the HTTP/1.1 Status Code Definitions.
So every successful operation against table service will return 2XX status code. To find out about the exact code returned, I would recommend checking out each operation on the REST API Documentation page. For example, Create Table operation returns 201 status code if the operation is successful.
Similarly, for errors in table service you will get error code in 400 range (that would mean you provided incorrect data e.g. 409 (Conflict) error if you're trying to create a table which already exists) or in 500 range (for example, table service is unavailable). You can find the list of all Table Service Error Codes here: https://msdn.microsoft.com/en-us/library/azure/dd179438.aspx.
Basically, any return in 2xx is "OK". In this example:
https://msdn.microsoft.com/en-us/library/system.net.httpstatuscode%28v=vs.110%29.aspx
203 Non-Authoritative Information:
Indicates that the returned metainformation is from a cached copy
instead of the
origin server and therefore may be incorrect.
This Azure white paper elaborates further:
http://go.microsoft.com/fwlink/?LinkId=153401
9.6.5 Error handling and reporting
The REST API is designed to look like a standard HTTP server interacting with existing HTTP clients
(e.g., browsers, HTTP client libraries, proxies, caches, and so on).
To ensure the HTTP clients handle errors properly, we map each Windows
Azure Table error to an HTTP status code.
HTTP status codes are less expressive than Windows Azure Table error
codes and contain less information about the error. Although the HTTP
status codes contain less information about the error, clients that
understand HTTP will usually handle the error correctly.
Therefore, when handling errors or reporting Windows Azure Table
errors to end users, use the Windows Azure Table error code along with
the HTTP status code as it contains more information about the error.
Additionally, when debugging your application, you should also consult
the human readable element of the XML error
response.
These links are also useful:
Microsoft Azure: Status and Error Codes
Clean way to catch errors from Azure Table (other than string match?)
If you are using Azure Storage SDK accessing Azure Table Storage, the SDK would throw a StorageException on the client side for unexpected Http Status Codes returned from the table storage service. To extract the actual HttpStatusCode you would need to wrap your code in a try {} catch(StorageException ex){} block. And then parse the actual exception object to extract the HttpStatusCode embedded in it.
Have a look at Azure Storage Exception parser I implemented in Nuget:
https://www.nuget.org/packages/AzureStorageExceptionParser/
This extracts HttpStatusCode and many other useful fields from Azure StorageExceptions. You can use the same library accross table, blob, queue clients etc. as they all follow the same StorageException pattern.
Note that there will be some exceptions thrown by the Azure Storage SDK that are not StorageExceptions, those are mostly client side request validation type of exceptions and naturally they do not contain any HttpStatusCode. (Hence you would need to have a catch for specifically StorageExceptions to extract HttpStatusCode s).
As a separate note, Azure Storage SDK has a fairly robust retry mechanism for failed requests. Below is the snippet from SDK source code where they decide if the failed response is retrieable or not.
https://github.com/Azure/azure-storage-net/blob/master/Lib/Common/RetryPolicies/ExponentialRetry.cs
if ((statusCode >= 300 && statusCode < 500 && statusCode != 408)
|| statusCode == 501 // Not Implemented
|| statusCode == 505 // Version Not Supported
|| lastException.Message == SR.BlobTypeMismatch)
{
return false; //aka. do not Retry if w are here otherwise Retry if within max retry count..
}