403 when calling API from Azure App Service - azure

I have a strange problem. I have a .NET Core App which works fine on local machine and passes unit tests.
Inside the app it basically calls our platform web service:
using( WebClient client = new WebClient() )
{
NetworkCredential creds = new NetworkCredential(_userName, _password);
CredentialCache credCache = new CredentialCache();
credCache.Add(new System.Uri(_baseUrl), "Basic", creds);
client.Credentials = credCache;
var url = _baseUrl + "/api/v1/Pricing/Rates";
client.Headers.Add(HttpRequestHeader.ContentType, "application/json");
var request = JsonConvert.SerializeObject(data);
System.Console.Out.WriteLine(request);
var response = client.UploadString(url, request);
var responseObject = JObject.Parse(response);
var products = responseObject["PricingProducts"].Children();
var result = new Dictionary<string, double>();
foreach( var product in products )
{
result.Add(product.Value<string>("LoanProgramName"),
product.Value<double>("Rate"));
}
return result;
}
When I execute this on local machine using dotnet run, everything works fine. Unit tests work great too. The logs on the App Service don't tell me much except that I am getting a 403 from the platform web service.
ers.RatesController.Get (AlexaRates) with arguments ((null)) - ModelState is Valid
2018-02-24 06:37:44.418 +00:00 [Information] Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker: Executed action AlexaRates.Controllers.RatesController.Get (AlexaRates) in 201.3483ms
2018-02-24 06:37:44.447 +00:00 [Error] Microsoft.AspNetCore.Server.Kestrel: Connection id "0HLBRA4B41EO8", Request id "0HLBRA4B41EO8:00000002": An unhandled exception was thrown by the application.
System.Net.WebException: The remote server returned an error: (403) Forbidden.
at System.Net.HttpWebRequest.GetResponse()
at System.Net.WebClient.GetWebResponse(WebRequest request)
at System.Net.WebClient.DownloadBits(WebRequest request, Stream writeStream)
at System.Net.WebClient.UploadBits(WebRequest request, Stream readStream, Byte[] buffer, Int32 chunkSize, Byte[] header, Byte[] footer)
at System.Net.WebClient.UploadDataInternal(Uri address, String method, Byte[] data, WebRequest& request)
at System.Net.WebClient.UploadString(Uri address, String method, String data)
at Rates.RetrieveLatest() in D:\home\site\repository\AlexaRates\Rates.cs:line 50
at AlexaRates.Controllers.RatesController.Get() in D:\home\site\repository\AlexaRates\Controllers\RatesController.cs:line 22
at lambda_method(Closure , Object , Object[] )
at Microsoft.Exten
Has anyone experienced anything similar? I see a bunch 403 posts, but they are mostly about people calling a REST API hosted on the service not calling out.

The 403 forbidden error usually means the server understood the request but refuses to authorize it.
According to your error message, it seems that the error happens in Rates class and RatesController class, which you haven’t showed for us. You could set a break point to check the code in these classes by using remote debugging.
You say the project is working fine locally, but get error in Azure, so please make sure you have published all your projects and data sources to Azure. Check whether the ‘_baseUrl ‘ is from Azure. And make sure you have started the Azure App Service.
There may be other causes of 403 forbidden error. Such as page cache and logging in of cookie. You could refer to this article to learn how to fix the 403 Forbidden Error.
Cause of 403 Forbidden Errors
403 errors are almost always caused by issues where you're trying to access something that you don't have access to.

My fix was that I realized that our infrastructure guys added a IP restriction on the azure app. That is why the app was bouncing back with a 403.
I removed the IP restrictions on the "Networking" -> "Access Restrictions" page.

After trying to add headers and doing various other things the end result was the same - I was getting a 403 error on calling out to a web service.
The solution was to convert from a Web App to a VM and deploy the application there using the old school setup. The application worked there.

Related

Azure app service some requests returns 400 Bad Request. The request could not be understood by the server due to malformed syntax

So we have a simple .net core 5.0 service that only serves some simple pages with mvc. We are starting to get 400 Errors (details below) on some of the requests. Our frontend is embedded in an iframe which forces us to use our own domain for our api-calls. The 400 errors disappears when we use the azure internal-urls. (*.azurewebsites.net instead of *.ourdomain.net). When I get to the "diagnose and solve problems" -> "availability and performance" -> HTTP 4XX ERRORS i can se below errors. Any ideas on what can cause this error?
Bad Request. The request could not be understood by the server due to malformed syntax. The client should not repeat the request without modifications.
So, the biggest problem above is that we do not get the correct errormessage. After a lot of experimentation we activated the ConnectionLogging for Kestrel.
WebHost.CreateDefaultBuilder(args)
.UseKestrel(options =>
{
options.ConfigureEndpointDefaults(listenOptions =>
{
listenOptions.UseConnectionLogging();
});
})
And after that we found some more intressting logs. One that said:
Connection id "0HMFSA73IA4LS" bad request data: "Malformed request: invalid headers."
Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Malformed request: invalid headers.
After some more investigation we could diff a succesful request from a failing request. And the problem was related to the certificate of *.ourdomain.se. In a part of the certificate we hade a string thats named "Stockholms län" in the cases where it failed the string decoded to l�n and when it succeeded the string decoded to l%C3%A4n. We are now investigating if this is a load balancer problem. But this app is running hostingmodel outofprocess. By changing this to inprocess and wrap our Kestrel in IIS the errors disapears.

Azure App Service Hijacks Response for HTTP STATUS 500 Errors

I have an app service that is working as expected in that I can get to pages, log in, and perform a search.
In the event of errors I have the below code in place in the MVC Controller (not an API endpoint)
{
_log.Error("Customer Search Failed", ex);
Response.StatusCode = 500;
return new JsonResult()
{
Data = new { Success = false, Error = "An error occured.", Amount = 0 },
JsonRequestBehavior = JsonRequestBehavior.AllowGet,
ContentEncoding = System.Text.Encoding.UTF8
};
}
Locally, I get the following response along with an HTTP Status Code of 500:
{"Success":false,"Error":"An error occured.","Amount":0}
In Azure, though, I get this response:
The page cannot be displayed because an internal server error has occurred.
I have written a test app to reproduce this in my own Azure environment so that there wouldn't be anything company Azure Resources that could cause this and I got the same behavior. No application gateways, VNets, anything like that. The app service itself is effectively seeing the 500 HTTP Status Code and then sending its own response vs what I want to send.
Does anyone know why this is happening and/or how to prevent this from happening?
The problem was that Azure must set up standard responses for non-200 HTTP Status Codes. This overrode what my Controllers were returning.
I had to explicitly remove the 500 httpHeader via the web.config in order to see the JSON response I was expecting.
<httpErrors errorMode="Detailed">
<remove statusCode="500" />
</httpErrors>

OData PATCH request results in IllegalArgumentException

Since last week we started using SDK version 3.34.1 (and also tested this with 3.35.0). When we send a PATCH request to a SAP service we get a HTTP 204 No-Content response back from our SAP service (SAP Gateway). When the SDK tries to read that response, it tries to parse the response body which is empty. This leads to the following exception:
2020-12-17 16:13:51.767 ERROR 106363 --- [ut.sap.cases]-0] .s.c.s.d.o.c.r.ODataRequestResultGeneric :
Failed to buffer HTTP response. Unable to buffer HTTP entity.
java.lang.IllegalArgumentException: Wrapped entity may not be null
at org.apache.http.util.Args.notNull(Args.java:54)
at org.apache.http.entity.HttpEntityWrapper.<init>(HttpEntityWrapper.java:59)
at org.apache.http.entity.BufferedHttpEntity.<init>(BufferedHttpEntity.java:59)
at com.sap.cloud.sdk.datamodel.odata.client.request.ODataRequestResultGeneric.lambda$getHttpResponse$4f00ca4e$1(ODataRequestResultGeneric.java:180)
at io.vavr.control.Try.of(Try.java:75)
at com.sap.cloud.sdk.datamodel.odata.client.request.ODataRequestResultGeneric.getHttpResponse(ODataRequestResultGeneric.java:180)
at com.sap.cloud.sdk.datamodel.odata.client.request.ODataHealthyResponseValidator.requireHealthyResponse(ODataHealthyResponseValidator.java:44)
at io.vavr.control.Try.andThenTry(Try.java:250)
at com.sap.cloud.sdk.datamodel.odata.client.request.ODataRequestGeneric.tryExecute(ODataRequestGeneric.java:194)
at com.sap.cloud.sdk.datamodel.odata.client.request.ODataRequestGeneric.tryExecuteWithCsrfToken(ODataRequestGeneric.java:225)
at com.sap.cloud.sdk.datamodel.odata.client.request.ODataRequestUpdate.execute(ODataRequestUpdate.java:136)
at com.sap.cloud.sdk.datamodel.odata.helper.FluentHelperUpdate.executeRequest(FluentHelperUpdate.java:372)
at com.alliander.gvrn.pmd.adapter.out.sap.cases.SapCasesClient.updateCase(SapCasesClient.java:103)
at com.alliander.gvrn.pmd.adapter.out.sap.cases.SapCasesClient.persistOn(SapCasesClient.java:81)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at ....
We use generated typed OData V2 client, which are generated by providing our EDMX files a per the SDK documentation.
Below a code snippet of the function that's calling the service. The matrixCase is a autogenerated object. The OData PATCH is properly handled by the SAP service.
private void updateCase(final ExternalId caseId, final PMDFlow pmdFlow, String jwtToken) {
final HttpDestination sapMatrix = httpDestinationProvider.providePrincipalPropagationDestination(jwtToken);
// "Create matrixCase object with key
MatrixCase matrixCase = MatrixCase.builder()
.psReference(caseId.getValue())
.build();
// Set PmdAppControl explicitly, otherwise the generated client doesn't know which fields are updated.
matrixCase.setPMDAppControl(pmdFlow.getSapNotation());
try {
casesService
.updateMatrixCase(matrixCase)
.executeRequest(sapMatrix);
} catch (ODataException e) {
OdataLogger.logODataException(e);
throw new SapClientException(e);
}
}
We've updated to SDK 3.34.1 due to other issues, however before we used 3.32.0 and I don't remember having this issue in version 3.32.0
Any ideas?
Danny
Yes, your observation is correct that 204 represents a valid answer and is not worth logging an error. Hence, the Cloud SDK team will adjust the log entry level to be less alarming.
Regards,
Tanvi

Getting HTTP error 400- Header too long error

I am getting HTTP 400- Header too long error, I have tried most of solution over internet, but none of them seemed to be working (solutions are like adding reg entries, etc).
I have a sample web application which is calling a web api on my system, but I am getting below error in HTTPERR log file:
2019-11-16 16:58:59 ::1%0 1213 ::1%0 80 HTTP/1.1 GET
/WebApplication2/api/values - 400 - RequestLength -
Here is my code which is calling the web API:
HttpWebRequest GETRequest = (HttpWebRequest)WebRequest.Create(URL);
GETRequest.Method = "GET";
GETRequest.ContentType = "application/json";
GETRequest.Headers.Add("Authorization", "<Something Big value>");
WebResponse GETResponse = await GETRequest.GetResponseAsync();
It is recommended to capture ETL log via
https://blogs.msdn.microsoft.com/wndp/2007/01/18/event-tracing-in-http-sys-part-1-capturing-a-trace/
Then you could use network monitor to analyze both request payload and etl log.
It will help you find the root cause.
Besides, can you find the 400 error in IIS log? Now that the 400 error appear in httperr log. It looks like http.sys blocked the request before it reach IIS server.
If you can find the 400 error in IIS , then you could try to modify the limitation in system.web/httpRuntime and request filter.
If the 400 error only can be found in httperr log. Then you may have to try to make some change in http.sys registry.
https://support.microsoft.com/en-us/help/820129/http-sys-registry-settings-for-windows
Please remember to reboot server to activate these registry!

SignalR 500 errors during "ping" on Azure website

I've got an ASP.NET 4.5 web app using SignalR 1.0.0-rc1 which I've pushed out to an Azure website for some quick-and-dirty testing (I'm mostly interested in this page here: http://alantaappbeta.azurewebsites.net/api/v3.0/Tests/Sample.htm).
The problem is that the page doesn't seem to want to talk to the SignalR service. The hubs and everything are registered correctly, as http://alantaappbeta.azurewebsites.net/signalr/hubs returns the correct client-side hubs file, and the call to /signalr/negotiate returns some reasonable looking JSON.
{
"Url":"/signalr",
"ConnectionId":"a15023f9-c675-4fc2-9fd6-403a297f10c0",
"KeepAlive":15.0,
"DisconnectTimeout":40.0,
"TryWebSockets":false,
"WebSocketServerUrl":null,
"ProtocolVersion":"1.1"
}
But when it calls /signalr/ping, it returns a 500 error, with the message "Protocol error: Unknown transport". The stack trace returned in the error page looks like this:
[InvalidOperationException]: Protocol error: Unknown transport.
at Microsoft.AspNet.SignalR.PersistentConnection.ProcessRequestAsync(HostContext context)
at Microsoft.AspNet.SignalR.Hubs.HubDispatcher.ProcessRequestAsync(HostContext context)
at Microsoft.AspNet.SignalR.Owin.CallHandler.Invoke(IDictionary`2 environment)
at Microsoft.AspNet.SignalR.Owin.Handlers.HubDispatcherHandler.Invoke(IDictionary`2 environment)
at Microsoft.Owin.Host.SystemWeb.OwinCallContext.Execute()
at Microsoft.Owin.Host.SystemWeb.OwinHttpHandler.BeginProcessRequest(HttpContextBase httpContext, AsyncCallback callback, Object extraData)
at Microsoft.Owin.Host.SystemWeb.OwinHttpHandler.System.Web.IHttpAsyncHandler.BeginProcessRequest(HttpContext context, AsyncCallback cb, Object extraData)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
Because this is on Azure, I don't have access to all the troubleshooting tools you'd normally have, but what logs I have been able to get access to don't give me any information beyond the above.
I suppose it goes without saying that this runs fine on my local IIS instance :-).
Any suggestions?
Edit: This is how I'm opening my connection:
$.connection.hub.start({
transport: 'auto',
xdomain: true
}).done(function () {
console.log('Connected with hub.id=' + $.connection.hub.id);
}).fail(function (e) {
console.log('Unable to connect to SignalR Hubs: ' + e);
});
But neither the .done() nor the .fail() handlers are being called.
Oddly enough, if I set the transport to 'longPolling', I can get it to fail in a similar manner using IIS Express on my local box. But it doesn't seem to make any difference what I set the transport to on Azure: I still get the same error.
It turns out that the issue was that I had a mix of different versions of SignalR on my machine: I was using the server-side DLLs from the NuGet package (1.0.0-rc1), but the JavaScript client file was from the dev branch. Once I got them all synchronized, everything worked. I'm still not sure precisely why the issue only showed up with Azure, but I suspect it has something to do with the precise transports supported by the different versions of IIS.

Resources