I have an Azure App Service hosting an OData endpoint that is behind an Azure API Management (APIM) instance. To prevent calling the App Service directly it is protected by a certificate that only the APIM has.
When I call the APIM URL through Chrome or Postman, it behaves as expected. Just one request with no redirects or funny business, and it returns the OData root.
Here is a Fiddler log of a request to the APIM using Postman
However, when using the same URL as an OData source in Power Query using OData.Feed(), it returns a 301 which forwards to the backend URL, which obviously fails because that URL is protected by a certificate. Here is a Fiddler log of a request to the APIM using Power Query in Excel
I've configured the subscription key to be passed in the headers, but I've also tried it as a query param and it doesn't work in Power Query either way. I've also tried using an OData entity endpoint directly (to avoid the $metadata call) with no luck.
The user agent Power Query uses is Microsoft.Data.Mashup, but I haven't found any documentation about its compatibility with APIM, but that shouldn't matter, right?
In typical fashion after working on this for two days, I discovered the answer right after posting on StackOverflow. I'll leave this question up in case anyone has the same issue.
The problem was that the Power Query connector automatically follows #odata.context links for metadata, and #odata.nextLink links for paging. These links still had the app service site as the host instead of the APIM host.
So a quick edit of the outbound rules in APIM was able to fix the issue
<outbound>
<base />
<set-variable name="backendBaseUrl" value="#(context.Request.OriginalUrl.Scheme + "://" + context.Request.OriginalUrl.Host.ToString() + context.Api.Path)" />
<find-and-replace from="#("http://" + context.Request.Url.Host.ToString())" to="#((string)context.Variables["backendBaseUrl"])" />
<find-and-replace from="#("https://" + context.Request.Url.Host.ToString())" to="#((string)context.Variables["backendBaseUrl"])" />
</outbound>
Here I have to rules to replace http and https URLs just in case some configuration changes.
Related
We've an Azure based solution that tracks user actions on a SharePoint web site. A SharePoint extension produces a JSON payload that is sent to an API Management resource with the task to log it to Event Hub.
Useless I describe next parts of the architecture since will go beyond the purpose of the question.
We're used to deploy the whole solution automating DevOps with a custom software.
In the last deploy the APIM seems unable to work properly. The request took about 3 minutes and ends with 502 - Web server received an invalid response while acting as a gateway or proxy server.
I get this error using postman where it should normally return a mocked 200 after logging to Event Hub.
This is the XML policy used:
<set-body>#{
var body = context.Request.Body.As<JObject>();
body.Add(new JProperty("event_id", Guid.NewGuid().ToString()));
body.Add(new JProperty("ip_address", context.Request.IpAddress));
body.Add(new JProperty("ingest_status", "unknown"));
return body.ToString();
}</set-body>
<log-to-eventhub logger-id="pagesLogger">#(
context.Request.Body.As<string>(preserveContent: true)
)</log-to-eventhub>
<mock-response status-code="200" content-type="application/json" />
This is the top-level ALL APIs policy:
<cors>
<allowed-origins>
<origin>*</origin>
</allowed-origins>
<allowed-methods>
<method>GET</method>
<method>POST</method>
<method>OPTIONS</method>
</allowed-methods>
<allowed-headers>
<header>*</header>
</allowed-headers>
</cors>
The weird thing is that also commenting log-to-eventhub I get an error response: 500 - The request timed out.
I tried to create another API in the same APIM resource but I'm still unable to get a 200 from a mocked response. Also creating a brand new APIM resource it's not able to solve the problem.
Enabling application insights on APIM did not help to get further informations. The same also with Event Hub diagnostics.
I don't expect a solution but I would really appreciate hints that help me diagnose the problem.
It was an Azure issue. Without changing anything after two days everything
is back to normality
I have a frontend url say - 'https:\test.helloweb.com\account' which is defined a post request /person and I want to direct any request that comes on that url to a backend url like so
'https:\secure.hiddenapi\vi\api\person'.
The above scenario is easy and works now the place where I'm stuck is if there is a request for
Say -
/person/<id>/membership
/person/<id>/membership/<memid>
/person/<id>/accountdetails/
these requests needs to be directed to
https:\\secure.hiddenapi\vi\api\person\<id>\membership
https:\\secure.hiddenapi\vi\api\person\<id>\membership\<memid>
https:\\secure.hiddenapi\vi\api\person\<id>\accountdetails.
I tried defining the operation as /person/* then if i make a request to /person/<id>/membership then the backend url called is https:\\secure.hiddenapi\vi\api\person\membership without the id.
Seems like a simple issue but cannot make any progress!
Here is how APIM maps URLs. Consider:
An API with:
API URL suffix set to "myapi"
Web service URL set to "https://contoso.com/api"
An operation with:
URL template: "/myoperation"
When APIM receives a request it will split it into two parts:
https://service.azure-api.net/myapi/myoperation
^ ^
API base URL operation path
It will then replace API base URL with "Web service URL" specified for matched API and this will become request to backend:
https://contoso.com/api/myoperation
In your case, given that you end up with "person/person" it is likely you have "person" in both Web service URL and operation URL template. So you have two options.
First option is to configure Web service URL as "https://secure.hiddenapi/vi/api" and let all your operation templates start with "/person".
Another way is to set service URL as "https://secure.hiddenapi/vi/api/person" and do not include "/person" into your operation URL templates. It's perfectly fine if operation URL template starts with a variable like "{id}/membership/{memid}".
It's a good practice to split your whole backend API into different APIs in APIM for control and management. So personally I'd prefer first option, and if later there would be a need to make request to backend with prefix different than "/person" I'd create another API.
Try using rewrite URL policy to route to different backend URL for specific operation as follows:
<inbound>
<base />
<rewrite-uri template="/accountdetails/" />
</inbound>
We're trying to use a client certificate to authenticate when calling an OData service in SAP S/4HANA. And we're calling from an azure APIM instance. As certificate we've created a self-signed certificate and configured SAP S/4HANA according to this blog (https://blogs.sap.com/2020/05/03/x.509-certificate-based-logon-to-odata-services/)
Then we test this from the browser it works like a charm.
But calling from azure APIM we get the following response from SAP S/4HANA:
<?xml version="1.0" encoding="utf-8"?> <error xmlns:xsi="http://www.w3.org/2001/XMLSchema-Instance">
<code>HTTP/404/E/Not Found</code>
<message> Service /sap/opu/odata/sap/xxxxyyyy/xxyyzz call was terminated because the corresponding service is not available.The termination occurred in system UFI with error code 404 and for the reason Not found. Please select a valid URL. If it is a valid URL, check whether service /sap/opu/odata/sap/xxxxyyyy/xxyyzz is active in transaction SICF. If you do not yet have a user ID, contact your system administrator. </message>
SAP S/4HANA support says that then calling from browser they can 'see' certificate in payload but then calling from APIM, the payload is 'empty'.
I've got the trace logs from the SAP S/4HANA gateway server and I've noticed this subtly difference calling from browser vs calling from APIM:
Browser call (successfull):
[Thr 140708195055360] HttpModGetDefRules: determined the defactions: COPY_CERT_TO_MPI (1)
APIM call (failed):
[Thr 140708197697280] HttpModGetDefRules: determined the defactions: NOTHING (0)
So the certificate is obviously reaching SAP S/4HANA gateway server but not the SAP S/4HANA Odata server. So somehow, for some reason it's lost on the SAP S/4HANA gateway server only then it comes from azure APIM.
I've tried to make the calls 100% identical (same headers same values) but I can't control the way the certificate is added in azure apim or can one ?
I read that one can set the certificate body using policy below:
<authentication-certificate body="#(context.Variables.GetValueOrDefault<byte[]>("byteCertificate"))" password="optional-certificate-password" />
but I can't figure out how to get a proper value for "byteCertificate".
Has anyone done this? Or has anyone had a similar issue?
We finally found the solution!
Thanks to microsoft APIM support team, thanks a lot :)
APIM acts like a reverse proxy and adds headers related to this role. The header "X-Forwarded-For" causes SAP to deny the request with the above misleading error message. We found a solution that SAP could configure:
ICM parameter "icm/HTTPS/accept_ccert_for_x_forwarded_for_requests" has to be set to "true" - per default it's set to "false".
(The header can't be deleted with a policy on APIM side.)
I am having difficulty adding text into my url path between my frontend and backend when using Azure Api Management.
I have a Azure functions api with a url like this:
e.g. https://pXXXXX-myapi.azurewebsites.net/api/pXXXXX/alm/{name}
The default api created by Azure API Management is this:
e.g. https://myapi.azure-api.net/pXXXXX/pXXXXX/alm/{name}
Ideally I'd like to Frontend to remove the redundant pXXXXX:
e.g. https://myapi.azure-api.net/pXXXXX/alm/{name}
If I remove it from the front end e.g. GET /pXXXXX/alm/{name} becomes GET /alm/{name}.
Then I receive a 500 error.
I assume this is because removing "/pXXXXX" removes it from the backend path as well.
So I have tried to use the rewrite-uri policy like so:
<rewrite-uri template="/pXXXXX/alm/{name}" copy-unmatched-params="false" />
I've tried multiple variations of this with no success.
Assuming
you've changed the frontend for your operation to "GET /alm/{name}"
and your backend function app service url is "https://pXXXXX-myapi.azurewebsites.net/api"
and your API Base URL is "https://myapi.azure-api.net/pXXXXX"
then rewrite-uri exactly like you pasted is correct. You want to add leading "pXXXXX" to the url.
I've reproduced exact same scenario on my side and it works.
500 error does not mean there is something wrong with the path. If you would wrongly configure your url you would get 404. Most probably there is an internal error inside your function app or other policies in the APIM. Just call this API with Ocp-Apim-Trace header set to true and subscription key that allows tracing and you will see what is wrong. Or you can paste trace here if you want so we can help.
Posting my own answer but thanks to those who tried to help.
I believe the issue came from an error from APIM itself.
I assume it was not setting up the policies correctly, because now it is automatically including set-backend-service in the inbound policy whereas it did not over the past 2 days when I have been repeatably re-creating my api.
Now my inbound policy looks like this:
<inbound>
<base />
<set-backend-service id="apim-generated-policy" backend-id="pXXXXX-myapi" />
<rewrite-uri template="/pXXXXX/alm/{name}" copy-unmatched-params="false" />
</inbound>
And my front end api looks as before:
/pXXXXX/alm/{name}
I am using Azure API Management as front end for my Logic App. The "subscription required" setting needs to be enabled as we do need the protection. However, we must send the key via query parameter because our calling application only supports GET, not POST.
So my API call was sent to Azure using the format of https://my.azure-api.net/myapi/manual/paths/invoke?subscription-key=mykey
Now in Azure API setting I did create a policy set to delete action on the "subscription-key" query parameter, but here's the problem:
Even though the parameter is removed from the request body into Logic App, upon digging into the "RAW" outputs in Logic App where it shows various headers, we can see the subscription-key in these two headers:
"X-WAWS-Unencoded-URL": "/myapi/manual/paths/invoke?subscription-key=xxx
"X-Original-URL": /myapi/manual/paths/invoke?subscription-key=xxx
In other words, the full original query URL was made available to Logic App before the parameter was removed. This exposes the API subscription key to the Logic App.
Is there any workaround for this?
Ah I see now that those headers were actually sent automatically by Azure API Management to the backend Logic App API, so all I had to do was to set header policies to remove them in addition to the query parameter policy.
<set-query-parameter name="subscription-key" exists-action="delete" />
<set-header name="X-WAWS-Unencoded-URL" exists-action="delete" />
<set-header name="X-Original-URL" exists-action="delete" />
This takes care of it.