I developed a couple of microservice using Azure functions, every service has independent use case and different programming language.
Now I have a use case to use all service in below order, So I developed one more Azure function to use all service in given order. below code running well.
public static async Task<HttpResponseMessage> Run([HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)]HttpRequestMessage req, TraceWriter log)
{
string returnValue = string.Empty;
dynamic data = await req.Content.ReadAsStringAsync();
if (data == null)
{
return req.CreateResponse(HttpStatusCode.BadRequest, "Please pass a value in the request body");
}
else
{
string body = data.ToString();
var transformResult = await HttpRestHelper.CreateRequestAsync(AppConfiguration.TransformServiceEndPoint, body, HttpMethod.POST);
var validationResult = await HttpRestHelper.CreateRequestAsync(AppConfiguration.ValidationServiceEndPoint, transformResult.Result.ToString(), HttpMethod.POST);
if (validationResult.Result != null && Convert.ToBoolean(validationResult.Result))
{
var encryptionResult = await HttpRestHelper.CreateRequestAsync(AppConfiguration.EncryptionServiceEndPoint, transformResult.Result.ToString(), HttpMethod.POST);
var storageResult = await HttpRestHelper.CreateRequestAsync(AppConfiguration.StorageServiceEndPoint, encryptionResult.Result.ToString(), HttpMethod.POST);
returnValue = storageResult.Result.ToString();
}
else
{
returnValue = "Validation Failed";
}
return req.CreateResponse(HttpStatusCode.OK, returnValue, "text/plain");
}
}
Question
If every microservice takes 1 min to execution, I have to wait ~4min in my Super Service and billed for 4+ min. (We don't need to pay for waiting time :) https://www.youtube.com/watch?v=lVwWlZ-4Nfs)
I want to use Azure Durable functions here but didn't get any method to call external url.
Please help me or suggest a better solution.
Thanks In Advance
Durable Orchestration Functions don't work with arbitrary HTTP endpoints. Instead, you need to create individual functions as Activity-triggered.
Orchestration will use message queues behind the scenes rather than HTTP. HTTP is request-response in nature, so you have to keep the connection and thus pay for it.
Queue-based orchestrator can also give you some extra resilience in face of intermittent failures.
Related
I am currently making an API that will be hosted via Azure Functions. I'm running .net core 3.1. The way I have the project routed right now is defining the accepted methods as a parameter for the HttpTrigger then I have an if statement for determining how the endpoint was called. I am attempting to use the OpenAPI package to create API definitions, but when I assign Methods to the function, the Swagger document only picks up the first Method listed (PUT). I am unsure of the intended structure / usage of endpoints that have multiple possible request methods.
See code below. (OpenAPI tags are placeholder descriptions)
namespace Dyn.Sync.Func.PractifiSync
{
public class Prospect
{
[FunctionName("Prospect")]
[OpenApiOperation(operationId: "Run", tags: new[] { "name" })]
[OpenApiSecurity("function_key", SecuritySchemeType.ApiKey, Name = "code", In = OpenApiSecurityLocationType.Query)]
[OpenApiParameter(name: "name", In = ParameterLocation.Query, Required = true, Type = typeof(string), Description = "The **Name** parameter")]
[OpenApiResponseWithBody(statusCode: HttpStatusCode.OK, contentType: "text/plain", bodyType: typeof(string), Description = "The OK response")]
public async Task<IActionResult> Create([HttpTrigger(AuthorizationLevel.Anonymous, "post", "put", Route = null)] HttpRequest req, ILogger log)
{
string primarySecretsContainerName = "Main";
DynUser user = await DynAuthManager.CreateDynUserAsync(req);
DynProspect prospect = JsonSerializer.Deserialize<DynProspect>(req.Body);
PFIConnection pfiConnector = PFIConnectionsCache.GetConnection(user, DynSecretsCache.GetSecretsContainer(primarySecretsContainerName));
try
{
if (!pfiConnector.IsConnected) { await pfiConnector.Connect(); }
if (req.Method == "POST") { return await pfiConnector.CreateProspect(prospect); }
if (req.Method == "PUT") { return await pfiConnector.UpdateProspect(prospect); }
else { return new ObjectResult("Invalid method.") { StatusCode = 400 }; }
}
catch (Exception ex)
{
DynError dynError = new DynError(ex);
log.LogError(ex, "Exception " + dynError.RequestID.ToString() + " occured.");
return (IActionResult)new ExceptionResult(ex, true);
}
}
}
}
My question is this: When the swagger document is created, it only lists whatever method I defined first (in other words, it ignores the "put" method). What is the intended way to structure an API when creating it in Azure functions? I tried creating a separate method in the same class for each HTTP method that it would accept, but then I couldn't even hit the endpoint when making requests. Does microsoft want us to create a new function class for each endpoint? So then instead of:
PUT http://myapi.azure.com/api/prospect
POST http://myapi.azure.com/api/prospect
it would be:
PUT http://myapi.azure.com/api/updateprospect
POST http://myapi.azure.com/api/prospect
I should note that this will eventually live under and Azure API Management instance, which makes me even more worried to implement it in a "one function per method" fashion as when loading azure functions the way I have done it, it correctly assigns the methods in APIM and I'd prefer not to have to manually configure it.
I have been searching for documentation on this specific issue with no luck. Anyone have any ideas how Microsoft intended this to be used?
Thanks.
I want to schedule splunk report to an azure web-hook and persist it into Cosmos DB.(after from processing ) This tutorial gave me some insight on how to process and persist data into cosmos db via the azure functions ( in java ). To solve the next part of the puzzle I"m reaching out for some advise on how to go about:
How to setup and host a webhook on Azure ?
Should I set a HttpTrigger , inside the EventHubOutput function and deploy it into the function app.? Or should I use the Webhook from Azure Event Grid ?(not clear on how to do this ). I'm NOT looking to stream any heavy volumes of data and want to keep the consumption cost low. So , which route should I take here?. Any pointers to tutorials will be of help here.
How do I handle a webhook data processing on #EventHubOutput ( referring the java example in the tutorial) ?. What is the setup and configuration I need to do here ? Any working examples will be of help .
I ended up using just the #HttpTrigger and binding the output using #CosmosDBOutput to persist the data. Something like this , would like to know if there are any better approaches.
public class Function {
#FunctionName("PostData")
public HttpResponseMessage run(
#HttpTrigger(
name = "req",
methods = {HttpMethod.GET, HttpMethod.POST},
authLevel = AuthorizationLevel.ANONYMOUS)
HttpRequestMessage<Optional<String>> request,
#CosmosDBOutput( name = "databaseOutput", databaseName = "SplunkDataSource",
collectionName = "loginData",
connectionStringSetting = "CosmosDBConnectionString")
OutputBinding<String> document,
final ExecutionContext context) {
context.getLogger().info("Java HTTP trigger processed a request.");
// Parse the payload
String data = request.getBody().get();
if (data == null) {
return request.createResponseBuilder(HttpStatus.BAD_REQUEST).body(
"Please pass a name on the query string or in the request body").build();
} else {
// Write the data to the Cosmos document.
document.setValue(data);
context.getLogger().info("Persisting payload to db :" + data);
return request.createResponseBuilder(HttpStatus.OK).body(data).build();
}
}
Info :
I have below 2 method which is part of Web API (not core API) and it is deployed in Azure
Method 1 :
public async Task<bool> ProcessEmployee(list<employee> EmployeeList)
var tasks = new List<Task<EmployeeResponseModel>>();
HttpClient localHttpClient = new HttpClient();
localHttpClient.Timeout = TimeSpan.FromSeconds(100);
foreach (var employee in EmployeeList) // **having 1000 calls**
{
tasks.Add(GetAddressResponse(employee.URL,localHttpClient));
}
var responses = await Task.WhenAll(tasks);
}
Method 2 :
private async Task<EmployeeResponseModel> GetAddressResponse(url, HttpClient client)
{
var response = new EmployeeResponseModel();
try
{
using (HttpResponseMessage apiResponse = await client.GetAsync(**url**))
{
if (apiResponse.IsSuccessStatusCode)
{
var res= await apiResponse.Content.ReadAsStringAsync();
response = JsonConvert.DeserializeObject<EmployeeResponseModel>(res);
}
}
return response;
}
catch (Exception ex)
{
}
return response;
}
If i monitor from Azure -> Diagnose and Solve Problem -> Web App Slow all external API calls is showing latency issue
But if i am calling same external API from Postman is is quite fast and having less latency
method 1 and method 2 is part of one web api and it is deployed on Azure AppService.
getAddress is external API which is been deployed in other environment and don't have much information
if we are calling external API i.e 'getAddress' from 1) we are facing high latency more than 5 sec.
if we are calling external API i.e 'getAddress' from Postman we receive response in 303 ms.
I guess it results from the location of the service plan.
If the location of the service plan is far away from you position, it may cause the latency. But it can't rule out other possibilities, so my suggestion is debug in localhost first to rule out the possibility of the code.
Background:
I'm building a .net core webapi does practically nothing more than checking if a given URL exists and returns the result. If a URL exists and is a redirect (301, 302), the api follows the redirect and returns that result as well. The webapi is called by an SPA which does an api-call for every given url in a checkrequest-queue. So, if someone adds 500 urls to the queue the SPA will loop through it and will send 500 calls to the API – something I could improve upon.
The problem:
My IIS application pool is being shut down on a regular basis due to high CPU usage and/or memory usage:
A worker process serving application pool 'api.domain.com(domain)(4.0)(pool)' has requested a recycle because it reached its private bytes memory limit.
The only way to get my API going again is to manually restart the application. I don't think the operations performed by the API are that demanding, but I surely must be doing something wrong here. Can somebody help me please? The code called by the SPA is:
var checkResponse = new CheckResponse();
var httpMethod = new HttpMethod(request.HttpMethod.ToUpper());
var httpRequestMessage = new HttpRequestMessage(httpMethod, request.Url);
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls;
var httpResponseMessage = await httpClient.SendAsync(httpRequestMessage);
checkResponse.RequestMessage = httpResponseMessage.RequestMessage;
checkResponse.Headers = httpResponseMessage.Headers;
checkResponse.StatusCode = httpResponseMessage.StatusCode;
switch (httpResponseMessage.StatusCode)
{
case HttpStatusCode.Ambiguous:
case HttpStatusCode.Found:
case HttpStatusCode.Moved:
case HttpStatusCode.NotModified:
case HttpStatusCode.RedirectMethod:
case HttpStatusCode.TemporaryRedirect:
case HttpStatusCode.UseProxy:
var redirectRequest = new CheckRequest
{
Url = httpResponseMessage.Headers.Location.AbsoluteUri,
HttpMethod = request.HttpMethod,
CustomHeaders = request.CustomHeaders
};
checkResponse.RedirectResponse = await CheckUrl(redirectRequest);
break;
}
The Action on my ApiController:
[HttpPost]
public async Task<IActionResult> Post([FromBody] CheckRequest request)
{
if (!ModelState.IsValid)
{
return BadRequest(ModelState);
}
var result = await CheckService.CheckUrl(request);
return Ok(result);
}
I am making a small App that should list the number of items in my Azure queues.
When I use FetchAttributesAsync and ApproximateMessageCount in a Console App, I get the expected result in ApproximateMessageCount after a call to FetchAttributesAsync (or FetchAttributes).
When I use the same in a Universal Windows app, ApproximateMessageCount remains stuck at null after a call to FetchAttributesAsync (FetchAttributes is not available there).
Console code:
CloudStorageAccount _account;
if (CloudStorageAccount.TryParse(_connectionstring, out _account))
{
var queueClient = _account.CreateCloudQueueClient();
Console.WriteLine(" {0}", _account.QueueEndpoint);
Console.WriteLine(" ----------------------------------------------");
var queues = (await queueClient.ListQueuesSegmentedAsync(null)).Results;
foreach (CloudQueue q in queues)
{
await q.FetchAttributesAsync();
Console.WriteLine($" {q.Name,-40} {q.ApproximateMessageCount,5}");
}
}
Universal App code:
IEnumerable<CloudQueue> queues;
CloudStorageAccount _account;
CloudQueueClient queueClient;
CloudStorageAccount.TryParse(connectionstring, out _account);
queueClient = _account.CreateCloudQueueClient();
queues = (await queueClient.ListQueuesSegmentedAsync(null)).Results;
foreach (CloudQueue q in queues)
{
await q.FetchAttributesAsync();
var count = q.ApproximateMessageCount;
// count is always null here!!!
}
I have tried all kinds of alternatives, like Wait()'s and such on the awaitables. Whatever I try, the ApproximateMessageCount stays a null with dertermination :-(.
Am I missing something?
I think you have discovered a bug in the storage client library. I looked up the code on Github and essentially instead of reading the value for Approximate Message Count header, the code is reading the value for Lease Status header.
In QueueHttpResponseParsers.cs class:
public static string GetApproximateMessageCount(HttpResponseMessage response)
{
return response.Headers.GetHeaderSingleValueOrDefault(Constants.HeaderConstants.LeaseStatus);
}
This method should have been:
public static string GetApproximateMessageCount(HttpResponseMessage response)
{
return response.Headers.GetHeaderSingleValueOrDefault(Constants.HeaderConstants.ApproximateMessagesCount);
}
I have submitted a bug for this: https://github.com/Azure/azure-storage-net/issues/155.