I currently have a couple of WebApi projects that use a few class libraries such as address lookup, bank validation, image storage etc.
Currently they are all in a shared solution but I'm planning to split them up. I thought about moving the libraries into NuGet packages so that they are separate from the API projects and is properly shared.
However, if I make a change to one of these components I will need to build and redeploy the API service even though it's a separate component which has changed.
I thought about putting these components into a separate service but seems a bit of overhead for what it is.
I've been looking at Azure WebJobs and think I may be able to move these components into this instead. I have two questions related to this:
Are WebJobs suitable for calling on demand (not using a queue)? The request will be activated from a user on a web site which calls my API service which then calls the Web Job so it needs to be quick.
Can a WebJob return data? I've seen examples where it does some processing and updates a database but I need a response (ideally Json) back to my API service.
Thanks
According to your requirement, I assume that you could try to leverage Azure Functions by creating a function using the HTTP trigger, which could be triggered by accessing the Function URL with parameters and return the response as you expected. You could follow this tutorial for getting started with Azure Functions.
Related
I want add a functionality in my ADF pipeline which will send me email notification in case of failure. On searching the internet, I came to know that Azure Logic Apps helps with this. I am trying to follow below link to achieve this.
https://microsoft-bitools.blogspot.com/2018/03/add-email-notification-in-azure-data.html
I have tried searching up many tutorials, guides and the official docs as well. However, all of them have some templates already there in the Logic Apps Designer. I cannot find the templates and the 'When a HTTP request is received' trigger is also not available in the drop-down.
Please let me know how to proceed.
EDIT :
If you start with a blank Logic App, search for 'HTTP' or 'Request' and select Request.
On the next screen under triggers, select "When a HTTP request is received" and you should be good to go.
EDIT:
It looks like you created a Standard Logic App, which works in a slightly different way. For instance it can contain multiple workflows, which means you create workflows yourself. In the Consumption model, there's one workflow withing a Logic App, so you can open up the editor for that one directly. If there's no explicit reason for you to use Standard, a Consumption Logic App will be easier to work with.
If you really need a Standard Logic App, go to 'Workflows' and create a new workflow:
Then click the newly created workflow to edit it, go to 'Designer' and search for 'HTTP' to add an HTTP trigger:
Here's some information on the Consumption model for Logic Apps:
Resource type
Benefits
Resource sharing and usage
Limits management
Logic App (Consumption) Host environment: Multi-tenant Azure Logic Apps
- Easiest to get started - Pay-for-what-you-use - Fully managed
A single logic app can have only one workflow. Logic apps created by customers across multiple tenants share the same processing (compute), storage, network, and so on.
Azure Logic Apps manages the default values for these limits, but you can change some of these values, if that option exists for a specific limit.
See Resource type and host environment differences for a comparison with the other hosting options.
I was able to solve this. I wasn't able to view a few functionalities because of another error : Functions runtime error Microsoft.WindowsAzure.Storage: Value cannot be null. (Parameter 'connectionString').
AzureWebJobsStorage App Setting was missing which caused the error. I added that and now I can see the triggers and other stuff.
Thanks #rickvdbosch
I've been working on developing an API to serve a machine learning model using Azure Machine Learning (AML) webservice deployments on a Kubernetes target as outlined here: https://learn.microsoft.com/en-us/azure/machine-learning/service/how-to-deploy-and-where#prepare-to-deploy
My particular use case requires more than simple scoring of data through the model. The API needs to have multiple endpoints to perform different actions related to the model. For example, an endpoint to upload data, an endpoint to delete data, an endpoint to list existing data, an endpoint to score previously uploaded data, an endpoint to change preprocessing parameters, etc...
I can build all of this logic, but I am struggling with the fact that AML web services only provides one endpoint (The service URI ending in "/score"). Is there a way to add more endpoints to an AML service? For example, I would like to have a way for users to be able to POST, GET, DELETE, PUT "/data", GET "/predictions", and POST, GET, DELETE, PUT "/parameters", etc..
Is there a way to do this in AML or is this not the right tool for what I am trying to accomplish? Is there a better solution within Azure that is more suited for my needs?
Thank you!
Azure ML allows controlled rollout/traffic splitting, but doesn't directly support your API design.
I might need to know more about your use case to make a recommendation. Are you looking at implementing incremental learning? What is the motivation for separate endpoints?
-Andon
Your proposal seems like a stateful web server which is more than a REST API service. For example, you need to keep a piece of logic to maintain "ids" of data: if there are two POST /data calls with different data, and the DELETE /data need to operate on the proper one. This is much more than a single performance optimized machine learning service.
I would recommend you creating a separate server with all these logic pieces and only reach Azure Machine Learning service whenever you need it. You could also build a cache in your service to only call Azure ML service when a new data coming in or the local cache expired. It will save you additional money from Azure :-)
I'm having a problem similar to this one. I'm attempting to deploy an API connection via an ARM template, and I'm getting 'ParameterNotDefined' errors when I deploy.
However, instead of creating an API connection to an azure table, I'm connecting to an azure queue.
Instead of getting a specific answer, I'd like guidance on how I can use documentation or other methods to answer this myself: I'm unable to find documentation on the existing APIs that are supported via the "MICROSOFT.WEB/CONNECTIONS" type, as well as the associated 'parameterValues' that each API expects you to supply.
Searching for a solution here is quite difficult because the terms are very overloaded. Here's what I've already tried:
I've tried using the Azure Resource Explorer, but it looks like this browses resources, not available APIs. If I can use it to view APIs, it's not clear how.
I've browsed through samples on the azure quickstart project. I see examples for azure blobs but not queues.
I've read some documentation on this page about creating API connections for logic apps. This appears relevent, and the author mentions making GET requests to "https://management.azure.com/.../" to view API details, but I get authorization header errors when doing this and I'd hope I can get documentation without operating this low on the stack.
I'm going to be creating a lot of API connections with logic apps in the near future and I need a more sophisticated method of discovery then google-searches, stack-overflow, and reuse of code samples. How do I view the entire set of API connections, along with the supported 'parameterValues' that are to be used with each one?
You can use ARMClient to retrieve the needed paremeters.
armclient.exe get https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.Web/locations/{region}/managedApis/{Api}?api-version=2016-06-01
For more information https://www.bruttin.com/2017/06/13/deploy-logic-app-with-arm.html
For anyone, checking it now, you can create the Api Connection manually and check the JSON View of it, it contains all the parameters.
I understand that Microservices is about independent loosely coupled services. I have read https://en.wikipedia.org/wiki/Microservices.
When it comes to Azure, I understand there are many components like Azure Service Fabric, AKS and also have the option of deploying containers within Azure VMs using Docker or any other containerization tools. However, since Microservices is about developing atmoic individually scalable services, can this also be achieved by deploying each service as an Azure Web API APP within an App Service Plan and configure Auto-Scale based on Performance metrics (though each API APP may not be individually scalable, they can still be individually manageable in terms of deployment, configuration etc)?
Can someone please suggest if this thought process is correct?
Microservices aren't a platform or technology so if you can make small independently deployable services then they are microservices. Sure - some tech helps but it depends on your situation.
If you only need a few services you probably don't need anything complex. Make sure services are well modeled, own their own data and ideally have a good monitoring and deployment pipeline setup. Design for service failure where possible.
Do you need to scale each part independently? Ideally, you should be able to but do services have very different requirements? You could have many small App service plans but that comes at cost of unused resources so split when you need to.
This question and of course the answers are going to be opinion based, but generally when thinking in terms of micros services, think not in terms as things like loads of API's and VM's etc. Instead think in terms of. When i upload an image, its needs to be resized, and the table updated to give a url for the thumb. or when XXX record is updated in database, Run XXX in order to create a report, or update Azure search. and that each service, just knows how to do a single thing only. I.E Resize an image.
Now one could say. I have a system, A repo library, and some functions library. When an image is posted, I upload, then call this, and that etc.
With Micor services. You would instead just add the image to a queue. Create an azure function that has a queue trigger. that would resize and save both the large and the thumb to storage. this would then either update the database, or in true micro service, it would add a queue to store the new info, another function would watch that queue and insert into the database.
You can use the DB queue from anything. You can use the Blob queue from anything. Your main API, does not care how images are handled. You can change your functions one day to maybe save to dropbox, instead of azure blob. All really easy, with no re-build of the API, because the API does not care.
A good example I use it for is email and SMS. My systems dont know how to send an email, or an SMS. They only know how to add to a queue. My microservices. SendEmail and SendSMS do know how to do it, and I can change how and who i send that content with, really easy. I can tomorrow change from Twilio to send grid, without ever telling the API that i've done it.
On a more complex thing. I have approval, at the moment that approval sends an email or SMS to either user or admin, and that can change over time. So I have an SMS server, Email Service and and approvalService. when approval happens, it just adds a config to the queue, The rest is done by a logic app, that knows to send an email to XXX and an SMS to XXX and then update database. My api, is just a post, that creates a queue.
Basically what I am saying here is to get started, maybe porting an existing app. Start with the workflow stuff, like send an email, resize an image, create a report, create a PDF, email 50 subscribers etc. and take all that code out and put into there own micro service that just knows how to do one thing. Then when you grow with confidence, create a workflow from all of these services with Logic Apps, let azure take care of the rest, thats what they want to do.
I have a time trigger azure function deployed on portal. It runs daily at 10:00 am. However, there is now a requirement that function should also be invoked and run on some other time dynamically as well.
I know how to set the trigger in function.json file dynamically via Kudu Api using the steps in answer mentioned here. So using those steps, I can set the trigger for the next minute and run the function.
But this isn't real-time, this seems a workaround. Isn't there any direct way to invoke and manually run azure function directly via apis?
Isn't there any direct way to invoke and manually run azure function directly via apis?
We could trigger the deployed Azure function with REST API. I test it with Time Trigger C# Azure function on my side.
Post https://{FunctionAppName}.azurewebsites.net/admin/functions/{functionName}
Note: I trace it from Azure portal, I don't find any official document mentioned this, if you want to use this API in the product environment, please pay more attention to this.
We need x-functions-key as header. And we could get the function key from the function Application.
We also could use bearer token as authorization, about how to get the authorization for this Rest API please refer to another SO thread.
Updated:
Add the body info.
For the requirement above, my recommendation would be to create two functions that share the same logic (if CSX, either by importing the common implementation using #load or adding a reference to a common assembly, or by having a common type).
You'd have one function using a timer trigger and another using a different trigger type that enables you to invoke the function on demand without a dependency on any of the Kudu or Admin APIs (e.g. HTTP, queue, SB, etc.), the function entry point (your Run method) would just invoke the common logic you bring in.