I am trying to integrate my azure solution with different cloud providers. I need to pull data from different cloud providers. They will expose the data via Rest services. I need to pull the data and publish it to my Azure platform. I am thinking of using the Azure Service Bus to pull the data and publish to an Azure Event Hub. Do you think this is better approach or any other good way to connect to a different cloud from Azure.
If the cloud providers you're trying to integrate with made data available via a RESTful API, I don't think Service Bus is going to help you. To make use of Service Bus, the third party cloud providers would need to enqueue data to a Service Bus Queue or Topic.
If you're trying to consume data periodically from RESTful APIs, how about using an Azure WebJob that calls those third party RESTful APIs on an interval, then processes the data?
Related
Follow up to other question posted here:
Azure microservices: authenticating downstream APIs from B2C to Azure AD, how to configure AD?
All our apps and microservices are hosted in a AKS cluster.
Want to have a single webapp decoupled from other services: using a message broker seems a good approach.
How should a public-facing portal be setup to be able to push data down into a queue or endpoint
How can calls to specific API endpoints and microservices to and from the webapp be secured from the public FE?
How would this be configured in terms of authentication, app reg etc.?
Lets answer your questions one after another which you have.
The first question
How should a public-facing portal be setup to be able to push data down into a queue or endpoint ?
For this first we need to use a Web API to push data to Azure Service Bus Queue. You can create a new ASP.NET Core Web API project and install Azure Service Bus NuGet package. Check this document for the way to do it.
Then we can use Service Bus Queue trigger Azure function to read the message from the queue and the process it or store it depending on the requirement. Check Azure Service Bus trigger for Azure Functions for more information.
The second question
How can calls to specific API endpoints and microservices to and from the webapp be secured from the public FE ?
We can achieve this requirement by securing the front end using Application Gateway. We need to configure Application Gateway in front of the Web API.
Check this Configure App Service with Application Gateway document from Microsoft for more information.
Lastly, the third question
How would this be configured in terms of authentication, app reg etc.?
In this case we can use Azure Managed Identity. Managed identities provide an identity for applications to use when connecting to resources that support Azure Active Directory (Azure AD) authentication.
So, we can use Azure Managed Identities with the Azure Resources like AKS. Check this Use managed identities in Azure Kubernetes Service for more information.
I am planning to deploy multiple applications on Azure kubernetes and each application will have multiple services.
I will have Azure API management in front of the kubernetes services.
As each application will have multiple services, how do I register those services in Azure API management? Would it be one by one? Is there a way to generate swagger document for the kubernetes services and register at once?
If you want to register them as a different services, yes you need to do one by one. But there is a way You can create a single API in API Management that points to multiple different backend APIs. Use the set-backend-service policy to change the backend API for certain operations.
I have implemented Rest API call using HttpTrigger based Azure functions, now customer has asked to implement the same using Azure Web Jobs instead of Azure Functions (owing to cost).
Are there any options available, please provide some reference how to achieve it in Azure web jobs
Thanks! in advance
No, that is not what WebJobs are for. You might be lookin into building an ASP.NET Web API and hosting it in an App Service Plan.
WebJobs is a feature of Azure App Service that enables you to run a program or script in the same context as a web app, API app, or mobile app. There is no additional cost to use WebJobs.
Taken from Run Background tasks with WebJobs in Azure App Service
Also, WebJobs do NOT support the HttpTrigger. The triggers that are supported by WebJobs:
Timer
Azure Storage queues and blobs
Azure Service Bus queues and topics
Azure Cosmos DB
Azure Event Hubs
File system
Source: What are Microsoft Flow, Logic Apps, Functions, and WebJobs? - Comparison table
EDIT:
As far as your solution goes: Either run it in a Function or in an App Service. If it's just one HttpTriggered call and (minor) startup times after some inactivity of the API aren't an issue: go for the Function implementation.
When retrieving a 'huge amount' of records (what is a huge amount?), as long as your Functions don't have any state and you pass in all information to determine the records to receive, there shouldn't be any issues. The Function app timeout duration in a consumption plan is 5 minutes by default.
we are working on an integration with service bus. We don't have an account with azure, so we can't run our own server, but our client does.
For testing purposes, are there any openly available service bus topics I can subscribe to and test our code? the content actually does not matter, we just want to make sure our code can connect and read messages.
If this is not possible, how is the approach for testing our subscription code for azure service bus? Do we need to open a azure account?
thanks!
You do need your own subscription to create and test Service Bus Topics.
To create your free Azure Subscription refer here.
I am looking for scalable hosting for an API but I am getting a little confused with Azure. Ideally I want something that means I don't have to manage OS updates and security, just deploy my API code.
Is this possible with Azure? What do I need to sign up for?
In general, you are looking for the Platform as a service offer. Check What is PaaS?. For the API deployment you should check the Azure App Services. Azure API Management is for advanced API management or when you want to expose multiple APIs using a single gateway.