I am looking for recommendation for moving Integration stack from Oracle Fusion middleware to Microsoft Azure. Is Logic App a fair substitute for it. Can I reuse most of XSL transformations that I have in Fusion middleware. Is it easy enough to monitor the payload and transformations from source to destination.
Most used components are file watchers, db adapters and soap/rest service invocations
Are there other better options. Would it be worthwhile to move the entire integration stack to something like axis2 and then expose it via Azure Service Bus and Azure Relay
Related
How does a Azure function differ from a ASP.NET Core Worker Service?
Does both of these cover the same use cases?
How do I decide which one to use?
Update:
Azure function is designed for azure. So when you use azure, or other azure services which are integrated with azure function, you should consider azure function, which could simplify your code and logic by the built-in features of azure function. But it will also have charges for azure function. And yes, if you don't use azure, just use asp.net core worker services.
Azure function are totally different from ASP.NET Core Worker Service.
The benefit of Azure function is that it supports a lot of triggers like azure blob trigger / azure event hub trigger, and is integrated with other azure services.
With these built-in features, it's easy to create a proper azure function to complete a proper task. For example, if you upload an image to azure blob storage, and then want to resize the image, you can easily create a blob trigger azure function with less code.
Worker services are the perfect use case for any background processing. And if you want to operation some azure services like azure blob / azure event hub etc. you may achieve this but need to do a lot of work.
At last, it depends on your use case to choose which one should be used, and select the simple / efficient one.
I think the answer Fred was looking for is, why would someone use Azure functions over Asp.net Core hosted service? And the answer isn't only because you have Azure account so everything is easy and integrated. It depends. I.e Traffic, cost, expertise, etc.
Actually, there are guidelines set by Microsoft when to use an Azure Functions versus let's say over a IHosted service running in a linux container hosted in AKS for example. and I'll post the link, but it's basically if your workload is mission critical or not. If you want the best performance, lowest cost, reliability a hosted dotnet core service running in Linux container would be an excellent alternative.
Here is the mission critical guidelines from Microsoft, hope this helps!
Design recommendations
Azure Functions:
Consider Azure Functions for simple business process scenarios which don't have the same stringent business requirements as business-critical system flows.
Low-critical scenarios can also be hosted as separate containers within AKS to drive consistency, provided affinity and anti-affinity requirements are fully considered when collocating containers on nodes.
Excerpt taken from here:
https://learn.microsoft.com/en-us/azure/architecture/framework/mission-critical/mission-critical-application-platform
I've been working on developing an API to serve a machine learning model using Azure Machine Learning (AML) webservice deployments on a Kubernetes target as outlined here: https://learn.microsoft.com/en-us/azure/machine-learning/service/how-to-deploy-and-where#prepare-to-deploy
My particular use case requires more than simple scoring of data through the model. The API needs to have multiple endpoints to perform different actions related to the model. For example, an endpoint to upload data, an endpoint to delete data, an endpoint to list existing data, an endpoint to score previously uploaded data, an endpoint to change preprocessing parameters, etc...
I can build all of this logic, but I am struggling with the fact that AML web services only provides one endpoint (The service URI ending in "/score"). Is there a way to add more endpoints to an AML service? For example, I would like to have a way for users to be able to POST, GET, DELETE, PUT "/data", GET "/predictions", and POST, GET, DELETE, PUT "/parameters", etc..
Is there a way to do this in AML or is this not the right tool for what I am trying to accomplish? Is there a better solution within Azure that is more suited for my needs?
Thank you!
Azure ML allows controlled rollout/traffic splitting, but doesn't directly support your API design.
I might need to know more about your use case to make a recommendation. Are you looking at implementing incremental learning? What is the motivation for separate endpoints?
-Andon
Your proposal seems like a stateful web server which is more than a REST API service. For example, you need to keep a piece of logic to maintain "ids" of data: if there are two POST /data calls with different data, and the DELETE /data need to operate on the proper one. This is much more than a single performance optimized machine learning service.
I would recommend you creating a separate server with all these logic pieces and only reach Azure Machine Learning service whenever you need it. You could also build a cache in your service to only call Azure ML service when a new data coming in or the local cache expired. It will save you additional money from Azure :-)
We currently have a window service which send some notification emails to users after doing some processing on database(SQL database). Runs once in day.
We want to move this on azure cloud. One alternate is to put it on Azure VM as is. but I am finding some other best possible solution for that.
I study about recurring and on demand Web jobs but I am not sure is this is best solution.
Also is there any possibility to update configuration of service code in App.config without re-deploy the code of service on cloud. I means we can manage configuration from Azure portal.
Thanks in advance.
Update 11/4/2016
Since this was written, there are 2 additional features available in Azure that are both excellent choices depending on what functionality you need:
Azure Functions (which was based on the WebJobs described below): Serverless code that can be trigger/invoked in various ways, and has scaling support.
Azure Service Fabric: Microservice platform, with support for actor model, stateful and stateless services.
You've got 3 basic options:
Windows service running on VM
WebJob
Cloud service
There's a lot of information out there on the tradeoffs between these choices, but here's a brief summary.
VM - Advantages: you can move your service basically as it is without having to change much or any of your code. They also have the easiest connectivity with other resources in Azure (blob storage, virtual networks, etc). The disadvantage is you're giving up all the of PaaS advantages and are still stuck managing your own VM infrastructure
WebJob - Advantages: Multiple invocation options (queues, blobs, manually, queue receive loops, continuous while-loop style, etc), scheduled (would cover your case). Easy to deploy (can go with website, as a console app, automatically through Kudu), has some built in logging in Azure portal - and yes, to answer your question, you can alter the configuration in the portal itself for connection strings and app settings.
Disadvantages - you'll need to update code, you don't have access to underlying resources (if you need that), and more of something to keep in mind than a disadvantage - it uses the same resources as the webapp it's deployed with.
Web Jobs are the newest of the options, but at the same time appear to have active development going on to increase the functionality and usefulness.
Cloud Service - like a managed VM, has some deployment options, access to underlying VM if needed. Would require some code changes from your existing service.
There's nothing you've mentioned in your use case that makes me think a Web Job shouldn't be first thing you try.
(Edit: Troy Hunt has a great and relatively recent blog post illustrating most of the points I've mentioned about Web Jobs above: http://www.troyhunt.com/2015/01/azure-webjobs-are-awesome-and-you.html)
I am looking for a solution for real-time data integration between few on-premise databases. There is no much transformation of data involved.
I am evaluating various ESBs available. I am thinking that data integration using Azure Service Bus as quick to develop a solution. Is it advisable to use Azure service bus for integration of all on-premise databases?
Unless there is extraordinary complexity in the integration, BizTalk is probably not the right tool for the job here. On the other hand, sending data out to the cloud just to transform it back to another database (on the same LAN?) is also not the right approach - this will introduce latency and traffic cost.
(Near) Real Time integration of databases sounds like a job for something like:
SQL Server Integration Services (SSIS)
If DB's are Sql-Server and the schemas are similar, Sql Server Replication
And similar technologies exist for other RDMBS technologies , e.g. Oracle Streams
If you really want to build a service bus, either build a local AMQP based bus as Sam suggests (e.g. Windows Service Bus or Rabbit), or buy an existing product (NServiceBus etc).
If you have all your applications 'on prem', you introduce an extra risk by moving your integration layer to the cloud (suddenly your internet connection could bring down your integration layer)
but the good news is that you can use Service Bus for Windows Server, that you run locally (even with Windows Azure pack!)
The same programming model, similar messaging features, so that might be a good option.
Comparing with BizTalk... Service Bus is light weight, messaging only. BizTalk provides much more rich features (transformations, pipelines, BAM, Business Rules, adapters).
Good luck
If you are only looking to integrate between a few on premise databases then you might consider using Sql Server's Service Broker (http://msdn.microsoft.com/en-gb/library/bb522893.aspx).
It provides a reliable asynchronous way of passing data between databases in a real time way. It can manage the message order and can have numerous conversations running at the same time on the same queue, each being processed by its own instance of the receiver.
There's a good overview here...
http://technet.microsoft.com/en-us/library/ms166104(v=sql.105).aspx
We are looking to use Windows Azure to host our existing SaaS platform and extend our functionality and capability. WE will be taking adavantage of both the data storage and application and web service functionality of Azure.
My question is as follows:
Some of our clients will not want Public CLoud access. Since our datastore stores sensitive client data many of them will require our whole system to be hosted internally on their own network and servers.
If we setup a full Azure setup of database and connected applications and processes how difficult is it to be able to duplicate that system for a specific client on their own servers and network using existing Microsoft technologies?
I know its a vague question and I also have a liminted understanding of Azure so whatever information you can provide here would be most appreciated.
Thank you
It sounds like you need the flexibility of a hybrid cloud/on-prem solution. Likely the best solution is the Windows Azure Service Bus. Essentially, you configure a WCF web service in the cloud (SOAP, REST, etc) that performs asynchronous brokered messaging between your on-premise application and your web application. This can be performed using queue messages, for example:
The web application (cloud) requests resources from the brokering service (cloud) by sending a queue message
The service handles the queue message and makes it available to the consuming (on-prem) service
On-prem service checks for new messages from the brokering service, gets the request for data, and returns desired data from DB
On-prem service sends message to brokering service with desired data
Web app (cloud) checks for new messages from the brokering service, then uses the data from on-prem service
Service bus is secure, asynchronous, fault-tolerant, and ensures that both components are decoupled.
Another method is to use Windows Azure Connect, which is a VPN solution that sets up network-level connnectivity. I recommend Service Bus because it promotes a more robust and scalable architecture, and fault-tolerance is high.