So I am getting confused with what is the right approach to implementing code that fires on a scheduled bases within azure.
Originally we were using a standard console app that would be put in the webjob folder on deployment. I found this a bit noddy as we had logic looping and waiting for the right time to fire.
I then tried the azure webjob package https://github.com/Azure/azure-webjobs-sdk-extensions, but see this has gone quiet and the master branch is currently broken! I like because it has a CRON type approach with a function.cs, but now not sure if this is being maintained.
So do people have a preference on how a background process would run, e.g. a scheduled task that would run at 2am every day against a database?
Too much choice and not enough consensus on what the right way is?
Much appreciated in advance
I can think of three options, all of which are valid and can suit your needs. Which one to choose in the end comes down to your requirement specifics and your technical expertise.
WebJobs. These are the most powerful and most difficult to build and maintain. You typically use a dedicated project template in Visual Studio to author these. You can ignore that GitHub link - that's not what you need. Make sure you have the Azure workload enabled in Visual Studio and create a WebJob project.
Azure Functions. These are a more lightweight alternative to WebJobs. There is Visual Studio tooling available for this as well but you also have the option of writing your code directly in the portal. Azure Functions will time out after some period of time, so if your job runs more more than a minute or two this might not be the best option.
Logic Apps. This is more of a power user tool with an easy to use (debatable) designer interface. But it's also incredibly powerful and you can call WebJobs or Functions if you need to from a Logic App.
I could add links but I'm sure you could find them easily enough.
Related
We are planning to migrate over from TFS 2015 to Azure DevOps, and the task assigned to me is to find out the way to do a comparison after the migration between what we have on TFS and Azure to ensure that all the tasks, bugs, etc was successfully migrated over. I've checked with the Guide from Azure and found nothing about such post migration checking and comparison. Is there any tool for this or we can only do the whole checking and comparison manually?
There is no such tool.
I have never experienced a partial migration. Due to the way the import works that is also VERY unlikely. Either the complete import fails, or the data is going to be there. I've done many of these migrations as well as server migrations/upgrades and the kind of data-loss you're worried about has never happened.
The one thing you'd need to be careful of are the changes to the retention policies.
We have multiple small projects within our organization. Ever since we adopted Azure DevOps recently, we started creating the individual Azure DevOps projects for everyone of these projects. The ALM process has been going on very well, with end to end traceability established with in the online tool.
However, as we get to end of each of these projects, we started to realize that these individual project and its code needs to be maintained further for the bug fixes and hotfixes. Unfortunately, Microsoft doesn't give a clear road-map for a project code, once it is completed.
Since we have a separate devops project for all the maintenance applications, this becomes much more confusing.
So, Could you suggest me on what is the best practice to maintain the code after the project is completed? Here are some options that I can think of.
Just keep the code in the same project and keep doing the bug fixes there. But, this will create an administrative nightmare to keep track of multiple projects for our maintenance application landscape (especially, since we already have a single projects with multiple repos and teams maintained).
Migrate the code from the completed project repository to the maintenance project. But, this again is a migration from one repo to another. So, I am not sure if this is right.
I am learning azure data factory and would really like to do its development in Visual studio environment. I have VS 2019 installed on my machine and I don't see an option to develop ADF in it.
Is there any version of VS that ADF can be developed in or we are right now stuck with developing it in web UI for the time?
I know BI development tools needed additional plug in to VS environment to work. Does ADF need something similar to that too.
If not, how can we back up our work done in web ADF. Is there an option to link it somehow with the azure repo or GIT?
Starting with ADF V2, development is really intended to be done completely in the web interface. I had the same question as you at the time, but now the web tools are quite good and I don't give it a second thought. While I'm sure there are other options for developing and deploying the ARM templates, do yourself a favor and use the web UI.
By default, Data Factory only saves code changes on "Publish". An optional configuration allows source control via Git integration. You can use either either Azure DevOps or Github. I highly recommend this approach, even if you only ever work in the main branch (fine for lone developers, a bad idea for collaboration). In this case, Publish takes the current state of the main branch and surfaces your artifacts to the ADF service. That means you will still need to Publish for your changes to be live.
NOTE: Git integration is also supported in Azure Synapse, where it has tremendous value for collaboration across a wide variety of artifact types.
I am using this - https://msdn.microsoft.com/en-us/pnp_articles/getting-started-with-building-azure-webjobs-for-your-office365-sites approach to create a web job and I am not able to see below screen. Is ‘Run on Schedule’ not an option anymore?
Below is the screen I am getting:
The web job works perfectly on demand but what I really need is to schedule. Any help on this would be highly appreciated.
You can add a settings.job file to your WebJob. This can include a CHRON expression that specifies when the job should run. The appropriate docs are here.
Alternatively, you can use a TimerTriggerAttribute. This is a WebJobs SDK Extension. Docs and examples for doing so are here. Note that if you use TimerTrigger, you'll need to make sure your WebJob is deployed as continuous. The TimerTrigger will wake up and call the designated methods based on the schedule that you provide.
I've got a large Azure IOT Suite implementation and I'd like to back it up or export the various jobs/components for safe keeping. We had a few instances where someone deleted something incorrectly and it took some time to recreate it.
Thanks!
Nick
Are you using azureiotsuite.com to deploy your Azure IoT suite?
Microsoft provides with an alternative approach of deploying it locally, and you can also customize and extend the solution to meet your specific requirements.
See to this repository for more details.
Use a DevOps approach i.e. script all the components and maintain the scripts in GitHub or some other version control tool.
Unfortunately, support for reverse engineering existing components is very limited.
I think I finally figured this out. With the new Resource Manager, you can click on the "Automation Script" and it will build out the ARM template that can be used to recreate the resources / settings as needed.