As title really, I can't see how you can Roll Back an update to a Windows Azure service/site?
Each "update" has a title, so I'd thought you could see a list of these updates, but I just can't see it listed anywhere?
I use https://manage.windowsazure.com, and these are cloud services.
Details on Rollbacks can be found here: http://msdn.microsoft.com/en-gb/library/windowsazure/hh472157.aspx#RollbackofanUpdate
To quote:
Windows Azure provides flexibility in managing services during an update by letting you initiate additional operations on a service, after the initial update request is accepted by the Windows Azure Fabric Controller. A rollback can only be performed when an update (configuration change) or upgrade is in the in progress state on the deployment. An update or upgrade is considered to be in-progress as long as there is at least one instance of the service which has not yet been updated to the new version. To test whether a rollback is allowed, check the value of the RollbackAllowed flag, returned by Get Deployment and Get Hosted Service Properties operations, is set to true.
Note
It only makes sense to call Rollback on an in-place update or upgrade because VIP swap upgrades involve replacing one entire running instance of your service with another. For more information about swapping VIPs, see How to Deploy a Service Upgrade to Production by Swapping VIPs in Windows Azure.
There is no stack maintained of your previous deployments, if however your present deployment somehow fails than the fabric may use your previous deployment(It tries to initialize previous states). Your roles are just VHD(Virtual hard disk) with a copy of OS and your installation files running virtually, whenever you update, a new VHD is allotted and the previous is destroyed(if update was successful). Thus there is no way to get the previous deployments(its destroyed). You should store your previous builds locally or on a foundation server just in case you need it like you now.
Related
We are running a larger backend application in NodeJS/TS on Firebase with about 180 cloud functions and Firestore as database. Firebase has been good for our needs so far, but we are getting to a level of usage where even small amounts of down-time can cause a lot of damage. Due to the amount of cloud functions a full deploy could take up to 30 minutes, we therefore usually only do partial deploys of changed functions only, which still take about 10 minutes. I am trying to find a way to be able to do quick rollback to previous version of a given function in case a bug is discovered after a production deploy. Firebase does not seem to provide rollback functionality, so the only option is to re-deploy the code with the previous version. One issue is the deploy time (up to 10 min for a single function), and the other is git versioning when there are partial deploys. Normally there would be a branch reflecting exactly what is in prod that could be used, but with partial deploys this is no longer the case. The only alternative for maintaining good git versioning with one to one branch with prod is to do a full deploy every time, but this takes a prohibitive amount of time (30+ minutes not including retries). The firebase deploy usually fail or exceed deployment quota as well, which makes things like CI pipelines very difficult (it would have to automatically retry failed functions, and the time is still an issue since 30+ min to deploy is not acceptable in the case of down-time). Has anyone found a good solution for roll-back (versioning) and a git structure that works well with firebase at scale?
Cloud Functions for Firebase is based on Cloud Functions and their behavior are the same. And today, it's not possible to route the traffic to a previous version (and to perform a rollback). (And I can also told you that NodeJS16 is now GA, instead of Beta as still mentioned in the Cloud Functions for Firebase documentation)
The next Cloud Functions runtime is cooking (and available in preview). That runtime is based on Cloud Run under the hood, that allow traffic splitting/routing, and therefore accept rollback.
So, for now, you haven't solution to perform a simple rollback with Firebase functions. A great change could be to use Cloud Functions V2 runtime directly, or event Cloud Run, but it's a big change in your code base.
Another solution could be to use a load balancer in front of all your functions and to:
Deploy new function under new name (no update of the current deployment, create a new service each time that you deploy a new version)
Create a new serverless backend with the new functions
Update the URL map to take into account the new backend.
After a while, delete the old function versions.
That also requires a lot of work to put that in action. And the advertising delay when you update your URL map should be between 3 and 5 minutes, not a such great advantage compare to your current solution.
it looks like your not the only one. previous questions answered. I recommend setting up some version control.I would solve the failing deploy issues first which should reduce the deploy time and redeploy times specifically if its multiple . You could use a different deploy branch or setup a staging environment as well. I would invest the time in getting the GIT control setup/turnkey.
Per user Ariel:
Each time you make a deploy to a cloud function you get an output line like this:
sourceArchiveUrl: gs://my-store-bucket/us-central1-function_name-xxoxtdxvxaxx.zip
I entered my Google Cloud Platform Developer Console -> Cloud Functions -> function_name -> Source tab
and there almost at the bottom it says: Source location
my-store-bucket/us-central1-function_name-xxoxtdxvxaxx.zip
the same as it was shown in the CLI, but without gs:// that link lead me to the following: https://storage.cloud.google.com/my-store-bucket/us-central1-function_name-........
I removed from the link everything that came after
https://storage.cloud.google.com/my-store-bucket
and that lead me to a huge list of files that each one of them represented a an image of all my cloud functions at the time point of each time i have made a deploy, exactly what i needed!
The only thing left to do was to locate the file with the last date before my mistaken deploy
source: Retrieving an old version of a Google Cloud function source
as of 2019
Rolling back to an older version of a firebase function (google cloud function)
2021:
Roll back Firebase hosting and functions deploy jointly?
You can roll back a Firebase Hosting deployment, but not the functions without using a GIT Version control etc. Using partials you can deploy multiple functions/Groups. You can checkout remote config templates to rollback and its kept for up to 90 days.
https://firebase.google.com/docs/remote-config/templates
Firebase partial deploy multiple grouped functions
https://firebase.google.com/docs/cli#roll_back_deploys
My company's application experienced database connection issues this morning resulting in me having to failover to our secondary database. Within our Azure App Services, this was an easy step of changing the connection string in the configuration, however I could not find an easy way of changing these settings on our Service Fabric services without redeploying.
I'm considering options to allow failover at runtime for these services to a secondary database but don't know what the 'best practices' would be. A couple options I have:
I could create a dns entry for our database server that i manage and then just switch that to the new server name when I need to fail over.
I could have some sort of rest api to call on my app services that would return whether or not to go to the secondary database.
Any other ideas? I'd like to make failover to the secondary as seamless as possible so it can be done quickly.
Have you considered putting both your primary and secondary database connection strings into your application's config and writing some code that automatically switches between them if it detects a problem? Both of the options you presented puts a human in the path, which means your users are going to experience downtime until the human fixes the problem (maybe the human is asleep, or on vacation, or on vacation and asleep).
In Service Fabric, Application (and system) upgrades are always rolling upgrades. Rolling upgrades have the advantage of preventing global outages. For example, suppose at some point you updated your config with the wrong connection string. A global config change might be quick and easy, but now you have a global outage and some upset customers. A rolling upgrade would have caught the error in the first upgrade domain and then rolled back, so only a fraction of your application would have been affected.
You can do a config-only rolling upgrade. This is where you make a change to your config package and then create a differential upgrade package so that only the config changes go out and your service process doesn't have to restart.
Just to post an update to my issue here. SQL Azure now has automatic failover groups. This is described here
I've update production deployment yesterday morning then I've made changes to service files using remote connection
add and update files and everything was OK.
today morning all the changes I've done after deployment was undone and customers use the old version and this cost us hundreds of thousand of pounds
i need to know what's happen nothing appeared in operations log
Probably what has happened is that Microsoft has updated your servers at the Cloud Centre and re-deployed your application from the original deployment. This is in their terms and conditions, you should not make any important manual changes to the deployment after it is deployed unless they are stored in the portal (environment settings etc.), otherwise they might be lost during updates or reboots.
I learned this the hard way too. I had a cache role with only one instance (I thought it only made sense with one instance) and while updates happened, my whole site went down several times over several days!
PaaS services are stateless, which means the VMs running your service can be destroyed and recreated at any time, at which point the VM will be recreated with the content from your original .cspkg.
For more information see http://blogs.msdn.com/b/kwill/archive/2012/09/19/role-instance-restarts-due-to-os-upgrades.aspx and http://blogs.msdn.com/b/kwill/archive/2012/10/05/windows-azure-disk-partition-preservation.aspx.
As others have said, PaaS Web Roles are stateless. If you're making manual configuration changes to your deployed solution package after it has been auto-deployed then any re-deployment by the Azure fabric will simply deploy the package minus your manual changes. To solve this issue you could use startup tasks to apply your manual changes using a PowerShell script or similar (depending on what you're changing). See http://msdn.microsoft.com/en-us/library/jj129544.aspx.
Note that startup tasks don't just run when a machine gets re-imaged or rebooted.
I've got 6 web sites, 2 databases and 1 cloud environment setup on my account
I used the cloud to run some tasks via Windows Task Manager, everything was installed on my D drive but between last week and today the 8 of March my folder containing the "exe" to run as been removed.
Also I've installed SVN tortoise to get the files deployed and it not installed anymore
I wonder if somebody has a clue about my problem
Best Regards
Franck merlin
If you're using Cloud Services (web/worker roles), these are stateless virtual machines. That is: Windows Azure provides the operating system, then brings your deployment package into the environment after bootup. Every single virtual machine instance booted this way starts from a clean OS image, along with the exact same set of code bits from you.
Should you RDP into the box and manually install anything, anything you install is going to be temporary at best. Your stuff will likely survive reboots. However, if the OS needs updating (especially the underlying host OS), your changes will be lost as a fresh OS is brought up.
This is why, with Cloud Services, all customizations should be done via startup tasks or the OnStart() event. You should never manually install anything via RDP since:
Your changes will be temporary
Your changes won't propagate to additional instances; you'll be required to RDP into every single box to perform the same changes.
You may want to download the Azure Training Kit and look through some of the Cloud Service labs to get a better feel for startup tasks.
In addition to what David said, check out http://blogs.msdn.com/b/kwill/archive/2012/10/05/windows-azure-disk-partition-preservation.aspx for the scenarios where the different drives will be destroyed.
Also take a look at http://blogs.msdn.com/b/kwill/archive/2012/09/19/role-instance-restarts-due-to-os-upgrades.aspx which points you to the RSS feed and MSDN article where you can see that a new OS is currently being deployed.
I tried to change the role size (upwards) in an Azure role and got the following error after uploading
"The role size specified for role 'Website' in the newly uploaded package differs from the role size for this role in the currently deployed service. Changing the size of the role will cause all local data on the role instance to be lost. Please use the Force flag if you want to allow the loss of local data."
which leads to the question - is there a force flag? Where is it? How do I set it?
Its just appeared!
An update to the Management Portal today (20 Oct 2011) has added an "Allow VM size or role count to be updated" tick box to the Upgrade Deployment dialog. So I guess that's the shiny new Force Flag!
The information I've heard on the Azure forums is that it is not possible to modify the role size without a full redeploy (and this has been my experience as well).
Over time Microsoft has allowed more things to be modified during an upgrade and Mark Russinovich may have suggested in a Mix or TechEd presentation that role size modification would be supported some point in the future.
The error message you are receiving may be an early artefact of the implementation of a size upgrade enhancement. The research I've done (and I have done a LOT of asking around) would indicate that the "Force Flag" mentioned is not actually implemented yet - although I'm more than happy to be proven wrong.
Right now, the vmsize is inside of the package (cspkg) which is sent to AppFabric. It's a decision which is part of the build/packaging. So, to change it, you need to change the VMSize, build the package and then [re]deploy.
If you have a running system, you can avoid down time by using VIP Swap. It will basically stand up another AT[s] in the staging slot with your new VM size and then swap over to it.
So, there's more moving parts involved but you can still change it with virtually no impact.