I am not able to upgrade my storage account (Gen1) to Gen2.
I got the error-json-file. The problem is with soft delete. I disabled the function but there are still some "soft deleted" blobs. How can I get rid of them?
Related
I am upgrading Azure blob storage to DataLake Gen 2. I have already created a few pipelines in ADF in which there was a delete activity with logging enabled, and I have given a path to a blob container for the logged files.
Now when I tried to upgrade to DataLake Gen 2, validation failed saying that "Incompatible feature soft delete". I disabled "Enable Logging" in the ADF pipeline, removed the path of the blob folder and deleted that folder from blob storage. However I am still getting this validation failed message.
I am not able to get what changes are still needed. see the error:
This question is answered in Microsoft Q&A platform. Thank you #NandanHegde-7720 for the solution, posting it as an answer here to help the other community members.
Disable soft delete for containers in your storage account.
• In the Azure portal, navigate to your storage account.
• Locate the Data protection settings under Data management.
• Uncheck soft delete for containers.
Any soft-deleted containers will be permanently deleted at the expiration of the retention period that was in effect at the time that the container was deleted.
Refer to this link for more information on how container soft delete works.
Alternatively, a quick workaround is to copy the files creating issues to the new storage account and then delete those files from the original storage account. Then try to upgrade data lake gen2.
As of now the blob feature 'soft delete' is not yet supported for ADLS Gen2 (hierarchical namespaces turned on). Soft delete is really good for accidental deletes either from human error or programmatic deletion. Considering soft delete is not yet supported for ADLS Gen2, is there any easy workaround for this? We want to really use ADLS Gen2 for its hierarchical namespace feature, but then don't want to lose our data if unintended deletes take place -- similar to soft delete we wan to retain data post its deletion for a few days (e.g. 15 days).
There is no a easy way to mimic the soft-delete feature in ADLS Gen2.
Here are some suggestions you can take a look.
1.Back up all the files to another ADLS Gen2 account. For example, you can create a blob trigger azure function, use blob storage output binding.
2.Use some tools like azcopy, periodically copy the files to local.
So when if it was deleted accidentally, just copy it back.
Soft Delete for ADLS is now available.
https://azure.microsoft.com/en-us/updates/soft-delete-for-blobs-capability-for-azure-data-lake-storage-is-now-generally-available/
I'm trying to locate Azure DataLake Store Gen2 using the Azure portal and for some reason cannot find it:
I've been searching the docs and the portal and cannot seem to find it, has anyone else run into this problem? It has been in global GA since Feb, so I don't think that's the issue. I've reviewed the docs for how to create a Storage Account, is that all that's needed to create the gen2 instance?
ADLS gen 2 is a feature of Azure Storage. When you are creating a Storage account, go to the Advanced tab:
Then enable Hierarchical namespace (this provides you ADLS Gen 2):
I'm fairly new to Azure, and just trying out Azure Data Lake Analytics.
I created a new Azure Data Lake Analytics account for testing purposes and would like to delete it now, however I used an existing Azure Data Lake Storage (ADLS) account as the default storage account during setup. I now know I probably should have added the existing ADLS as associated data store.
I assume I can safely delete the Azure Data Lake Analytics account now without affecting the underlying default storage account, but I want to check before I do this as it would be a massive problem if this the existing ADLS gets deleted.
Any pointers would be much appreciated. thanks
The two are separate. Deleting the Azure Data Lake Analytics service will not affect the Azure Data Lake Store.
As a disclaimer, test test test. Set up another instance of both in the same way and then confirm the delete behaviour, just to be 110% sure.
Azure Data Lake Team here. I can positively confirm that deleting the Azure Data Lake Analytics account will NOT delete the default or any linked Azure Data Lake Store account associated with it.
I have a VHD in a container which was mounted by an instance (a DotNetNuke accelerator). But now the instance is stopped so the VHD is not mounted anymore.
The problem is I can't rename this VHD with any UI client. "CloudBerry Explorer for Azure Blob Storage" shows an error when I try to rename it of
This operation is not permitted because the blob has snapshots.
I tried to list snapshots in VisualStudio but it only returns the blob's name...
I tried to mount the VHD in an instance and create a file and it works!?
I'm going crazy. :)
I've managed to remove blob which has snapshots by using Microsoft Azure Storage Explorer. Steps are:
You navigate to the blob in question
Right click -> Manage Snapshots
Delete all the snapshots and the blob will disappear also
The error is accurate. If you get that, it means the blob actually has snapshots. When you mount a blob you are most likely mounting the current version and not a past snapshot. You can enumerate the snapshots using the REST API (use the 'include=snapshots option) to prove this to yourself.
If you really want to delete the blob with minimal hassle, I would suggest using https://www.myazurestorage.com (safe site from Microsoft DPE). It will enumerate your snapshots automatically and allow you to delete them. Once they are all deleted, you can also delete the blob.