I am under the impression that the Azure Data Lake Store does not currently offer any encryption at rest (the way Azure Blob Storage does). I managed to found some vague mention of this on the official website, suggesting this is coming soon.
Is this your understanding as well? Does this cover the databases stored under the Azure Data Lake Analytics as well?
Actually encryption is available in preview on ADL Storage right now. If you contact us we can give you access to the preview.
Azure Data Lake Store Encryption is currently Generally Available, not in preview any more. You can specify encryption at rest when creating ADLS in portal. Options are these:
Related
we are looking to migrate our delta lake from Azure to GCP.
So far we are thinking about moving delta files from one ADLS bucket to GCS bucket. We believe there might be more to that.
We are looking for a methodology, best practices and hints on to do that migrationo. Can anybody help on that please.
You might like to check sources and sinks of the Cloud Storage Transfer Service. One of the sources - is an Azure Blob Storage, including Azure Data Lake Storage Gen2 - I don't know if that can help in your case. And some documentation about access configuration.
All other details - depends on you case, and it is very difficult to provide a general answer.
I'm wanting to know if you can apply Azure Information Protection on files that are in Azure Data Lake or Blob storage.
I cant seem to find any documentation that confirms this.
Cheers
Tim
When creating a HDInsights Hadoop cluster in Azure there are two storage options. Either Azure Data Lake Store (ADLS) or Azure Blob Storage.
What are the real differences between these two options and how do they affect the performance?
I found this page https://learn.microsoft.com/en-us/azure/data-lake-store/data-lake-store-comparison-with-blob-storage
But it is not very specific, only uses very general terms like "ADLS is optimized for analytics".
Does it mean that its better for storing the HDInsights file system? And if ADLS is indeed faster then why not use it for non-analytics data as well?
As per this document, an Azure Storage account can hold up to 4.75 TB, though individual blobs (or files from an HDInsight perspective) can only go up to 195 GB. Azure Data Lake Store can grow dynamically to hold trillions of files, with individual files greater than a petabyte. For more information, see Understanding blobs and Data Lake Store.
Also, check Benefits of Azure Storage and Use Data Lake Store for more details and comparisons.
Hope this helps.
In addition to Ashok's answer: ADLS is currently only available in a few regions, compared to Azure Storage. So if you need your HDInsight account in a specific region, you should make sure your storage is in the same region.
Another benefit of ADLS over Azure Storage is its POSIX-based security model at the file/folder level that uses AAD security principals instead of Shared Access Keys.
The reason why you may not want to use ADLS for non-analytics data is primarily cost. Because of some of the additional capabilities, it is currently a bit more expensive.
In addition to the other answers its not possible to use the Spark Data Factory activity on HDInsights clusters that use Data Lake as the primary storage. This limitation applies to both ADFv1 and v2 as seen here: https://learn.microsoft.com/en-us/azure/data-factory/v1/data-factory-spark and https://learn.microsoft.com/en-us/azure/data-factory/transform-data-using-spark
I want to create blob from Azure Table. AzCopy supports this functionality but I couldn't find any document stating that Data movement API also supports it.
IS this option available ?
https://azure.microsoft.com/en-us/blog/introducing-azure-storage-data-movement-library-preview-2/
Currently, Azure Storage Data Movement Library doesn't support Azure Storage Table.
my question is about Microsoft Azure blob storage.
Can I manage blob snapshots from user console (azure portal)? Take snapshot,delete,recover etc.
Thanks
No, currently the portal does not offer this functionality. You may want to check out Azure Explorers available in the market. Most of them have support for managing blob snapshots.