How do I specify an additional storage account with SAS keyfrom Azure portal while creating HDInsgith cluster? It's expecting actual storage key , not SAS key. Ideally I want to do that and export a template out of it. My goal is to get ARM template example for attaching storage with SAS key to HDInsight cluster. But I am not able to find this template anywhere. I just need an example that I can use.
Unfortunately, you don't have option to attach additional storage accounts with SAS key while creating HDInsight cluster from the Azure portal.
I would request you to provide the feedback here:
https://feedback.azure.com/forums/217335-hdinsight
All of the feedback you share in these forums will be monitored and reviewed by the Microsoft engineering teams responsible for building Azure.
Related
The synapse link for Dataverse is running fine when the storage account access key is disabled. We can able to create new records, there is no problem here.
But it fails to set up a new synapse link for Dataverse when the storage account key is disabled. Has anyone seen this issue before?
expecting synapse link to work when the storage account access key is disabled
As per my analysis looks like the Storage account key access should be enabled at the time of Synapse link creation and once it is created successfully, then you can disable the storage access key and the behavior should be similar to that your existing Synapse link service.
Is it possible to setup auto process of continuous export of data in Azure Data Explorer to Azure Data Lake? based on official docs it is required every time use of AAD token, so I could not configure once this export.
https://learn.microsoft.com/en-us/azure/data-explorer/kusto/api/connection-strings/storage
You can set it up using the Continuous data export process. For the storage provide the storage account key to avoid needing to renew it.
I have created ticket with MS support and having answer:
I have reached our engineering team and you are correct - this scenario is currently not yet supported. Very few customers use ADLS gen1, so there has been very little demand for feature this so far. ADX does not support refresh tokens, so we cannot renew the token OBO of the service principal. I have sent your feedback on the need of this feature to the engineering team – and for long term (unfortunately no ETA right now), we plan to support using managed identities in continuous export, and then this will be supported.
In the meanwhile, our suggestion is to move to ADLS gen2, where you can configure the export with an account key (or a very long living SAS key).
Our software uses Azure blob & Azure table storage.
I would like developers to be able to look through our production data with the Microsoft Azure Storage Explorer, but not be allowed to accidentaly edit it's data.
I don't want to allow anonymous access to the data (read only) as suggested here.
What would be a good way to achieve this?
Make use of Shared Access Signature option to connect to Azure Blob Storage from the Storage Explorer.
Find more details about SAS here.
Find more details about SAS in Storage Explorer here.
As a newbie of Azure, I plan to build a cloud computing service with a free trial account.
I first created a Storage account. The Deployment model is Resource Manager as recommended so that I chose Blob storage as the Account kind.
Then I created an HDInsight cluster. But in the Data source configuration, the aforementioned Blob storage account can not be selected but with a warning - Could not reach the storage!. However, If I have created the Storage account with Classic as the Deployment model, the created Storage account can be selected as the Data source.
Anyone have any idea about why is it so?
Thanks in advance! I got stuck up here for long time
If you have selected 'Resource Manger' as the Deployment model, then the storage account should be of type 'general purpose azure blob storage account', you might have created azure blob only storage type account.
For an HDInsight cluster there has to be at least one azure storage account which is its default storage account -- it is required so that it is treated as its fs (filesystem). This I get. But what about optional linked azure storage accounts? From ADF (Azure Data Factory) perspective at least, do we need to have a storage account added as linked storage account to an HDInsight cluster? Anyway the Azure storage account is accessible purely by providing just two pieces of information --- the account name and the key. Both these things are specified in Linked Servers in ADF. This guarantees the access of the storage account. What is the real benefit of having some account added as linked storage account, from ADF point of view or otherwise? Basically, what I am asking is -- is there anything that we can't do purely using account name and key without adding the account as linked storage for the given HDInsight cluster?
The main reason to have additional accounts is because they have limits. A storage account can have 500 TB of data in it and 20000 request per second. Depending on the size of your cluster and work load you might hit the request limit. If you are worried about those limits and you don't want to manage alot of storage accounts you should look into Azure Data Lake.
I think I sort of figured out the answer. With linked storage accounts the cluster, when used as a compute, can directly access BLOBS on those storage accounts without requiring us to separately specify the storage keys in queries. That's the use case for which linked storage is a must have.