Can't Find Data Lake Store Gen2 - azure

I'm trying to locate Azure DataLake Store Gen2 using the Azure portal and for some reason cannot find it:
I've been searching the docs and the portal and cannot seem to find it, has anyone else run into this problem? It has been in global GA since Feb, so I don't think that's the issue. I've reviewed the docs for how to create a Storage Account, is that all that's needed to create the gen2 instance?


ADLS gen 2 is a feature of Azure Storage. When you are creating a Storage account, go to the Advanced tab:
Then enable Hierarchical namespace (this provides you ADLS Gen 2):

Related

Accessing ADLS Gen2 using Azure Databricks

I am new to Cloud(Azure), trying to connect to ADLS Gen2 using Azure Databricks but not able to access it.
Step tried -
got the access key from Azure portal, which was for specific Storage account
Tried to create Score as well, which was looking for Key Vault.
Is there any other way by which we can directly read the data from ADLS using python or any other python module which can help us to achieve this.

Are you able to apply Azure Information Protection on Azure Data Lake or Blob Storage?

I'm wanting to know if you can apply Azure Information Protection on files that are in Azure Data Lake or Blob storage.
I cant seem to find any documentation that confirms this.
Cheers
Tim

How to check whether the storage account V2 created is having data lake gen2 property or not in Azure?

I'm very new to Azure and would like to know how can i check an existing Storage account V2 available in resource group is having type Data lake Gen2 or not.
I know the process to create data lake gen 2 by using the option Hierarchical namespace enabled == Data Lake Gen2 while creation.
But how can i check after creation:
Any where in portal.
Azure CLI - any CLI commands to check
Thanks in advance.

On portal, select the storage account and click on Configuration. You should be able to see if hierarchical namespace has been enabled on the right hand side as shown in the picture below.

Azure Data Lake Storage Gen2 label appearing as Containers and NOT File System

I just started with Data Lakes in Azure and countered an issue with the ADLS Gen2 screens in the Azure Portal.
Using the Azure Portal, I created a new Storage account to setup a new Azure Data Lake Gen2 storage by following the online instructions. At the time of creating the storage account, the option Hierarchical namespace was enabled and the storage set to StorageV2(general purpose). This created the Data Lake. However the name of the container still appears as Container. In the videos i've seen the Services panel display the Container with the label 'Data Lake Gen2 File System'. However in the one i created it still shows the label as Container. Furthermore the collapsible panel to the left also shows Container as against File System. Please refer to the screen grab below.
Can anyone tell me on whether i am missing anything or is it just that Azure had changed the names in the immediate past that i am not aware of?

You're not missing anything. Some time ago, we also noticed that the portal UI of ADLS GEN2 made the change: change the label Data Lake Gen2 File System to Containers.
It does not break any feature, just a simple UI change.

Read from ADLS gen 2 with SSIS

Does anyone know which connection and Data Flow Component to use for ADLS (Azure Data Lake Store) gen2?
I've managed to use the blob connector in the connection manager and successfully connect to ADLS Gen2, but when I try to use the blob source component I get a 400 bad request. Works fine if it's just a blob storage without HNS.
The ADLS components states it's just for ADLS gen 1.
So how to read and write to/from ADLS Gen 2?

A current version of SSIS Azure Feature Pack supports ADLS Gen2. It can be used as a data source or destination in dataflow:
The screenshot is to show it as a destination, but the ADLSgen2 works well also as a source via corresponding "Flexible File Destination" and "Flexible File Source"

First of all, based on the great link provided by #rickvdbosch it looks like that there are many temporary limitations with Azure Data Lake Storage Gen2 concerning the BLOB Storage API. Which means that it is not a component limitation and maybe you should wait until it will be integrated with SSIS.
Microsoft SQL SERVER Feature pack for Azure
If you meant these components when you mentioned that:
The ADLS components states it's just for ADLS gen 1.
Then ignore this part.
I am not pretty sure if it supports Gen2, but I think you can use the Azure Data Lake Store components which are a part of the Microsoft SQL SERVER feature pack for Azure. For more information you can refer to:
Azure Data Lake Store in SSIS
Azure Data Lake Store Source
Azure Data Lake Store Destination
Download Link
Azure Feature Pack for Integration Services (SSIS)
Other methods
If the suggestion above didn't work then you should use Azure Data Factory or a command line by Installing AWS CLI and using AzCopy v10

I got the following info:
"At the moment Gen 2 don’t support BLOB API (but it will in a short time) and hence, SSIS is not able to connect."
So for SSIS it's currently either ADLS Gen 1, or blob store

I used the Script Task to write files or System.Objects (converted to csv in Memory) to Azure Storage Gen 2 (Hierarchical Namespace Enabled) using the Rest API. I did this as a demo until the SSIS components are released.

You can't write to ADLS Gen2 using the old components from the Azure Feature Pack, but you can connect to the blob Gen2 (non-hierarchical) using the Azure Blob Destination Component.

Resources