Is there a way of downloading a container and all of its content from Azure Blob Storage?
I use CloudBerry Explorer for Azure Blob Storage to manage my containers, files and folders.
I have a container with over 100GB of data which I would like to download, but cannot find a way of downloading the container, only individualy files.
If you want, you can use the AzCopy tool to download an entire blob container. Assuming you have the latest version of the Azure SDK installed, you can find this tool in the C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy folder.
You can try the following command:
AzCopy /Source:"https://[accountname].blob.core.windows.net/[containername]/" /Dest:"[folder path e.g. D:\temp\" /SourceKey:"[account key]" /S
Replace [accountname], [containername], [folder path], [account key] with the appropriate values.
To download the Container (All Files) from Azure Blob Storage
az login
az account set --subscription <Sub ID>
az storage blob download-batch --account-name <storageaccountname> --source <containername> --destination <C:\Users\Downloads\***>
To Delete All files at a time
az storage blob delete-batch --account-name <storageaccountname> --source <containername>
az storage blob download-batch --account-name --source --destination <C:\Users\Downloads***>
Related
Currently, I'm using az storage blob directory download but every time I run it, I get: This command is implicitly deprecated because command group 'storage blob directory' is deprecated and will be removed in a future release. Use 'az storage fs directory' instead.
I checked the docs and I can't seem to find what the values for --file-system should be. Can someone share an example of downloading the content of a directory (a folder) inside a container inside a blob from Azure Storage to a Windows machine?
--file-system is your container name
az storage fs directory download -f myfilesystem --account-name mystorageaccount -s SourceDirectoryPath -d "<local-path>" --recursive
Above will download the entire directory
Follow this command:
az storage fs directory download -f <container name> --account-name <storage account name> -s <the source dir you want to download> -d "./" --recursive --account-key <The access key of the storage account>
Before run the above command, please make sure your storage account has already enable hierarchical namespace, otherwise the file level of storage account will be flat in your side.
It works on my side:
This is the structure on my side:
I can download the specific directory to current directory:
Your can refer to this official document:
Examples of az storage fs directory
In azure cli the command
az storage blob upload-batch --source <source folder> --destination-path <destination path> --destination <destination>
is it available the same api in the javascript sdk #azure/storage-blob or in another package?
Thank you!
AFAIK, this feature is not available in any SDK.
If you are using #azure/storage-blob, you would need to list files in a folder yourself (using fs module) and then upload each file using uploadFile(string, BlockBlobParallelUploadOptions) method which is what az storage blob upload-batch is doing.
I have a custom vhd which I have in my Azure VM. It's a linux machine. I need to use azure cli to create an Image from this vhd file. This image will then be used to create VMs which have a username and Password. I have successfully used the vhd to create an Image using the Portal and created a VM out of it. However, I am unable to create an image using cli commands. This is what I get when I run the command -
jenkins#Jenkins-vm:~/testFolder$ az image create -g myRG -n myImage --os-type Linux --source ./myCustom.vhd
usage: az image create [-h] [--verbose] [--debug] [--only-show-errors]
[--output {json,jsonc,yaml,yamlc,table,tsv,none}]
[--query JMESPATH] [--subscription _SUBSCRIPTION]
--resource-group RESOURCE_GROUP_NAME --name NAME
--source SOURCE [--os-type {Windows,Linux}]
[--data-disk-sources DATA_DISK_SOURCES [DATA_DISK_SOURCES ...]]
[--location LOCATION]
[--storage-sku {Standard_LRS,Premium_LRS,StandardSSD_LRS,UltraSSD_LRS}]
[--hyper-v-generation {V1,V2}]
[--os-disk-caching {None,ReadOnly,ReadWrite}]
[--data-disk-caching {None,ReadOnly,ReadWrite}]
[--tags [TAGS [TAGS ...]]]
[--zone-resilient [{true,false}]]
az image create: error: 'NoneType' object has no attribute 'os_disk'
I don't understand the meaning of the error displayed.
I'm very new to Azure so feeling a little lost.
You can see the description of the CLI command az image create:
Create a custom Virtual Machine Image from managed disks or snapshots.
And the description of the parameter --source:
OS disk source from the same region, including a virtual machine ID or
name, OS disk blob URI, managed OS disk ID or name, or OS snapshot ID
or name.
It means the CLI command only can create VM images from Azure, not matter the VM blob URI, managed disk, or snapshot, all of them should exist in Azure. So you need to upload the VHD file to Azure Storage Blob, and then use the VHD URI to create the VM image via the CLI command. And one thing you need to watch out is that the VHD file should be the OS disk.
I am trying to practice on the following tasks:
Create Storage Account:
az storage account create --name heyatafroz25 --resource-group user-fottsascvuzj
Get Storage Account Key:
az storage account keys list -g user-fottsascvuzj -n heyatafroz25
Create Share Account:
az storage share create --account-name heyatafroz25 --name key1
az storage share create --account-name heyatafroz25 --name key2
Create Storage Directory:
az storage directory create --account-name heyatafroz25 --name heyatdir1 --share-name key1
az storage directory create --account-name heyatafroz25 --name heyatdir2 --share-name key2
Uploading the File
I was asked to create a index.php file which i created using the touch command.
Post that I am not sure what details to be considered for path and source.
For path i took the present working directory
az storage file upload --account-name heyatafroz25 --account-key N+PKe3ihto+G0h9CvVRV/bJ5KeEFF6RFB0aKf2qcfcyJA1uOyCBUO06Tlh9KHUzhA+PyugmDLwlrceXW5V31Xw== --path /home/scrapbook/tutorial/index.php --share-name key1 --source /home/scrapbook/tutorial/index.php
Please suggest on the corrections in the 5th command.
Thanks in advance.
Looking at the documentation for az storage file upload:
--source: Path of the local file to upload as the file content.
Essentially this is the path of the local file you want to upload.
--path: The path to the file within the file share. If the file name is omitted, the source file name will be used.
So assuming if you're uploading a file, it would be the name of the file you want the local file to be saved in the storage.
To elaborate further, let's say you have a local file called "image.png" and you want to save it as "logo.png", you will use the following command:
az storage file upload --account-name <account-name> --account-key <account-key> --share-name <share-name> --path logo.png --source image.png
Try running the following command:
az storage file upload --account-name mystorageaccount --account-key NT9ewNtqU1CB+Z7Lzm5f3UOvWbywC8b0Bk8TWnp06zwzDCoe3vGV2u/wQmupT04//pqpIyOwsn/Q9rtSDBdVdg== --share-name myfileshare --path "myDirectory/index.php" --source "/home/scrapbook/tutorial/php-docs-hello-world/index.php"
I'm trying to copy from azure blob with this command:
az storage blob download-batch --destination / --source my-continer --pattern "my/pattern/here/*"
to the / path, yet it copies the entire blob path.
So in / I see my/pattern/here structure instead of just the content of /here folder
Is there a way to override this behavior and make it copy just the path that matches the pattern that I entered?
It seems it's impossible that just download blobs without the path my/pattern/here through the CLI command az storage blob download-batch. See screenshot of the blob:
Its name shows with the path, it means you download the blob with name my/pattern/name to the current directory locally. Then in Linux, the name means that it will create the directories until there is no /.
So I suggest you can use the CLI command az storage blob download to download the files with a loop.