I have azure blob container contains a folder with name (data_source), now i'm trying to copy file with the same name to the same container, so i have both the file and the folder together with the same name.
I use this command:
az storage copy -s "$SRC" -d "$DEST" --recursive
Note that i use --recursive because i have other folders to copy as well.
The problem is, everything get copied successfully except the file with the same name as the folder in the container. I tried several ways even with the --recursive flag and didn't work.
May be the az storage copy is confusing files with directories, but i have no clue.
Note: I cannot change the file name, they must be the same.
I created two folders with same name and same blob file name with same contents. I changed one value inside one of my blob file keeping the blob file and folder name same as the other folder and blob file present in another storage account and performed copy operation like below:-
Storage account venkat123 with same folder and file, but file has one extra value test added, Refer below: -
Portal:
Storage Account venkat326 has the same folder data_source and file data_source.json but does not contain test value, refer below:-
Portal:
Now, I used the az storage copy command to copy the folders with same names and the content with value test was added from venkat123 storage account to venkat326 storage account like below:-
Command:-
az storage copy -s "https://venkat123.blob.core.windows.net/test/data_source/data_source.json<SAS-token>" -d "https://venkat326.blob.core.windows.net/demo/data_source/data_source.json<SAS-token>" --recursive
Output:-
Portal:
Related
I am trying to copy an existing directory from a blob storage to a directory already existing in an azure file share via the Azure CLI in the Azure portal
I get the following error
failed to perform copy command due to error: cannot transfer
individual files/folders to the root of a service. Add a container or
directory to the destination URL
What I have tried
azcopy copy ' https://myazurename.blob.core.windows.net/subdirectory' 'https://myazurename.file.core.windows.net/blob-mirror/subdirectory' --recursive
azcopy copy ' https://myazurename.blob.core.windows.net/subdirectory/*' 'https://myazurename.file.core.windows.net/blob-mirror/subdirectory' --recursive
azcopy copy ' https://myazurename.blob.core.windows.net/subdirectory/*' 'https://myazurename.file.core.windows.net/blob-mirror/subdirectory/*' --recursive
Yet everything gives the same error
I tried in my environment and got below results:
Initially I have tried the same commands and got same error:
command:
azcopy copy "https://<storage account name>.blob.core.windows.net/<c ontainer name>/directory1/?[SAS]" "https://<storage account name>.file.core.windows.net/fileshare1/directory1/" --recursive
Console:
Portal:
Blob container:
File share:
After I added SAS in both blob and file url copied files successfully from blobstorage to fileshare:
command:
azcopy copy "https://<storage account name>.blob.core.windows.net/<c ontainer name>/directory1/?[SAS]" "https://<storage account name>.file.core.windows.net/fileshare1/directory1/?[SAS]" --recursive=true
Console:
Portal:
Fileshare:
Update
You can get SAS for both blob and file
Home -> storageaccount -> shared access signature -> check the allowed resource group -> click the generate SAS and Connection string.
I have a local directory what needs to be synced to a specific folder to my Azure Blob Storage.
Unfortunately azcopy always creates a subfolder in the target directory.
local folder:
C:/local/vacation
Uploadfolder:
xxx.blob.core.windows.net/files/images/jpg
I want all JPGs that are within the local folder into the upload folder.
But azcopy always creates the folder "vacation" in my upload folder :(
This is the code:
$localFolder = "C:/local/vacation"
$targetFolder= "xxx.blob.core.windows.net/files/images/jpg" (# here is the actual SAS URI)
azcopy copy $localfolder $targetfolder --include-pattern "*.jpg" --recursive=true
You can use Sync operation to perform the copy without the directory being created.
azcopy sync "C:\local\vacation" "https://*****.blob.core.windows.net/files/images/jpg/?SAStoken" --include-pattern "*.jpg" --recursive=true
Example:
For the first command as you can see below I ran to sync the jpg files to the destination for the first time , then I added another jpg file in my local machine folder and ran the command for second time it synced the 1 new file to the destination.
The Above highlighted files are present on my local machine directory and when I performed sync they were only copied from local machine to azure and the directory was not created as subfolder.
Reference:
Synchronize with Azure Blob storage by using AzCopy v10
You should be able to use one of the following:
$localFolder = "C:/local/vacation/**"
$targetFolder= "xxx.blob.core.windows.net/files/images/jpg"
azcopy copy $localfolder $targetfolder --include-pattern "*.jpg" --recursive=true
or
$localFolder = "C:/local/vacation/**/*.jpg"
$targetFolder= "xxx.blob.core.windows.net/files/images/jpg"
azcopy copy $localfolder $targetfolder --recursive=true
azcopy copy './*' 'https://xxx.blob.core.windows.net/path/to/dir' --recursive
After trying plenty of stuff. It seems that within the azcopy sync it is not possible.
Only with azcopy copy this works. What a pitty :/
Uploaded a .sh file into storage account and trying to get that into rhel vm. I have azcopy version 10.6.1. Command i am using azcopy copy ”https://.blob.core.windows.net/container/test.sh>[SAS token]" . When i run this, it is moving to next line and giving this ">" output. Tried to do in many ways but no luck.
How can we troubleshoot this issue?
The syntax to use "azcopy copy" command is
azcopy copy [source] [destination] [flags]
To download a single file by using a SAS token:
azcopy cp "https://[account].blob.core.windows.net/[container]/[path/to/blob]?[SAS]" "/path/to/file.txt"
To download a single file by using a SAS token and then piping the output to a file (block blobs only):
azcopy cp "https://[account].blob.core.windows.net/[container]/[path/to/blob]?[SAS]" > "/path/to/file.txt"
This is the source of above information.
Related references:
Download blobs from Azure Blob storage by using AzCopy v10
Troubleshoot Azcopy related issues
I'm trying to copy files to and from an Azure Fileshare using AZCopy v10. I have had this successfully working using v8.1 but I keep getting errors using v10.
From the command line I'm using this to copy a file from the local drive to the fileshare;
c:\Temp\azcopy.exe copy "c:\temp\sample.txt" "https://myfiles.file.core.windows.net/dbfiles/sample.txt?SASKeyText"
This generates the error message;
failed to perform copy command due to error: cannot transfer individual files/folders to the root of a service. Add a container or directory to the destination URL
I have tried adding a directory to the fileshare and adding that to the command string but I get the same error.
If I reverse the copy from the fileshare to the local drive I get the error;
failed to perform copy command due to error: account copies are an inherently recursive operation, and thus --recursive is required
I have followed the guide at https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-files but haven't been able to see what's wrong.
Thanks in advance for any help.
The error here was with the SAS token and not the form of the command.
I suppose this should be marked up amongst examples of unhelpful error messages.
Thanks to everyone who took the time to have a look.
I had this same issue when trying to do a copy from my local machine to Azure Blob storage.
This was the command I was running:
azcopy copy --from-to=LocalBlob "C:\AzureStorageTest\my-app\*" "https://myapptest.blob.core.windows.net/%24web" --recursive
But I got the error below:
failed to perform copy command due to error: cannot transfer individual files/folders to the root of a service. Add a container or directory to the destination URL
Here's how I solved it:
I was missing the ?[SAS] argument at the end of the Blob storage location. So instead of this:
azcopy copy --from-to=LocalBlob "C:\AzureStorageTest\my-app\*" "https://myapptest.blob.core.windows.net/%24web" --recursive
I had this:
azcopy copy --from-to=LocalBlob "C:\AzureStorageTest\my-app\*" "https://myapptest.blob.core.windows.net/%24web?[SAS]" --recursive
Note:
The format is azcopy copy "/path/to/dir" "https://[account].blob.core.windows.net/[container]/[path/to/directory]?[SAS]" --recursive. You only need to modify the "/path/to/dir", [account] and [container]/[path/to/directory]. Every other thing remains the way they are.
Specify the source-destination routing using the --from-to=LocalBlob (if you're copying from local to blob storage) argument to be explicit about the copy operation.
My actual Blob storage location is https://myapptest.blob.core.windows.net/$24web but I used https://myapptest.blob.core.windows.net/%24web, since $ will throw some error when used, so %24 was used.
That's all.
I hope this helps
Here is a sample azcopy script which worked for me
az storage azcopy blob upload
-c'https://$AZURE_STORAGE_ACCOUNT_NAME.blob.core.windows.net/\\\$web' \
--account-name $AZURE_STORAGE_ACCOUNT_NAME \
-s "build/*" \
--account-key $AZURE_STORAGE_ACCOUNT_ACCESS_KEY \
--recursive
If you get this error while trying to copy to $web container:
"failed to perform copy command due to error: cannot transfer individual files/folders to the root of a service. Add a container or directory to the destination URL"
Per a solution listed here, we need to add an escape character (\) before $web. Following command (to copy all files and subfolders to web container) worked for me:
azcopy copy "<local_folder>/*" "https://******.blob.core.windows.net/\$web/?<SAS token>" --recursive
Without the escape character, the following command fails with the above error.
azcopy copy "<local_folder>/*" "https://******.blob.core.windows.net/$web/?<SAS token>" --recursive
Can I copy a directory from a share with subdirectories so that blob name would have the path name?
\\testshare\July\File1.txt
\\testshare\May\File1.txt
\\testshare\June\File3.txt
\\testshare\June\Week3\File3.txt
In the blob container, I want to have
July/File1.text
May/File1.text
June/File3.txt
June/Week3/File3.txt
Never mind, AZCopy does that by default.
AzCopy /Dest:https://myblob.blob.core.windows.net/files /Source:C:\myfolder /DestKey:xzxzxzxzzxzxzxzxzxzxzxzxzxzxzxzxzxzxzxzxz== /S