I can not find the API for Node.js to copy everything in one fileShare to another fileShare in different storage account. Does anyone run into this problem before?
All I can find is the azCopy solution.
Thank you very much
I can not find the API for Node.js to copy everything in one fileShare
to another fileShare in different storage account.
This is because no such API exists. Node SDK for Azure Storage is a wrapper over Azure Storage REST API and the REST API only supports copying files (neither directories nor shares).
What you would need to do is list all files from a share and then copy these files individually which is what azcopy does.
Related
We have 5 vendors that are SFTPing files to Blob Storage. When the files come in, I need to copy them to another container and create a folder in that container named with the date to put the files in. From the second container, I need to copy the files to a file share on an Azure server. What is the best way to go about this?
I'm very new to Azure and unsure what the best way is to accomplish what I am being asked to do. Any help would be greatly appreciated.
I'd recommend using Azure Synapse for this task. It will let you move data to and from different storage securely and with little-to-no code.
Specifically, I'd put a blob storage trigger on the SFTP blob container so that the Synapse Pipeline to move data automatically runs when your vendors drop their files.
Note that when you look for documentation on how to do things in Synapse, most of the time the Azure Data Factory documentation will also be applicable, since most of Data Factory's functionality is now in Synapse.
The ADF and Synapse YouTube channels are excellent resources, as well as the Microsoft Learn courses on Data Engineering.
I need to copy them to another container and create a folder in that container named with the date to put the files in.
You can use Azcopy to copy a files to another container by using SAS token.
command:
azcopy copy 'https://<storage account>.blob.core.windows.net/test/files?SAS' 'https://<storage account >.blob.core.windows.net/mycontainer/12-01-2023?SAS' --recursive
Console:
Portal:
I need to copy the files to a file share on an Azure server
You can also copy the files from container to file share by using Azcopy.
Command:
azcopy copy 'https://<storage account>.blob.core.windows.net/test?SAS' 'https://<storage account >.file.core.windows.net/fileshare/12-01-2023?SAS' --recursive
Console:
Portal:
You can get the SAS token through portal:
Go to portal -> your storage account -> shared access signature -> check the resource types -> click generate SAS and Connection-string.
Portal:
Probably azcopy is a good way to move all or part of the blobs from one container to another one. But I would suggest to automate it with Azure Functions. I think it can be atomated triggering an Azure Function every time a blob or set of blobs (Azure could process a batch of blobs) are updoladed to the source container.
Note on Azure Functions, depends on the quantity of blobs to be moved and the time that it could take, durable functions should be better solution to skip timeout exception. Durable function returns inmediate response but are running in "background".
Consider this article to have a better approach to this solution:
https://build5nines.com/azure-functions-copy-blob-between-azure-storage-accounts-in-c/
I am looking for a way to upload a flat file to file.core.windows.net in Azure Storage. Uploading to a Blob storage is easy enough and straight forward using SSIS's Azure Blob Upload task, however I need the file to go to the File shares not the blob containers. I have tried to use AZCopy however I cannot seem to get it to work, so I was curious if anyone knew of an easier way to upload to the File Shares? I have access to the account name and Storage key for the upload.
I figured it out using AZCopy, I realized the instructions i received was from an older version of AZCopy with different keywords (Version 8 instead of the newest V10)
I'm facing weird issue. I have one Azure Blob storage container which consists of 2 folders, let's say folder A and folder B. If I copy the files using WinSCP only to folder A, the file is copied in folder A as well as folder B and vice versa.
I see this issue only using WinSCP with Blob storage and not with Azcopy or Azure storage explorer. Also i don't see this issue with Azure file storage using WinSCP.
Any help appreciated.
I was looking through the azure file service apis for python and for rest but I didn't see a method to just move files. I would like to move files on azure file storage without mounting a network drive is there a way to do this?
As I know, your requirement couldn't be implemented directly. However you could use the combination of copy and delete API.
These are copy and delete API of REST.And this part is about copy and delete API of python.And an Azure Storage File is an SMB-compatible share, so you should be able to make files copies with normal file I/O operations.You could refer to this answer.
If this couldn't meet your requirement or still have questions, please let me know.
I have a Azure VM (Win 2016) where I have a folder where we have file coming every 5 minute.
Now I want to create and Window Service which will run on Azure VM and if any file exist, it will move to Azure File storage.
Could someone guide whats need to do or any other approach?
As I see it, you have 2 options:
Mount File Storage Share as a network drive. Once you mount the share as a network drive (and get a drive letter) you can simply use System.IO namespace to perform IO operations. Please see this link for more details: https://learn.microsoft.com/en-us/azure/storage/files/storage-how-to-use-files-windows.
Use Microsoft's Azure Storage SDK which is a wrapper over Azure Storage REST API and upload files from the local folder to the share in Azure File Storage. Please note that once the file is uploaded in Azure File Storage, you would need to manually delete the file from your server to achieve move operation.