I have a container called 'services'.Inside the Container , i have a directory called 'Test'.how we can get the list of blobs inside the directory test.I have few kept few CSVs inside the test. Need the rest api to get the list of the files.
However i am able to get the list of items inside the container easily using below Rest API
https://myaccount.blob.core.windows.net/services?restype=container&comp=list
i tried
https://myaccount.blob.core.windows.net/services/Test?restype=directory&comp=list
but it is not working.
Please help to get the coorect param value or rest API to find the list of items inside the directory
https://myaccount.blob.core.windows.net/services/Test?restype=directory&comp=list&prefix=Test/
The doc:
https://learn.microsoft.com/en-us/rest/api/storageservices/list-blobs#uri-parameters
Just add a param prefix is ok.
Related
Hi I am working in Azure Logic app. I am trying to get multiple files from azure data Lake gen v2 and attach these multiple files in an email. As a first step I have added http request and I am giving required information along with file path. It works fine for one file. but I am trying to input folder path and inside that folder, all the files I want to get and attach in email.
Logic app Flow Diagram
Added sample screenshot for attachment
tried to add attchment
In the above diagram, Get blob content step which works fine for one file but I am finding difficult to attach multiple files in email. Can some one help me to figure out the solution. Any help would be appreciated. Thank you
You can use List blobs action to list all blobs in the folder you want:
Then you can define a variable to append the attachments array.
Use For Each to loop the blobs from List Blobs action. Within For Each you can use Get blob content to get blob content, and then use Append to array variable to append attachments.
The expressions of Path, DisplayName and File Content are as follows:
Path : items('For_each')?['Path']
DisplayName : items('For_each')?['DisplayName']
File Content : body('Get_blob_content')
Finally, please fill in the attachment in the email:
==========================update===================
If you send an email with 400 response, please use expression in Append to array variable as below:
base64(body('Get_blob_content'))
Im learning logic apps and im trying to create a simple flow from azure blob storage, perform a liguid parsing and then save parsed file to another blob container.
How should it work:
1. Whenever new file is added to blob container ("from") [containing xml files]
2.Liquid action takes place (XML -> JSON)
3.New file .json is saved to blob container ("too") :)
What i have learned:
1. I manage to write a liguid template for xml files - tested - working
2. I know how to copy file between blob containers - tested - working
For each loop:
https://i.imgur.com/ImaT3tf.jpg "FE loop"
Completed:
https://i.imgur.com/g6M9eLJ.jpg "Completed..."
Current LA:
https://i.imgur.com/ImaT3tf.jpg "Current"
What I dont know how to do:
1. How to "insert" current file content in for each into liquid action? It looks like logic apps is skipping that step?
The main problem is you could not use Current item as the xml content, you need to get the content with Get blob content action in For_each, then parse xml to json. After this create the blob in another container with json value.
You could refer to my workflow.
I'm unable to restrict the sharepoint search to only retrieve folders.
I'm running rest requests to the search api
https://(sharepoint url)/ian/_api/search/query?querytext='metatest3'
This retrieves only the folder that is on this collection but when I use
https://(sharepoint url)/ian/_api/search/query?querytext='metatest3 and IsContainer:1'
This doesn't retrieve the folder and no results are found.
I'm perplexed because when the first API call is returned I have
<d:element m:type="SP.KeyValue">
<d:Key>IsContainer</d:Key>
<d:Value>true</d:Value>
<d:ValueType>Edm.Boolean</d:ValueType>
</d:element>
We can use the REST API below to search only folders.
https://(sharepoint url)/ian/_api/search/query?querytext='metatest3 contenttypeid:0x0120*'
Or
https://(sharepoint url)/ian/_api/search/query?querytext='metatest3 IsContainer:true'
I have multiple JSON format files which is being pushed to the Azure storage account under a specific container. There are n number of files in the container.
And 4 to 8 nodes which will be accessing the Azure storage container to downloaded the files locally, the download code is written in java.
Since there are n number of files and multiple file accessing the container at the same time, how to avoid the situation that the same file is downloaded by another server?
Example:
Azure container has 1.json, 2.json, 3.json, etc which are > 35 MB size.
batch-process-node1 -> starts downloading 1.json
batch-process-node2 -> starts downloading 2.json
batch-process-node3 -> should not start downloading the 1.json
Is there any logic to be built for each node which has the java process to download the file uniquely?
Is there any setting that can be set in the Azure storage container?
--
Trying to use the Camel Azure-bolb component, using the block blob (blobType).
New to Azure storage blob, any help is appreciated.
Since we are already using Apache camel in the code, we tried to use camel azure-blob component to address the issue. Below is the approach we used, still the race condition is acceptable for our scenario.
Camel route started with timer consumer, and producer to get the list of blob from container using below endpoint,
azure-blob://<account>/<container>?credentials=#storagecredentials&blobType=blockBlob&operation=listBlobs
Note: storagecredential is a bean of type StorageCredentialsAccountAndKey class.
Created a java class implementing the Processor of camel, and in process() method, using the exchange.getIn().getBody() => which provides an iterable object with has ListBlobItem.
first i set the meta data of the blob using below endpoint
azure-blob://<account>/<container>/*<blobName>*?credentials=#storagecredentials&blobType=blockBlob&operation=updateBlockBlob&blobMetadata=#blobMetaData1
Note: blobMetaData1 is bean created in the context file.
<util:map id="blobMetaData1" map-class="java.util.HashMap">
<entry key="someKey" value="someValue"/>
</util:map>
Key thing: In this class process method
validate the metadata is being set or not, if set then the
process is already picked the blob. so it won't be picked again
assuming if the process executed in different server.
got the blob name from the ListBlobItem individual blob item.
using getURI() and forming the endpoint within this processor class.
in order to invoke the custom endpoint, used to set it an customer
header value of In message.
using the recipientList camel option which invokes the metadata endpoint to update the specific blob.
Then used another processor to form the download blob endpoint
azure-blob://<account>/<container>/*<blobName>*?credentials=#storagecredentials&blobType=blockBlob&operation=getBlob
and using the recipientList to get the processor endpoint from message header.
finally forming another delete endpoint which will delete once its downloaded.
steps i already did using SFTP connector (how can i access files while looping through list files in folder in azure logic app):
I added foreach loop
I added list files in folder
I passed Body as parameter in foreach loop
then i added action to create new file with new name for all files.
but i am not able to get file name and content while iterating sftp folder using foreach loop?
Please see example image, showing the logic app design. I am iterating sftp folder posting file content to http end point: