Im learning logic apps and im trying to create a simple flow from azure blob storage, perform a liguid parsing and then save parsed file to another blob container.
How should it work:
1. Whenever new file is added to blob container ("from") [containing xml files]
2.Liquid action takes place (XML -> JSON)
3.New file .json is saved to blob container ("too") :)
What i have learned:
1. I manage to write a liguid template for xml files - tested - working
2. I know how to copy file between blob containers - tested - working
For each loop:
https://i.imgur.com/ImaT3tf.jpg "FE loop"
Completed:
https://i.imgur.com/g6M9eLJ.jpg "Completed..."
Current LA:
https://i.imgur.com/ImaT3tf.jpg "Current"
What I dont know how to do:
1. How to "insert" current file content in for each into liquid action? It looks like logic apps is skipping that step?
The main problem is you could not use Current item as the xml content, you need to get the content with Get blob content action in For_each, then parse xml to json. After this create the blob in another container with json value.
You could refer to my workflow.
Scenario:
I copy .css file from one subdirectory to another in Azure Storage Container. It is done from C# code level in my application. This is css style file for my website. Unfortunately I received error in my browser console during loading page:
Error
Resource interpreted as Stylesheet but transferred with MIME type application/octet-stream:
"SOME_PATH/template/css/styles.css?d=00000000-0000-0000-0000-000000000000".
Knowledge:
I know that it is why my file is sended as octet-stream instead of text/css. What can I do to say Azure to treat this file as text/css?
Edit: My code
string newFileName = fileToCopy.Name;
StorageFile newFile = cmsDirectory.GetStorageFileReference(newFileName);
using (var stream = new MemoryStream())
{
fileToCopy.DownloadToStream(stream);
stream.Seek(0, SeekOrigin.Begin);
newFile.UploadFromStream(stream);
}
where DownloadToStream and UploadToStream are methodes in my class:
CloudBlob.DownloadToStream(target);
and
CloudBlob.DownloadToStream(target);
CloudBlob is CloudBlockBlob type
You can set content type of blob via property ContentType
look at:
https://learn.microsoft.com/en-us/dotnet/api/microsoft.windowsazure.storage.blob.blobproperties.contenttype
Download AzCopy - http://aka.ms/azcopy
If you specify /SetContentType without a value, AzCopy sets each blob or file's content type according to its file extension.
Run this command on Windows
AzCopy /Source:C:\myfolder\ /Dest:https://myaccount.blob.core.windows.net/myContainer/ /DestKey:key /Pattern:ab /SetContentType
More details: https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy?toc=%2fazure%2fstorage%2fblobs%2ftoc.json
Use the Microsoft Azure Storage Explorer to modify the content-type string by hand for already existing file. Right click the blob file in explorer and the Left click on properties, scroll down to change the file format.
I'm currently having issue with Azure File storage when I build up a URL with a shared access signature (SAS) Token. The file will download in the browser, but the content-type is always application/octet-stream rather than changing to match the mime type of the file. If I put the file in Azure BLOB storage and build up a URL with a SAS Token, it sends the correct content-type for my file (image/jpeg).
I've upgraded my storage account from V1 to V2 thinking that was the problem, but it didn't fix it.
Does anyone have a clue what I could try that might get Azure File storage to return the correct content-type using a URL with SAS Token to download the file?
So far these are the only fixes for the content-type that I've found:
Use the Microsoft Azure Storage Explorer to modify the content-type string by hand. You have to right click the file and the left-click properties to get the dialog to appear.
Programmatically modify the file using Microsoft's WindowsAzure.Storage Nuget package.
Surface file download via my own web site and not allow direct access.
For me, none of these are acceptable choices. The first two can lead to mistakes down the road if a user uploads a file via the portal or Microsoft Azure Storage Explore and forgets to change the content type. I also don't want to write Azure Functions or web jobs to monitor and fix this problem.
Since blob storage does NOT have the same problems when uploading via Microsoft Azure Storage Explore or via the portal, the cost is much lower AND both work with SAS Tokens, we are moving towards blob storage instead. We do lose the ability to mount the drive to our local computers and use something like Beyond Compare to do file comparisons, but that is a disadvantage that we can live with.
If anyone has a better solution than the ones mentioned above that fixes this problem, I will gladly up-vote it. However, I think that Microsoft will have to make changes for this problem to be fixed.
When I upload a jpeg file to file share through portal, content-type is changed to application/octet-stream indeed. But I can't reproduce your download problem.
I didn't specify content-type in my SAS request uri, but the file just download as a jpeg file. Have tested in SDK(Account SAS/Stored Access Policy/SAS on file itself) or REST API, both work even without content-type.
You can try to specify the content-type using the code below.
SharedAccessFileHeaders header = new SharedAccessFileHeaders()
{
ContentDisposition = "attachment",
ContentType = "image/jpeg"
};
string sasToken = file.GetSharedAccessSignature(sharedPolicy,header);
Azure blob falls to the default value of 'application/octet-stream' if nothing is provided. To get the correct mimetypes, this is what I did with my flask app:
#app.route('/', methods=['GET', 'POST'])
def upload_file():
if request.method == 'POST':
f = request.files['file']
mime_type = f.content_type
print (mime_type)
print (type(f))
try:
blob_service.create_blob_from_stream(container, f.filename, f,
content_settings=ContentSettings(content_type=mime_type))
except Exception as e:
print (str(e))
pass
mime_type was passed to ContentSettings to get the current mimetypes of files uploaded to azure blob.
In nodeJS:
blobService.createBlockBlobFromStream(container, blob, stream, streamLength, { contentSettings: { contentType: fileMimeType } }, callback)
where:
fileMimeType is the type of the file being uploaded
callback is your callback implementation
Reference to method used:
https://learn.microsoft.com/en-us/javascript/api/azure-storage/azurestorage.services.blob.blobservice.blobservice?view=azure-node-latest#createblockblobfromstream-string--string--stream-readable--number--createblockblobrequestoptions--errororresult-blobresult--
Check this out - Microsoft SAS Examples
If you don't want to update the content-type of your file in Azure or it's too much of a pain to update the content-type of all your existing files, you can pass the desired content-type w/ the SAS token as well. The rsct param is where you would specify the desired content-type.
e.g. - https://myaccount.file.core.windows.net/pictures/somefile.pdf?sv=2015-02-21&st=2015-07-01T08:49Z&se=2015-07-02T08:49Z&sr=c&sp=r&rscd=file;%20attachment&rsct=application%2Fpdf&sig=YWJjZGVmZw%3d%3d&sig=a39%2BYozJhGp6miujGymjRpN8tsrQfLo9Z3i8IRyIpnQ%3d
This works with java using com.microsoft.azure azure-storage library. Uploading to Shared Access Signature resource.
InputStream is = new FileInputStream(file);
CloudBlockBlob cloudBlockBlob = new CloudBlockBlob(new URI(sasUri));
cloudBlockBlob.getProperties().setContentType("application/pdf");
cloudBlockBlob.upload(is, file.length());
is.close();
For anyone looking to upload files correctly with a declared Content Type, the v12 client has changed setting Content type. You can use the ShareFileHttpHeaders parameter of file.Create
ShareFileClient file = directory.GetFileClient(fileName);
using FileStream stream = File.OpenRead(#"C:\Temp\Amanita_muscaria.jpg");
file.Create(stream.Length, new ShareFileHttpHeaders { ContentType = ContentType(fileName) });
file.UploadRange(new HttpRange(0, stream.Length),stream);
where ContentType(fileName) is a evaluation of filename, eg:
if (fileName.EndsWith(".txt")) return "text/plain";
// etc
// here you define your file content type
CloudBlockBlob cloudBlockBlob = container.GetBlockBlobReference(file.FileName);
cloudBlockBlob.Properties.ContentType = file.ContentType; //content type
I know that I'm not answering the question, but I do believe the answer is applicable. I had the same problem with a storage account that I need it to have it as a static website. Whenever I upload a blob to a container, the default type is "application/octet-stream" and because of this the index.html get downloaded instead of being displayed.
To change the file type do the following:
# Get Storage Account for its context
$storageAccount = Get-AzStorageAccount -ResourceGroupName <Resource Group Name> -Name <Storage Account Name>
# Get Blobs inside container of storage account
$blobs = Get-AzStorageBlob -Context $storageAccount.Context -Container <Container Name>
foreach ($blob in $blobs) {
$CloudBlockBlob = [Microsoft.Azure.Storage.Blob.CloudBlockBlob] $blob.ICloudBlob
$CloudBlockBlob.Properties.ContentType = <Desired type as string>
$CloudBlockBlob.SetProperties()
}
Note: for Azure File storage you might wanna change the library to [Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob]
I have not tried this, but ideally, you could use ClientOptions to specify a different header. It'd would look something like this:
ClientOptions options = new ClientOptions();
HttpHeader httpHeaders = new HttpHeader("Content-Type", "application/pdf");
options.setHeaders(Collections.singleton(httpHeaders));
blobClient = new BlobClientBuilder()
.endpoint(<SAS-URL>)
.blobName("hello")
.clientOptions(options)
.buildClient();
This way we can provide the our own mime_type as 'content-type'
with open(file.path,"rb") as data:
#blob_client.upload_blob(data)
mime_type =mimetypes.MimeTypes().guess_type(file.name)[0]
blob_client.upload_blob(data,content_type=mime_type)
print(f'{file.name}' " uploaded to blob storage")
Based on this answer: Twong answer
Example if you are using .NET (C#) API to proxy/generate SAS url from ShareFileClient (ShareFileClient class description):
if (downloadClient.CanGenerateSasUri)
{
var sasBuilder = new ShareSasBuilder(ShareFileSasPermissions.Read, DateTimeOffset.Now.AddDays(10))
{
ContentType = "application/pdf",
ContentDisposition = "inline"
};
return downloadClient.GenerateSasUri(sasBuilder);
}
Above example setup 10 days long token for pdf file which will be open into new browser tab (especially on Apple iOS).
Solution in Java is to specify the content-type when generating the signature image url:
blobServiceSasSignatureValues.setContentType("image/jpeg");
I tried uploading an HTML file to my azure blob storage, and retrieved the link.
Unfortunately, when entering the URL into a web browser, it does not load the page, it tries to download it.
How can I make HTML files on Azure CDN load as web pages, not downloads?
Thanks
FIXED! Turns out in Azure, I need to edit the properties of the html file, and set the content type to text/html. :)
We need to set it's property Content type through Blob Options class.
PHP :
namespace - use MicrosoftAzure\Storage\Blob\Models\CreateBlobOptions;
//use code where you are creating blob
$opts = new CreateBlobOptions();
//$opts->setCacheControl('test');
$opts->setContentEncoding('UTF-8');
$opts->setContentLanguage('en-us');
//$opts->setContentLength(512);
$opts->setContentMD5(null);
$opts->setContentType($mimeType);
$blobRestProxy->createBlockBlob($containerName, $indexFile, $content,$opts);
It will work in git package : "microsoft/windowsazure": "^0.5"
In C#
entryData.DestinationBlob.Properties.ContentType = "image/jpeg";
entryData.DestinationBlob.SetProperties();
How can I create a sub container in the azure storage location?
Windows Azure doesn't provide the concept of heirarchical containers, but it does provide a mechanism to traverse heirarchy by convention and API. All containers are stored at the same level. You can gain simliar functionality by using naming conventions for your blob names.
For instance, you may create a container named "content" and create blobs with the following names in that container:
content/blue/images/logo.jpg
content/blue/images/icon-start.jpg
content/blue/images/icon-stop.jpg
content/red/images/logo.jpg
content/red/images/icon-start.jpg
content/red/images/icon-stop.jpg
Note that these blobs are a flat list against your "content" container. That said, using the "/" as a conventional delimiter, provides you with the functionality to traverse these in a heirarchical fashion.
protected IEnumerable<IListBlobItem>
GetDirectoryList(string directoryName, string subDirectoryName)
{
CloudStorageAccount account =
CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
CloudBlobClient client =
account.CreateCloudBlobClient();
CloudBlobDirectory directory =
client.GetBlobDirectoryReference(directoryName);
CloudBlobDirectory subDirectory =
directory.GetSubdirectory(subDirectoryName);
return subDirectory.ListBlobs();
}
You can then call this as follows:
GetDirectoryList("content/blue", "images")
Note the use of GetBlobDirectoryReference and GetSubDirectory methods and the CloudBlobDirectory type instead of CloudBlobContainer. These provide the traversal functionality you are likely looking for.
This should help you get started. Let me know if this doesn't answer your question:
[ Thanks to Neil Mackenzie for inspiration ]
Are you referring to blob storage? If so, the hierarchy is simply StorageAccount/Container/BlobName. There are no nested containers.
Having said that, you can use slashes in your blob name to simulate nested containers in the URI. See this article on MSDN for naming details.
I aggree with tobint answer and I want to add something this situation because I also
I need the same way upload my games html to Azure Storage with create this directories :
Games\Beautyshop\index.html
Games\Beautyshop\assets\apple.png
Games\Beautyshop\assets\aromas.png
Games\Beautyshop\customfont.css
Games\Beautyshop\jquery.js
So After your recommends I tried to upload my content with tool which is Azure Storage Explorer and you can download tool and source code with this url : Azure Storage Explorer
First of all I tried to upload via tool but It doesn't allow to hierarchical directory upload because you don't need : How to create sub directory in a blob container
Finally, I debug Azure Storage Explorer source code and I edited Background_UploadBlobs method and UploadFileList field in StorageAccountViewModel.cs file. You can edit it what you wants.I may have made spelling errors :/ I am so sorry but That's only my recommend.
If you are tying to upload files from Azure portal:
To create a sub folder in container, while uploading a file you can go to Advanced options and select upload to a folder, which will create a new folder in the container and upload the file into that.
Kotlin Code
val blobClient = blobContainerClient.getBlobClient("$subDirNameTimeStamp/$fileName$extension");
this will create directory having TimeStamp as name and inside that there will be your Blob File. Notice the use of slash (/) in above code which will nest your blob file by creating folder named as previous string of slash.
It will look like this on portal
Sample code
string myfolder = "<folderName>";
string myfilename = "<fileName>";
string fileName = String.Format("{0}/{1}.csv", myfolder, myfilename);
CloudBlockBlob blob = container.GetBlockBlobReference(fileName);