BlockBlobClient.UploadAsync fails when trying to modify the incoming WatchableReadStream object - azure

I hope you're having some advice because I'm pretty much out of options. What I'm trying to do:
upload an image to a blobcontainer via the azure portal
when doing that, the blobtrigger below fires.
I want to resize the incoming image which is a Microsoft.Azure.WebJobs.Host.Blobs.WatchableReadStream object.
I want to upload the result to a new blob container.
The issue is, when I am passing any other Stream, e.g. a MemoryStream than the unmodified myBlob object the UploadAsync method seems to silently fail and the BlobTrigger fires again, starting an endless loop. Writing to the current myBlob object also silently fails. Only thing which works is passing the myBlob object in BlockBlobClient.UploadAsync without ANY modification whatsoever, which is useless. I am doing a remote debugging session in Azure.
Also, when I try to feed a MemoryStream object to the UploadAsync method the same problem occurs. The method below is the blob trigger:
public async Task UploadImage([BlobTrigger("mediafullsizecontainer/{name}", Connection = "AzureWebJobsStorage")]Stream myBlob, string name, ILogger log)
{
BlockBlobClient fullsizeBlobClient = _mediaFullsizeContainerClient.GetBlockBlobClient(name);
Response<BlobProperties> blobPropertiesResponse = await fullsizeBlobClient.GetPropertiesAsync(null, default);
BlobProperties blobProperties = blobPropertiesResponse.Value;
//Only process blobs when the correct metadata properties are set
if (blobProperties.Metadata.Any(property => property.Key == "category" & property.Value != String.Empty))
{
string category = blobProperties.Metadata["category"];
Stream s = await ResizeImage(myBlob);
BlockBlobClient thumbnailBlobclient = _mediaThumbnailContainerClient.GetBlockBlobClient(name);
Response<BlobContentInfo> uploadResponse = await thumbnailBlobclient.UploadAsync(s, new BlobUploadOptions(), default);
BlobContentInfo blobContentInfo = uploadResponse.Value;
}
}
The ResizeImage method uses the SixLabors.ImageSharp image processing library.
public async Task<Stream> ResizeImage(Stream inputStream)
{
(Image image, IImageFormat imageFormat) imageWithFormat = await Image.LoadWithFormatAsync(inputStream);
int height = imageWithFormat.image.Height;
int width = imageWithFormat.image.Width;
imageWithFormat.image.Mutate(operation => {
operation.Resize(width / 4, height / 4);
});
MemoryStream outputStream = new MemoryStream();
imageWithFormat.image.Save(outputStream, imageWithFormat.imageFormat);
return outputStream;
}
So when I do a mock change on the myBlob object like this, myBlob.Write(buffer) fails:
byte[] buffer = new byte[myBlob.Length];
myBlob.Read(buffer, 0, (int)myBlob.Length);
myBlob.Position = 0;
myBlob.Write(buffer);
When I copy the myBlob contents to a MemoryStream and pass the memoryStream to UploadAsync, UploadAsync fails:
MemoryStream ms = new MemoryStream();
myBlob.CopyTo(ms);
BlockBlobClient thumbnailBlobclient = _mediaThumbnailContainerClient.GetBlockBlobClient(name);
Response<BlobContentInfo> uploadResponse = await thumbnailBlobclient.UploadAsync(ms, new BlobUploadOptions(), default);
Only thing which works is passing the myBlob object without any modification to UploadAsync which is useless since I need to modify the incoming stream:
public async Task UploadImage([BlobTrigger("mediafullsizecontainer/{name}", Connection = "AzureWebJobsStorage")]Stream myBlob, string name, ILogger log)
{
BlockBlobClient fullsizeBlobClient = _mediaFullsizeContainerClient.GetBlockBlobClient(name);
Response<BlobProperties> blobPropertiesResponse = await fullsizeBlobClient.GetPropertiesAsync(null, default);
BlobProperties blobProperties = blobPropertiesResponse.Value;
//Only process blobs when the correct metadata properties are set
if (blobProperties.Metadata.Any(property => property.Key == "category" & property.Value != String.Empty))
{
string category = blobProperties.Metadata["category"];
BlockBlobClient thumbnailBlobclient = _mediaThumbnailContainerClient.GetBlockBlobClient(name);
Response<BlobContentInfo> uploadResponse = await thumbnailBlobclient.UploadAsync(myBlob, new BlobUploadOptions(), default);
BlobContentInfo blobContentInfo = uploadResponse.Value;
}
}
How can I pass a custom stream to the UploadAsync method? The documentation on https://learn.microsoft.com/en-us/dotnet/api/azure.storage.blobs.specialized.blockblobclient.uploadasync?view=azure-dotnet says you just need a stream object - but this does not work. Thanks in advance.

Ok looks like I found the answer myself, I got rid of the dreaded UploadAsync method and did it like this by reading the outputStream of the resized image into a buffer then opening a Stream for writing to the thumbnailBlobclient object and passing the buffer to the stream object. It makes sense that trying to modify the incoming myBlob object which came with the trigger, fires the trigger again and creating a loop.
Stream outputStream = await ResizeAndCropImage(myBlob);
byte[] buffer = new byte[outputStream.Length];
outputStream.Read(buffer);
BlockBlobClient thumbnailBlobclient = _mediaThumbnailContainerClient.GetBlockBlobClient(name);
Stream s = thumbnailBlobclient.OpenWrite(true, new BlockBlobOpenWriteOptions { HttpHeaders = new BlobHttpHeaders { ContentType = "image/jpg" } }, default); //set http headers here as well?
await s.WriteAsync(buffer);
s.Close();
I will set the content type in a more generic way but this concept is working.

Related

OpenXml Cannot Read or Write To Azure Blob Storage with FileMode or FileAccess value is invalid for the stream

We are working with DocumentFormat.OpenXml.WordDocument and opening a template of a Wordfile inside the Azure Blob, writing to it and saving it. There are two issues:
During the opening of the Document
using (WordprocessingDocument wordDoc = WordprocessingDocument.Open(streamToWrite, true))
Here the issue is like this ->
DocumentFormat.OpenXml.Packaging.OpenXmlPackageException: 'The stream was not opened for writing.'
If I replaces the IsEditable=trueto false, the issue will go. But the another issue occurs subsequently during the reading of the stream, after the changing the texts.
// Writing the changed document using (StreamWriter sw = new StreamWriter(wordDoc.MainDocumentPart.GetStream(FileMode.OpenOrCreate)))
The Error is like this-> System.ArgumentException: 'Stream was not writable.'
Final Code is like this, which is referred from this link : -https://learn.microsoft.com/en-us/office/open-xml/how-to-search-and-replace-text-in-a-document-part
public async Task<IActionResult> UpdateDocumentAsync([FromBody] ProcessingQuery query)
{
BlobClient readblob = new BlobClient(new Uri("https://XXXXXX/docxviewer/outputviewer/blob.docx?sp=racwd"));
using (HttpClient client = new HttpClient())
{
using (Stream streamToWrite = await client.GetStreamAsync(readblob.Uri))
{
try
{
using (WordprocessingDocument wordDoc = WordprocessingDocument.Open(streamToWrite, false))
{
string docText = null;
using (StreamReader sr = new StreamReader(wordDoc.MainDocumentPart.GetStream()))
{
docText = sr.ReadToEnd();
}
//Writing of Single Queries this will become dictionary key-value pair
Regex regexText1 = new Regex("{first_name}");
docText = regexText1.Replace(docText, query?.first_Name);
Regex regexText2 = new Regex("{last_name}");
docText = regexText2.Replace(docText, query?.last_Name);
Regex regexText3 = new Regex("{phone}");
docText = regexText2.Replace(docText, query?.phone);
// Writing the changed document
using (StreamWriter sw = new StreamWriter(wordDoc.MainDocumentPart.GetStream(FileMode.OpenOrCreate)))
{
sw.Write(docText);
sw.Flush();
sw.Close();
}
wordDoc.Close();
}
}
catch (Exception ex)
{
return BadRequest(ex.Message);
}
streamToWrite.Close();
}
}
return Ok(readblob);
}
Please help us on this codes with relevant codes and documents. If there is any issue in blob storage token Kindly help us in that too.. :)

azure blob client returns file not fully

i'm trying to download file from azure blob storage, but it returns only part of file. What i'm doing wrong ? File in storage is not corrupted
public async Task<byte[]> GetFile(string fileName)
{
var blobClient = BlobContainerClient.GetBlobClient(fileName);
var downloadInfo = await blobClient.DownloadAsync();
byte[] b = new byte[downloadInfo.Value.ContentLength];
await downloadInfo.Value.Content.ReadAsync(b, 0, (int)downloadInfo.Value.ContentLength);
return b;
}
I'm using Azure.Storage.Blobs 12.4.2 package. I tried this code and it works for me
public async Task<byte[]> GetFile(string fileName)
{
var blobClient = BlobContainerClient.GetBlobClient(fileName);
using (var memorystream = new MemoryStream())
{
await blobClient.DownloadToAsync(memorystream);
return memorystream.ToArray();
}
}
I am not able to full understand your code as the current BlobClient as of v11.1.1 does not expose any download methods. As #Guarav Mantri-AIS mentioned the readAsync can behave in that manner.
Consider an alternative using the DownloadToByteArrayAsync() which is part of the API. I have include the code required to connect but of course this is just for the purpose of demonstrating a full example.
Your method would be condensed as follows:
public async Task<byte[]> GetFile(string containerName, string fileName)
{
//i am getting the container here, not sure where or how you are doing this
var container = GetContainer("//your connection string", containerName);
//Get the blob first
ICloudBlob blob = container.GetBlockBlobReference(fileName);
//and now download it straight to a byte array
return await blobClient.DownloadAsync();
}
public CloudBlobContainer GetContainer(string connectionString, string containerName)
{
//1. connect to the account
var account = CloudStorageAccount.Parse(connectionString);
//2. create a client
var blobClient = _account.CreateCloudBlobClient();
//3. i am getting the container here, not sure where or how you are doing this
return = _blobClient.GetContainerReference(containerName);
}

DocumentFormat.openxml doesn't seems to be working with Azure functions

I have been trying to intergrate the openxml package in Azure function. The code compiles fine but when i try to reach the function url it doesn't downloads the file but fails the call and there is no execution error in the code.
public static async Task<HttpResponseMessage> Run([HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)]HttpRequestMessage req, TraceWriter log, [Inject]TrainingManager trainingManager)
{
//dynamic data = await req.Content.ReadAsAsync<object>();
//string trainingCourseId = data?.trainingCourseId;
log.Info("Function App Started");
HttpResponseMessage response = req.CreateResponse(HttpStatusCode.OK);
using (MemoryStream mem = new MemoryStream())
{
// Create Document
using (WordprocessingDocument wordDocument =
WordprocessingDocument.Create(mem, WordprocessingDocumentType.Document, true))
{
// Add a main document part.
MainDocumentPart mainPart = wordDocument.AddMainDocumentPart();
// Create the document structure and add some text.
mainPart.Document = new DocumentFormat.OpenXml.Wordprocessing.Document();
Body docBody = new Body();
// Add your docx content here
Paragraph para = docBody.AppendChild(new Paragraph());
Run run = para.AppendChild(new Run());
run.AppendChild(new Text("Hi"));
mainPart.Document.Save();
}
mem.Seek(0, SeekOrigin.Begin);
response.Content = new StreamContent(mem);
}
response.Content.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("attachment")
{
FileName = "AttendanceList.docx"
};
response.Content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/msword");
log.Info("Sending Response");
return response;
}
Let me know if anyone has faced this issue or have a solution to it.
Here is what i see in network tab]1
Thanks!

Append to CloudBlockBlob stream

We have a file system abstraction that allows us to easily switch between local and cloud (Azure) storage.
For reading and writing files we have the following members:
Stream OpenRead();
Stream OpenWrite();
Part of our application "bundles" documents into one file. For our local storage provider OpenWrite returns an appendable stream:
public Stream OpenWrite()
{
return new FileStream(fileInfo.FullName, FileMode.Open, FileAccess.ReadWrite, FileShare.ReadWrite, BufferSize, useAsync: true);
}
For Azure blob storage we do the following:
public Stream OpenWrite()
{
return blob.OpenWrite();
}
Unfortunately this overrides the blob contents each time. Is it possible to return a writable stream that can be appended to?
Based on the documentation for OpenWrite here http://msdn.microsoft.com/en-us/library/microsoft.windowsazure.storage.blob.cloudblockblob.openwrite.aspx, The OpenWrite method will overwrite an existing blob unless explicitly prevented using the accessCondition parameter.
One thing you could do is read the blob data in a stream and return that stream to your calling application and let that application append data to that stream. For example, see the code below:
static void BlobStreamTest()
{
storageAccount = CloudStorageAccount.DevelopmentStorageAccount;
CloudBlobContainer container = storageAccount.CreateCloudBlobClient().GetContainerReference("temp");
container.CreateIfNotExists();
CloudBlockBlob blob = container.GetBlockBlobReference("test.txt");
blob.UploadFromStream(new MemoryStream());//Let's just create an empty blob for the sake of demonstration.
for (int i = 0; i < 10; i++)
{
try
{
using (MemoryStream ms = new MemoryStream())
{
blob.DownloadToStream(ms);//Read blob data in a stream.
byte[] dataToWrite = Encoding.UTF8.GetBytes("This is line # " + (i + 1) + "\r\n");
ms.Write(dataToWrite, 0, dataToWrite.Length);
ms.Position = 0;
blob.UploadFromStream(ms);
}
}
catch (StorageException excep)
{
if (excep.RequestInformation.HttpStatusCode != 404)
{
throw;
}
}
}
}
There is now a CloudAppendBlob class that allows you to add content to an existing blob :
var account = CloudStorageAccount.Parse("storage account connectionstring");
var client = account.CreateCloudBlobClient();
var container = client.GetContainerReference("container name");
var blob = container.GetAppendBlobReference("blob name");
In your case you want to append from a stream:
await blob.AppendFromStreamAsync(new MemoryStream());
But you can append from text, byte array, file. Check the documentation.

GDI+ Generic Error

When my images are being loaded from my database on my web server, I see the following error:
A generic error occurred in GDI+. at
System.Drawing.Image.Save(Stream stream, ImageCodecInfo encoder,
EncoderParameters encoderParams) at
System.Drawing.Image.Save(Stream stream, ImageFormat format) at
MyWeb.Helpers.ImageHandler.ProcessRequest(HttpContext context)
All my code is attempting to do is load the image, can anybody take a look and let me know what I'm doing wrong?
Note - This works if I test it on my local machine, but not when I deploy it to my web server.
public void ProcessRequest(HttpContext context)
{
context.Response.Clear();
if (!String.IsNullOrEmpty(context.Request.QueryString["imageid"]))
{
int imageID = Convert.ToInt32(context.Request.QueryString["imageid"]);
int isThumbnail = Convert.ToInt32(context.Request.QueryString["thumbnail"]);
// Retrieve this image from the database
Image image = GetImage(imageID);
// Make it a thumbmail if requested
if (isThumbnail == 1)
{
Image.GetThumbnailImageAbort myCallback = new Image.GetThumbnailImageAbort(ThumbnailCallback);
image = image.GetThumbnailImage(200, 200, myCallback, IntPtr.Zero);
}
context.Response.ContentType = "image/png";
// Save the image to the OutputStream
image.Save(context.Response.OutputStream, ImageFormat.Png);
}
else
{
context.Response.ContentType = "text/html";
context.Response.Write("<p>Error: Image ID is not valid - image may have been deleted from the database.</p>");
}
}
The error occurs on the line:
image.Save(context.Response.OutputStream, ImageFormat.Png);
UPDATE
I've changed my code to this, bit the issue still happens:
var db = new MyWebEntities();
var screenshotData = (from screenshots in db.screenshots
where screenshots.id == imageID
select new ImageModel
{
ID = screenshots.id,
Language = screenshots.language,
ScreenshotByte = screenshots.screen_shot,
ProjectID = screenshots.projects_ID
});
foreach (ImageModel info in screenshotData)
{
using (MemoryStream ms = new MemoryStream(info.ScreenshotByte))
{
Image image = Image.FromStream(ms);
// Make it a thumbmail if requested
if (isThumbnail == 1)
{
Image.GetThumbnailImageAbort myCallback = new Image.GetThumbnailImageAbort(ThumbnailCallback);
image = image.GetThumbnailImage(200, 200, myCallback, IntPtr.Zero);
}
context.Response.ContentType = "image/png";
// Save the image to the OutputStream
image.Save(context.Response.OutputStream, ImageFormat.Png);
} }
Thanks.
Probably for the same reason that this guy was having problems - because the for a lifetime of an Image constructed from a Stream, the stream must not be destroyed.
So if your GetImage function constructs the returned image from a stream (e.g. a MemoryStream) and then closes the stream before returning the image then the above will fail. My guess is that your GetImage looks a tad like this:
Image GetImage(int id)
{
byte[] data = // Get data from database
using (MemoryStream stream = new MemoryStream(data))
{
return Image.FromStream(data);
}
}
If this is the case then try having GetImage return the MemoryStream (or possibly the byte array) directrly so that you can create the Image instance in your ProcessRequest method and dispose of the stream only when the processing of that image has completed.
This is mentioned in the documentation but its kind of in the small print.

Resources