Is there a way to know if a blob file exists inside a container without getting the whole list of blob objects ?
Thanks,
If you know the address of the blob, a tip from the Azure SDK is to first build a CloudBlockBlob (or a CloudPageBlob) and then call FetchAttributes. This call will throw a StorageClientException if it cannot locate the blob.
From the CloudBlobClient.GetBlockBlobReference documentation:
The FetchAttributes method executes a HEAD request to populate the
blob's properties and metadata and as such is a lightweight option for
determining whether the blob exists.
Starting from Windows Azure Storage Client Library 2.0, the blob contains method Exists(), e.g: blob.Exists()
the same is true for the BlobContainer.
This is the code that I'm using
public static bool Exists(this CloudBlob blob)
{
try
{
blob.FetchAttributes();
return true;
}
catch (StorageClientException e)
{
if (e.ErrorCode == StorageErrorCode.ResourceNotFound)
{
return false;
}
else
{
throw;
}
}
}
Related
I'm trying out Azure Blob Change Feed feature and it behaves strange to me with Append Blobs: append events are missing in the feed.
My scenario is:
Create storage account, enable change feed feature:
Change feed enabled
Create Append Blob if not exists (1) and appending some input into it (2).
private void WriteBlob(string input)
{
MemoryStream stream = new MemoryStream(Encoding.UTF8.GetBytes(input));
try
{
if (client == null)
{
var credential = new ClientSecretCredential("...", "...");
client = new AppendBlobClient(new Uri("..."), credential);
}
client.CreateIfNotExists(); // (1)
client.AppendBlock(stream); // (2)
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
Fetch Change Feed entries in separate console app.
public static List<BlobChangeFeedEvent> GetChanges()
{
var credential = new ClientSecretCredential("...", "...");
BlobChangeFeedClient blobChangeFeedClient = new BlobChangeFeedClient(new Uri("..."), credential);
List<BlobChangeFeedEvent> events = new List<BlobChangeFeedEvent>();
foreach (BlobChangeFeedEvent changeFeedEvent in blobChangeFeedClient.GetChanges())
{
events.Add(changeFeedEvent);
}
return events;
}
The problem is that after a few runs of WriteBlob method I only get single change feed event that corresponds to the blob creation, and subsequent appends are missing in the feed, however inputs are being appended successfully to the blob resource.
The question is why it is working this way? I didn't find anything special about Append Blob blob type regarding Change feed in docs.
Currently, the append event for an append blob is not supported.
As per this doc, only the following event types are supported:
BlobCreated
BlobDeleted
BlobPropertiesUpdated
BlobSnapshotCreated
And in the source code of Azure.Storage.Blobs.ChangeFeed package, there is no append event type.
A feature request of this is submitted, hope it can be added in the future release.
I try to get properties after or before download by BlobHelper.GetBlobReference() for loging , at last I try with blob.FetchAttributes(); but doestn work my properties are null. My container and my blob have not permissions
public static CloudBlob GetBlobReference(string containerName,string fileName)
{
var blobClient = GetBlobClient();
if (blobClient != null)
{
var contRef=blobClient.GetContainerReference(containerName);
return contRef.GetBlobReference(fileName);
}
return null;
}
var blob = BlobHelper.GetBlobReference(SelectedContainer,
fileName);
if (blob.Properties != null)
{
//I try to get Lenght of blob but it is -1
}
This is expected behavior. GetBlobReference simply creates an instance of CloudBlob on the client and doesn't make a network request. From the documentation link:
Call this method to return a reference to a blob of any type in this
container. Note that this method does not make a request against Blob
storage. You can return a reference to the blob whether or not it
exists yet.
If you want to get the properties populated, you must call FetchAttributes or use GetBlobReferenceFromServer.
I am using Azure storage to store and retrieve images. I have this method to save an image (blob):
public void SaveBlob(string containerName, string blobName, byte[] blob)
{
// Retrieve reference to the blob
CloudBlockBlob blockBlob = GetContainer(containerName, true).GetBlockBlobReference(blobName);
using (var stream = new MemoryStream(blob, writable: false))
{
blockBlob.UploadFromStream(stream);
}
}
This is the GetContainer method:
public CloudBlobContainer GetContainer(string containerName, bool createIfNotExist)
{
if (string.IsNullOrEmpty(containerName))
return null;
// Retrieve a reference to a container. If container doesn't exist, optionally create it.
CloudBlobContainer container = this._blobClient.GetContainerReference(containerName);
if (container == null && !createIfNotExist)
return null;
// Create the container if it doesn't already exist. It will be private by default.
container.CreateIfNotExists(BlobContainerPublicAccessType.Off, null, null);
return container;
}
What's happening here is that when I attempt to save a blob, I get a reference to a container first. If the container doesn't exist, it is created, and then the blob is saved. Will I run into timing issues here if I have to create the container first and then save a blob to it immediately? My concern is that I might be attempting to save the blob to the container before Azure is finished creating it. Or maybe this isn't an issue?
Looking at your code, I don't think you will run into any timing issues because CreateIfNotExists method is a sync method and will only return when the container is created. Also with Azure Blob Storage (unlike Amazon S3), the method will either create the container immediately or throw an error if it fails to do so (or in other words Azure Storage is Strongly Consistent).
Also I think this piece of code is redundant:
if (container == null && !createIfNotExist)
return null;
As container will never be equal to null and thus this if condition will never be true.
I am writing automatic tests and I need to check if the Upload succeeded.
How do I do that? How come there is no FileExist method?
Exists method for blobs have been added in the new Storage Client Library 2.0 release. Since you are using an older library version, you can instead use FetchAttributes. It will throw an exception if the blob does not exist.
On the other hand, as Magnus also mentioned, Upload* methods throw an exception if they do not succeed.
I recommend checking file size for case that forexample connection to server was closed before completing data transfer.
public bool WriteDocumentStream(String documentId, Stream dataStream, long length)
{
CloudBlobContainer container = BlobClient.GetContainerReference(ContainerName);
CloudBlob blob = container.GetBlobReference(documentId);
blob.UploadFromStream(dataStream);
blob.FetchAttributes();
bool success = blob.Properties.Length == length;
if (!success)
blob.Delete();
return success;
}
//length should be like this: context.Request.ContentLength
//(if request have ContentLength defined at headers)
After I use CloudBlob.BeginUploadFromStream() method to upload a file, I later get a StorageClientException with StorageErrorCode.ResourceNotFound when trying to retrieve the file for a download. If I upload the same file using CloudBlob.UploadFromStream() method, then the blob DOES exist and i can download it.
here's my download code:
var client = _storageAccount.CreateCloudBlobClient();
var container = client.GetContainerReference(BLOB_CONTAINER_DOCUMENTS_ADDRESS);
container.CreateIfNotExist();
string blobName = id.ToString();
var newBlob = container.GetBlobReference(blobName);
if (newBlob.Exists())
{
var stream = newBlob.OpenRead();
return stream;
}
else
{
throw new Exception("Blob does not exist!");
}
Exists is an extension method. I'm getting the StorageClientException with the error code ResourceNotFound when I use the BeginUploadFromStream() method
public static bool Exists(this CloudBlob blob)
{
try
{
blob.FetchAttributes();
return true;
}
catch (StorageClientException e)
{
if (e.ErrorCode == StorageErrorCode.ResourceNotFound)
{
return false;
}
else
{
throw;
}
}
}
And my call to upload
var blob = container.GetBlobReference(blobName);
This will NOT throw an exception when i later check if the blob exists
blob.UploadFromStream(fileStream);
This will
AsyncCallback uploadCompleted = new AsyncCallback(OnUploadCompleted);
blob.BeginUploadFromStream(fileStream, uploadCompleted, documentId);
EDIT
As suggested, i didn't have a call to EndUploadFromStream() method. Here is my updated call to upload:
blob.BeginUploadFromStream(fileStream, uploadCompleted, blob);
And my handler
private void OnUploadCompleted(IAsyncResult result)
{
var blob = (CloudBlob) result.AsyncState;
blob.EndUploadFromStream(result);
}
Running this, the EndUploadFromStream() method throws a WebException with the msg: "The request was aborted: The request was canceled." The InnerException is "Cannot close stream until all bytes are written."
Anyone have any idea what's going on here?
BeginUploadFromStream uploads the blob asynchronously, so your method proceeds while the blob uploads on a thread in the background. If the blob hasn't finished uploading -- or if Azure hasn't been told that the upload has completed -- you won't see the blob in storage. Only blobs uploaded through successfully completed transactions are visible.
Could you post the code for OnUploadCompleted?
It looks at first glance as if either the blob is still uploading -- or you've forgotten to call EndUploadFromStream() in your OnUploadCompleted method.
What it sounds like is happening is IIS is cancelling the thread that is being initiated to make the BeginUploadFromStream. Since the storage API is really just manipulating a bunch of REST calls under the hood you can think of these storage calls as web service calls and not like traditional IO.
Check out this topic on HttpKeepAlives, this might solve your problem but as the article pointed out it may impact performance of your site. So you may want to add logic to only enable the keep alive for the requests that are performing the upload.
http://www.jaxidian.org/update/2007/05/05/8/