I need to store multiple images link to 1000s of items in SQL Server. For each record there can be multiple Images to be stored.
I will be using a single account for storing all documents. Though, within accounts Blob Storage allows segregation only by Containers.
Should I create 1000s of containers for each item to separate images? Or '/' notation is recommended in this scenario (link below with details on using forward slash to achieve hierarchy)? If images are stored with '/' notation, can the name of the image be preserved while rendering or when users are accessing it, without '/' in the name?
Creating an Azure Blob Hierarchy
Here is an example scenario -
ItemID Images
1 A1.jpg; A2.jpg; A32018.jpg
2 B1.jpg; B2.jpg; A32018.jpg
3 C1.jpg; C2.jpg; A32018.jpg
As specified here, multiple items can have images with same names, but should be stored separately.
The Azure blob storage have account/contianer/blob(s)
There is one account, one container (could be called folder) but can have multiple blob (folders or files). This means that whatever is after container and whatever is the depth all are called as blob.
I don't understand still know that how you want to store the images but may be this will help.
In case I have two images image1.png and image2.png, it can be stored in below ways
Suppose account : imagedatastore
container : imagescontainer
You can keep both the images on imagescontainer such as
imagescontainer/image1.png
imagescontainer/image2.png
or can create separate containers for both images.
This is totally dependant on your segregation logic.
Related
I need to read blobs from a azure container which are added after a particular date.
Basically, I have a windows service which runs once a day and gets the list of blobs added after the first run.
I do not see any such option in CloudBlobContainer.ListBlobsSegmentedAsync function or via the Get Blob REST API call.
I could think of only one option- have timestamp in the filename and filter by prefix but would like to know other better options to achieve this.
Unfortunately there's very limited server-side filtering available in Azure Blob Storage and only filtering allowed today is by blob name prefix.
One solution to your problem is list all blobs in a container. Each blob has a property called Created Date/Time which tells you when the blob was first created (there's another property called Last Modified as well).
When you have the list, you can filter on the client side by this Created Date/Time property to get the desired list of blobs.
I want to create following folder structure on Azure:
mycontainer
-images
--2007
---img001.jpg
---img002.jpg
Now, one way is to use PUT Blob request and upload img001.jpg specifying the whole path as
PUT "mycontainer/images/2007/img001.jpg"
But, I want to first create the folders images and 2007 and then in a different request upload the blob img001.jpg.
Right now when I tried to doing this using PUT BLOB request:
StringToSign:
PUT
x-ms-blob-type:BlockBlob
x-ms-date:Tue, 07 Feb 2017 23:35:12 GMT
x-ms-version:2016-05-31
/account/mycontainer/images/
HTTP URL
sun.net.www.protocol.http.HttpURLConnection:http://account.blob.core.windows.net/mycontainer/images/
It is creating a folder but its not empty. By, default its creating an
empty blob file without name.
Now, a lot of people say we can't create a empty folder. But, then how come, we can make it using the azure portal as the browser must be sending some type of rest request to create the folder.
I think it has to do something with Content-Type i.e. x-ms-blob-content-type, which should be specified in order to tell azure that its a folder not a blob.
But, I am confused.
I want to first create the folders images and 2007 and then in a different request upload the blob img001.jpg
I agree with Brendan Green, currently, Azure blob storage just enable us to create virtual directory structure by naming blobs with path information in their names.
I think it has to do something with Content-Type i.e. x-ms-blob-content-type, which should be specified in order to tell azure that its a folder not a blob. But, I am confused.
You could check the description of Request Headers that could be set for Put Blob operation and you will find it does not support creating an empty folder by specifying some request headers.
Besides, as Gaurav Mantri said, if you really want to create an empty folder structure without content, you could try to use Azure File storage and it also enables us to use REST API to access Azure File storage. And the Create Directory operation cloud be used to create a new directory under the specified share or parent directory.
PUT https://myaccount.file.core.windows.net/myshare/myparentdirectorypath/mydirectory?restype=directory
This is not possible - the folder structure is virtual only.
See Get started with Azure Blob storage using .NET. You can only create a container, and everything else held in that container is a blob.
Excerpt:
As shown above, you can name blobs with path information in their
names. This creates a virtual directory structure that you can
organize and traverse as you would a traditional file system. Note
that the directory structure is virtual only - the only resources
available in Blob storage are containers and blobs.
Is there a way to generate a SAS token or policy for a virtual path within blob container ?
E.g. I have a blob container called mycontainer. Inside it have the following blobs
FolderA/PathA/file.pdf
FolderA/PathA/file2.mpg
FolderA/PathC/file.doc
FolderB/PathA/file.pdf
I want to generate SAS token such that the client/application can perform operations inside of FolderA only within container mycontainer
Is that possible ?
Alternate approach is either to
a) Create a list of SAS tokens for each file (i.e. blockblob) within FolderA
b) Re-design such that FolderA is a blob container instead
I want to generate SAS token such that the client/application can
perform operations inside of FolderA only within container mycontainer
Is that possible ?
No, it is not possible because Folder inside a blob container is a virtual entity. Azure blob storage supports only two level hierarchy - container and blob. A folder is simply the prefix in a blob's name.
Both of the solutions you mentioned below are good alternatives and you would be able to use either of them depending on your use case. My recommendation would be to use approach (b) as it provides a nice isolation for individual users in the sense that each user gets her/his own container where they can save their own files.
I've inherited a project built using the Azure Storage Client 1.7 and am upgrading it as Microsoft have announced that this will no longer be supported from December this year.
References to the files in Blob storage are stored in a database with the following fields:
FilePath - a string in the form of uploadfiles/xxx/yyy/Image-20140117170146.jpg
FileURI - A string in the form of https://zzz.blob.core.windows.net/uploadfiles/xxx/yyy/Image-20140117170146.jpg
GetBlobReferenceFromServer will throw an exception if the file doesn't exist, so it seems you should use GetBlockBlobReference if you know the container and the Blob type.
So my question(s):
Can I assume any Blobs currently uploaded (using StorageClient 1.7) will be BlockBlobs?
As I need to know the container name to call GetBlockBlobReference can I reliably say that in the examples above my container would always be uploadfiles
Can I assume any Blobs currently uploaded (using StorageClient 1.7)
will be BlockBlobs?
Though you can't be 100% sure that the blobs uploaded via Storage Client library 1.7 are Blob Blobs because 1.7 also supported Page Blobs however you can make some intelligent guesses. For example, if the files are image files and other commonly used files (pdf, document etc.), you can assume that they are block blobs. Typically you would see vhd files uploaded as page blobs. Again if these are uploaded by the users of your application, more than likely they are block blobs.
Having said this, I think you should use GetBlobReferenceFromServer method. What you could do is list all blobs from the database and for each of them call GetBlobReferenceFromServer method. If the blob exists, then you will get the blob type. If the blob doesn't exist, this method will give you an error. This would be the quickest way to identify the blob type of existing entries in the database. If you want, you can store the blob type back in the database along with existing record if you find both block and page blobs when you check the blob type so that if in future you need to decide between creating a CloudBlockBlob or CloudPageBlob reference, you can look at this field.
As I need to know the container name to call GetBlockBlobReference can
I reliably say that in the examples above my container would always be
uploadfiles
Yes. In the examples you listed above, you can say that the blob container is upload files.
Question background:
This may be a simple question but I cant find an answer to it. I've just started using Azure storage (for storing images) and want to know if one 'blob' holds a maximum of one file?
This is my container called fmfcpics:
Within the container I have a block blob named myBlob and within this I have one image:
Through the following code, if I upload another image file to the myBlob block blob then it overwrites the image already in there:
CloudBlockBlob blockBlob = container.GetBlockBlobReference("myblob");
using (var fileStream = System.IO.File.OpenRead(#"C:\Users\Me\Pictures\Image1.jpg"))
{
blockBlob.UploadFromStream(fileStream);
}
Is this overwriting correct? Or should I be able to store multiple files at the myBlob?
Each blob is a completely separate entity, direct-addressable via uri:
http(s)://storageaccountname.blob.core.windows.net/containername/blobname
If you want to manage multiple entities (such as image jpg's in your case), you would upload each one to a separate blob name (and you're free to store as many as you want within a single container, and you may have as many containers as you want).
Note: These are block blobs. There are also page blobs that have random-access capability, and this is the basis for vhd storage (and in that case, the vhd would have a formatted file system within it, with multiple files).
In blob Azure Documentation you understand how blob service works and the concepts about this storage service.
In some minutes you can use the service easily