Does OpenEBS provide any kind of storage class that enables shared storage among containers? let's say I have 5 containers that need to access the exact same data. I have several containers doing AI training that need access to a shared image database, so it be like ReadWrite Many way
No. Currently OpenEBS is working on block storage.
Related
I want to implement a multi-region architecture in azure. My current architecture is running in one region and I am using blob storage to save my data. I was wondering if the blob storage can be shared between users in different regions? because I have seen that the replication is read-only otherwise I have to create another blob storage for the other regions ? and how I synchronize it to have the same data so the users in different regions can see the same content in the software?
I was wondering if the blob storage can be shared between users in
different regions?
Blob storage can certainly be shared between users in different regions. Blob storage resources are accessible over HTTP protocol so it doesn't really matter where your users are.
However please note that you may incur extra charges for data egress if the blob storage data is consumed by the application in your secondary regions.
Furthermore, you will notice some increased latency for both reads and writes. You can reduce the read latency by fronting your blob storage with CDN (but then you will pay extra for the CDN).
we are using Azure VM Scale sets to compute a larger job. In some stages we want the machines to share data with each other. We have tried Blob Storage for this but it's way to slow.
We are looking at either make the computers talking to each other or a simpler solution having them share a network drive (a fast one, being close to the actual hardware). Is this available in Azure? How we have understood it Azure Files is as slow as Blob storage because it's on top of blob storage.
Is it possible to create a disk that is shared between VM's in an Azure Scale Set?
No, this is not possible. you might use network shares.
well, you can implement Software Defined Storage cluster, but thats probably overkill
My task is to migrate our data store which is currently located on a network drive to Azure Data Lake Store or Blob Storage, as well as to migrate the ingestion and postprocessing software.
If I can mount Azure Data Lake Store or Blob Storage as a drive, it would make my task much easier.
You can easily mount an Azure File Share to Windows and Linux boxes: https://learn.microsoft.com/en-us/azure/storage/files/storage-how-to-use-files-windows
Additionally, if you're looking to go directly to blob from a Linux box, you can use Fuse (https://learn.microsoft.com/en-us/azure/storage/blobs/storage-how-to-mount-container-linux and https://azure.microsoft.com/en-us/blog/linux-fuse-adapter-for-blob-storage/).
I would advise against this approach. It makes more sense to abstract the details of storage in your software, therefore your application has no clue what type of storage is being used. It sounds like you have a massive coupling issues (technical debt) as your root cause. While mounting may work, it may not scale. So your mileage may vary.
You can mount Azure blob/datalake using goofys: https://github.com/kahing/goofys/blob/master/README-azure.md
Is it's possible to attach the same premium storage to multiple VMs so the files stored in the storage can be access in all of them.
The idea is to have a VM optimized for CPU that will calculate something and write results to the storage and have a low cost VM that will read the results and do other operations.
So if by saying "same" you mean same storage account - yes, you can do that, if by "same" you mean same VHD, no, you cant simultaneously attach same VHD to different VM's.
But you can have Azure Storage Files take on that role, it works like an SMB share, were you can store the results and other nodes will read them. Or you could just create a share on some VM that is supposed to read the results and store the results there.
Either way, its perfectly doable.
I have a VM on Azure which is my content management system using nodejs and mongodb.
One of things the CMS does is have a social sharing function where html pages are created and users are given the url to this page.
I expect a large volume of users (probably 5000 at a given time) access the http pages. I do not want this load to be on the same server as my CMS.
So I was thinking about moving the html pages to another server. My question is do I need to look at Azure blob storage to do this or should I just use another VM and put files there?
The files are very small and minified. I want to keep my costs down whilst at the same time if I get more than 5000 requests, the server should auto scale.
The question itself is somewhat subjective/opinion-soliciting. And how you solve this problem is really up to you.
But from an objective perspective:
Blobs themselves are not the same as local file storage. If you're going to store content in them, either your CMS needs to support them natively or you're going to need to build that support into it (if that's even possible). Since they have their own REST API (and related SDKs) you cannot simple do file I/O operations against them. They are, however, accessible via URI (which may be made private or public).
Azure VMs store their disks (vhd's) in page blobs (so, you're already using blob storage technically speaking). And each VM may have attached disks (1TB each) also in page blobs, two disks per core (so a dual-core VM supports 4 attached 1TB disks). Just like your OS disk, these attached disks are durable, in blob storage. A CMS may access an attached disk once it's formatted and given a drive letter (Windows) or mounted (Linux). EDIT - forgot to mention: If you go with the attached-disk approach, you need to consider the fact that these disks are per-VM. That is, they are not shared across multiple VM's (in the event you scale your CMS to multiple instances).
Azure File Service is an SMB share sitting atop Azure Blob Storage. Again, durable storage, and drive-mappable. EDIT unlike attached disks, Azure File Service SMB shares are accessible across multiple VM's.