Background:
I am moving a legacy app that was storing images and documents on local disk on the web server, over toward a PaaS Azure web app and using Azure File Storage to store and serve the files.
Question:
I have noticed that sometimes the url for file download fails the first time, either image links on a page are broken until I refresh or a download fails the first time, then succeeds the next. I am guessing that this is due to some issue with how Azure File Storage works and it hasn't started up or something. The only consistent thread I have observed is that this seems to happen once or twice in the morning when I am first working with it. I am guessing maybe my account has to ramp up or something, so its not ready on the first go round. I tried to come up with steps to reproduce, but I could not reproduce the symptom. If my hunch is correct, I will have to wait until tomorrow morning to try. I will post more detailed error information if/when I can.
var fullRelativePath = $"{_fileShareName}/{_fileRoot}/{relativePath}".Replace("//","/");
return $"{_fileStorageRootUrl}{fullRelativePath}{_fileStorageSharedAccessKey}";
Thanks!
So its been a while but I remember I was able to resolve this, so I'll be writing this from memory. To be able to access an image from file storage via a URL, you need to use a SAS token. I already had, which is why I was perplexed about this. I'm not sure if this is the ideal solution, but what I wound up doing was just appending some random characters to the end of the url, after the SAS token, and that make it work. My guess is this somehow made it unique, which may have helped it bypass some caching mechanism that was behaving erratically.
I'll see if I can dig up working example from my archive. If so, I'll append it to this answer.
Related
Posting here as server fault doesn't seem to have the detailed Azure knowledge.
I have a Azure storage account, a file share. This file share is connected to a Azure VM through mapped drive. A FTP server on the VM accepts a stream of files and stores them in the File Share directly.
There are no other connections. Only I have Azure admin access, limited support people have access to the VM.
Last week, for unknown reasons 16 million files, which are nested in many sub-folders (origin, date) moved instantly into a unrelated subfolder, 3 levels deep.
I'm baffled how this can happen. There is a clear instant cut off when files moved.
As a result, I'm seeing increased costs on LRS. I'm assuming because internally Azure storage is replicating the change at my expense.
I have attempted to copy the files back using a VM and AZCOPY. This process crashed midway through leaving me with a half a completed copy operation. This failed attempt took days, which makes me confident I wasn't the support guys dragging and moving a folder by accident.
Questions:
Is it possible to just instantly move so many files (how)
Is there a solid way I can move the files back, taking into account the half copied files - I mean an Azure backend operation way rather than writing an app / power shell / AZCOPY?
So there a cost efficient way of doing this (I'm on Transaction Optimised tier)
Do I have a case here to get Microsoft to do something, we didn't move them... I assume something internally messed up.
Thanks
A tool that supports server-side copy (like AzCopy) can move the files quickly because only the metadata is updated. If you wants to investigate the root cause, I recommend opening a support case. (To sort this out – Your best bet is to connect with our Azure support team by filing a ticket, our support team on best effort basis can help you guide on this matter. )
I'm trying to write a script that will create a new google cloud project, then provision firebase resources to it, all using the Node SDK.
The first step is calling google.cloudresourcemanager("v1").projects.create, and I use an organization-level service account for that with the proper permissions which returns the proper Operation object on success.
The issue is that after this call, there's often a delay of up to several hours before a call to google.cloudresourcemanager("v1").projects.get or google.firebase({version: "v1beta1"}).projects.addFirebase works, and if you check in the console the project isn't there. The issue is not with permissions (authorization/authentication) as when I manually verify that a project exists then call those two functions, they work as expected.
Has anybody else experienced something like this?
thanks!
On the Official API Documentation it is mentioned this:
Request that a new Project be created. The result is an Operation which can be used to track the creation process. This process usually takes a few seconds, but can sometimes take much longer. The tracking Operation is automatically deleted after a few hours, so there is no need to call operations.delete.
This means that this method works in an asynchronous way. As the response can be used to track the creation of the project, but it hasn't been fully created when the API answers.
Also it Mentiones that it can take long time.
As this is how the API works all the SDKs including the NodeJS one will share this behavior.
I've investigated this and found lots of related information, but nothing that answers my question.
A little background. A small set of files are shared by IIS 10 statically. These files need to be updated usually weekly, but not more than once an hour (unless someone manually runs an update utility for testing). The files are expected to be a couple K bytes in size, no larger then 10 kilobytes. The update process can be run on the IIS server and will be written in PowerShell or C#.
My plan for updating files that are actively being served as static files by IIS is:
Copy the files to a temporary local location (on the same volume)
Attempt to move the files to the IIS static site location
The move may fail if the file is in use (by IIS). Implement a simple retry strategy for this.
It doesn't cause a problem if there is a delay in publishing these files. What I really want to avoid is IIS trying to access one of the files at just the wrong time, a race condition while my file replacement is in process. I have no control over the HTTP client which might be a program that's not tolerant of the type of error IIS could be expected to return, like an HTTP status 404, "Not Found".
I have a couple random ideas:
HTTP GET the file from IIS before I replace it with the intention of getting the file into IIS's cache in hopes that will improve the situation.
Just ignore this potential issue an hope for the best
I can't be the only developer who's faced this. What's a good way to address this issue? (Or is it somehow not an issue at all and I'm just over thinking it?)
Thanks in advance for any help.
I have a file which is around 1.2 GB size and I want to call an instance to it while formulating results for my website. Is it possible to make the instance the same for all users of the website? According to my understanding, for eg Heroku, as all create separate instances of the website for every user, is there any way to make it happen. I apologize in advance if the question is naive !!
Bummer, heroku only allows you to have 500mb of files in your deploys.
If not for that, you could just commit that big file to your repo and use it on your site.
Heroku doesn't create an instance of the app for every user but for every deploy. Once you deploy your application, your entire old server is thrown away and a new one is created.
Let's say you need a temporary file, you could put it on /tmp/something and then use it for a while. Once you make a deploy, as the server is discarded and a new one is spawn, that file wouldn't be there and you'd have to recreate it for the first request.
Anyway, that doesn't look good. Even if heroku would let you store the file there, you'd also have to parse and dig through it in order to display your results for your site, which is likely to make you run out of memory too.
I would recommend you to review your approach to this problem, maybe break that file in small pieces or store it in a database to perform smaller calculations.
If there is absolutely no way around it, you could have a server in some other server like digital ocean and build a small api to perform the calculations there and make your site call it.
If you want, I can give you a hand on switching the approach for it, just post a different question, comment the link to it and I'll give it a shot.
I would really like to measure connection speed between two exact sites. Naturally one of the sites is our site. Somehow I need to prove that not our internet connection is flaky, but that a site at the other end, is overcrowded.
At our end I have windows and linux machines available for this.
I imagine I would run a script at certain times of day which - for example - tries to download an image from that site and try to measure download time. Then put the download time into a database then create a graph from the records in the database. (I know that this is really simple and not sophisticated enough, but hence my question)
I need help on the time measurement.
The felt speed differences are big, sometimes the application works flawlessly, but sometimes we get timed out errors.
Now I use speedtest to check if our internet connection is OK, but this does not show that the site that is not working is slow, and now I can't provide hard numbers to assist my situation.
Maybe it is worth mentioning that the application we try to use at the other end is java based.
Here's how I would do it in Linux:
Use wget to download whatever URL you think represents your sites best. Parse the output into a file (sed, awk), use crontab to trigger the download multiple times.
wget www.google.com
...
2014-02-24 22:03:09 (1.26 MB/s) - 'index.html' saved [11251]