I would like to query application URLs hitting my CDN resources in Azure. I can list all sorts of columns by default like requestUri_s or userAgent_s or even clientIP_s etc. But I can't find any column which would contain the client/user's URL or a domain, which requested the resource.
I tried to look in the query examples in Azure, but none of them had this information.
Any ideas?
Thx.
Related
I am hosting Azure static web site using Azure Storage, CDN Profile and End Point. I want to see the outgoing traffic. Currently I can only see that for the storage account but I can't see where I can actually count unique site visitors. Please, help. I am OK if it can only be done through powershell script.
There is no built in feature available to get the user visits count, However there are two ways to achieve that:
By Enabling Application Insights: Once you enable you will be able to view the user device,name,count etc
View Storage Analytics Logs: This is like a backend view, You can view the storage analytics logs to view the operations performed on blobs
I've got a static web site hosted in Azure Blob Storage with Cloudflare as my CDN. It's such a small site (not even 1Mb and only 1 blog post), but I'm getting 1.1-1.2Gb of requests each month for the past 6 months or so with no explanation. Is there a way to find out what is being requested? In Azure, I can only find information about the performance, up-time, etc, but nothing about url's and I need to pay to get this info from Cloudflare (I believe). Has anyone else experience such strange requests?
I suggest you open Diagnostic settings and download Azure Storage Explorer to view the logs.
When you finished settings, u can check logs by tools. You can see request urls and http status info.
The previous data should not be visible, but you can monitor and analyze future requests.
When I did a lookup on those two IP's, they were both registered to Cloudflare, one makes sense given I'm using their service, but to have what appears to be their bot hit my site with this frequency doesn't... Wonder if there's a setting
What we already have in place
Azure Blob Storage with static files, ir our case images
Azure CDN in front of the Blob Storage
ASP.NET Core Service
uploading images to the blob storage
saving meta data about the logical collection of images, called Asset
let's call this the media service
How we use that
Let's say we have another service called the product service.
Each product has a product picture associated with it.
We create a new asset in the media service and get back an AssetId.
The product service now stores this AssetId in his local database
What we're looking for
As long as we operate with the assets meta data in the back end life's good.
But at the end of the day we want to visualize the pictures in a WebUI. To do that we need to have the Url pointing back to the image on the blob storage / CDN.
Right now the back end actively resolves the Urls before the data flows to the WebUI. But there must be better way of doing this.
One feasible approach that comes to mind is to have a simple resolver middleware
/images/{assetid}/{assetType} -> Redirect to CDN Url
but then each call flows over this middleware instead of going to the CDN in first place before falling back to a resolver to get the correct Url.
What do I miss? What's the best practice any way to solve such basic thing in the micro-service area?
Thanks
I would not use a middleware component to redirect the URLs, because as you mentioned, the client will always contact first your middleware component, and there is an additional unnecessary call and overhead. The nature of the CDN is to serve the assets in a global distributed network, so your assets will be indexed to the CDN edges and it will be served from the closest edge to the client. You should be able to use the WebUI component and use the CDN Urls (https://.azureedge.net/*). In case you need to use redirect rules, you can use the CDN features to perform redirect rules. Note: Depending on the CDN service you are using, you may need to upgrade CDN pricing tier to Premium. Eg. Verizon you need to use Premium tier to have ability for redirect rules.
I've created a cdn service and set the origin url to a azure website which runs iis-node that serves a NodeJS web application, when trying to access static resources the cdn always return 404 while the same pathname works when accessing the resource through the origin url,
anything that i might be missing here?
thanks!
Turns out it takes at least an hour until the content is being populated, during that time the CDN return 404.
Based on the article
A content delivery network (CDN) is a distributed network of servers
that can efficiently deliver web content to users. CDNs store cached
content on edge servers in point-of-presence (POP) locations that are
close to end users, to minimize latency.
Azure Content Delivery Network (CDN) offers developers a global
solution for rapidly delivering high-bandwidth content to users by
caching their content at strategically placed physical nodes across
the world
What you can do though to fasten the process is to just log in to Azure portal, go to the
CDN profiles section, pick your CDN, And Click on Purge on the top menu.
It should automatically update the content of the cache memory for your endpoint/s and you shouldn't wait hours.
Hope it helps. Cheers
In case anyone else comes across this thread - I built an Azure CDN (Verizon Standard) and it took almost 24 hours for the images/files to pull up using the Endpoint Hostname.
Unfortunately, in my case I thought I was doing something wrong so I kept changing it which probably prolonged it. When I finally realized I did everything correct I decided to wait it out.
I had the same issue but the cause for me was that the Azure CDN only serves content under the /CDN path. So if you are trying to access anything on http://yourdomain.com/here.jpg you'll get a 404 (unless you have a URL rewwrite in place)
I have a website hosted in Azure as a cloud service (not as a website), and I need to get the hit count for every web page of the site.
I enabled Azure Diagnostics, and I see the IIS logs copied to my blob storage, however this logs contain very few data (only one hit to a javascript file).
Furthermore, putting "Verbose" or "All" in the diagnostics configuration of the web role doesn't seem to affect the results, I get only one line (an access to a css file, or an image file, etc).
I'm using Azure SDK 2.0.
Is it possible to use the included IIS logs generated by azure to get a hit count? What should I need to change in the diagnostics configuration?
Or should I need a different approach to achieve this?
The IIS logs it produces are the same ones you'd find on a Windows Server anywhere. Note that depending on the settings you provided to the diagnostics it might take a little while before the data is moved to the storage account. Setting the level of verbosity for the configuration determines what is moved from the instances over to the storage account. Did you give it plenty of time to move the data over before looking at the file in storage again? Sometimes it just brings over what it has, and of course, there could be buffering which means when the file was brought over not everything was in it, etc.
You should be able to get this information from the logs, and yes, you should be able to do it from the IIS logs. That being said, if what you are after is a hits per page I would suggest actually a different approach. Look at an analytics provider like Google Analytics or one of the competitors to that. You'll get a massive amount of information beyond just page hits and no need to worry about parsing log files, etc.