I have a lot of ASP.NET apps / websites on my server (Some custom built and some open source). I was wondering if there was a free (Or built into Win2003) way of finding out which applications use the most resources throughout the day?
Sort of a breakdown of CPU(s) & Memory... As I have a suspicion one of the open source apps is hogging the CPU from time to time?
I think Windows Perfmon is what you are looking for. Usually gives you everything you need in terms of process, overall usage etc etc. Covers just about everything, network, disk, memory and makes some nice graphs too. You can also export the results of the logging to CSV so you can build even nicer charts in Excel if you want.
Oh I forgot, I am not sure how to do this anymore, it was a long time ago, but you can also have your application publish counters to Perfmon. I did that in Java but I am sure that .Net will provided a way to do the same.
Windows 2003 includes a tool called Performance Monitor (perfmon.msc) which can keep track of various resources. It's a fairly powerful and complex tool, if you Google for "windows 2003" perfmon counters you can find a lot of information to get started and how to use it effectively.
Run your websites in different application pools, then you can isolate easily!
http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/67e39bd8-317e-4cf6-b675-6431d4425248.mspx?mfr=true tells how application pools work in IIS6
Related
We want to build a web service to return some images (like google map tiles).
And the source data is organized as the esri compact cache format,the key of our service is to read the tiles from the bundles.
I am not sure how to choose the platform,windows or linux?
It is said that the linux have a bettor IO reading/writing performance than that of windows.
However java is our only choose if we choose linux,so I want to know if there is any points we should know to impove the IO reading performnce in linux?
PS:
In winodws platform,we will build the service based on .net4 using c#,and deploy the service use iis.
In linux,we will build the service using java (maybe based on spring mvc or some other mvc framework),and deploy the service using tomcat.
Update:
We may have the following source compact files in different folds:
L1
RxxCxx.bundle
RxxCxx.bundlx
L2
RxxCxx.bundle
RxxCxx.bundlx
And the request from the client may looks like this:
http://ourserver/maptile?row=123&col=234&level=1.png
For this requst,we will go in to the fold L1 since the level is 1,then read the RxxCxx.bundlx file first,since this file is the metadata that till tell us the position(the offset and length in RxxCxx.bundle) of the data for render the image(row=123&col=234),then we will read the RxxCxx.bundle according to the offset and length. Then we render the data to an image by write them to the response and set the content type to "image/png" or something else.
This is a whole procceed to handle a request.
Then I wonder if there is any documents or exist demos which can show me how to handle these type of IO reading?
The only situation where you have to have Windows servers in your environment is when you choose MS SQL Server DBMS (it is almost a Sybase but is a way cheaper), in which case have Windows box for the DB and *nix server for middle tier.
There are many situations where Windows can be used. Beginning with the declaration "have to have Windows" reveals an existing bias and is then followed by many groundless statements. But at least you clearly recognized this as the case.
Java is the best technology for millisecond grade middleware, mainly for the amount of mature standartized open source technologies available. Everything from coding (Eclipse, NetBeans, Idea) to manual (ant, maven) and automatic (teamcity, hudson/jenkins) builds, testing, static code analysis is there, is standartized, is open source, and is backed up by a multimillion size community.
I feel it necessary to say Visual Studio/C# (because OP mentioned as an alternative) offers everything you mentioned above with the exception of being open source. That said, the .NET Framework (or .NET Core) is now open source. Get information here. Based on your above comment, I think I can conclude that the only viable solutions are available through the open source community.
Quote I once heard that has a lot of truth: "It's only free if your time is worthless."
Also, counting the entirety of the open source community is a bogus argument. You'd have to take one development tool/API and compare the community support with another. For example, compare the community size/quality for Visual Studio with that of Eclipse. Or that of the .NET Framework vs. Java.
By the way, I've experienced no better intellisense implementation than with Visual Studio/Windows. When Eclipse does work you have rely on the quality of the open source libraries you reference to have anything meaningful. I've found the .NET Framework requires fewer 3rd party libraries than Java to accomplish the same goal.
Linux is the best server side platform for performance, stability, ease of maitenance, quality of the development environment - an extremely powerful command line based IDE. You can expect multimonth uptime from a Linux server, but not from Windows.
We have many Windows servers running services processing "big data" that have a system up-time since 5/30/2014 (nearly a year) and several more running without interruption since 2013. The only times we experience up-time problems is when hardware is aged/failing or the application-layer software we wrote contains bugs.
Tomcat/Servlet (or Jetty/Servlet) is a classic industrial combination in many financial institutions where stability is the #1 priority.
IIS is also used: job posting for IIS developer at financial institution
And lastly, the IO performance concern: a high quality user space non-blocking IO code will be CPU and hardware bandwidth bound, so OS will not be determining factor. Though fancy things like interrupts affinity, threads pinning, informed realtime tuning, kernel bypass I believe are easier to do on Linux.
Most of these variables are defined by each OS. It sounds like you have a lot of experience with threads, but also I would posit the developer can optimize at the application layer just as easily in both environments. Changing thread priority, implementing a custom thread pool, configuring BIOS, etc. are all available in the Windows world as well. Unless you want to customize the kernel which Unix/Linux allows, but then you have to support your own custom build of Unix/Linux.
I don't think commercial software should be vilified or avoided in favor of open source as a rule.
I understand this may sound as a groundless statement, but use *nix unless you have to use Windows. The only situation where you have to have Windows servers in your environment is when you choose MS SQL Server DBMS (it is almost a Sybase but is a way cheaper), in which case have Windows box for the DB and *nix server for middle tier.
Java is the best technology for millisecond grade middleware, mainly for the amount of mature standartized open source technologies available. Everything from coding (Eclipse, NetBeans, Idea) to manual (ant, maven) and automatic (teamcity, hudson/jenkins) builds, testing, static code analysis is there, is standartized, is open source, and is backed up by a multimillion size community.
Linux is the best server side platform for performance, stability, ease of maitenance, quality of the development environment - an extremely powerful command line based IDE. You can expect multimonth uptime from a Linux server, but not from Windows.
Tomcat/Servlet (or Jetty/Servlet) is a classic industrial combination in many financial institutions where stability is the #1 priority.
And lastly, the IO performance concern: a high quality user space non-blocking IO code will be CPU and hardware bandwidth bound, so OS will not be determining factor. Though fancy things like interrupts affinity, threads pinning, informed realtime tuning, kernel bypass I believe are easier to do on Linux.
Say I have a bunch of webservers each serving 100's of requests/s, and I want to see real time stats like:
Request rate over last 5s, 60s, 5 min etc
Number of unique users seen again per time window
Or in general for a bunch of timestamped events, I want to see real-time derived statistics - what's the best way to go about it?
I've considered having each GET request update a global counter somewhere, then sampling that at various intervals, but at the event rates I'm seeing it's hard to get a distributed counter that's fast enough.
Any ideas welcome!
Added: Servers are Linux running Apache/mod_wsgi, with a Python (Django) stack.
Added: To give a sense of the event rates I want to track stats for, they're coming in at over 10K events/s. Even incrementing a distributed counter at that rate is a challenge.
You might like to help us try out the beta of our agent for application performance monitoring in Python web applications.
http://newrelic.com
It delves more into the application performance rather than just the web server, but since any bottlenecks aren't generate going to be the web server, but your application then that is going to be more useful anyway.
Disclaimer. I work for New Relic and this is the project I am working on. It is a paid product, but the beta means it is free for now with all features. Later when that changes, if you didn't want to pay for it, their is still a Lite subscription level which is free and which gives you basic web metrics reporting which still covers some of what you are after. Anyway, right now would be a great opportunity to make use of it to debug your performance while you can.
Virtually all good servers provide this kind of functionality out of the box. For example, Apache has the mod_status module and Glassfish supports JMX. Furthermore, there are many commercial packages for monitoring clusters, such as Hyperic and Zenoss.
What web or application server are you using? It is difficult to provide a solution without that information.
Look at using WebSockets, their overhead is much smaller than a HTTP request, they are very well suited to real-time web applications. See: http://nodeknockout.com/ for Node based websocket examples.
http://en.wikipedia.org/wiki/WebSocket
You will need to run a daemon if you want to run it on your apache server.
Also take a look at:
http://kaazing.com/ if you wan't less hassle, but are willing to fork out some cash.
On the Windows side, Perfmonance monitor is the tool you should investigate.
As Jared O'Connor said, you should precise what kind of web server you want to monitor.
What I am looking specifically for is software thats runs on Linux (CentOS) that can do the following:
Show human readable CPU, Memory, Disk, Apache, MySQL utilization/performance.
Provide historic reports on the above metrics (today, week, month, year etc...)
Provide this data in an easy to view web based report or at least exportable to excel/csv.
I have looked at Cacti and I don't think its really an enterprise solution. I don't care if this is free or paid for software, though open source would be nice I am really just looking for the best solution.
Does anything like this exist for Linux? The problem this company is faced with is we have no way of measuring how the changes we make in our code and server configurations impact overall performance. So when I saw lets do this - then do it, I can't shows the benefits or revert back cause it was a negative in terms of performance. I am not a linux guru, just a developer with some linux skills, but am open to all suggestions. Thanks for reading.
Even though there are lot of open source projects but the main drawback they suffer is that they are away harder to configure. I have some across a free to called SeaLion which is way easier to install and configure. And it has awesome timeline base to representing outputs. Also there are different paid tools line new relic, server density, solar wind which you can also give a look.
Check out the eginnovations monitoring tool
http://www.eginnovations.com
Monitors Linux, Apache, mySQL and other applications and is web-based, so you dont have to be a linux expert.
M.
Cacti is a simple one. OpenNMS is more complete.
You are not limited to linux, using SNMP you can fetch this data from a remote host and use any NMS you like.
IMHO one of the best "freemium" tools is Zenoss (http://community.zenoss.org/).
The community edition is free. It will do everything you need, and comes with a simple RPM based installation process. It's a lot easier than Cacti or Nagios to setup and use. I would give it a try.
I use munin. I'ts much much simpler to set up than cacti. It's better to compile it yourself than pull it with apt-get (or other) because that way it has more built-in data-gathering scripts.
Basically there is no single dashboard where you can get all reports metrics. There are a range of opensource softwares which and can serve your need.
For server performance many people recommends munin, you will have to learn how to read teh report data. You can also write custom scripts to get certain report parameters of Mysql. Additionally if your server host provides an API, you can then do lot more related to reports in your admin panel.
you have a look at following url which can provide you more idea about choosing best fit to your need.
https://serverfault.com/questions/44/what-tool-do-you-use-to-monitor-your-servers
http://sixrevisions.com/tools/10-free-server-network-monitoring-tools-that-kick-ass/
What is the limit of IIS 6.0? like for example if i need to host 100,000 or 200,000 websites on IIS 6.0, how many machines would i need? or is IIS7 would be a better choice in this case for some reason?
As mentioned in the comments above the scale isn't so much the number of websites you create in IIS, but how complex and how busy those sites are.
In IIS6 one website does not necessarily equate to one executing process on the server. Application pools can group multiple websites into a single executing process to group and/or isolate applications. Alternately a single app pool can spawn multiple executing processes to make better use of server hardware.
It might help if you were to provide more detail in your question about what exactly you're trying to accomplish. If you're going to be serving hundreds of thousands of sites it would probably be a good idea to partner with a hosting company, or get some assistance from someone who knows the ins and outs of IIS, or another platform in detail and has operational experience with working through large-scale hosting scenarios.
IIS7 is not radically different from IIS6 in any performance-related way; with one exception: you can run ASP.NET in a "native" pipeline mode that bypasses some processing steps. I prefer IIS7 (if I can choose) because of its manageability advantages. But like everyone else said here: the question is impossible to answer without more information.
Hosting that many websites with IIS will be cost-prohibitive in licensing fees. Most large scale web hosting is done on Linux using Apache.
I'd like to start a free budget/personal finance site and will need plenty of horsepower and storage. I'm definitely a nubee, so how does one get started in terms of hardware infrastructure? Do I need to get a dedicated IP from my ISP and obtain my own servers? Do I go with amazon or Sql Server Data Services/Azure or something like that? Is the latter services free or a discount offering available to non-profit/free services such as the budget/personal finance site I'm looking to start?
If you don't mind writing your web application in python, then I's suggest using Google App Engine. See: What Is Google App Engine?
What I like to do when I have new ideas for a site is to find an inexpensive hosting solution ($10 per month). This allows me to test the idea and see if the site is going to be successful. If it is a flop, I haven't wasted much money and if it is successful I can upgrade to better hosting (dedicated server).
There are many hosting options available and several of them have great tools such as an online SQL Server management studio. Your other option would be to host it yourself if you are prepared to deal with firewall issues, backups, storage, etc.
Whether it is feasible to DIY varies a lot by country...if you have a decent broadband connection with a fixed IP this can be the cheapest route to play around with first, especially if you need an awful lot of storage.
Note however that many fast broadband connections are only fast for downloads - when you're running a server, the speed your users will see is the upload speed, which is usually a lot less. Also, you'll need to do your own admin and backup etc.
Apart from this most hosting options have a price tag on top, varying from virtual hosts (sharing a real machine), to colocation (your machine in somebody's data center), to cloud services like amazon et al (which have a good scaling ability)- and you will need to shop around for the software stack and hardware features you really need.
There's really two ways to answer this question, what differentiates them is budget.
One is to properly design this solution, prototype it, benchmark the prototype, extrapolate anticipated user load, add overhead and scale accordingly. This takes time, costs but gives you a supportable solution that serves your customers well.
The other is to just give something, anything a go and fix the problems as they come along. This is quicker and cheaper but might be a headache for a while and might p*** off your customers.
Basically it comes down to budget.
Best of luck.