i have a website what can be used by 50 users at the same time. Those users will be in the same room.
My problem is to know how much bandwith (in Mb/s) do I need to rent for that room so that they can access my website comfortably (speed up and down) ?
The average page size of my website is 1MB.
I searched for answers on the internet and all I got was bandwith used in a month (for servers).
Sorry if my question is "vague", I did my best to make it clear.
Thank you in advance for your answers.
Using https://gtmetrix.com/ you can test your websites speed, page size, and load times
There are several alternatives you just have to do the research
The more important issue you should focus on is why your page is 1Mb that should be your first priority to resolve and using tools like gtmetrix can help
I recommend load testing your site to figure that out. If you're at all familiar with JMeter, you can use it to create a script that simulates a user navigating your site, then run multiple instances of that user (in your case, 50) to see how the site holds up under load.
You can learn more about JMeter here:
https://jmeter.apache.org/
If you're not familiar with creating JMeter scripts, you can record and auto-generate basic scripts using the Blazemeter Chrome Extension, here.
For low-load testing (50 users is pretty low), you can upload your JMeter script to Blazemeter, and with a free tier Blazemeter account, you can perform some basic tests to see how your site holds up. If you go that route, I recommend focusing on avg. response time and hits/second in order to determine what your bandwidth need truly is under load.
Related
I am developing a website (basically a public facing site).
How can I simulate multiple users are surfing my site and doing various activities so that I can understand how my site will behave in a real time environment?
I am using Apache server and PHP.
As mentioned in previous posts, you will need a load test tool. The good news is that there are many tools and services in this field: open source load test ones like JMeter and Gatling, commercial one like Loadrunner and Neoload, the bad news is that you have to answer some questions and make some decisions.
One key decision you need to make is whether to test your application in the lab or in the cloud.
cloud based testing: Blazemeter, soasta, neustar ...
in-lab testing: JMeter, Gatling, Neoload, loadrunner, webperformer ....
In addition, you need to answer the following questioins:
how many virtual clients you want to emulate to stress the server
how much budget do you have
how complex is the web application
how much skills do the tester(s) have.
If you have high budget, complex web application and testers with good skills (like a developer), you can consider Loadrunner, NeoLoad.
If you have low budget but your tester(s) have good skills, you can consider Jmeter and Gating.
If you need to emulate lots of virtual clients (say 10000) to stress your complex web site and your tester(s) don't have the skill of a developer/programer, you may want to consider NetGend. There is a blog site where you can find out how complex performance testing can be (like filling HTML forms, extract values from JSON messages etc) and how easy it is on NetGend platform. By the way, you don't need high budget for NetGend.
Good lucky in your load/performance testing!
What you want is a load testing tool, there are several but id check out Neoload. You could also use Selenium and the various way to run selenium tests automatically
As this is your first time engaging in this task you would be well advised to find someone who has been there, done that and developed the battle scars from this activity. It is not a trivial effort to performance test a piece of software. If you listen to the traditional software vendors they will tell you that "any business analyst can use this tool and be effective" as if the tool is 85-95% of the skills you will need to be able to successfully performance test an application or a site. This is marketing foo to remove barriers to a sale.
In actuality the tool you select is anywhere from 5-15% of the total skill set you will need to be successful. Also, if the financial risk of failure is sufficiently high to warrant a performance test then it almost matters not which tool you pick, for the cost of the tool and the expertise will be dwarfed by your financial risk of not scaling.
If you don't have time to develop the skills or enough lead time to get a solid performance tester then you may want to consider some of the managed services offerings in the market, such as SOASTA, which can provide the expertise and the tool bundled within the deal. Here are some things you will want to look at in advance of any test (common issues)
Load Balancer misconfiguration resulting in distorted load to one node
Not appropriately managing your cache age for your static resources (.jpg, .css, ...) resulting in higher than expected load
All of your lookup queries to the database should be index optimized. Use a database profiler to check this
Holding onto resources too long. If your 95th percentile page to page request delay is five minutes then don't set your timeout at 30 minutes or 90 minutes for the HTTP session. This will hold onto resources far too long for the dead session. I use a rule of thumb of 95th percentile value times 1.5.
if this is a shopping site then don't hand out a default cart to everyone who shows up. Make sure they are on the revenue path before you hand them a cart, such as looking at the cart or placing something in the cart. Otherwise you have just built a 1:1 relationship with every customer and just about every piece of your architecture from web server to app server to the database server where the cart is created and managed
Also on the cart front, implement a 100x100 rule. If someone has a 100 items in the cart pick up the phone and call them to personalize the sale. If you have persistent carts which never expire then consider implementing a 100 day rule for evacuating from the cart items of that age or killing a cart altogether which hasn't been touched in that period. These people are clearly not on the revenue path.
Consider your design for ecommerce. Every step between add to cart and checkout is an opportunity to abandon the sale. The fewer the steps the greater the conversion rate: This is the genius behind the Amazon one click checkout. Minimize your number of steps and you will see a higher revenue flow as a result.
I would recommend gatling for load testings. It's scala based but a recorder is provided to generate workload test cases.
=> http://gatling-tool.org/
It's important to set some dimensions for test tool assessment to simulate user traffic:
SLA details (performance goals such as pages per second (PPS), http request per second (HPS), throughput, CPU usage etc.)
Virtual user size to simulate (You will need this info to decide number of
slave PCs/VMs, in other words that's called load generators, according to virtual user number
range)
Maintenance cost of scripts over changes
Effort to develop and execute test scripts
Number of test scenarios
Scheduling needs (You may want to schedule tool in regular basis or
execute test on-demand if needed)
Budget for test tool licenses and ROI (return of investment) calculations (Price, tool
expertise cost, utilizaton of test tool on other web applications etc)
Metrics provided by test tools
Monitoring requirements of network, servers and client
Integration with current test infrastructure (If HP ALM exist, you may be interested in Loadrunner)
If you are in hurry and don't have time to evaluate which tool to select, you may start with JMeter.
Selenium can be used for test automation of regression tests, I would like to highlight it's not effective for performance test due to its API. Sahi is another option for test automation.
I think that you are definitely looking for a load testing tool like Blazemeter. I recently discover this webinar which shows you how to do load testing on your application using a PaaS provider as a development and runtime environment, where you can deploy your application to run the load tests. They combine Blazemeter with a monitoring tool, New Relic in this case, to see the way in which you detect the new users surfing on your website. It is really cool and very interesting since you can know what is the performance of your application with a specific infrastructure.
Simple.
Set up server / application monitoring - New Relic is the easiest and most powerful. Free for 14 days.
Record a typical user's activity - Use JMeter to set up a proxy on your laptop and route web requests, mobile app usage etc through it. Sounds difficult but it's really easy. JMeter can act as the man-in-the-middle and capture all the requests sent by the browser/app to the server(s).
Now you "clone" the above user as many times as you need/could and blast the server. Initially you'd run the load test from your dev machine. Mine can take up to 80 concurrent users before cpu/ram runs out. Beyond this level explore BlazeMeter (free 50 concurrent users), jmeter-ec2 script (free), flood.io etc. Upload your script and blast away at your server. Ideally you should run incremental stress tests at your server. 10 users, 50 users, 100 users, 200 users etc.
Analyse, fix issues & ramp up the stress - In between each stress blast, go over your new relic. How is the application & server performing? What's failing? How are the alerts working?
If you are also searching for UI-Testing, you should check out Sahi Pro 6 automated testing tool, it also can be integrated with Jenkins.
==> http://sahipro.com/
It is really easy to record user actions with it on any browser and just playback the recorded scripts.
You can run scripts simultaneously on multiple browsers, thus simulate multiple users browsing your page:
https://sahipro.com/docs/using-sahi/playback-desktop.html#Distributed%20Runs%20-%20More%20Information
I am trying to simulate user load on a hardware router. I am specifically trying to emulate the average load of a home router.
What i need to to do is load it up over a week long period at different times and perform the following:
Data Transfer
Torrent Downloads
HTTP/HTTPS Pages requests to different pages. Static content, dynamic content. etc.
I would need to this repeat at my specific intervals and be able to test multiple routers at once.
Anyone know of any software or scripts that will achieve this.
Cheers
Sure. You might be surprised to learn that the load on an average home router is probably pretty low most of the time. Do the math: even downloading at maximum DSL or cable router speed (even if it were small packet sizes, which in higher loads is not usually the case) is just not a significant load on a modern CPU these days.
Scripting loads is easy. I have a script that I bang against Comcast sometimes when I doubt their last mile link to my home. It simply uses wget (or try curl) to download a file of reasonable size repeatedly and records the download statistics (time and/or data rate) of the transfers. Just find a .pdf or other file of the size you need from around the net somewhere, or use a busy website with lots of content. Just avoid the little guys who might have to pay for that bandwidth you are consuming in your test. Better yet, Amazon S3 storage (and transfer bandwidth) is very cheap these days and easy to use. You could put some files of your own choosing up there, and download those repeatedly for your test environment instead of stealing bandwidth from someone else! ;)
Never played with any torrent clients, so I can't help you there, but I bet there are some you can script.
Also, you might check out netperf. I don't know the status of that project, but I've used it in the past to generate very high network loads. Google for it.
Have fun and good luck!
-Chris
My webhost is aking me to speed up my site and reduce the number of files calls.
Ok let me explain a little, my website is use in 95% as a bridge between my database (in the same hosting) and my Android applications (I have around 30 that need information from my db), the information only goes one way (as now) the app calls a json string like this the one in the site:
http://www.guiasitio.com/mantenimiento/applinks/prlinks.php
and this webpage to show in a web view as welcome message:
http://www.guiasitio.com/movilapp/test.php
this page has some images and jquery so I think this are the ones having a lot of memory usage, they have told me to use some code to create a cache of those files in the person browser to save memory (that is a little Chinese to me since I don't understand it) can some one give me an idea and send me to a tutorial on how to get this done?. Can the webview in a Android app keep caches of this files?
All your help his highly appreciated. Thanks
Using a CDN or content delivery network would be an easy solution if it worked well for you. Essentially you are off-loading the work or storing and serving static files (mainly images and CSS files) to another server. In addition to reducing the load on your your current server, it will speed up your site because files will be served from a location closest to each site visitor.
There are many good CDN choices. Amazon CloudFront is one popular option, though in my optinion the prize for the easiest service to setup is CloudFlare ... they offer a free plan, simply fill in the details, change the DNS settings on your domain to point to CloudFlare and you will be up and running.
With some fine-tuning, you can expect to reduce the requests on your server by up to 80%
I use both Amazon and CloudFlare, with good results. I have found that the main thing to be cautious of is to carefully check all the scripts on your site and make sure they are working as expected. CloudFlare has a simple setting where you can specify the cache settings as well, so there's another detail on your list covered.
Good luck!
I was wondering how much do you win by putting all of your css scripts and stuff that needs to be downloaded in one file?
I know that you would win a lot by using sprites, but at some point it might actually hurt to do that.
For example my website uses a lot of small icons and most of the pages has different icons after combining all those icons together i might get over 500kb in total, but if i make one sprite per page it is reduced to almost 50kb/page so that's cool.
But what about scripts js/css how much would i win by making a script for each page which has just over ~100 lines? Or maybe i wouldn't win at all?
Question, basically i want to know how much does a single request cost to download a file and is it really bad to to have many script/image files with todays modern browsers and a high speed connections.
EDIT
Thank you all for your answers, it was hard to chose just one because every answer did answer my question, I chose to reward the one that in my opinion answered my question about request cost the most directly, I will not accept any answer as correct because everyone was.
Multiple requests means more latency, so that will often make a difference. Exactly how costly that is will depend on the size of the response, the performance of the server, where in the world it's hosted, whether it's been cached, etc... To get real measurements you should experiment with your real world examples.
I often use PageSpeed, and generally follow the documented best practices: https://developers.google.com/speed/docs/insights/about.
To try answering your final question directly: additional requests will cost more. It's not necessarily "really bad" to have many files, but it's generally a good idea to combine content into a single file when you can.
Your question isn't answerable in a real generic way.
There are a few reasons to combine scripts and stylesheets.
Browsers using HTTP/1.1 will open multiple connections, typically 2-4 for every host. Because almost every site has the actual HTML file and at least one other resource like a stylesheet, script or image, these connections are created right when you load the initial URL like index.html.
TCP connections are costly. That's why browsers open directly multiple connections ahead of time.
Connections are usually limited to a small number and each connection can only transfer one file at a time.
That said, you could split your files across multiple hosts (e.g. an additional static.example.com), which increases the number of hosts / connections and can speed up the download. On the other hand, this brings additional overhead, because of more connections and additional DNS lookups.
On the other hand, there are valid reasons to leave your files split.
The most important one is HTTP/2. HTTP/2 uses only a single connection and multiplexes all file downloads over that connection. There are multiple demos online that demonstrate this, e.g. http://www.http2demo.io/
If you leave your files split, they can also be cached separately. If you have just small parts changing, the browser could just reload the changed file and all others would be answered using 304 Not Modified. You should have appropriate caching headers in place of course.
That said, if you have the resources, you could serve all your files separately using HTTP/2 for clients that support it. If you have a lot of older clients, you could fallback to combined files for them when they make requests using HTTP/1.1.
Tricky question :)
Of course, the trivial answer is that more requests takes more time, but that is not necessarily this simple.
browsers open multiple http connections to the same host, see http://sgdev-blog.blogspot.hu/2014/01/maximum-concurrent-connection-to-same.html Because that, not using parallel download but rather downloading one huge file is considered as a performance bottleneck by http://www.sitepoint.com/seven-mistakes-that-make-websites-slow/
web servers shall use gzip content-encoding whenever possible. Therefore size of the text resources such as HTML, JS, CSS are quite compressed.
most of those assets are static content, therefore a standard web server shall use etag caching on them. It means that next time the download will be like 26 bytes, since the server tells "not changed" instead of sending the 32kbyte of JavaScript over again
Because of the etag cache, the whole web site shall be cacheable (I assume you're programming a game or something like that, not some old-school J2EE servlet page).
I would suggest making 2-4 big files and download that, if you really want to go for the big files
So to put it together:
if you have only static content, then it is all the same, because etag caching will shortcut any real download from the server, server returns 304 Not modified answer
if you have some generated dynamic content (such as servlet pages), keep the JS and CSS separate as they can be etag cached separately, and only the servlet page needs to be downloaded
check that your server supports gzip content encoding for compression, this helps a lot :)
if you have multiple dynamic content (such as mutliple dynamically changing images), it makes sense to have them represented as 2-4 separate images to utilize the parallel http connections for download (although I can hardly imagine this use case in the real life)
Please, ensure that you're not serving static content dynamically. I.e. try to load the image to a web browser, open the network traffic view, reload with F5 and see that you get 304 Not modified from the server, instead of 200 OK and real traffic.
The biggest performance optimization is that you don't pull anything from the server, and it comes out of the box if used properly :)
I think #DigitalDan has the best answer.
But the question belies the real one, how do I make my page load faster? Or at least , APPEAR to load faster...
I would add something about "above the fold": basically you want to inline as much as will allow your page to render the main visible content on the first round trip, as that is what is perceived as the fastest by the user, and make sure nothing else on the page blocks that...
Archibald explains it well:
https://www.youtube.com/watch?v=EVEiIlJSx_Y
How much you win if you use any of these types might vary based on your specific needs, but I will talk about my case: in my web application we don't combine all files, instead, have 2 types of files, common files, and per page files, where we have common files that needed globally for our application, and other files that is used for its case only, and here is why.
Above is a chart request analysis for my web application, what you need to consider is this
DNS Lookup happens only once as it cached after that, however, DNS name might be cached already, then.
On each request we have:
request start + initial connection + SSL negotiation+ time to first byte + content download
The main factor here which takes majority of request time in most cases is the content download size, so if I have multiple files that all of them needed to be used in all pages, I would combine them into one file so I can save the TCP stack time, on the other hand, if I have files needed to be used in specific pages, I would make it separate so I can save the content download time in other pages.
Actually very relevant question (topic) that many web developer face.
I would also add my answer among other contributors of this question.
Introduction before going to answer
High performance web sites depending on different factors, here is some consideration:
Website size
Content type of website (primary content Text, image, video or mixture)
Traffic on your website (How many people visiting your website average)
Web-host Location vs your primary visitor location (with in your country, region and world wide), it matters a lot if you have website for Europe and your host is in US.
Web-host server (hardware) technology, I prefer SSD disks.
How web-server (software) is setup and optimized
Is it dynamic or static web site
If dynamic, how your code and database is structured and designed
By defining your need you might be able to find the proper strategy.
Regarding your question in general
What regards your website. I recommend you to look at Steve Souders 14 recommendation in his Book High Performance Web Sites.
Steve Souders 14 advice:
Make fewer HTTP requests
Use a Content Delivery Network (CDN)
Add an Expires Header
Gzip Components
Put Style-sheets at the Top
Put Scripts at the Bottom
Avoid CSS Expressions
Make JavaScript and CSS External if possible
Reduce DNS Lookups
Minify JavaScript
Avoid Redirects
Remove Duplicates Scripts
Configure ETages
Make Ajax Cacheable
Regarding your question
So if we take js/css in consideration following will help a lot:
It is better to have different codes on different files.
Example: you might have page1, page2, page3 and page4.
Page1 and page2 uses js1 and js2
Page3 uses only js3
Page4 uses all js1, js2 and js3
So it will be a good idea to have JavaScript in 3 files. You are not interested in including every thing you have that you do not use.
CSS Sprites
CSS at top and JS at the end
Minifying JavaScript
Put your JavaScript and CSS in external files
CDN, in case you use jQuery for example do not download it to your website just use the recommended CDN address.
Conclusion
I am pretty sure there is more details to write. And not all advice are necessary to implement, but it is important to be aware of those. As I mentioned before, I suggest you reading this tiny book, it gives you more details. And finally there is no perfect final solution. You need to start some where, do your best and improved it. No thing is permanent.
Good luck.
the answer to your question is it really depends.
the ultimate goal of page load optimization is to make your users feel your page load is fast.
some suggestions:
do not merge common library js css files like jquery coz they might have already cached by brower when you visited other sites so u don't even need to download them;
merge resources, but at least separate first screen required resouces and the others coz the earlier user could see some meaningful stuff, the faster they feel about your page;
if several of your pages shared some resources, separate the merged files for shared resources and page specific resources so that when you visit the second page, the shared ones might have already been cached by browser, so the page load is faster;
user might be using a phone with slow or inconsistent speed 3g/4g network, so even 50k of data or 2 more requests does make them feel different a lot;
Is really bad to have a lot of 100-lines-files and is also really bad to have just one or two big files, though for each type css/js/markup.
Desktops have mostly high speed connection, and mobile has also high latency.
Taking all the theory about this topic, i think the best approach shall be more practical, less accurate and based upon actual connection speed and device types from a statistical point of view.
For example, i think this is the best way to go today:
1) put all the stuff needed to show the first page/functionality to the user, in one file, shall be under 100KB - this is absolutely a requirement.
2) after that, split or group the files in sizes so that the latency is no longer noticeable together with the download time.
To make it simple and concrete, if we assume: time to first byte is around ~200ms, the size of each file should be between ~120KB and ~200 KB, good for the most connections of today, averaged.
I am considering developing a web site which has many characteristics of a social networking site. The website, I am considering will have a lot of apps, which will interact with the database, and also, scraping other websites for information and a multiuser chat. Also, it will feature a forum, blog, and other similar CRUD applications. The key things I am looking at is
Response time
Max number of developers may be 1 to 3 during the initial stages
I expect the website to scale up to around 1000 concurrent users in a year, and then hopefully an exponential growth.
The users are expected to spend a lot of time, in the site.
With this requirements in mind, I looked at Django, and Web2Py, since I am knowledgable in Python. It fits the bill mostly, but, I am concerned about the scalability, and as it scales, I will require more servers to be added. This means, additional cost, and I don't have any ideas to monetize the app in the near future for various reasons. So, I have to be satisfied with a limited amount of resources.
Can you kindly advice me?
Thx
Ik
From what you had described, Node.js is perfect. Not only does it have a low memory footprint and can it handle thousands of concurrent clients out of the box, but you can definitely use it for scraping websites (see this and this), creating chats (check nodechat and this other nice tutorial)
The respond time depends on your application, but if you code the right way (don't block the event loop of Node.js, keep you 'heavy-lifting' outside the server process) Node.js is really fast.
This depends on you, but consider Node.js is JavaScript on the server-side, so there is already a great pool of developers that already know JS and could learn Node.js specific things fast.
There were some official benchmarks on the nodejs blog some weeks ago, look here: http://blog.nodejs.org/2011/11/05/node-v0-6-0/ A simple server with Node.js can handle 5-6 thousands of requests per second, so you can imagine that's really something.
Spending a lot of time on the site means that they'll be making many requests, so look at my point above 3).
http://highscalability.com/blog/2011/2/22/is-nodejs-becoming-a-part-of-the-stack-simplegeo-says-yes.html
Scaling node.js