Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
With multicores now more common, game developers are now leaning toward the use of threads, which is discussed in this question:
Why don't large programs (such as games) use loads of different threads?
To me, this idea seems analogous to the idea having multiple machines running things in clusters, or parallel computing.
Some games run on dedicated servers.
My question is: Can you use clusters to maximize parallel power, in the same way threading does on a multi-core system? Will it give the same benefit? Why/why not?
You can do that. But sharing the computation of a Game engine between clusters will introduce more bottleneck to the system. Because clusters will us the network and it is way slower than CPU and main memory.
Some games use simulation clients to share large computing loads. But they need to be pretty careful about the synchronization issues caused by the network delays.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I have to design and deploy an application for a relatively constrained device such as a Raspberry Pi 4 Model B. My idea is to deploy several processes that are able to communicate them through IPC (Inter Process Communication).
I am evaluating the several options and it's easy to understand that the Shared Memory solution has the best performance at the cost of bigger development time. With the following trend:
Shared Memory. Faster, but less portable and difficult to implement
Pipes.
Message Queues.
Sockets. Slower, but more portable and easy to implement
This is a reference, but there is other material on the web about.
All the 3 solutions are good options, but I am afraid that if one day the implementation must be altered, then there may be problems.
I am thinking that MPI could be a good option for keeping the right level of abstraction, but I am scared that the additional layer of machine agnostic introduced by it may slow down the application.
MPI, in general, isn't a good general purpose abstraction. MPI is at its best in scientific simulations where structured data needs to be distributed. Contrary, MPI doesn't provide particularly helpful abstractions for use cases which require RPCs or master/worker task queues. That said, you can map almost any IPC use-case to MPI, but you may be better of with a paradigm that provides a higher level of abstraction to your use-case.
Nevertheless, MPI is generally optimized for performance and implementations can leverage shared memory underneath. Of course you will always have the cost of explicit communication (API calls) as opposed to accessing memory directly.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
I have a use case where I get 10K to 15 K Messages/sec and it might be less than 5K also sometime and I push those into RabbitMQ now those messages I should parse,run some RE on that and do some sort aggregation and run some statistics. My product(the data pipeline) would be deployed in single machine as there are some business constrain. Have explored Spark Akka Storm and RxJava, Could you suggest me what to be used. But I don't want to do in plain Java, as by this way I have to handle all the threading etc.
Based on my experience I would go for Akka. You can create different pools of actors to perform some tasks concurrently on different messages. Also you can leverage the power of akka-camel to have a RabbitMQ consumer using the RabbitMQ Camel Component.
You might be able to do the same with Storm but I don't have that much experience with it to recommend it personally.
I wouldn't go for Apache Spark since you would need to user Apache Spark Streaming and you will need to learn about how to configure the buffering window correctly for your use case.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
What costs less money when deployed on Amazon Cloud Node.js or Java Web Services?
Or when it does matter. We take into consideration only one way traffic (to server) for many clients.
They're both going to cost roughly the same in terms of hosting costs. In terms of development costs, however, things might be different:
Node is just Javascript -- it has a huge ecosystem and lots of new developers are using it -- since it's quite 'hip', it's easier to find people to hop onto new projects.
Java is old school and has been around forever, there are tons of 'senior' guys you can hire (for good $$).
Node is quite a bit faster to develop with. If you're building a small application, you might spend much less time developing it with Node than Java.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I would like to become a Hadoop administrator. I have a copy of the book 'Hadoop Operations' and i would like to get my hands dirty with setups et al.
So here's the question: Should i invest in a physical server for practice? or is it all done in the cloud?
Don't invest in a physical server, unless you're sure (and I mean SURE) you want to spend hundreds of CPU-hours in practical exercises. A more cost-effective option may be to get an account with a IaaS provider (such as Amazon), and experiment with virtual machines. You can turn off unneeded VMs when not doing exercises, so your costs could be a lot smaller. Plus you can get many VMs for short periods of time without huge upfront investments.
Some of the most challenging aspects of administering Hadoop is dealing with large clusters and clusters that are highly utilized. Unfortunately this means that there is only so much you can learn on your own, as both of those scenarios can be very expensive and time-consuming to set up. So don't try going too deep on your toy cluster, instead get familiar with the basics & configuration options and then try to find a job, or a project where you could join an existing ops team.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
I would like to know why we prefer to make web servers multi-threaded
instead of make it multi-process web servers ....
Is it because of legacy issues.....
I would like to hear practical reasons as well as theoretical reasons
On *nix, to start up a process you need duplicate all the resources of the parent process. All the parents file descriptors are dup'ed, for example, and a new memory space is created to contain the new process. When the process terminates everything has to be torn down.
A thread, on the other hand, is essentially just a stack. Very quick to start and stop.
Early web servers didn't use threads for a simple reason: they weren't implemented yet.
Threads are usually cheaper than processes.