Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I need to update multiple processes with several different pieces of data, at varying rates, but as fast as 10 Hz. I don't want the receiving processes to have to actively get this data, but rather have it pushed to them, so that they only have to do anything about the new data when there actually is any (no polling).
I'm only sending probably a few bytes of data to each process. The data being transmitted will not likely need to be stored permanently, at least not before being received and processed by the recipients. Also, no data is updated less frequently than once every few seconds, so receiver crashes are not a concern (once a crashed receiver recovers, it can just wait for the next update).
I've looked at unix domain sockets and UDP and a little bit at pipes and shared memory, but it seems that they don't quite fit what I'm trying to do:
Domain sockets require the sender to send a separate message to each recipient (i.e., no broadcasting/multicasting)
Shared memory has the disadvantage of having the clients check that data has been updated (unless there's a mechanism I'm not familiar with that can notify them)
UDP doesn't guarantee that the messages will arrive (maybe not likely a problem for communication on the same computer?), and I have some concern about the overhead from the network stack (which domain sockets doesn't have)
The concern about TCP (and other protocols that support inter-device communication) is that there is functionality that's not needed for interprocess communication on a single device, and that that could create unnecessary overhead.
Any suggestions and direction to references and resources are appreciated.
Have you looked at zeroMQ? It is a lightweight messaging library that supports various push/pull access patterns over several transport mechanisms.
One option is to write flat files or SQLite database on the same box.
And have another control file with a process shared mutex, condition variable and record count mapped into memory of the publisher and subscribers. This is the notification mechanism.
This way you would have full history of records in the file or the database which makes it easy to replay records, debug and recover subscribers from crashes.
The publisher would:
Map the control file into memory.
Add new records to the file or the database.
Lock the mutex.
Update the record count.
notify_all on the condition variable.
Unlock the mutex.
The subscribers would:
Map the control file into memory.
Lock the mutex.
Wait on the condition variable till there are new records (each subscriber maintains its own count of already processed records).
Unlock the mutex.
Process the new records from the file or the database.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I would like to ask what is the best practice for Azure Service Bus message TTL (time to live) option - https://learn.microsoft.com/en-us/azure/service-bus-messaging/message-expiration.
We use Azure Service Bus to import data from one system to another, amount of records is a couple of millions.
Briefly saying, this option tells ASB how much time a message can stay in a queue or a topic before it moved to dead letter queue(if it is configured) - https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-dead-letter-queues#moving-messages-to-the-dlq.
Even so, I cannot find how TTL value impacts on ASB throughput and performance. What is difference between 5 minutes, 1 hour and 20 hours set for TTL in terms of ASB queue/topic performance?
Thank you in advance
Time to live property is used to set the expiration time window for messages in Service Bus.
Based the time configured for TTL, the messages either moved to dead-letter or lost from the Queue. The usage of this property may differ based on the use cases.
For example, if I am sure that my system will not go down and will pick the messages as soon as it is en-queued, I will configure the TTL to very minimal time window say 1 minute (helps to verify the system is working fine by monitoring the dead-letter length of the Queue). If my system is not reliable or the system runs only once a day to process the messages, then I should have a higher value for this property, so that the messages will be available in the Queue for a longer time, letting the system to process.
Coming to the performance, there will not be much lack in the performance in the Queue due to the higher values of TTL.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 5 years ago.
Improve this question
i want to know the purpose of the hostname in EventProcessHost and how to set partition in consumer side . right now i am able to get data from consumer group but all partitions goes to the output stream .
Questions:
1.How to set partition via code java.
2.Use of hostname in EventProcessHost
3.example for multi consumer each has it's own partition in java code.
I highly appreciate any help.
There is a complete Java example, see the docs
You don't need to set a partition when you use an EventProcessHost. Instead, each instance will lease a partition it will work on. So if you created the event hub using ,say 4 partitions, you should instantiate 4x EventProcessHost to get better troughput. See the linked docs as well:
This tutorial uses a single instance of EventProcessorHost. To increase throughput, it is recommended that you run multiple instances of EventProcessorHost, preferably on separate machines. This provides redundancy as well. In those cases, the various instances automatically coordinate with each other in order to load balance the received events.
Leases are given out for a specific time only. After that another receiver can take over that lease. If you give it a while you should notice all instances will retrieve data.
About the hostname:
When receiving events from different machines, it might be useful to specify names for EventProcessorHost instances based on the machines (or roles) in which they are deployed.
In my embedded system running Linux (Ubuntu armhf) I have to communicate between processes.
I'm doing it with TCP sockets. It works great but due the high frequency of my requests I have a very high processor usage (94% average measured whit nmon).
There is a way to lower it using that kind of communication in a more efficient manner?
shared memory and message queues can be used to exchange information between processes. The difference is in how they are used. both have some advantage and disadvantage.
Shared memory
it's an area of storage that can be read and written by more than one process. It provides no inherent synchronization; in other words, it's up to the programmer to ensure that one process doesn't clobber another's data. But it's efficient in terms of throughput: reading and writing are relatively fast operations.
A message queue is a one-way pipe:
one process writes to the queue, and another reads the data in the order it was written until an end-of-data condition occurs. When the queue is created, the message size (bytes per message, usually fairly small) and queue length (maximum number of pending messages) are set. Access is slower than shared memory because each read/write operation is typically a single message. But the queue guarantees that each operation will either processes an entire message successfully or fail without altering the queue. So the writer can never fail after writing only a partial message, and the reader will either retrieve a complete message or nothing at all.
If you wish to stick with your basic architecture, you can switch from TCP sockets to Unix domain sockets (AF_UNIX/AF_LOCAL). Since it's a strictly local protocol, it doesn't have the overhead of TCP.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 9 years ago.
Background:File upload
My scenario: I need to upload larget amount of files to Azure blob, it maybe 10,000 to 100,000 files.
each file sized 10KB-50KB.
I used solution in discussion above, I see files quickly uploading, however, I have so many files so that I found that my application leads to very high CPU usage, always 100%...what's worse is that next step I need to run hundreds of processes, I mean I need to run hundreds of processes, each process needs to upload 10,000 files or more. I have tested it until now, unfoutuntely, I see many weird problems, like exception "connection is closed" etc...
Do you guys have ideas to decrease CPU usage of Tasks...
The problem here that I see is that you spin so many threads that you will overload the machine resource-wise by simply having to manage all the queued threads even if technically they don't try to run all at the same time. They will take RAM and in the absence of RAM, will use SWAP space - which will then bring down the machine in a blaze of non-glory.
I would use a queue (azure queue, msmq, Systems.Collections.Queue) to queue up all the objects, use a limited number of Threads that will process the file using Async methods described in your background link and then the thread is done executing check for the next item in the queue and process that one. My recommendation, is to use a non-memory queue - I will explain below. The main benefit is to save ram so that your software doesn't crash or slow down because the queue is too big.
Parallel.ForEach and such are great time savers but can really ruin the performance of your machine when you are dealing with a lot of items - and if the machine ever goes down then you cannot recover from it unless you have a checkpoint somewhere. Using a persistent queue will allow you to properly manage not just machine resources but also where you are in the process.
You can then scale this across multiple machines by using a persistent queue like MSMQ or if in the cloud, Azure queues. If you use a service that checks how big the azure queue is, you can even bring up instances from time to time to reduce the load and then terminate the extra instances.
This is the scenario that I would implement:
Use the standard ThreadPool size
When you detect a new file/batch - submit to the queue
Have an event fire every time you insert a new item in the queue (if memory queue)
Have a process check the queue (if persistent queue)
If a new item is in the queue, first check if you have space in the ThreadPool ignore if you don't (use a PEEK approach so you don't remove the item) - Add a worker to ThreadPool if there is space
The process thread (which runs under ThreadPool) should execute and then check if you have another item in the queue - if not - the thread dies, which is fine
Using this method, you could run this with 1 machine, or 50,000 - provided you use a persistent queue for more than 1 machine you won't have any problems. Of course, make sure you do a proper job of testing for duplicate items if you are using Azure Queues as you could be handed a queued item that's been given to another machine.
It's a simple approach, scalable and can if using a persistent queue (even a file system) recover from failure. It will not however overload the machine by abusing the resources by forcing it to manage a ThreadPool with 1 million+ items.
Hope this helps
Simply, go for a thread pool implementation, let there be 20 threads (because that's probably around what your network bandwidth can handle simultaneously), each upload is going to take 2-3 seconds, it is going to take around 4-5 hours, which is acceptable. Make sure that you don't share storage or container instances between uploads, that might cause the "connection is closed" errors.
I'm a Microsoft Technical Evangelist and I have developed a sample and free tool (no support/no guarantee) to help in these scenarios.
The binaries and source-code are available here: https://blobtransferutility.codeplex.com/
The Blob Transfer Utility is a GUI tool to upload and download thousands of small/large files to/from Windows Azure Blob Storage.
Features:
Create batches to upload/download
Set the Content-Type
Transfer files in parallel
Split large files in smaller parts that are transferred in parallel
The 1st and 3rd feature is the answer to your problem.
You can learn from the sample code how I did it, or you can simply run the tool and do what you need to do.
I know the term "Load Balancing" can be very broad, but the subject I'm trying to explain is more specific, and I don't know the proper terminology. What I'm building is a set of Server/Client applications. The server needs to be able to handle a massive amount of data transfer, as well as client connections, so I started looking into multi-threading.
There's essentially 3 ways I can see implementing any sort of threading for the server...
One thread handling all requests (defeats the purpose of a thread if 500 clients are logged in)
One thread per user (which is risky to create 1 thread for each of the 500 clients)
Pool of threads which divide the work evenly for any number of clients (What I'm seeking)
The third one is what I'd like to know. This consists of a setup like this:
Maximum 250 threads running at once
500 clients will not create 500 threads, but share the 250
A Queue of requests will be pending to be passed into a thread
A thread is not tied down to a client, and vice-versa
Server decides which thread to send a request to based on activity (load balance)
I'm currently not seeking any code quite yet, but information on how a setup like this works, and preferably a tutorial to accomplish this in Delphi (XE2). Even a proper word or name to put on this subject would be sufficient so I can do the searching myself.
EDIT
I found it necessary to explain a little about what this will be used for. I will be streaming both commands and images, there will be a double-socket setup where there's one "Main Command Socket" and another "Add-on Image Streaming Socket". So really one connection is 2 socket connections.
Each connection to the server's main socket creates (or re-uses) an object representing all the data needed for that connection, including threads, images, settings, etc. For every connection to the main socket, a streaming socket is also connected. It's not always streaming images, but the command socket is always ready.
The point is that I already have a threading mechanism in my current setup (1 thread per session object) and I'd like to shift that over to a pool-like multithreading environment. The two connections together require a higher-level control over these threads, and I can't rely on something like Indy to keep these synchronized, I'd rather know how things are working than to learn to trust something else to do the work for me.
IOCP server. It's the only high-performance solution. It's essentially asynchronous in user mode, ('overlapped I/O in M$-speak), a pool of threads issue WSARecv, WSASend, AcceptEx calls and then all wait on an IOCP queue for completion records. When something useful happens, a kernel threadpool performs the actual I/O and then queues up the completion records.
You need at least a buffer class and socket class, (and probably others for high-performance - objectPool and pooledObject classes so you can make socket and buffer pools).
500 threads may not be an issue on a server class computer. A blocking TCP thread doesn't do much while it's waiting for the server to respond.
There's nothing stopping you from creating some type of work queue on the server side, served by a limited size pool of threads. A simple thread-safe TList works great as a queue, and you can easily put a message handler on each server thread for notifications.
Still, at some point you may have too much work, or too many threads, for the server to handle. This is usually handled by adding another application server.
To ensure scalability, code for the idea of multiple servers, and you can keep scaling by adding hardware.
There may be some reason to limit the number of actual work threads, such as limiting lock contention on a database, or something similar, however, in general, you distribute work by adding threads, and let the hardware (CPU, redirector, switch, NAS, etc.) schedule the load.
Your implementation is completely tied to the communications components you use. If you use Indy, or anything based on Indy, it is one thread per connection - period! There is no way to change this. Indy will scale to 100's of connections, but not 1000's. Your best hope to use thread pools with your communications components is IOCP, but here your choices are limited by the lack of third-party components. I have done all the investigation before and you can see my question at stackoverflow.com/questions/7150093/scalable-delphi-tcp-server-implementation.
I have a fully working distributed development framework (threading and comms) that has been used in production for over 3 years now across more than a half-dozen separate systems and basically covers everything you have asked so far. The code can be found on the web as well.