I'm trying to find the best way to run a Task from a dedicated background thread.
The context of usage is consuming from a Kafka topic and raising an async event handler to handle the ConsumeResult<TKey, TValue> instance.
A Kafka Consumer (the consumer instance below) blocks the thread until a message is consumed or the CancellationToken it is passed has been cancelled.
consumeThread = new Thread(Consume)
{
Name = "Kafka Consumer Thread",
IsBackground = true,
};
This is the implementation of the Consume method I came up with, which is started by the dedicated thread above:
private void Consume(object _)
{
try
{
while (!cancellationTokenSource.IsCancellationRequested)
{
var consumeResult = consumer.Consume(cancellationTokenSource.Token);
var consumeResultEventArgs = new ConsumeResultReceivedEventArgs<TKey, TValue>(
consumer, consumeResult, cancellationTokenSource.Token);
_ = Task.Run(async () =>
{
if (onConsumeResultReceived is null) continue;
var handlerInstances = onConsumeResultReceived.GetInvocationList();
foreach (ConsumeResultReceivedEventHandler<TKey, TValue> handlerInstance in handlerInstances)
{
if (cancellationTokenSource.IsCancellationRequested) return;
await handlerInstance(this, consumeResultEventArgs).ConfigureAwait(false);
}
}, cancellationTokenSource.Token);
}
}
catch (OperationCanceledException)
{
}
catch (ThreadInterruptedException)
{
}
catch (ThreadAbortException)
{
// Aborting a thread is not implemented in .NET Core.
}
}
I'm not sure this is the recommened way to run a Task from a dedicated Thread, so any advice would be very much appreciated.
It's not clear to me why you need a dedicated thread at all. The code as it currently stands starts a thread and then that thread blocks for consumption and then raises the event handler on a thread pool thread.
The _ = Task.Run idiom is a "fire and forget", which is dangerous in the sense that it will silently swallow any exceptions from your event raising code or event handlers.
I'd recommend replacing Thread with Task.Run, and just raising the event handlers directly:
consumeTask = Task.Run(ConsumeAsync);
private async Task ConsumeAsync()
{
while (true)
{
var consumeResult = consumer.Consume(cancellationTokenSource.Token);
var consumeResultEventArgs = new ConsumeResultReceivedEventArgs<TKey, TValue>(
consumer, consumeResult, cancellationTokenSource.Token);
if (onConsumeResultReceived is null) continue;
var handlerInstances = onConsumeResultReceived.GetInvocationList();
foreach (ConsumeResultReceivedEventHandler<TKey, TValue> handlerInstance in handlerInstances)
{
if (cancellationTokenSource.IsCancellationRequested) return;
await handlerInstance(this, consumeResultEventArgs).ConfigureAwait(false);
}
}
}
Related
Let's say there is a library that handles events asynchronously, e.g. UDP broadcasting. I would like to be able to pass a delegate to this library and make sure that delegate is executed in the thread where it was defined.
public void Run(Action<string> action)
{
var PORT = 10000;
var udpClient = new UdpClient();
// Start listener
udpClient.Client.Bind(new IPEndPoint(IPAddress.Any, PORT));
var endpoint = new IPEndPoint(0, 0);
//var scheduler = TaskScheduler.Current; // No exception but executed in another thread
var scheduler = TaskScheduler.FromCurrentSynchronizationContext(); // Raises exception
Task.Factory.StartNew(() =>
{
while (true)
{
action(Encoding.UTF8.GetString(udpClient.Receive(ref endpoint)));
}
}, CancellationToken.None, TaskCreationOptions.None, scheduler);
// Start sender
var data = Encoding.UTF8.GetBytes("PING");
udpClient.Send(data, data.Length, "255.255.255.255", PORT);
var aTimer = new System.Timers.Timer();
aTimer.Elapsed += new ElapsedEventHandler(OnTimer);
aTimer.Interval = 1000;
aTimer.Enabled = true;
void OnTimer(object source, ElapsedEventArgs e)
{
var data = Encoding.UTF8.GetBytes("PONG");
udpClient.Send(data, data.Length, "255.255.255.255", PORT);
}
}
Then I check in which thread my delegate was executed.
Console.WriteLine("Start on Thread : " + Thread.CurrentThread.ManagedThreadId);
new Beacon().Run(response =>
{
Console.WriteLine("Continue on Thread : " + Thread.CurrentThread.ManagedThreadId);
});
If I use TaskScheduler.Current nothing changes, the delegate is executed in another thread. If I use TaskScheduler.FromCurrentSynchronizationContext, I get an exception below.
The current SynchronizationContext may not be used as a TaskScheduler
Questions
Why I can't use synchronization context in .NET core or how to use it properly?
Is there a better way of marshaling delegate call to the main thread?
Are BackgroundWorker and EventLoopScheduler obsolete now?
I have situation where i have to Use Task.Run In my ForEach loop
Requirement:
I'm going to be forced to manually kill thread
I have button where i can start and stop this Thread or Task.Run in For loop.
Problem
My problem is when i start the Task.Run method Its running but when i try to stop with using CancellationTokenSource or runningTaskThread.Abort(); it will not kill. its just stop when i start new Task.Run at that time it run with old thread so it become multiple thread every start process.
Code:
Below is my code for start Thread
var messages = rootObject.MultiQData.Messages.Where(m => m.TimeStamp > DateTime.Now).OrderBy(x => x.TimeStamp).ToList();
//Simulate MultiQ file in BackGroud
if (messages.Count > 0)
{
cancellationTokenSource = new CancellationTokenSource();
cancellationToken = cancellationTokenSource.Token;
Task.Factory.StartNew(
() =>
{
runningTaskThread = Thread.CurrentThread;
messages.ForEach(
m => SetUpTimer(m, rootObject.MultiQData.Connection.FleetNo));
}, cancellationToken);
}
For stop Task.Run
if (cancellationTokenSource != null)
{
if (cancellationToken.IsCancellationRequested)
return;
else
cancellationTokenSource.Cancel();
}
I have also use Thread with Thread.Abort but it is not working
Please Help to solve this issue
I got solution using timer.Stop(),timer.Dispose(). On creation of Thread i am calling SetUpTimer and this SetupTimer i have created multiple timer.
So on call of stop thread i have dispose timer and its work for me
For reference see below code
private void SetUpTimer(Message message, string fleetNo)
{
var ts = new MessageTimer();
var interval = (message.TimeStamp - DateTime.Now).TotalMilliseconds;
interval = interval <= 0 ? 100 : interval;
ts.MessageWrapper = new MessageWrapper(message, fleetNo);
ts.Interval = interval;
ts.Elapsed += ts_Elapsed;
ts.Start();
//Add timer in to the lost for disposing timer at time of stop Simulation
lsTimers.Add(ts);
}
private void StopTask()
{
try
{
// Attempt to cancel the task politely
if (cancellationTokenSource != null)
{
if (cancellationToken.IsCancellationRequested)
return;
else
cancellationTokenSource.Cancel();
}
//Stop All Timer
foreach (var timer in lsTimers)
{
timer.Stop();
timer.Dispose();
}
}
catch (Exception ex)
{
errorLogger.Error("Error while Stop simulation :", ex);
}
}
Sample code here
public override void Run()
{
while (true)
{
IAsyncResult result = CUDClient.BeginReceive(TimeSpan.FromSeconds(10), OnMessageReceive, CUDClient);
Thread.Sleep(10000);
}
}
I have tested this Azure worker role. I kept 100 messages in the Service bus Queue. It's doing entities updates as a operation(Entity framework). It took 15 minutes to process all the queues and looks like taking longer time. Any suggestion to improve this?
Thanks in Advance
Actually Service Bus is very fast enough in my experience. What wrong with you is "Thread.Sleep(10000)";
Sleeping 10 sec for each message.
For 100 messages 100*10 = 10000 seconds = 16.67 minutes
So this is a problem for the delay...
Solution:
Dont use Thread.Sleep(10000); (Its not suitable for BeginReceive, only suitable for Receive)
public override void Run() //This should not be a Thread...If its a thread then your thread will terminate after receiving your first message
{
IAsyncResult result = CUDClient.BeginReceive(**TimeSpan.MaxValue**, OnMessageReceive, CUDClient);
}
//Function OnMessageReceive
{
//Process the Message
**IAsyncResult result = CUDClient.BeginReceive(TimeSpan.MaxValue, OnMessageReceive, CUDClient);**
}
using TimeSpan.MaxValue your connection to the SB will be preserved for longtime. so no frequent null message(less cost)...
Try using XecMe Parallel task for processing the message reading.
XecMe # xecme.codeplex.com
Try this one...
//Somefunction
IAsyncResult result = CUDClient.BeginReceive(OnMessageReceive, CUDClient);
while (true)
Thread.Sleep(1000); //In case you are using thread
//Somefunction End
public static void OnMessageReceive(IAsyncResult result)
{
CUDClient.BeginReceive(OnMessageReceive, CUDClient);
SubscriptionClient queueClient = (SubscriptionClient)result.AsyncState;
IBusinessLogicProvider Obj;
try
{
//Receive the message with the EndReceive call
BrokeredMessage receivedmsg = queueClient.EndReceive(result);
//receivedmsg = CUDClient.Receive();
if (receivedmsg != null)
{
switch (receivedmsg.ContentType)
{
case "Project":
Obj = new ProjectsBL();
Obj.HandleMessage(receivedmsg);
receivedmsg.BeginComplete(OnMessageComplete, receivedmsg);
break;
}
}
}
}
I tried this.
while (true)
{
//read all topic messages in sequential way....
IAsyncResult result = CUDClient.BeginReceive(OnMessageReceive, CUDClient);
Thread.Sleep(1000);
}
public static void OnMessageReceive(IAsyncResult result)
{
SubscriptionClient queueClient = (SubscriptionClient)result.AsyncState;
IBusinessLogicProvider Obj;
try
{
//Receive the message with the EndReceive call
BrokeredMessage receivedmsg = queueClient.EndReceive(result);
//receivedmsg = CUDClient.Receive();
if (receivedmsg != null)
{
switch (receivedmsg.ContentType)
{
case "Project":
Obj = new ProjectsBL();
Obj.HandleMessage(receivedmsg);
receivedmsg.BeginComplete(OnMessageComplete, receivedmsg);
break;
}
}
}
}
It processed all the 100 messages in 1 minute(00:01:02) . A lot better than previous one.
I have a timer to verify one condition every time and show pop up form only once if the condition is verified. I want to verify in parallel all instances, so i used parallel.for, but i have this error "Cross-thread operation not valid: Control 'CameraViewVS' accessed from a thread other than the thread it was created on." in line " frm.WindowState = FormWindowState.Normal;"
this is my code:
public void timer1_Tick(object source, EventArgs e)
{
Parallel.For(0, nbre, l =>
{
cameraInstanceList[l].Start();
if (cameraInstanceList[l].MoveDetection == true)
{
//show the the form S once
foreach (Form S in Application.OpenForms)
{
var frm = S as Formes.CameraViewVS;
if (frm != null && frm.IP == cameraInstanceList[l].adresse)
{
cameraInstanceList[l].MoveDetection = false;
frm.WindowState = FormWindowState.Normal;
frm.Activate();
return;
}
}
f1 = new Formes.CameraViewVS(cameraInstanceList[l],
adresseIPArray[l]);
f1.Show(this);
}
}
);
Most properties on WinForm object instances need to be accessed from the thread that they were created on. You can use the Control.InvokeRequired property to determine if you need to use the control (or form) Invoke method to execute the code on the UI thread.
It is also a good practise to create most WinForm controls on the main UI thread, and not on any thread pool threads. In WinForms applications, you can use the SynchronizationContext to ensure some code, such as creating a form, is called on the UI thread.
EDIT: changed so that the method doesn't return after movement detected.
public void timer1_Tick(object source, EventArgs e)
{
// assume this is being called on the UI thread, and save the thread synchronization context
var uiContext = SynchronizationContext.Current;
Parallel.For(0, nbre, l =>
{
while (true)
{
Thread.Sleep(250); // <--- sleep for 250 ms to avoid "busy" wait
cameraInstanceList[l].Start();
if (cameraInstanceList[l].MoveDetection == true)
{
// capture instances used in closures below
var cameraInstance = cameraInstanceList[l];
var ipAdresse = adresseIPArray[l];
//show the the form S once
foreach (Form S in Application.OpenForms)
{
var frm = S as Formes.CameraViewVS;
if (frm != null)
{
// create delegate to be invoked on form's UI thread.
var action = new Action(() =>
{
if (frm.IP == cameraInstance.adresse)
{
cameraInstance.MoveDetection = false;
frm.WindowState = FormWindowState.Normal;
frm.Activate();
}
};
if (frm.InvokeRequired)
frm.Invoke(action);
else
action();
continue; // <--- go back to the top of the while loop
// and wait for next detection
}
}
// create delegate to create new form on UI thread.
var createNewFormCallback = new SendOrPostCallback((o) =>
{
f1 = new Formes.CameraViewVS(cameraInstance, ipAdresse);
f1.Show(this);
};
// and invoke the delegate on the ui thread
uiContext.Send(createNewFormCallback, null);
}
}
}
);
}
Thomas is very close to right answer ,Because Every Control runs in a different thread .You should just write a code for context-switching of resources which is being used by Controls
Thread ..Don't worry you have a lot of facility for this in c sharp.Just use BeginInvoke and Invoke and i hope you would be able to resolve your problem.Write this in place of your old code block ..
var action = new Action(() =>
{
if (frm.IP == cameraInstance.adresse)
{
cameraInstance.MoveDetection = false;
frm.WindowState = FormWindowState.Normal;
frm.Activate();
}
};
if (frm.InvokeRequired)
frm.BeginInvoke(action);
else
frm.Invoke(action);
I'm experiencing an issue managing threads on .Net 4.0 C#, and my knowledge of threads is not sufficient to solve it, so I've post it here expecting that somebody could give me some piece of advise please.
The scenario is the following:
We have a Windows service on C# framework 4.0 that (1)connects via socket to a server to get a .PCM file, (2)then convert it to a .WAV file, (3)send it via Email - SMTP and finally (4)notify the initial server that it was successfully sent.
The server where the service had been installed has 8 processors and 8 GB or RAM.
To allow multiprocessing I've built the service with 4 threads, each one of them performs each task I mentioned previously.
On the code, I have classes and methods for each task, so I create threads and invoke methods as follows:
Thread eachThread = new Thread(object.PerformTask);
Inside each method I'm having a While that checks if the connection of the socket is alive and continue fetching data or processing data depending on their porpuse.
while (_socket.Connected){
//perform task
}
The problem is that as more services are being installed (the same windows service is replicated and pointed between two endpoints on the server to get the files via socket) the CPU consumption increases dramatically, each service continues running and processing files but there is a moment were the CPU consumption is too high that the server just collapse.
The question is: what would you suggest me to handle this scenario, I mean in general terms what could be a good approach of handling this highly demanded processing tasks to avoid the server to collapse in CPU consumption?
Thanks.
PS.: If anybody needs more details on the scenario, please let me know.
Edit 1
With CPU collapse I mean that the server gets too slow that we have to restart it.
Edit 2
Here I post some part of the code so you can get an idea of how it's programmed:
while(true){
//starting the service
try
{
IPEndPoint endPoint = conn.SettingConnection();
string id = _objProp.Parametros.IdApp;
using (socket = conn.Connect(endPoint))
{
while (!socket.Connected)
{
_log.SetLog("INFO", "Conectando socket...");
socket = conn.Connect(endPoint);
//if the connection failed, wait 5 seconds for a new try.
if (!socket.Connected)
{
Thread.Sleep(5000);
}
}
proInThread = new Thread(proIn.ThreadRun);
conInThread = new Thread(conIn.ThreadRun);
conOutThread = new Thread(conOut.ThreadRun);
proInThread.Start();
conInThread.Start();
conOutThread.Start();
proInThread.Join();
conInThread.Join();
conOutThread.Join();
}
}
}
Edit 3
Thread 1
while (_socket.Connected)
{
try
{
var conn = new AppConection(ref _objPropiedades);
try
{
string message = conn.ReceiveMessage(_socket);
lock (((ICollection)_queue).SyncRoot)
{
_queue.Enqueue(message);
_syncEvents.NewItemEvent.Set();
_syncEvents.NewResetEvent.Set();
}
lock (((ICollection)_total_rec).SyncRoot)
{
_total_rec.Add("1");
}
}
catch (SocketException ex)
{
//log exception
}
catch (IndexOutOfRangeException ex)
{
//log exception
}
catch (Exception ex)
{
//log exception
}
//message received
}
catch (Exception ex)
{
//logging error
}
}
//release ANY instance that could be using memory
_socket.Dispose();
log = null;
Thread 2
while (_socket.Connected)
{
try{
_syncEvents.NewItemEventOut.WaitOne();
if (_socket.Connected)
{
lock (((ICollection)_queue).SyncRoot)
{
total_queue = _queue.Count();
}
int i = 0;
while (i < total_queue)
{
//EMail Emails;
string mail = "";
lock (((ICollection)_queue).SyncRoot)
{
mail = _queue.Dequeue();
i = i + 1;
}
try
{
conn.SendMessage(_socket, mail);
_syncEvents.NewResetEvent.Set();
}
catch (SocketException ex)
{
//log exception
}
}
}
else
{
//log exception
_syncEvents.NewAbortEvent.Set();
Thread.CurrentThread.Abort();
}
}
catch (InvalidOperationException e)
{
//log exception
}
catch (Exception e)
{
//log exception
}
}
//release ANY instance that could be using memory
_socket.Dispose();
conn = null;
log = null;
Thread 3
while (_socket.Connected)
{
int total_queue = 0;
try
{
_syncEvents.NewItemEvent.WaitOne();
lock (((ICollection) _queue).SyncRoot)
{
total_queue = _queue.Count();
}
int i = 0;
while (i < total_queue)
{
if (mgthreads.GetThreatdAct() <
mgthreads.GetMaxThread())
{
string message = "";
lock (((ICollection) _queue).SyncRoot)
{
message = _queue.Dequeue();
i = i + 1;
}
count++;
lock (((ICollection) _queueO).SyncRoot)
{
app.SetParameters(_socket, _id,
message, _queueO, _syncEvents,
_total_Env, _total_err);
}
Thread producerThread = new
Thread(app.ThreadJob) { Name =
"ProducerThread_" +
DateTime.Now.ToString("ddMMyyyyhhmmss"),
Priority = ThreadPriority.AboveNormal
};
producerThread.Start();
producerThread.Join();
mgthreads.IncThreatdAct(producerThread);
}
mgthreads.DecThreatdAct();
}
mgthreads.DecThreatdAct();
}
catch (InvalidOperationException e)
{
}
catch (Exception e)
{
}
Thread.Sleep(500);
}
//release ANY instance that could be using memory
_socket.Dispose();
app = null;
log = null;
mgthreads = null;
Thread 4
MessageVO mesVo =
fac.ParseMessageXml(_message);
I would lower the thread priority and have all threads pass through a Semaphore that limits concurrency to Environment.ProcessorCount. This not a perfect solution but it sounds like it is enough in this case and an easy fix.
Edit: Thinking about it, you have to fold the 10 services into one single process because otherwise you won't have centralized control about the threads that are running. If you have 10 independent processes they cannot coordinate.
There should normally be no collapse because of high cpu usage. While any of the threads is waiting for something remote to happen (for instance for the remote server to response to the request), that thread uses no cpu resource. But while it is actually doing something, it uses cpu accordingly. In the Task you mentioned, there is no inherent high cpu usage (as the saving of PCM file as WAV requires no complex algorithm), so the high cpu usage seems to be a sign of an error in programming.