Npgsql LISTEN Thread Crashing Server - multithreading

I have a long running PostgreSQL function. For simplicity, something like this:
CREATE FUNCTION pg_function()
RETURNS void
AS
$$
BEGIN
PERFORM pg_notify('channel1', 'pg_function() started.');
PERFORM pg_sleep(5);------PERFORM task1();-----------------------------------------
PERFORM pg_notify('channel1', 'Task1 payload.');
PERFORM pg_sleep(5);------PERFORM task2();-----------------------------------------
PERFORM pg_notify('channel1', 'Task2 payload.');
PERFORM pg_sleep(5);------PERFORM task3();-----------------------------------------
PERFORM pg_notify('channel1', 'Task3 payload.');
PERFORM pg_notify('channel1', 'pg_function() completed.');
END;
$$
LANGUAGE "plpgsql";
On C#, I have:
public bool listening;
public void PgFunction()
{
this.listening = true;
ThreadStart listenerStart = delegate
{
using (NpgsqlConnection connection = new NpgsqlConnection(this.connectionString))
{
connection.Open();
connection.Notification += Listen;
using (NpgsqlCommand listenChannel1 = new NpgsqlCommand("LISTEN channel1;", connection))
{
listenChannel1.ExecuteNonQuery();
}
while (this.listening)
{
using (NpgsqlCommand pollingCommand = new NpgsqlCommand("SELECT 0;", connection))
{
pollingCommand.ExecuteNonQuery();
}
Thread.Sleep(5000);
}
}
};
Thread listenerThread = new Thread(listenerStart) { IsBackground = false };
listenerThread.Start();
ThreadStart pgFunctionThreadStart = () => ExecuteNonQuery(new NpgsqlCommand("SELECT pg_function();"));
pgFunctionThreadStart += () =>
{
Thread.Sleep(5000);
this.listening = false;
};
Thread pgFunctionThread = new Thread(pgFunctionThreadStart) { IsBackground = true };
pgFunctionThread.Start();
}
private void Listen(object sender, NpgsqlNotificationEventArgs e)
{
string payload = e.AdditionalInformation;
//SignalR stuff here
}
When I run the program debugging, this code works okay. But when it is tested on IIS server or browsed with Visual Studio 2013 integrated IIS, the application crashes. Since I have very little knowledge of tasks and threads in C#, I would like to know what I am doing wrong here? Please advise.
Edit
Upon debugging it again, I came with a NpgsqlException, which happens to happen once in a while:
Additional information: Cannot write to a BufferedStream while the read buffer is not empty if
the underlying stream is not seekable. Ensure that the stream underlying this BufferedStream
can seek or avoid interleaving read and write operations on this BufferedStream.

Related

.NET 4.7.2 => c-sharp => UI async/await

I'm trying to load a content of a text file into text box asynchronously.
Therefore my goal is to not block the UI-Thread.
To check that, the code gives feedback with Console.Writeline("..."+Thread.CurrentThread.ManagedThreadId)
My first intention shows the code below, which will give me Thread 1 in both Console.WriteLines().
That means UI-Thread is blocked, right?
private async void cmdProgrammLaden_Click(object sender, EventArgs e)
{
Console.WriteLine("Button-Thread-ID: " + Thread.CurrentThread.ManagedThreadId);
this.txtSendData.Text = await DncProgrammLadenAsync();
}
async Task<string> DncProgrammLadenAsync()
{
string path = String.Empty;
string content = String.Empty;
using (OpenFileDialog openFileDialog = new OpenFileDialog())
{
openFileDialog.InitialDirectory = #"c:\Test\";
openFileDialog.Filter = "Textfile (*.txt)|*.txt";
openFileDialog.FilterIndex = 0;
openFileDialog.RestoreDirectory = true;
if (openFileDialog.ShowDialog() == DialogResult.OK)
{
path = openFileDialog.FileName;
var filestream = openFileDialog.OpenFile();
using (StreamReader reader = new StreamReader(filestream))
{
content = await reader.ReadToEndAsync();
Console.WriteLine("Task-Thread-ID: " + Thread.CurrentThread.ManagedThreadId);
//Thread.Sleep(3000);
}
}
return content;
}
}
Then I tried to rework this code like *(short version)*
Task.Run(() =>
{
string content = String.Empty;
using (StreamReader reader = new StreamReader(#"c:\Test\File C.txt"))
{
content = reader.ReadToEnd();
Console.WriteLine("Task-Thread-ID: " + Thread.CurrentThread.ManagedThreadId);
}
});
Then I get different Thread IDs.
But this seems not right to me.
What do I not understand correctly?
That means UI-Thread is blocked, right?
No. By default, await captures the current context and resumes executing the async method in that context when that await completes. The UI thread is not blocked during the await, but the code resumes executing on the UI thread after the await.
However, file streams are tricky. They are only asynchronous if opened with an asynchronous flag, and I have no idea if OpenFile passes that flag. Asynchronous APIs called on a synchronous file handle just run the operation on a thread pool thread (I think).

NetMQ in Publisher and subscriber not working

I'm trying to implement the NetMQ Pub/Sub Model, but the Subscriber is not receiving any messages. What possibly is wrong here?
private static void ServerTask()
{
using (var context = NetMQContext.Create())
{
using (var socket = context.CreateSubscriberSocket())
{
socket.Bind("tcp://10.120.19.109:5000");
socket.Subscribe(string.Empty);
while (true)
{
Thread.Sleep(100);
string receivedMessage = socket.ReceiveString();
Console.WriteLine("Received: " + receivedMessage);
}
}
}
}
public static void ClientTask()
{
using (NetMQContext ctx = NetMQContext.Create())
{
using (var socket = ctx.CreatePublisherSocket())
{
socket.Connect("tcp://10.120.19.109:5000");
string obj = "hi";
socket.Send(obj);
}
}
}
Both are in different apps.
If you are new to NetMQ I suggest reading the zeromq guide http://zguide.zeromq.org/page:all.
Bottom line is that you are sending the message before the subscriber sent the subscription.
Pubsub in zeromq and NetMQ is like radio, you will only get messages from the moment you start listen.
To simple way to do it (not a real life solution) is to sleep for some time after the connect.
For real life solution I need to understand what are you trying to achieve
issue is like
using (NetMQContext ctx = NetMQContext.Create())
{
using (var publisher = ctx.CreatePushSocket())
{
publisher.Bind("tcp://localhost:5000");
int i = 0;
while (true)
{
try
{
publisher.Send(i.ToString(), dontWait:true);
Console.WriteLine(i.ToString());
}
catch (Exception e)
{
}
finally
{
i++;
}
}
}
Now , this code works. But if I move my while(true) loop outside. and call this code from some other function Which forces push socket and context to be created as new everytime.. this doesnot work.

Error Cross-thread operation not valid: Control 'CameraViewVS' accessed from a thread other than the thread it was created on. parallel.for

I have a timer to verify one condition every time and show pop up form only once if the condition is verified. I want to verify in parallel all instances, so i used parallel.for, but i have this error "Cross-thread operation not valid: Control 'CameraViewVS' accessed from a thread other than the thread it was created on." in line " frm.WindowState = FormWindowState.Normal;"
this is my code:
public void timer1_Tick(object source, EventArgs e)
{
Parallel.For(0, nbre, l =>
{
cameraInstanceList[l].Start();
if (cameraInstanceList[l].MoveDetection == true)
{
//show the the form S once
foreach (Form S in Application.OpenForms)
{
var frm = S as Formes.CameraViewVS;
if (frm != null && frm.IP == cameraInstanceList[l].adresse)
{
cameraInstanceList[l].MoveDetection = false;
frm.WindowState = FormWindowState.Normal;
frm.Activate();
return;
}
}
f1 = new Formes.CameraViewVS(cameraInstanceList[l],
adresseIPArray[l]);
f1.Show(this);
}
}
);
Most properties on WinForm object instances need to be accessed from the thread that they were created on. You can use the Control.InvokeRequired property to determine if you need to use the control (or form) Invoke method to execute the code on the UI thread.
It is also a good practise to create most WinForm controls on the main UI thread, and not on any thread pool threads. In WinForms applications, you can use the SynchronizationContext to ensure some code, such as creating a form, is called on the UI thread.
EDIT: changed so that the method doesn't return after movement detected.
public void timer1_Tick(object source, EventArgs e)
{
// assume this is being called on the UI thread, and save the thread synchronization context
var uiContext = SynchronizationContext.Current;
Parallel.For(0, nbre, l =>
{
while (true)
{
Thread.Sleep(250); // <--- sleep for 250 ms to avoid "busy" wait
cameraInstanceList[l].Start();
if (cameraInstanceList[l].MoveDetection == true)
{
// capture instances used in closures below
var cameraInstance = cameraInstanceList[l];
var ipAdresse = adresseIPArray[l];
//show the the form S once
foreach (Form S in Application.OpenForms)
{
var frm = S as Formes.CameraViewVS;
if (frm != null)
{
// create delegate to be invoked on form's UI thread.
var action = new Action(() =>
{
if (frm.IP == cameraInstance.adresse)
{
cameraInstance.MoveDetection = false;
frm.WindowState = FormWindowState.Normal;
frm.Activate();
}
};
if (frm.InvokeRequired)
frm.Invoke(action);
else
action();
continue; // <--- go back to the top of the while loop
// and wait for next detection
}
}
// create delegate to create new form on UI thread.
var createNewFormCallback = new SendOrPostCallback((o) =>
{
f1 = new Formes.CameraViewVS(cameraInstance, ipAdresse);
f1.Show(this);
};
// and invoke the delegate on the ui thread
uiContext.Send(createNewFormCallback, null);
}
}
}
);
}
Thomas is very close to right answer ,Because Every Control runs in a different thread .You should just write a code for context-switching of resources which is being used by Controls
Thread ..Don't worry you have a lot of facility for this in c sharp.Just use BeginInvoke and Invoke and i hope you would be able to resolve your problem.Write this in place of your old code block ..
var action = new Action(() =>
{
if (frm.IP == cameraInstance.adresse)
{
cameraInstance.MoveDetection = false;
frm.WindowState = FormWindowState.Normal;
frm.Activate();
}
};
if (frm.InvokeRequired)
frm.BeginInvoke(action);
else
frm.Invoke(action);

Handling threads for multiprocessing

I'm experiencing an issue managing threads on .Net 4.0 C#, and my knowledge of threads is not sufficient to solve it, so I've post it here expecting that somebody could give me some piece of advise please.
The scenario is the following:
We have a Windows service on C# framework 4.0 that (1)connects via socket to a server to get a .PCM file, (2)then convert it to a .WAV file, (3)send it via Email - SMTP and finally (4)notify the initial server that it was successfully sent.
The server where the service had been installed has 8 processors and 8 GB or RAM.
To allow multiprocessing I've built the service with 4 threads, each one of them performs each task I mentioned previously.
On the code, I have classes and methods for each task, so I create threads and invoke methods as follows:
Thread eachThread = new Thread(object.PerformTask);
Inside each method I'm having a While that checks if the connection of the socket is alive and continue fetching data or processing data depending on their porpuse.
while (_socket.Connected){
//perform task
}
The problem is that as more services are being installed (the same windows service is replicated and pointed between two endpoints on the server to get the files via socket) the CPU consumption increases dramatically, each service continues running and processing files but there is a moment were the CPU consumption is too high that the server just collapse.
The question is: what would you suggest me to handle this scenario, I mean in general terms what could be a good approach of handling this highly demanded processing tasks to avoid the server to collapse in CPU consumption?
Thanks.
PS.: If anybody needs more details on the scenario, please let me know.
Edit 1
With CPU collapse I mean that the server gets too slow that we have to restart it.
Edit 2
Here I post some part of the code so you can get an idea of how it's programmed:
while(true){
//starting the service
try
{
IPEndPoint endPoint = conn.SettingConnection();
string id = _objProp.Parametros.IdApp;
using (socket = conn.Connect(endPoint))
{
while (!socket.Connected)
{
_log.SetLog("INFO", "Conectando socket...");
socket = conn.Connect(endPoint);
//if the connection failed, wait 5 seconds for a new try.
if (!socket.Connected)
{
Thread.Sleep(5000);
}
}
proInThread = new Thread(proIn.ThreadRun);
conInThread = new Thread(conIn.ThreadRun);
conOutThread = new Thread(conOut.ThreadRun);
proInThread.Start();
conInThread.Start();
conOutThread.Start();
proInThread.Join();
conInThread.Join();
conOutThread.Join();
}
}
}
Edit 3
Thread 1
while (_socket.Connected)
{
try
{
var conn = new AppConection(ref _objPropiedades);
try
{
string message = conn.ReceiveMessage(_socket);
lock (((ICollection)_queue).SyncRoot)
{
_queue.Enqueue(message);
_syncEvents.NewItemEvent.Set();
_syncEvents.NewResetEvent.Set();
}
lock (((ICollection)_total_rec).SyncRoot)
{
_total_rec.Add("1");
}
}
catch (SocketException ex)
{
//log exception
}
catch (IndexOutOfRangeException ex)
{
//log exception
}
catch (Exception ex)
{
//log exception
}
//message received
}
catch (Exception ex)
{
//logging error
}
}
//release ANY instance that could be using memory
_socket.Dispose();
log = null;
Thread 2
while (_socket.Connected)
{
try{
_syncEvents.NewItemEventOut.WaitOne();
if (_socket.Connected)
{
lock (((ICollection)_queue).SyncRoot)
{
total_queue = _queue.Count();
}
int i = 0;
while (i < total_queue)
{
//EMail Emails;
string mail = "";
lock (((ICollection)_queue).SyncRoot)
{
mail = _queue.Dequeue();
i = i + 1;
}
try
{
conn.SendMessage(_socket, mail);
_syncEvents.NewResetEvent.Set();
}
catch (SocketException ex)
{
//log exception
}
}
}
else
{
//log exception
_syncEvents.NewAbortEvent.Set();
Thread.CurrentThread.Abort();
}
}
catch (InvalidOperationException e)
{
//log exception
}
catch (Exception e)
{
//log exception
}
}
//release ANY instance that could be using memory
_socket.Dispose();
conn = null;
log = null;
Thread 3
while (_socket.Connected)
{
int total_queue = 0;
try
{
_syncEvents.NewItemEvent.WaitOne();
lock (((ICollection) _queue).SyncRoot)
{
total_queue = _queue.Count();
}
int i = 0;
while (i < total_queue)
{
if (mgthreads.GetThreatdAct() <
mgthreads.GetMaxThread())
{
string message = "";
lock (((ICollection) _queue).SyncRoot)
{
message = _queue.Dequeue();
i = i + 1;
}
count++;
lock (((ICollection) _queueO).SyncRoot)
{
app.SetParameters(_socket, _id,
message, _queueO, _syncEvents,
_total_Env, _total_err);
}
Thread producerThread = new
Thread(app.ThreadJob) { Name =
"ProducerThread_" +
DateTime.Now.ToString("ddMMyyyyhhmmss"),
Priority = ThreadPriority.AboveNormal
};
producerThread.Start();
producerThread.Join();
mgthreads.IncThreatdAct(producerThread);
}
mgthreads.DecThreatdAct();
}
mgthreads.DecThreatdAct();
}
catch (InvalidOperationException e)
{
}
catch (Exception e)
{
}
Thread.Sleep(500);
}
//release ANY instance that could be using memory
_socket.Dispose();
app = null;
log = null;
mgthreads = null;
Thread 4
MessageVO mesVo =
fac.ParseMessageXml(_message);
I would lower the thread priority and have all threads pass through a Semaphore that limits concurrency to Environment.ProcessorCount. This not a perfect solution but it sounds like it is enough in this case and an easy fix.
Edit: Thinking about it, you have to fold the 10 services into one single process because otherwise you won't have centralized control about the threads that are running. If you have 10 independent processes they cannot coordinate.
There should normally be no collapse because of high cpu usage. While any of the threads is waiting for something remote to happen (for instance for the remote server to response to the request), that thread uses no cpu resource. But while it is actually doing something, it uses cpu accordingly. In the Task you mentioned, there is no inherent high cpu usage (as the saving of PCM file as WAV requires no complex algorithm), so the high cpu usage seems to be a sign of an error in programming.

How to grab value from a thread?

Hi i am trying to grab a value from my threading but it seem work not so find to me course i found that my code structure are unstable enough..here is my code i name my thread class as "clsThreadCount" and below is my implementation
public volatile bool Grab = false;
public volatile int count = 0;
public void Initialization(int i)
{
count = i;
}
public void Play()
{
Grab = false;
_shouldStop = false;
ThreadTest();
}
public void Stop()
{
_shouldStop = true;
workerThread.Join(1);
workerThread.Abort();
}
private void ThreadTest()
{
workerThread = new Thread(DoWork);
workerThread.Start();
while (!workerThread.IsAlive) ;
}
private void DoWork()
{
try
{
while (!_shouldStop)
{
if (Grab)
{
count++;
Grab = false;
}
}
}
catch (Exception)
{
Play();
}
finally
{
}
}
when my program(main menu) are starting to run i will trigger the initialize function at pass the parameter as 7
ObjThreadCount.Initialization(7); // count = 7
ObjThreadCount.Play(); // the thread are running
ObjThreadCount.Grab = true; // the grab equal to true, count++ are trigger
Thread.Sleep(100); // wait awhile
lblResult.Text = ObjThreadCount.count.ToString(); // sometime i can get count++ result (e.g. 8)
ObjThreadCount.Stop(); // thread stop
sometime my program can able to get a right counting from the thread but sometime are not.
i realize at my while loop implementation there are something are missing..
something like waitone or waitautoevent..can i ignore Thread.Sleep(100) ?? what are the suitable code should i add in the while loop ?
Please help me~ :S
** sorry in the first upload i forgot to write down "volatile" into the variable
thank you..
If C# (and C and java, and probably C++), you need to declare _shouldStop and Grab as volatile.

Resources