While doing some testing with OrmLite I encountered some problem with multithreading.
In some cases, using my Repo from different threads I encountered a random "concurrency" error on the FieldDefinitionMap.
Reproducing the error is quite easy, just start 10 threads with massive Read / Write operations to get a random error:
'Field Definition Id was not found'
Is very important in order to reproduce the error to use the update/insert functions for the first time on the new threads.
The error comes from the function:
public virtual void SetParameterValues<T>(IDbCommand dbCmd, object obj)
I was able to solve it putting a lock on the FieldDefinitionMap like follow:
namespace ServiceStack.OrmLite
public class ModelDefinition
public Dictionary<string, FieldDefinition> FieldDefinitionMap
{
get
{
lock (this) // Locking the get works without problems
{
if (fieldDefinitionMap == null)
{
fieldDefinitionMap = new Dictionary<string, FieldDefinition>();
foreach (var fieldDef in FieldDefinitionsArray)
{
fieldDefinitionMap[fieldDef.FieldName] = fieldDef;
}
}
return fieldDefinitionMap;
}
}
}
By the way I do not know if there is any better solution.
Related
The application was developed on ASP NET Core 3. To log user actions, I decided to use a single method in the Project class. Faced the problem of using one singleton dbContext from different threads.
Startup.cs
public void ConfigureServices(IServiceCollection services)
{
string connection = Configuration.GetConnectionString("ConnectionDB");
services.AddDbContext<DataBaseContext>(options => options.UseSqlServer(connection), ServiceLifetime.Transient, ServiceLifetime.Singleton);
services.AddSingleton<Project>();
}
Project.cs
public async Task AddUserLog(string action, string message, int userId)
{
try
{
UserLog userLog = new UserLog()
{
Action = action,
Message = message,
UserId = userId
Datepoint = DateTime.Now
};
_dbContext.UserLog.Add(userLog);
await _dbContext.SaveChangesAsync();
}
catch (Exception ex)
{
await AddSystemLog("Project", "AddUserLog", ex.Message);
}
}
SchemeController.cs
public class SchemeController : ControllerBase
{
private readonly Project _project;
public SchemeController(Project project)
{
_project = project;
}
[Authorize(Policy = "AdvancedControl")]
[HttpPost("[action]")]
public async Task SomeMethode()
{
for (int i = 0; i < 10; i++)
{
await _project.AddUserLog("Text", "Message", 42);
}
}
}
Already at the second iteration of the loop, I catch an exception in the AddUserLog method:
"A second operation started on this context before a previous operation completed. This is usually caused by different threads using the same instance of DbContext."
I suggest several solutions:
Add the log to the buffer table and then save it to the database by timer. But this is not the best way out;
Block the method while it is being saved to the database.
But I don’t like any of the options.
Please tell me the correct approach in solving this issue.
So, you trying to use shared resource (singleton Project class) to perform parallel operations (save UserLogs) while your shared resource implementation is not thread-safe (exceptions raised).
You have at lease three ways to solve this:
Do not use shared resource: register Project per scope instead of singletone;
Do not perform operations in parallel: seems hard to achieve because you making webapp and you can't force user(s) to wait
Refactor your resource to be thread-safe: add locks/mutexes/buffering... inside Project
There is no one "correct" way - all 3 are correct. Choose one you like (or combine several).
Usually using scoped dbcontext is recommended (because connections are pooled), but it's the creator of app who should decide.
My C# app has lots of forms to perform various tasks. To keep things simple, I have a static class FormsCollection where the instances of each of the other forms are kept, so that they are shown and hidden from one place.
Now i was getting that classic debug time "Cross-thread" error. I tried to fix it the following way:
public class FormsCollection : Control // inherited from Control only to be
// able to call "this.Invoke"
{
public delegate void ShowFormDelegate(Form form);
public static Main mainForm;
// and many other forms...
public void ShowForm(Form form)
{
if (form.InvokeRequired)
{
ShowFormDelegate delegateFunc = new ShowFormDelegate(ShowForm);
this.Invoke(delegateFunc, new object[] { form });
}
else
{
previousForm = currentForm;
currentForm.Hide();
currentForm = form;
currentForm.Show();
}
}
}
Inside the user/caller forms, i simply make an object of FormsCollection and call the ShowForm method (almost a 100 such calls):
FormsCollection f = new FormsCollection();
f.ShowForm(FormsCollection.mainForm);
And after all this ordeal, what i get is that the same error appears at the very same spot as before! What an irony! :)
What am i doing wrong? Please help me out....
I got extremely hard-to-find answer to this problem from this slightly irrelevant page. Following is the updated code for any struggling programmers looking for a work around to this problem:
public void Show(Form nextForm)
{
Thread thread = new Thread(
new ThreadStart(() =>
{
PreviousForm.BeginInvoke(
new Action(() =>
{
PreviousForm = CurrentForm;
CurrentForm = nextForm;
PreviousForm.Hide();
CurrentForm.Show();
}
));
}
));
thread.Start();
}
This seems to have resolved the cross-thread problem that i was facing despite several work-arounds. It is working as yet without raising any exceptions and I have tested almost all the scenarios of my app going back and forth from form to form.
I have faced this problem while changing a code block.
List<Entity> entities = new List<Entity>();
//Some values added to the list ....
foreach(var entity in entities)
{
Thread th = new Thread(new ThreadStart(SomeMethod));
th.Start(entity);
}
SomeMethod is taking Entity and changing on it. somthing like
private void SomeMethod(Entity entity)
{
//Some operation on entity
}
This is a .NET 2.0 code, while changing it to 4.0 I did a small change i.e.
foreach(var entity in entities)
{
Thread th = new ThreadStart(() => SomeMethod(entity));
th.Start();
}
This is not working, from error it looks like entity variable getting changed between threads and resulting some null reference exception. I havnt changed anything else, as soon as code reverted to the previous way, it is fine. Can anyone explain this?
Please try to use a new local variable in such a way:
foreach(var entity in entities)
{
var copy = entity;
Thread th = new ThreadStart(() => SomeMethod(copy));
th.Start();
}
The approach with a new variable will not work if you introduce the variable in such a way
Thread th = new ThreadStart(() => {var copy = entity; SomeMethod(copy)) };
since that code will still capture the loop variable that is changed in each iteration of the loop.
Also please note that you can get the error you described if the entities collection contains the same element twice (that is two references to the same Entity object). Please make sure that it's not the case.
Sorry for big chunk of code, I couldn't explain that with less.Basically I'm trying to write into a file from many tasks.
Can you guys please tell me what I'm doing wrong? _streamWriter.WriteLine() throws the ArgumentOutOfRangeException.
class Program
{
private static LogBuilder _log = new LogBuilder();
static void Main(string[] args)
{
var acts = new List<Func<string>>();
var rnd = new Random();
for (int i = 0; i < 10000; i++)
{
acts.Add(() =>
{
var delay = rnd.Next(300);
Thread.Sleep(delay);
return "act that that lasted "+delay;
});
}
Parallel.ForEach(acts, act =>
{
_log.Log.AppendLine(act.Invoke());
_log.Write();
});
}
}
public class LogBuilder : IDisposable
{
public StringBuilder Log = new StringBuilder();
private FileStream _fileStream;
private StreamWriter _streamWriter;
public LogBuilder()
{
_fileStream = new FileStream("log.txt", FileMode.Create, FileAccess.ReadWrite, FileShare.ReadWrite);
_streamWriter = new StreamWriter(_fileStream) { AutoFlush = true };
}
public void Write()
{
lock (Log)
{
if (Log.Length <= 0) return;
_streamWriter.WriteLine(Log.ToString()); //throws here. Although Log.Length is greater than zero
Log.Clear();
}
}
public void Dispose()
{
_streamWriter.Close(); _streamWriter.Dispose(); _fileStream.Close(); fileStream.Dispose();
}
}
This is not a bug in StringBuilder, it's a bug in your code. And the modification you shown in your followup answer (where you replace Log.String with a loop that extracts characters one at a time) doesn't fix it. It won't throw an exception any more, but it won't work properly either.
The problem is that you're using the StringBuilder in two places in your multithreaded code, and one of them does not attempt to lock it, meaning that reading can occur on one thread simultaneously with writing occurring on another. In particular, the problem is this line:
_log.Log.AppendLine(act.Invoke());
You're doing that inside your Parallel.ForEach. You are not making any attempt at synchronization here, even though this will run on multiple threads at once. So you've got two problems:
Multiple calls to AppendLine may be in progress simultaneously on multiple threads
One thread may attempt to be calling Log.ToString at the same time as one or more other threads are calling AppendLine
You'll only get one read at a time because you are using the lock keyword to synchronize those. The problem is that you're not also acquiring the same lock when calling AppendLine.
Your 'fix' isn't really a fix. You've succeeded only in making the problem harder to see. It will now merely go wrong in different and more subtle ways. For example, I'm assuming that your Write method still goes on to call Log.Clear after your for loop completes its final iteration. Well in between completing that final iteration, and making the call to Log.Clear, it's possible that some other thread will have got in another call to AppendLine because there's no synchronization on those calls to AppendLine.
The upshot is that you will sometimes miss stuff. Code will write things into the string builder that then get cleared out without ever being written to the stream writer.
Also, there's a pretty good chance of concurrent AppendLine calls causing problems. If you're lucky they will crash from time to time. (That's good because it makes it clear you have a problem to fix.) If you're unlucky, you'll just get data corruption from time to time - two threads may end up writing into the same place in the StringBuilder resulting either in a mess, or completely lost data.
Again, this is not a bug in StringBuilder. It is not designed to support being used simultaneously from multiple threads. It's your job to make sure that only one thread at a time does anything to any particular instance of StringBuilder. As the documentation for that class says, "Any instance members are not guaranteed to be thread safe."
Obviously you don't want to hold the lock while you call act.Invoke() because that's presumably the very work you want to parallelize. So I'd guess something like this might work better:
string result = act();
lock(_log.Log)
{
_log.Log.AppendLine(result);
}
However, if I left it there, I wouldn't really be helping you, because this looks very wrong to me.
If you ever find yourself locking a field in someone else's object, it's a sign of a design problem in your code. It would probably make more sense to modify the design, so that the LogBuilder.Write method accepts a string. To be honest, I'm not even sure why you're using a StringBuilder here at all, as you seem to use it just as a holding area for a string that you immediately write to a stream writer. What were you hoping the StringBuilder would add here? The following would be simpler and doesn't seem to lose anything (other than the original concurrency bugs):
public class LogBuilder : IDisposable
{
private readonly object _lock = new object();
private FileStream _fileStream;
private StreamWriter _streamWriter;
public LogBuilder()
{
_fileStream = new FileStream("log.txt", FileMode.Create, FileAccess.ReadWrite, FileShare.ReadWrite);
_streamWriter = new StreamWriter(_fileStream) { AutoFlush = true };
}
public void Write(string logLine)
{
lock (_lock)
{
_streamWriter.WriteLine(logLine);
}
}
public void Dispose()
{
_streamWriter.Dispose(); fileStream.Dispose();
}
}
I think the cause is because you are accessing the stringBuilder in the Parellel bracket
_log.Log.AppendLine(act.Invoke());
_log.Write();
and inside the LogBuilder you perform lock() to disallow memory allocation on stringBuidler. You are changing the streamwriter to handle the log in every character so would give the parellel process to unlock the memory allocation to stringBuilder.
Segregate the parallel process into distinct action would likely reduce the problem
Parallel.ForEach(acts, act =>
{
_log.Write(act.Invoke());
});
in the LogBuilder class
private readonly object _lock = new object();
public void Write(string logLines)
{
lock (_lock)
{
//_wr.WriteLine(logLines);
Console.WriteLine(logLines);
}
}
An alternate approach is to use TextWriter.Synchronized to wrap StreamWriter.
void Main(string[] args)
{
var rnd = new Random();
var writer = new StreamWriter(#"C:\temp\foo.txt");
var syncedWriter = TextWriter.Synchronized(writer);
var tasks = new List<Func<string>>();
for (int i = 0; i < 1000; i++)
{
int local_i = i; // get a local value, not closure-reference to i
tasks.Add(() =>
{
var delay = rnd.Next(5);
Thread.Sleep(delay);
return local_i.ToString() + " act that that lasted " + delay.ToString();
});
}
Parallel.ForEach(tasks, task =>
{
var value = task();
syncedWriter.WriteLine(value);
});
writer.Dispose();
}
Here are some of the synchronization helper classes
http://referencesource.microsoft.com/#q=Synchronized
System.Collections
static ArrayList Synchronized(ArrayList list)
static IList Synchronized(IList list)
static Hashtable Synchronized(Hashtable table)
static Queue Synchronized(Queue queue)
static SortedList Synchronized(SortedList list)
static Stack Synchronized(Stack stack)
System.Collections.Generic
static IList Synchronized(List list)
System.IO
static Stream Synchronized(Stream stream)
static TextReader Synchronized(TextReader reader)
static TextWriter Synchronized(TextWriter writer)
System.Text.RegularExpressions
static Match Synchronized(Match inner)
static Group Synchronized(Group inner)
It is seems that it isn't problem of Parallelism. It's StringBuilder's problem.
I have replaced:
_streamWriter.WriteLine(Log.ToString());
with:
for (int i = 0; i < Log.Length; i++)
{
_streamWriter.Write(Log[i]);
}
And it worked.
For future reference: http://msdn.microsoft.com/en-us/library/system.text.stringbuilder(v=VS.100).aspx
Memory allocation section.
I am experiencing intermittent SaveChangesException when creating CRM entities using the early binding APIs (generated context) via a Threading.Task.
Using the code below I am able to recreate the exception - note that it is intermittent and I can have several runs without experiencing it.
As you can see there is no shared state in my code, I am creating the CRM Connection, service and context per operation.
The exception thrown is:
Microsoft.Xrm.Sdk.SaveChangesException: An error occured while processing this request. ---> System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection. Parameter name: index
Am I missing something here or are the early bound SDK classes unsafe to use in a multi-threaded environment?
class Program
{
static void Main(string[] args)
{
foreach (var index in Enumerable.Range(1, 400))
{
var capturedIndex = index;
Task.Factory.StartNew(() => CreateContact(capturedIndex));
}
Console.Read();
}
private static void CreateContact(int i)
{
var contact = new Contact {FirstName = "Test", LastName = string.Concat("Contact_", i)};
var context = new XrmServiceContext(new OrganizationService(new CrmConnection("Crm"))); // The generated context is taken from the SDK samples
context.AddObject(contact);
try
{
context.SaveChanges();
Console.WriteLine(string.Concat("Contact created - ", i));
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
}
}
The exception occurs at a lower level within the WCF configuration. For those using late-bound entities, a workaround is to disable the ProxyTypesEnabled property on the CrmConnection. Another workaround is to set the ServiceConfigurationMode to PerInstance but unfortunately, this comes with a performance hit. Note: both of these can be set through the connection string as well.
var connection1 = new CrmConnection("Crm")
{
ProxyTypesEnabled = false
}; // late-bound only
var connection2 = new CrmConnection("Crm")
{
ServiceConfigurationInstanceMode = ServiceConfigurationInstanceMode.PerInstance
};