I have a database connection with a query similar to as follows in a couple separate files in a project:
SqlTypes.fs
module SqlTypes
open FSharp.Data.TypeProviders
type DBType = SqlDataConnection<ConnectionStrings.DbConnStr>
let getDb() = DBType.GetDataContext() //will this cause a memory leak when called from outside here?
Utilities.fs
module Utilities
open System
open SqlTypes
open System.Linq
///Returns an IQueryable of Orders whose OrderStatus is less than 5 (Not yet released)
let private getOrdersNotYetReleased () =
let orders = query {
for order in getDb().Orders do //memleak here?
where(order.Status < 5)
sortBy (order.CreatedDateUtc)
select order
}
//return
orders
Q: Is it necessary to use a use clause (like in the following) to prevent a memory leak?
///Returns an IQueryable of Orders whose OrderStatus is less than 5 (Not yet released)
let private getOrdersNotYetReleased () =
use db = DBType.GetDataContext() //is this safer? Does it matter if this `use` is here?
let orders = query {
for order in db.Orders do
where(order.Status < 5)
sortBy (order.CreatedDateUtc)
select order
}
//return
orders
Or, will Dispose() be called on the Db object when the computation block exits?
The IDisposable pattern provides for a Dispose() method. The use keyword is there to take advantage of this pattern; if a class implements IDisposable, the author of the class is telling you that using the use keyword with their class is probably a good idea.
What the Dispose() method actually does depends entirely on what code the author of the class writes for the Dispose() method. There is no guarantee that it even does anything.
Microsoft's official word on IDisposable has always been that its purpose is to "clean up unmanaged resources," but in practice, Dispose() does whatever the author of the class believes it needs to do to gracefully shut down the object. What it generally does not do is recover memory from managed objects; that is the purpose of the garbage collector.
But, generally speaking, if a class implements IDisposable, that means it contains a Dispose() method that does something meaningful for the disposition of the object, like closing files or database connections, for example. So. Class implements IDisposable? Use use with it.
Further Reading
Managing Resources in F#
Related
I'm begginer in multithreading. I recently started to writing something like multithreaded observer. I need some clarification.
Let's say I'm working with Subject, and I'm changing its state. Then Observers (in example - GUI widgets) have to be notified, so they could perform the update() method.
And there is my question: how am i handling those getValue() performed by many Observers? If it's just a getter for some variable, do i have to run it in new thread? Does it require any locking?
Or mayby there is a metod to just send those new value to GUI thread, and letting widgets there access those value. And again, can it be a single loop, or do i have to create another threads for every widget to get those value?
That's a difficult subject. Here are couple of things that will guide and help you with it.
Embrace eventual consistency. When one object updates on one thread, others will receive change notifications and update to the correct state eventually. Don't try to keep everything in sync all the time. Don't expect everything to be up to date all the time. Design your system to handle these situations. Check this video.
Use immutability especially for collections. Reading and writing to a collection from multiple threads can result in disasters. Don't do it. Use immutable collections or use snapshotting. Basically one object that will called from multiple thread will return a snapshot of the state of the collection. when a notification for a change is received, the reader (GUI in your case) will request a snapshot of the new state and update it accordingly.
Design rich Models. Don't use AnemicModels that have only setters and getters and let others manipulate them. Let the Model protect it's data and provide queries for it's state. Don't return mutable objects from properties of an object.
Pass data that describes changes with change notifications. This way readers (GUI) may sync their state only from the change data without having to read the target object.
Divide responsibility. Let the GUI know that it's single threaded and received notifications from the background. Don't add knowledge in your Model that it will be updated on a background thread and to know that it's called from the GUI and give it the responsibility of sending change requests to a specific thread. The Model should not care about stuff like that. It raises notifications and let subscribers handle them the way they need to. Let the GUI know that the change notification will be received on the background so it can transfer it to the UI thread.
Check this video. It describes different way you can do multithreading.
You haven't shown any code or specified language, so I'll give you an example in pseudo code using a Java/C# like language.
public class FolderIcon {
private Icon mIcon;
public Icon getIcon() { return mIcon; }
public FolderIcon(Icon icon) {
mIcon = icon;
}
}
public class FolderGUIElement : Observer {
private Folder mFolder;
private string mFolderPath;
public FolderGUIElement(Folder folder) {
mFolder = folder;
mFolderPath = mFolder.getPath();
folder.addChangeListener(this);
}
public void onSubjectChanged(change c) {
if(c instanceof PathChange) {
dispatchOnGuiThread(() => {
handlePathChange((PathChange)change);
});
}
}
handlePathChange(PathChange change) {
mFolderPath = change.NewPath;
}
}
public class Folder : Subject {
private string mPath;
private FolderIcon mIcon;
public string getPath() { return mPath; }
public FolderIcon getIcon() { return mIcon; }
public void changePath(string newPath) {
mPath = patnewPath;
notifyChanged(new PathChange(newPath));
}
public void changeIcon(FolderIcon newIcon) {
mIcon = newIcon;
notifyChanged(new IconChange(newIcon));
}
}
Notice couple of things in the example.
We are using immutable objects from Folder. That means that the GUI elements cannot get the value of Path or FolderIcon and change it thus affecting Folder. When changing the icon we are creating a brand new FolderIcon object instead of modifying the old one. Folder itself is mutable, but it uses immutable objects for it's properties. If you want you can use fully immutable objects. A hybrid approach works well.
When we receive change notification we read the NewPath from the PathChange. This way we don't have to call the Folder again.
We have changePath and changeIcon methods instead of setPath and setIcon. This captures the intent of our operations better thus giving our model behavior instead of being just a bag of getters and setters.
If you haven't read Domain Driven Design I highly recommend it. It's not about multithreading, but on how to design rich models. It's in my list of books that every developer should read. On concept in DDD is ValueObject. It's immutable and provide a great way to implement models and is especially useful in multithreaded systems.
(Pseudo-)Code
Here is a non-compilable code-sketch of the concepts I am having trouble with:
struct Data {};
struct A {};
struct B {};
struct C {};
/* and many many more...*/
template<typename T>
class Listener {
public:
Listener(MyObject* worker):worker(worker)
{ /* do some magic to register with RTI DDS */ };
public:
// This function is used ass a callback from RTI DDS, i.e. it will be
// called from other threads when new Data is available
void callBackFunction(Data d)
{
T t = extractFromData(d);
// Option 1: direct function call
// works somewhat, but shows "QObject::startTimer: timers cannot be started
// from another thread" at the console...
worker->doSomeWorkWithData(t); //
// Option 2: Use invokeMethod:
// seems to fail, as the macro expands including '"T"' and that type isn't
// registered with the QMetaType system...
// QMetaObject::invokeMethod(worker,"doSomeGraphicsWork",Qt::AutoConnection,
// Q_ARG(T, t)
// );
// Option 3: use signals slots
// fails as I can't make Listener, a template class, a QObject...
// emit workNeedsToBeDone(t);
}
private:
MyObject* worker;
T extractFromData(Data d){ return T(d);};
};
class MyObject : public QObject {
Q_OBJECT
public Q_SLOTS:
void doSomeWorkWithData(A a); // This one affects some QGraphicsItems.
void doSomeWorkWithData(B b){};
void doSomeWorkWithData(C c){};
public:
MyObject():QObject(nullptr){};
void init()
{
// listeners are not created in the constructor, but they should have the
// same thread affinity as the MyObject instance that creates them...
// (which in this example--and in my actual code--would be the main GUI
// thread...)
new Listener<A>(this);
new Listener<B>(this);
new Listener<C>(this);
};
};
main()
{
QApplication app;
/* plenty of stuff to set up RTI DDS and other things... */
auto myObject = new MyObject();
/* stuff resulting in the need to separate "construction" and "initialization" */
myObject.init();
return app.exec();
};
Some more details from the actual code:
The Listener in the example is a RTI DataReaderListener, the callback
function is onDataAvailable()
What I would like to accomplish
I am trying to write a little distributed program that uses RTI's Connext DDS for communication and Qt5 for the GUI stuff--however, I don't believe those details do matter much as the problem, as far as I understood it, boils down to the following:
I have a QObject-derived object myObject whose thread affinity might or might not be with the main GUI thread (but for simplicity, let's assume that is the case.)
I want that object to react to event's which happen in another, non-Qt 3rd-party library (in my example code above represented by the functions doSomeWorkWithData().
What I understand so far as to why this is problematic
Disclaimer: As usual, there is always more than one new thing one learns when starting a new project. For me, the new things here are/were RTI's Connext and (apparently) my first time where I myself have to deal with threads.
From reading about threading in Qt (1,2,3,4, and 5 ) it seems to me that
QObjects in general are not thread safe, i.e. I have to be a little careful about things
Using the right way of "communicating" with QObjects should allow me to avoid having to deal with mutexes etc myself, i.e. somebody else (Qt?) can take care of serializing access for me.
As a result from that, I can't simply have (random) calls to MyClass::doSomeWorkWithData() but I need to serialize that. One, presumably easy, way to do so is to post an event to the event queue myObject lives in which--when time is available--will trigger the execution of the desired method, MyClass::doSomeWorkWithData() in my case.
What I have tried to make things work
I have confirmed that myObject, when instantiated similarly as in the sample code above, is affiliated with the main GUI thread, i.e. myObject.thread() == QApplication::instance()->thread().
With that given, I have tried three options so far:
Option 1: Directly calling the function
This approach is based upon the fact that
- myObject lives in the GUI thread
- All the created listeners are also affiliated with the GUI thread as they are
created by `myObject' and inherit its thread that way
This actually results in the fact that doSomeWorkWithData() is executed. However,
some of those functions manipulate QGraphicsItems and whenever that is the case I get
error messages reading: "QObject::startTimer: timers cannot be started from another
thread".
Option 2: Posting an event via QMetaObject::invokeMethod()
Trying to circumvent this problem by properly posting an event for myObject, I
tried to mark MyObject::doSomeWorkWithData() with Q_INVOKABLE, but I failed at invoking the
method as I need to pass arguments with Q_ARG. I properly registered and declared my custom types
represented by struct A, etc. in the example), but I failed at the fact the
Q_ARG expanded to include a literal of the type of the argument, which in the
templated case didn't work ("T" isn't a registered or declared type).
Trying to use conventional signals and slots
This approach essentially directly failed at the fact that the QMeta system doesn't
work with templates, i.e. it seems to me that there simply can't be any templated QObjects.
What I would like help with
After spending about a week on attempting to fix this, reading up on threads (and uncovering some other issues in my code), I would really like to get this done right.
As such, I would really appreciate if :
somebody could show me a generic way of how a QObject's member function can be called via a callback function from another 3rd-party library (or anything else for that matter) from a different, non QThread-controlled, thread.
somebody could explain to me why Option 1 works if I simply don't create a GUI, i.e. do all the same work, just without a QGraphcisScene visualizing it (and the project's app being a QCoreApplication instead of a QApplication and all the graphics related work #defineed out).
Any, and I mean absolutely any, straw I could grasp on is truly appreciated.
Update
Based on the accepted answer I altered my code to deal with callbacks from other threads: I introduced a thread check at the beginning of my void doSomeWorkWithData() functions:
void doSomeWorkWithData(A a)
{
if( QThread::currentThread() != this->thread() )
{
QMetaObject::invokeMethod( this,"doSomeWorkWithData"
,Qt::QueuedConnection
,Q_ARG(A, a) );
return;
}
/* The actual work this function does would be below here... */
};
Some related thoughts:
I was contemplating to introduce a QMutexLocker before the if statement, but decided against it: the only part of the function that is potentially used in parallel (anything above the return; in the if statement) is--as far as I understand--thread safe.
Setting the connection type manually to Qt::QueuedConnection: technically, if I understand the documentation correctly, Qt should do the right thing and the default, Qt::AutoConnection, should end up becoming a Qt::QueuedConnection. But since would always be the case when that statement is reached, I decided to put explicitly in there to remind myself about why this is there.
putting the queuing code directly in the function and not hiding it in an interim function: I could have opted to put the call to invokeMethod in another interim function, say queueDoSomeWorkWithData()', which would be called by the callback in the listener and then usesinvokeMethodwith anQt::AutoConnection' on doSomeWorkWithData(). I decided against this as there seems no way for me to auto-code this interim function via templates (templates and the Meta system was part of the original problem), so "the user" of my code (i.e. the person who implements doSomeWorkWithData(XYZ xyz)) would have to hand type the interim function as well (as that is how the templated type names are correctly resolved). Including the check in the actual function seems to me to safe typing an extra function header, keeps the MyClass interface a little cleaner, and better reminds readers of doSomeWorkWithData() that there might be a threading issue lurking in the dark.
It is ok to call a public function on a subclass of QObject from another thread if you know for certain that the individual function will perform only thread-safe actions.
One nice thing about Qt is that it will handle foreign threads just as well as it handles QThreads. So, one option is to create a threadSafeDoSomeWorkWithData function for each doSomeWorkWithData that does nothing but QMetaMethod::invoke the non-threadsafe one.
public:
void threadSafeDoSomeWorkWithData(A a) {
QMetaMethod::invoke("doSomeWorkWithData", Q_ARG(A,a));
}
Q_INVOKABLE void doSomeWorkWithData(A a);
Alternatively, Sergey Tachenov suggests an interesting way of doing more or less the same thing in his answer here. He combines the two functions I suggested into one.
void Obj2::ping() {
if (QThread::currentThread() != this->thread()) {
// not sure how efficient it is
QMetaObject::invoke(this, "ping", Qt::QueuedConnection);
return;
}
// thread unsafe code goes here
}
As to why you see normal behaviour when not creating a GUI? Perhaps you're not doing anything else that is unsafe, aside from manipulating GUI objects. Or, perhaps they're the only place in which your thread-safety problems are obvious.
So I'm trying to use the TPL features in .NET 4.0 and have some code like this (don't laugh):
/// <summary>Fetches a thread along with its posts. Increments the thread viewed counter.</summary>
public Thread ViewThread(int threadId)
{
// Get the thread along with the posts
Thread thread = this.Context.Threads.Include(t => t.Posts)
.FirstOrDefault(t => t.ThreadID == threadId);
// Increment viewed counter
thread.NumViews++;
Task.Factory.StartNew(() =>
{
try {
this.Context.SaveChanges();
}
catch (Exception ex) {
this.Logger.Error("Error viewing thread " + thread.Title, ex);
}
this.Logger.DebugFormat(#"Thread ""{0}"" viewed and incremented.", thread.Title);
});
return thread;
}
So my immediate concerns with the lambda are this.Context (my entity framework datacontext member), this.Logger (logger member) and thread (used in the logger call). Normally in the QueueUserWorkItem() days, I would think these would need to be passed into the delegate as part of a state object. Are closures going to be bail me out of needing to do that?
Another issue is that the type that this routine is in implements IDisposable and thus is in a using statement. So if I do something like...
using (var bl = new ThreadBL()) {
t = bl.ViewThread(threadId);
}
... am I going to create a race between a dispose() call and the TPL getting around to invoking my lambda?
Currently I'm seeing the context save the data back to my database but no logging - no exceptions either. This could be a configuration thing on my part but something about this code feels odd. I don't want to have unhandled exceptions in other threads. Any input is welcome!
As for your question on closures, yes this is exactly what closures are about. You don't worry about passing state, instead it is captured for you from any outer context and copied onto a compiler supplied class which is also where the closure method will be defined. The compiler does a lot of magic here to make you're life simple. If you want to understand more I highly recommend picking up Jon Skeet's C# in Depth. The chapter on closures is actually available here.
As for your specific implementation, it will not work mainly for the exact problem you mentioned: the Task will be scheduled at the end of ViewThread, but potentially not execute before your ThreadBL instance is disposed of.
After the previous question "What are the important rules in Object Model Design", now I want to ask this:
Is there any way to have dynamic properties for class instances?
Suppose that we have this schematic object model:
So, each object could have lots of properties due to the set of implemented Interfaces, and then become relatively heavy object. Creating all the possible -and of course reasonable- object can be a way for solving this problem (i.e. Pipe_Designed v.s. Pipe_Designed_NeedInspection), but I have a large number of interfaces by now, that make it difficult.
I wonder if there is a way to have dynamic properties, something like the following dialog to allow the end user to select available functionalities for his/hers new object.
What you want is Properties pattern. Check out long and boring but clever article from Steve Yegge on this
I think maybe you're putting too many roles into the "Road" and "Pipe" classes, because your need for dynamic properties seems to derive from various states/phases of the artifacts in your model. I would consider making an explicit model using associations to different classes instead of putting everything in the "Road" or "Pipe" class using interfaces.
If you mean the number of public properties, use explicit interface implementation.
If you mean fields (and object space for sparse objects): you can always use a property bag for the property implementation.
For a C# example:
string IDesigned.ApprovedBy {
get {return GetValue<string>("ApprovedBy");}
set {SetValue("ApprovedBy", value);}
}
with a dictionary for the values:
readonly Dictionary<string, object> propValues =
new Dictionary<string, object>();
protected T GetValue<T>(string name)
{
object val;
if(!propValues.TryGetValue(name, out val)) return default(T);
return (T)val;
}
protected void SetValue<T>(string name, T value)
{
propValues[name] = value;
}
Note that SetValue would also be a good place for any notifications - for example, INotifyPropertyChanged in .NET to implement the observer pattern. Many other architectures have something similar. You can do the same with object keys (like how EventHandlerList works), but string keys are simpler to understand ;-p
This only then takes as much space as the properties that are actively being used.
A final option is to encapsulate the various facets;
class Foo {
public bool IsDesigned {get {return Design != null;}}
public IDesigned Design {get;set;}
// etc
}
Here Foo doesn't implement any of the interfaces, but provides access to them as properties.
If you have a class with some plain get/set properties, is there any reason to use the getters within the class methods, or should you just use the private member variables? I think there could be more of an argument over setters (validation logic?), but I'm wondering just about getters.
For example (in Java) - is there any reason to use option 2?:
public class Something
{
private int messageId;
public int getMessageId() { return this.messageId; }
public void setMessage(int messageId) { this.messageId = messageId; }
public void doSomething()
{
// Option 1:
doSomethingWithMessageId(messageId);
// Option 2:
doSomethingWithMessageId(getMessageId());
}
}
Java programmers in general tend to be very consistent about using getter methods. I program multiple languages and I'm not that consistent about it ;)
I'd say as long as you don't make a getter it's ok to use the raw variable - for private variables. When you make a getter, you should be using only that. When I make a getter for a private field, my IDE suggests that it replace raw field accesses for me automatically when I introduce a getter. Switching to using a getter is only a few keystrokes away (and without any chance of introducing errors), so I tend to delay it until I need it.
Of course, if you want to stuff like getter-injection, some types of proxying and subclassing framworks like hibernate, you have to user getters!
With getters you wont accidentally modify the variables :) Also, if you use both getters and the "raw" variable, your code can get confused.
Also, if you use inheritance and redefined the getter methods in child classes, getter-using methods will work properly, whereas those using the raw variables would not.
If you use the getter method everywhere - and in the future perform a code-search on all calls of getMessageId() you will find all of them, whereas if you had used the private ones, you may miss some.
Also if there's ever logic to be introduced in the setter method, you wont have to worry about changing more than 1 location for it.
If the value that you are assigning to the property is a known or verified value, you could safely use the private variable directly. (Except perhaps in some special situations, where it would be obvious why that would be bad.) Whether you do or not is more a matter of taste or style. It's not a performance issue either, as the getter or setter will be inlined by the compiler if it's simple enough.
If the value is unknown to the class, you should use the property to set it, so that you can protect the property from illegal values.
Here's an example (in C#):
public class Something {
private string _value;
public string Value {
get {
return _value;
}
set {
if (value == null) throw new ArgumentNullException();
_value = value;
}
}
public Something() {
// using a known value
_value = "undefined";
}
public Something(string initValue) {
// using an unknown value
Value = initValue;
}
}
If you use the getter you're ensuring you'll get the value after any logic/decisions have been applied to it. This probably isn't your typical situation but when it is, you'll thank yourself for this.
Unless I have a specific use case to use the internal field directly in the enclosing class, I've always felt that it's important to use access the field the same way it is accessed publicly. This ensures consistency in the return values across the board should there ever be any need to add some post-processing to the field via the getter method, or property. I feel like it's perfectly fine to access the raw field if you want its raw value for one reason or another.
More often than not, the getter encapsulation is plain and simple boilerplate code -- you're most likely not returning anything other than the field's value itself. However, in the case where you may want to change the way the data is presented at some point in the future, it's one less refactoring you have to make internally.