I completed my entire project on building an application that basically fetches sql query result based on some user input combination in the GUI and I believe that there is some performance issue when calling a Q_INVOKABLE function in C++ code which performs some computation to produce the query and also run the query and return the result, so I wanted to know if there is a way to run that function in a separate thread from qml.
The skeleton is somewhat like this:
Result.h
class Result{
public:
Q_INVOKABLE void on_submit_button_clicked();
}
Result.cpp
Result::on_submit_button_clicked()
{
/*Code that prepare the query*/
/*Code that runs the query using a Qthread*/
}
main.qml
ApplicationWindow
{
id:appwindow
/*Other parts of the GUI*/
Button{
id:submitButton
onClicked:{
result_object.on_submit_button_clicked() //how to run this function in a new thread ?
/*code to load my table view columns*/
}
}
}
Related
I'm playing around with reactive flows using RxJava2, Micronaut and Cassandra. I'm new to rxjava and not sure what is the correct way to return a of List Person in the best async manner?
data is coming from a Cassandra Dao interface
public interface PersonDAO {
#Query("SELECT * FROM cass_drop.person;")
CompletionStage<MappedAsyncPagingIterable<Person>> getAll();
}
that gets injected into a micronaut controller
return Single.just(personDAO.getAll().toCompletableFuture().get().currentPage())
.subscribeOn(Schedulers.io())
.map(people -> HttpResponse.ok(people));
OR
return Single.just(HttpResponse.ok())
.subscribeOn(Schedulers.io())
.map(it -> it.body(personDAO.getAll().toCompletableFuture().get().currentPage()));
OR switch to RxJava3
return Single.fromCompletionStage(personDAO.getAll())
.map(page -> HttpResponse.ok(page.currentPage()))
.onErrorReturn(throwable -> HttpResponse.ok(Collections.emptyList()));
Not a pro of RxJava nor Cassandra :
In your first and second example, you are blocking the thread executing the CompletionStage with get, even if you are doing it in the IO thread, I would not recommand doing so.
You are also using a Single wich can emit, only one value, or an error. Since you want to return a List, I would sugest to go for at least an Observable.
Third point, the result from Cassandra is paginated, I don't know if it's intentionnaly but you list only the first page, and miss the others.
I would try a solution like the one below, I kept using the IO thread (the operation may be costly in IO) and I iterate over the pages Cassandra fetch :
/* the main method of your controller */
#Get()
public Observable<Person> listPersons() {
return next(personDAO.getAll()).subscribeOn(Schedulers.io());
}
private Observable<Person> next(CompletionStage<MappedAsyncPagingIterable<Person>> pageStage) {
return Single.fromFuture(pageStage.toCompletableFuture())
.flatMapObservable(personsPage -> {
var o = Observable.fromIterable(personsPage.currentPage());
if (!personsPage.hasMorePages()) {
return o;
}
return o.concatWith(next(personsPage.fetchNextPage()));
});
}
If you ever plan to use reactor instead of RxJava, then you can give cassandra-java-driver-reactive-mapper a try.
The syntax is fairly simple and works in compile-time only.
I get strange errors when I am trying to pass around NSManagedObject through several functions. (all are in the same VC).
Here are the two functions in question:
func syncLocal(item:NSManagedObject,completionHandler:(NSManagedObject!,SyncResponse)->Void) {
let savedValues = item.dictionaryWithValuesForKeys([
"score",
"progress",
"player"])
doUpload(savedParams) { //do a POST request using params with Alamofire
(success) in
if success {
completionHandler(item,.Success)
} else {
completionHandler(item,.Failure)
}
}
}
func getSavedScores() {
do {
debugPrint("TRYING TO FETCH LOCAL SCORES")
try frc.performFetch()
if let results = frc.sections?[0].objects as? [NSManagedObject] {
if results.count > 0 {
print("TOTAL SCORE COUNT: \(results.count)")
let incomplete = results.filter({$0.valueForKey("success") as! Bool == false })
print("INCOMPLETE COUNT: \(incomplete.count)")
let complete = results.filter({$0.valueForKey("success") as! Bool == true })
print("COMPLETE COUNT: \(complete.count)")
if incomplete.count > 0 {
for pendingItem in incomplete {
self.syncScoring(pendingItem) {
(returnItem,response) in
let footest = returnItem.valueForKey("player") //only works if stripping syncScoring blank
switch response { //response is an enum
case .Success:
print("SUCCESS")
case .Duplicate:
print("DUPLICATE")
case .Failure:
print("FAIL")
}
}
} //sorry for this pyramid of doom
}
}
}
} catch {
print("ERROR FETCHING RESULTS")
}
}
What I am trying to achieve:
1. Look for locally saved scores that could not submitted to the server.
2. If there are unsubmitted scores, start the POST call to the server.
3. If POST gets 200:ok mark item.key "success" with value "true"
For some odd reason I can not access returnItem at all in the code editor - only if I completely delete any code in syncLocal so it looks like
func syncLocal(item:NSManagedObject,completionHandler:(NSManagedObject!,SyncResponse)->Void) {
completionHandler(item,.Success)
}
If I do that I can access .syntax properties in the returning block down in the for loop.
Weirdly if I paste the stuff back in, in syncLocal the completion block keeps being functional, the app compiles and it will be executed properly.
Is this some kind of strange XCode7 Bug? Intended NSManagedObject behaviour?
line 1 was written with stripped, line 2 pasted rest call back in
There is thread confinement in Core Data managed object contexts. That means that you can use a particular managed object and its context only in one and the same thread.
In your code, you seem to be using controller-wide variables, such as item. I am assuming the item is a NSManagedObject or subclass thereof, and that its context is just one single context you are using in your app. The FRC context must be the main thread context (a NSManagedObjectContext with concurrency type NSMainThreadConcurrencyType).
Obviously, the callback from the server request will be on a background thread. So you cannot use your managed objects.
You have two solutions. Either you create a child context, do the updates you need to do, save, and then save the main context. This is a bit more involved and you can look for numerous examples and tutorials out there to get started. This is the standard and most robust solution.
Alternatively, inside your background callback, you simply make sure the context updates occur on the main thread.
dispatch_async(dispatch_get_main_queue()) {
// update your managed objects & save
}
I have a big doubt. The problem is Out of Memory Exception in my class. But it seems something strange here. I have class in a dll.
public class MyClass : IDisposible
{
List<ClassA> a_classLists = new .....// new instance.
List<ClassB> b_classLists = new .....// new instance.
public string Method1(int IDValue)
{
// do here some web service call and get some XML data from it.
// parse the xml.
// Iterate through a for loop and add each node value to a_classLists
// Usually contains 10 or 15 items
Method2(); // from here calling another method
FinalSaveToDB(); // finally save the data to DB
return "";
}
private void Method2()
{
// do here some web service call and get some XML data from it.
// Iterate through a forloop.
// parse the xml. [large xml data. ie, image in binary format]
// For each loop add image binary data and other xml to b_classLists
// Usually it contains 50 or 60 such large lists.
}
private void FinalSaveToDB()
{
// using sqlbulkcopy, i am saving the data in the 2 lists to 2 different
// tables in the DB.
// Tab lock is mentioned in sqlbulk class.
// Actually 2 sqlbulkcopy class for 2 lists.
// Only 1 sql connection opens, then do the sqlbulkcopy. [there is no dataset or datareader]
// sqlconnection closes. I am using "using" clause for sqlconnection, bulkcopy etc
// these all are working fine.
}
private void Dispose()
{
// here nulling everything
// proxy null
// all the lists null....
}
}
This is the class I am instantiating 1000 times using reactive framework's Observable.Start
method as shown below...
private IObservable<string> SendEmpDetails(Employee emp)
{
using (MyClass p = new MyClass())
{
return Observable.Start(() => p.Method1(emp.ID), Scheduler.ThreadPool);
}
// here I hope it will call the Dispose and release all objects in the class.
}
// This EmployeeLists contains 1000 employee objects
EmployeeLists.ToObservable().Select(x => SendEmpDetails(x).Select(y => new { emp = x, retval = y }))
.Merge(10)
.ObserveOn(Scheduler.CurrentThread)
.Subscribe(x =>
{
SendStatus(x.retval.Item1, x.retval);
});
Even though, why i am getting out of memory exception ??? After starting the app, when it
process the 200th (or above) MyClass object, it throws error.
I forgot to mention 1 more thing, I am using VS 2010 and C# 4.0 (win7, 64 bit OS).
I need to log each activity. [ie, i need to understand the each and every process the app has gone through]. SO i declared a class [MyClass] level private string variable and assign each process details like "called this method", "got 5 records from this web service" etc.
logdata = Environment.Newline() + "This method has completed";
So the error is thrown here saying out of memory with some evalution failed.
So I turned off the string evaluation check box from Options in VS.
Again, there is no use.
So I changed the string to StringBuilder and tried to append the activity string each time.
Still no use. I dont understand what is the problem in it.
Is this because all the threads are working parallel, do they exchange the MyClass resources ??? Why the objects are not released ???
Please help me in this matter.
I have read a lot about the new Task functionality in .net 4.0, but I haven't found a solution for the following problem:
I am writing a server application that processes requests from many users and I want to use Tasks to distribute these request on multiple cores. However, these Tasks should be synchronized on objects - for the beginning, users -, so that just one task is processed for each object at a time. This would be simple to achieve with Task.ContinueWith(), but it should also be possible to synchonize a task on multiple objects (e.g. when a user transfers money to another user, a variable should be decremented at user A and incremented at user B without other tasks interfering).
So, my first attempt is a class that receives delegates, creates tasks and stores them in a dictionary with the objects to sync on as keys. If a new task is scheduled, it can be appended to the last task of the given object with Task.ContinueWith(). If it should be synchronized on multiple objects, the new Task is created using TaskFactory.ContinueWhenAll(). The created task is stored in the dictionary for every object it is synchronized on.
Here is my first draft:
public class ActionScheduler:IActionScheduler
{
private readonly IDictionary<object, Task> mSchedulingDictionary = new Dictionary<object, Task>();
private readonly TaskFactory mTaskFactory = new TaskFactory();
/// <summary>
/// Schedules actions synchonized on one or more objects. Only one action will be processed for each object at any time.
/// </summary>
/// <param name="synchronisationObjects">Array of objects the current action is synchronized on</param>
/// <param name="action">The action that will be scheduled and processed</param>
public void ScheduleTask(object[] synchronisationObjects, Action action)
{
// lock the dictionary in case two actions are scheduled on the same object at the same time
// this is necessary since reading and writing to a dictionary can not be done in an atomic manner
lock(mSchedulingDictionary)
{
// get all current tasks for the given synchronisation objects
var oldTaskList = new List<Task>();
foreach (var syncObject in synchronisationObjects)
{
Task task;
mSchedulingDictionary.TryGetValue(syncObject, out task);
if (task != null)
oldTaskList.Add(task);
}
// create a new task for the given action
Task newTask;
if (oldTaskList.Count > 1)
{
// task depends on multiple previous tasks
newTask = mTaskFactory.ContinueWhenAll(oldTaskList.ToArray(), t => action());
}
else
{
if (oldTaskList.Count == 1)
{
// task depends on exactly one previous task
newTask = oldTaskList[0].ContinueWith(t => action());
}
else
{
// task does not depend on any previous task and can be started immediately
newTask = new Task(action);
newTask.Start();
}
}
// store the task in the dictionary
foreach (var syncObject in synchronisationObjects)
{
mSchedulingDictionary[syncObject] = newTask;
}
}
}
}
This even works if a task "multiSyncTask" was created for multiple objects, and afterwards tasks for each of the objects are scheduled. Since they are all created with multiSyncTask.ContinueWith(), they start synchronously:
static void Main()
{
IActionScheduler actionScheduler = new ActionScheduler();
var syncObj1 = new object();
var syncObj2 = new object();
// these two start and complete simultaneously:
actionScheduler.ScheduleTask(new[] { syncObj1 }, () => PrintTextAfterWait("1"));
actionScheduler.ScheduleTask(new[] { syncObj2 }, () => PrintTextAfterWait("2"));
// this task starts after the first two and "locks" both objects:
actionScheduler.ScheduleTask(new[] { syncObj1, syncObj2 }, () => PrintTextAfterWait("1 and 2"));
// these two - again - start and complete simultaneously after the task above:
actionScheduler.ScheduleTask(new[] { syncObj1 }, () => PrintTextAfterWait("1"));
actionScheduler.ScheduleTask(new[] { syncObj2 }, () => PrintTextAfterWait("2"));
}
static void PrintTextAfterWait(string text)
{
Thread.Sleep(3000);
Console.WriteLine(text);
}
What do you think - is this a good solution for my problem? I am a bit sceptic about the big lock on the dictionary, but it is necessary in case two tasks are scheduled on one object at once to prevent race conditions. Of course, the dictionary is just locked for the time it takes to create a task, not when it is processed.
Also, I would love to know if there are any already existing solutions or coding paradigms out there that solve my problem better using .net 4.0 Tasks that I have failed to track down.
Thank you and with best regards,
Johannes
If I got you right.. you would like to have a Task.ContinueWith(task1, task2, lambda)?
Something like the Join arbiter in CCR?
http://msdn.microsoft.com/en-us/library/bb648749.aspx
If so, probably the most elegant option is to use the JoinBlock in TPL dataflow (http://www.microsoft.com/download/en/confirmation.aspx?id=14782).
Or, maybe, have you tried to do use Task.WaitAll() as the first instruction of your dependent task?
I realized that when I am trying to process items in a concurrent queue using multiple threads while multiple threads can be putting items into it, the ideal solution would be to use the Reactive Extensions with the Concurrent data structures.
My original question is at:
While using ConcurrentQueue, trying to dequeue while looping through in parallel
So I am curious if there is any way to have a LINQ (or PLINQ) query that will continuously be dequeueing as items are put into it.
I am trying to get this to work in a way where I can have n number of producers pushing into the queue and a limited number of threads to process, so I don't overload the database.
If I could use Rx framework then I expect that I could just start it, and if 100 items are placed in within 100ms, then the 20 threads that are part of the PLINQ query would just process through the queue.
There are three technologies I am trying to work together:
Rx Framework (Reactive LINQ)
PLING
System.Collections.Concurrent
structures
Drew is right, I think the ConcurrentQueue even though it sounds perfect for the job is actually the underlying data structure that the BlockingCollection uses. Seems very back to front to me too.
Check out chapter 7 of this book*
http://www.amazon.co.uk/Parallel-Programming-Microsoft-NET-Decomposition/dp/0735651590/ref=sr_1_1?ie=UTF8&qid=1294319704&sr=8-1
and it will explain how to use the BlockingCollection and have multiple producers and multiple consumers each taking off the "queue". You will want to look at the "GetConsumingEnumerable()" method and possibly just call .ToObservable() on that.
*the rest of the book is pretty average.
edit:
Here is a sample program that I think does what you want?
class Program
{
private static ManualResetEvent _mre = new ManualResetEvent(false);
static void Main(string[] args)
{
var theQueue = new BlockingCollection<string>();
theQueue.GetConsumingEnumerable()
.ToObservable(Scheduler.TaskPool)
.Subscribe(x => ProcessNewValue(x, "Consumer 1", 10000000));
theQueue.GetConsumingEnumerable()
.ToObservable(Scheduler.TaskPool)
.Subscribe(x => ProcessNewValue(x, "Consumer 2", 50000000));
theQueue.GetConsumingEnumerable()
.ToObservable(Scheduler.TaskPool)
.Subscribe(x => ProcessNewValue(x, "Consumer 3", 30000000));
LoadQueue(theQueue, "Producer A");
LoadQueue(theQueue, "Producer B");
LoadQueue(theQueue, "Producer C");
_mre.Set();
Console.WriteLine("Processing now....");
Console.ReadLine();
}
private static void ProcessNewValue(string value, string consumerName, int delay)
{
Thread.SpinWait(delay);
Console.WriteLine("{1} consuming {0}", value, consumerName);
}
private static void LoadQueue(BlockingCollection<string> target, string prefix)
{
var thread = new Thread(() =>
{
_mre.WaitOne();
for (int i = 0; i < 100; i++)
{
target.Add(string.Format("{0} {1}", prefix, i));
}
});
thread.Start();
}
}
I don't know how best to accomplish this with Rx, but I would recommend just using BlockingCollection<T> and the producer-consumer pattern. Your main thread adds items into the collection, which uses ConcurrentQueue<T> underneath by default. Then you have a separate Task that you spin up ahead of that which uses Parallel::ForEach over the BlockingCollection<T> to process as many items from the collection as makes sense for the system concurrently. Now, you will probably also want to look into using the GetConsumingPartitioner method of the ParallelExtensions library in order to be most efficient since the default partitioner will create more overhead than you want in this case. You can read more about this from this blog post.
When the main thread is finished you call CompleteAdding on the BlockingCollection<T> and Task::Wait on the Task you spun up to wait for all the consumers to finish processing all the items in the collection.