I have a program where I want to draw data that is constantly being updated (it is microhpone-line-data incidentally). The data is an 8000-length array of doubles, I don't really care about 'losing' data which is overriden between takes of the paint method.
In my naive implementation it became obvous that there's synchronisation issues, where the audio-data is updated while the painting routine is under-way.
I'm also aware I'm slightly out-of-date on Java and the Concurrency package it has, but my first response was to just put synchronised blocks around the shared data code. Unsurprisingly this blocks the graphics thread sometimes, so I'm thinking there's probaly a much better way of doing this.
Essentially I just dont have much experience with synchronisation and am screwing things up a bit somewhere. I wonder if someone with a better understanding of these matters might be able to suggest a more elegant solution that doesn't block the graphics thread?
My naive code:
Object lock = new Object();
double[] audio = new double[8000]
// array size is always exactly 8000
public update( double[] audio ) {
synchronized( lock ) {
this.audio=audio; // and some brief processing
}
repaint();
}
public void paint( Graphics g ) {
synchronized( lock ) {
// draw the contents of this.audio
}
}
Self-answer, unless some more intelligent person can offer something better, I just save a reference to the audio array at the beginning of the routine and draw from that, then any updates to the audio buffer do their calculations in a separate array and then assign this.audio to the new array in a single step.
It appears to work, although the paint routine does get an occasional flicker it is nothing like it was, where it was flashing very noticeably about 10% of the time due to synchronised blocking. The audio-data does not update half-way through the drawing routine either. so... problem solved. probably.
double[] audio = new double[8000]
// array size is always exactly 8000
public update( double[] audio ) {
// do any brief processing
this.audio=audio; // the reference is re-assigned in one step
repaint();
}
public void paint( Graphics g ) {
audioNow = this.audio; // save the reference
// draw the contents of audioNow (not this.audio)
}
Related
I have many doubts related with Cairo and GTK# (that runs on .NET and Mono). I'm developing a GTK# application for MS Windows and Linux. I'm using GTK# 2.12 over .NET right now while I'm working on the application.
I've created a custom widget that uses Cairo.ImageSurface and Cairo.Context objects. As far as I know, I'm calling the Dispose method of every ImageSurface object and every Context object I create inside the widget code.
The widget responds to the "MouseOver" event, redrawing some parts of its DrawingArea.
The (first) problem:
almost every redrawing operation increases a little bit the amount of used memory. When the amount of used memory has increased 3 or 4 Kbytes the Monodevelop tracelog panel shows me the following message:
Cairo.Surface is leaking, programmer is missing a call to Dispose
Set MONO_CAIRO_DEBUG_DISPOSE to track allocation traces
The code that redraws a part of the widget is something like:
// SRGB is a custom struct, not from Gdk nor Cairo
void paintSingleBlock(SRGB color, int i)
{
using (Cairo.Context g = CairoHelper.Create (GdkWindow)) {
paintSingleBlock (g, color, i);
// We do this to avoid memory leaks. Cairo does not work well with the GC.
g.GetTarget().Dispose ();
g.Dispose ();
}
}
void paintSingleBlock(Cairo.Context g, SRGB color, int i)
{
var scale = Math.Pow (10.0, TimeScale);
g.Save();
g.Rectangle (x(i), y(i), w(i), h(i));
g.ClosePath ();
g.Restore ();
// We don't directly use stb.Color because in some cases we need more flexibility
g.SetSourceRGB (color.R, color.G, color.B);
g.LineWidth = 0;
g.Fill ();
}
The (second) problem: Ok, Monodevelop tells me that I should set MONO_CAIRO_DEBUG_DISPOSE to "track allocation traces" (In order to find the leak, I suppose)... but I don't know how to set this environment variable (I'm in Windows). I've tried using bash and executing something like:
MONO_CAIRO_DEBUG_DISPOSE=1 ./LightCreator.exe
But nothing appears in stderr nor stdout... (neither the messages that appear in the Monodevelop's applicationt trace panel). I also don't know how to get the debugging messages that see inside Monodevelop but without Monodevelop.
There's anyone with experience debugging GTK# or Cairo# memory leaks?
Thanks in advance.
Just wanted to throw my 2c here as I was fighting a similar leak problem in Cairo with surfaces. What I noticed is that if I create a Surface object the ReferenceCount property becomes 1 and if I attach this surface to a Context if becomes not 2 but 3. After disposing the Context the ReferenceCount comes back but to 2.
So I used some reflection to call the native methods in Cairo to decrease the ReferenceCount when I really want to Dispose a surface. I use this code:
public static void HardDisposeSurface (this Surface surface)
{
var handle = surface.Handle;
long refCount = surface.ReferenceCount;
surface.Dispose ();
refCount--;
if (refCount <= 0)
return;
var asm = typeof (Surface).Assembly;
var nativeMethods = asm.GetType ("Cairo.NativeMethods");
var surfaceDestroy = nativeMethods.GetMethod ("cairo_surface_destroy", BindingFlags.Static | BindingFlags.NonPublic);
for (long i = refCount; i > 0; i--)
surfaceDestroy.Invoke (null, new object [] { handle });
}
After using it I still have some leaks, but they seem to be related to other parts of Cairo and not with the surfaces.
I have found that a context created with CairoHelper.Create() will have a reference count of two.
A call to dispose reduces the reference count by one. Thus the context is never freed and keeps its target alive, too.
The native objects have manual reference counting, but the Gtk# wrappers want to keep a native object alive as long as there is a C# instance referencing it.
If a native object is created for a C# wrapper instance it does not need to increment the reference count because the wrapper instance 'owns' the native object and the reference count has the correct value of one. But if a wrapper instance is created for an already existing native object the reference count of the native object needs to be manually incremented to keep the object alive.
This is decided by a bool parameter when a wrapper instance is created.
Looking at the code for CairoHelper.Create() will show something like this
public static Cairo.Context Create(Gdk.Window window) {
IntPtr raw_ret = gdk_cairo_create(window == null ? IntPtr.Zero : window.Handle);
Cairo.Context ret = new Cairo.Context (raw_ret, false);
return ret;
}
Even though the native context was just created 'owned' will be false and the C# context will increment the reference count.
There is no fixed version right now, it can only be corrected by fixing the source and building Gtk# yourself.
CairoHelper is an auto-generated file, to change the parameter to true this attribute must be included in gdk/Gdk.metadata.
<attr path="/api/namespace/class[#cname='GdkCairo_']/method[#name='Create']/return-type" name="owned">true</attr>
Everything to build Gtk# can be found here.
https://github.com/mono/gtk-sharp
can anybody answer, i should change sprites image and i do it using my function
-(void) openKeyWithSprite:(id) sender withSpriteName:(NSString*)spriteName
can this lead to leaks of memory or it's ok?
in init
_spriteBonus=[CCSprite spriteWithFile:#"monstr_1_1.png"];
in schedule
-(void) openKeyWithSprite:(id) sender withSpriteName:(NSString*)spriteName
{
CCTexture2D* tex = [[CCTextureCache sharedTextureCache] addImage:spriteName];
[_spriteBonus setTexture: tex];
}
The code you've shown here should be fine and won't leak any memory. setTexture releases the old texture reference and retains the new one.
However, assuming you're not using ARC: [CCSprite spriteWithFile:#"monstr_1_1.png"]; is in the autorelease pool, so make sure you add it to a parent CCNode (before exiting whatever function you're creating it in) in order to keep it retained in memory.
If you want to create frame animation, use CCAnimate action. For example,
id yourAnimation = [CCAnimation animationWithFrames: arrayOfFrames];
id animateAction = [CCAnimate actionWithDuration: animationDuration
animation: yourAnimation
restoreOriginalFrame: YES];
[yourSprite runAction: animateAction];
if you need to repeat your animation, use CCRepeat of CCRepeatForever action, for example
id resultAction = [CCRepeatForever actionWithAction: animateAction];
[yourSprite runAction: resultAction];
I've been converting my own personal OGLES 2.0 framework to take advantage of the functionality added by the new iOS 5 framework GLKit.
After pleasing results, I now wish to implement the colour-based picking mechanism described here. For this, you must access the back buffer to retrieve a touched pixel RGBA value, which is then used as a unique identifier for a vertex/primitive/display object. Of course, this requires temporary unique coloring of all vertices/primitives/display objects.
I have two questions, and I'd be very grateful for assistance with either:
I have access to a GLKViewController, GLKView, CAEAGLLayer (of the GLKView) and an EAGLContext. I also have access to all OGLES 2.0
buffer related commands. How do I combine these to identify the color
of a pixel in the EAGLContext I'm tapping on-screen?
Given that I'm using Vertex Buffer Objects to do my rendering, is there a neat way to override the colour provided to my vertex shader
which firstly doesn't involve modifying buffered vertex (colour)
attributes, and secondly doesn't involve the addition of an IF
statement into the vertex shader?
I assume the answer to (2) is "no", but for reasons of performance and non-arduous code revamping I thought it wise to check with someone more experienced.
Any suggestions would be gratefully received. Thank you for your time
UPDATE
Well I now know how to read pixel data from the active frame buffer using glReadPixels. So I guess I just have to do the special "unique colours" render to the back buffer, briefly switch to it and read pixels, then switch back. This will inevitably create a visual flicker, but I guess it's the easiest way; certainly quicker (and more sensible) than creating a CGImageContextRef from a screen snapshot and analyzing that way.
Still, any tips as regards the back buffer would be much appreciated.
Well, I've worked out exactly how to do this as concisely as possible. Below I explain how to achieve this and list all the code required :)
In order to allow touch interaction to select a pixel, first add a UITapGestureRecognizer to your GLKViewController subclass (assuming you want tap-to-select-pixel), with the following target method inside that class. You must make your GLKViewController subclass a UIGestureRecognizerDelegate:
#interface GLViewController : GLKViewController <GLKViewDelegate, UIGestureRecognizerDelegate>
After instantiating your gesture recognizer, add it to the view property (which in GLKViewController is actually a GLKView):
// Inside GLKViewController subclass init/awakeFromNib:
[[self view] addGestureRecognizer:[self tapRecognizer]];
[[self tapRecognizer] setDelegate:self];
Set the target action for your gesture recognizer; you can do this when creating it using a particular init... however I created mine using Storyboard (aka "the new Interface Builder in Xcode 4.2") and wired it up that way.
Anyway, here's my target action for the tap gesture recognizer:
-(IBAction)onTapGesture:(UIGestureRecognizer*)recognizer {
const CGPoint loc = [recognizer locationInView:[self view]];
[self pickAtX:loc.x Y:loc.y];
}
The pick method called in there is one I've defined inside my GLKViewController subclass:
-(void)pickAtX:(GLuint)x Y:(GLuint)y {
GLKView *glkView = (GLKView*)[self view];
UIImage *snapshot = [glkView snapshot];
[snapshot pickPixelAtX:x Y:y];
}
This takes advantage of a handy new method snapshot that Apple kindly included in GLKView to produce a UIImage from the underlying EAGLContext.
What's important to note is a comment in the snapshot API documentation, which states:
This method should be called whenever your application explicitly
needs the contents of the view; never attempt to directly read the
contents of the underlying framebuffer using OpenGL ES functions.
This gave me a clue as to why my earlier attempts to invoke glReadPixels in attempts to access pixel data generated an EXC_BAD_ACCESS, and the indicator that sent me down the right path instead.
You'll notice in my pickAtX:Y: method defined a moment ago I call a pickPixelAtX:Y: on the UIImage. This is a method I added to UIImage in a custom category:
#interface UIImage (NDBExtensions)
-(void)pickPixelAtX:(NSUInteger)x Y:(NSUInteger)y;
#end
Here is the implementation; it's the final code listing required. The code came from this question and has been amended according to the answer received there:
#implementation UIImage (NDBExtensions)
- (void)pickPixelAtX:(NSUInteger)x Y:(NSUInteger)y {
CGImageRef cgImage = [self CGImage];
size_t width = CGImageGetWidth(cgImage);
size_t height = CGImageGetHeight(cgImage);
if ((x < width) && (y < height))
{
CGDataProviderRef provider = CGImageGetDataProvider(cgImage);
CFDataRef bitmapData = CGDataProviderCopyData(provider);
const UInt8* data = CFDataGetBytePtr(bitmapData);
size_t offset = ((width * y) + x) * 4;
UInt8 b = data[offset+0];
UInt8 g = data[offset+1];
UInt8 r = data[offset+2];
UInt8 a = data[offset+3];
CFRelease(bitmapData);
NSLog(#"R:%i G:%i B:%i A:%i",r,g,b,a);
}
}
#end
I originally tried some related code found in an Apple API doc entitled: "Getting the pixel data from a CGImage context" which required 2 method definitions instead of this 1, but much more code is required and there is data of type void * for which I was unable to implement the correct interpretation.
That's it! Add this code to your project, then upon tapping a pixel it will output it in the form:
R:24 G:46 B:244 A:255
Of course, you should write some means of extracting those RGBA int values (which will be in the range 0 - 255) and using them however you want. One approach is to return a UIColor from the above method, instantiated like so:
UIColor *color = [UIColor colorWithRed:red/255.0f green:green/255.0f blue:blue/255.0f alpha:alpha/255.0f];
I am trying to execute parallel functions on a list of objects using the new C# 4.0 Parallel.ForEach function. This is a very long maintenance process. I would like to make it execute in the order of the list so that I can stop and continue execution at the previous point. How do I do this?
Here is an example. I have a list of objects: a1 to a100. This is the current order:
a1, a51, a2, a52, a3, a53...
I want this order:
a1, a2, a3, a4...
I am OK with some objects being run out of order, but as long as I can find a point in the list where I can say that all objects before this point were run. I read the parallel programming csharp whitepaper and didn't see anything about it. There isn't a setting for this in the ParallelOptions class.
Do something like this:
int current = 0;
object lockCurrent = new object();
Parallel.For(0, list.Count,
new ParallelOptions { MaxDegreeOfParallelism = MaxThreads },
(ii, loopState) => {
// So the way Parallel.For works is that it chunks the task list up with each thread getting a chunk to work on...
// e.g. [1-1,000], [1,001- 2,000], [2,001-3,000] etc...
// We have prioritized our job queue such that more important tasks come first. So we don't want the task list to be
// broken up, we want the task list to be run in roughly the same order we started with. So we ignore tha past in
// loop variable and just increment our own counter.
int thisCurrent = 0;
lock (lockCurrent) {
thisCurrent = current;
current++;
}
dothework(list[thisCurrent]);
});
You can see how when you break out of the parallel for loop you will know the last list item to be executed, assuming you let all threads finish prior to breaking. I'm not a big fan of PLINQ or LINQ. I honestly don't see how writing LINQ/PLINQ leads to maintainable source code or readability.... Parallel.For is a much better solution.
If you use Parallel.Break to terminate the loop then you are guarenteed that all indices below the returned value will have been executed. This is about as close as you can get. The example here uses For but ForEach has similar overloads.
int n = ...
var result = new double[n];
var loopResult = Parallel.For(0, n, (i, loopState) =>
{
if (/* break condition is true */)
{
loopState.Break();
return;
}
result[i] = DoWork(i);
});
if (!loopResult.IsCompleted &&
loopResult.LowestBreakIteration.HasValue)
{
Console.WriteLine("Loop encountered a break at {0}",
loopResult.LowestBreakIteration.Value);
}
In a ForEach loop, an iteration index is generated internally for each element in each partition. Execution takes place out of order but after break you know that all the iterations lower than LowestBreakIteration will have been completed.
Taken from "Parallel Programming with Microsoft .NET" http://parallelpatterns.codeplex.com/
Available on MSDN. See http://msdn.microsoft.com/en-us/library/ff963552.aspx. The section "Breaking out of loops early" covers this scenario.
See also: http://msdn.microsoft.com/en-us/library/dd460721.aspx
For anyone else who comes across this question - if you're looping over an array or list (rather than an IEnumberable ), you can use the overload of Parallel.Foreach that gives the element index to maintain original order too.
string[] MyArray; // array of stuff to do parallel tasks on
string[] ProcessedArray = new string[MyArray.Length];
Parallel.ForEach(MyArray, (ArrayItem,loopstate,ArrayElementIndex) =>
{
string ProcessedArrayItem = TaskToDo(ArrayItem);
ProcessedArray[ArrayElementIndex] = ProcessedArrayItem;
});
As an alternate suggestion, you could record which object have been run and then filter the list when you resume exection to exclude the objects which have already run.
If this needs to be persistent across application restarts, you can store the ID's of the already executed objects (I assume here the objects have some unique identifier).
For anybody looking for a simple solution, I have posted 2 extension methods (one using PLINQ and one using Parallel.ForEach) as part of an answer to the following question:
Ordered PLINQ ForAll
Not sure if question was altered as my comment seems wrong.
Here improved, basically remind that parallel jobs run in out of your control order.
ea printing 10 numbers might result in 1,4,6,7,2,3,9,0.
If you like to stop your program and continue later.
Problems alike this usually endup in batching workloads.
And have some logging of what was done.
Say if you had to check 10.000 numbers for prime or so.
You could loop in batches of size 100, and have a prime log1, log2, log3
log1= 0..99
log2=100..199
Be sure to set some marker to know if a batch job was finished.
Its a general aprouch since the question isnt that exact either.
I want to process some data. I have about 25k items in a Dictionary. IN a foreach loop, I query a database to get results on that item. They're added as value to the Dictionary.
foreach (KeyValuePair<string, Type> pair in allPeople)
{
MySqlCommand comd = new MySqlCommand("SELECT * FROM `logs` WHERE IP = '" + pair.Key + "' GROUP BY src", con);
MySqlDataReader reader2 = comd.ExecuteReader();
Dictionary<string, Dictionary<int, Log>> allViews = new Dictionary<string, Dictionary<int, Log>>();
while (reader2.Read())
{
if (!allViews.ContainsKey(reader2.GetString("src")))
{
allViews.Add(reader2.GetString("src"), reader2.GetInt32("time"));
}
}
reader2.Close();
reader2.Dispose();
allPeople[pair.Key].View = allViews;
}
I was hoping to be able to do this faster by multi-threading. I have 8 threads available, and CPU usage is about 13%. I just don't know if it will work because it's relying on the MySQL server. On the other hand, maybe 8 threads would open 8 DB connections, and so be faster.
Anyway, if multi-threading would help in my case, how? o.O I've never worked with (multiple) threads, so any help would be great :D
MySqlDataReader is stateful - you call Read() on it and it moves to the next row, so each thread needs their own reader, and you need to concoct a query so they get different values. That might not be too hard, as you naturally have many queries with different values of pair.Key.
You also need to either have a temp dictionary per thread, and then merge them, or use a lock to prevent concurrent modification of the dictionary.
The above assumes that MySQL will allow a single connection to perform concurrent queries; otherwise you may need multiple connections too.
First though, I'd see what happens if you only ask the database for the data you need ("SELECT src,time FROMlogsWHERE IP = '" + pair.Key + "' GROUP BY src") and use GetString(0) and GetInt32(1) instead of using the names to look up the src and time; also only get the values once from the result.
I'm also not sure on the logic - you are not ordering the log events by time, so which one is the first returned (and so is stored in the dictionary) could be any of them.
Something like this logic - where each of N threads only operates on the Nth pair, each thread has its own reader, and nothing actually changes allPeople, only the properties of the values in allPeople:
private void RunSubQuery(Dictionary<string, Type> allPeople, MySqlConnection con, int threadNumber, int threadCount)
{
int hoppity = 0; // used to hop over the keys not processed by this thread
foreach (var pair in allPeople)
{
// each of the (threadCount) threads only processes the (threadCount)th key
if ((hoppity % threadCount) == threadNumber)
{
// you may need con per thread, or it might be that you can share con; I don't know
MySqlCommand comd = new MySqlCommand("SELECT src,time FROM `logs` WHERE IP = '" + pair.Key + "' GROUP BY src", con);
using (MySqlDataReader reader = comd.ExecuteReader())
{
var allViews = new Dictionary<string, Dictionary<int, Log>>();
while (reader.Read())
{
string src = reader.GetString(0);
int time = reader.GetInt32(1);
// do whatever to allViews with src and time
}
// no thread will be modifying the same pair.Value, so this is safe
pair.Value.View = allViews;
}
}
++hoppity;
}
}
This isn't tested - I don't have MySQL on this machine, nor do I have your database and the other types you're using. It's also rather procedural (kind of how you would do it in Fortran with OpenMPI) rather than wrapping everything up in task objects.
You could launch threads for this like so:
void RunQuery(Dictionary<string, Type> allPeople, MySqlConnection connection)
{
lock (allPeople)
{
const int threadCount = 8; // the number of threads
// if it takes 18 seconds currently and you're not at .net 4 yet, then you may as well create
// the threads here as any saving of using a pool will not matter against 18 seconds
//
// it could be more efficient to use a pool so that each thread takes a pair off of
// a queue, as doing it this way means that each thread has the same number of pairs to process,
// and some pairs might take longer than others
Thread[] threads = new Thread[threadCount];
for (int threadNumber = 0; threadNumber < threadCount; ++threadNumber)
{
threads[threadNumber] = new Thread(new ThreadStart(() => RunSubQuery(allPeople, connection, threadNumber, threadCount)));
threads[threadNumber].Start();
}
// wait for all threads to finish
for (int threadNumber = 0; threadNumber < threadCount; ++threadNumber)
{
threads[threadNumber].Join();
}
}
}
The extra lock held on allPeople is done so that there is a write barrier after all the threads return; I'm not quite sure if it's needed. Any object would do.
Nothing in this guarantees any performance gain - it might be that the MySQL libraries are single threaded, but the server certainly can handle multiple connections. Measure with various numbers of threads.
If you're using .net 4, then you don't have to mess around creating the threads or skipping the items you aren't working on:
// this time using .net 4 parallel; assumes that connection is thread safe
static void RunQuery(Dictionary<string, Type> allPeople, MySqlConnection connection)
{
Parallel.ForEach(allPeople, pair => RunPairQuery(pair, connection));
}
private static void RunPairQuery(KeyValuePair<string, Type> pair, MySqlConnection connection)
{
MySqlCommand comd = new MySqlCommand("SELECT src,time FROM `logs` WHERE IP = '" + pair.Key + "' GROUP BY src", connection);
using (MySqlDataReader reader = comd.ExecuteReader())
{
var allViews = new Dictionary<string, Dictionary<int, Log>>();
while (reader.Read())
{
string src = reader.GetString(0);
int time = reader.GetInt32(1);
// do whatever to allViews with src and time
}
// no iteration will be modifying the same pair.Value, so this is safe
pair.Value.View = allViews;
}
}
The biggest problem that comes to mind is that you are going to use multithreading to add values to a dictionary, which isn't thread safe.
You'll have to do something like this to make it work, and you might not get that much of a benefit from implementing it this was as it still has to lock the dictionary object to add a value.
Assumptions:
There is a table People in your
database
There are alot of people in
your database
Each database query adds overhead you are doing one db query for each of the people in your database I would suggest it was faster to get all the data back in one query then to make repeated calles
select l.ip,l.time,l.src
from logs l, people p
where l.ip = p.ip
group by l.ip, l.src
Try this with a loop in a single thread, I belive this will be much faster then your existing code.
With in your existing code another thing you can do is to take the creation of the MySqlCommand out of the loop, prepare it in advance and just change the parameter. This should speed up execution of the SQL. see http://dev.mysql.com/doc/refman/5.0/es/connector-net-examples-mysqlcommand.html#connector-net-examples-mysqlcommand-prepare
MySqlCommand comd = new MySqlCommand("SELECT * FROM `logs` WHERE IP = ?key GROUP BY src", con);
comd.prepare();
comd.Parameters.Add("?key","example");
foreach (KeyValuePair<string, Type> pair in allPeople)
{
comd.Parameters[0].Value = pair.Key;
If you are using mutiple threads, each thread will still need there own Command, at lest in MS-SQL this would still be faster even if you recreated and prepared the statment every time, due to the ability for the SQL server to be able to cache the execution plan of a paramertirised statment.
Before you do anything else, find out exactly where the time is being spent. Check the execution plan of the query. The first thing I'd suspect is a missing index on logs.IP.
18 minutes for something like this seems much too long to me. Even if you can cut the execution time in eight by adding more threads (which is unlikely!) you still end up using more than 2 minutes. You could probably read the whole 25k rows into memory in less than five seconds and do the necessary processing in memory...
EDIT: Just to clarify, I'm not advocating actually doing this in memory, just saying that it looks like there's a bigger bottleneck here that can be removed.
I think if you are running this on a multi core machine you could gain benefits from multi threading.
However the way I would approach it is to first look at unblocking the thread you are currently using by making asynchronous database calls. The call backs will execute on background threads, so you will get some multi core benefit there and you won't be blocking threads waiting for the db to come back.
For IO intensive apps like this example sounds like you are likely to see improved throughput depending on what load the db can handle. Assuming the db scales to handle more than one concurrent request you should be good.
Thanks everyone for your help. Currently I am using this
for (int i = 0; i < 8; i++)
{
ThreadPool.QueueUserWorkItem(addDistinctScres, i);
}
ThreadPool to run all the threads. I use the method provided by Pete Kirkham, and I'm creating a new connection per thread.
Times went down to 4 minutes.
Next I'll make something wait for the callback of the threadpool? before performing other functions.
I think the bottleneck now is the MySQL server, because the CPU usage has drops.
#odd parity I thought about that, but the real thing is waaay more than 25k rows. Idk if that'd work.
This sound like the perfect job for map/reduce, i am not a .Net-programmer, but this seems like a reasonable guide:
http://ox.no/posts/minimalistic-mapreduce-in-net-4-0-with-the-new-task-parallel-library-tpl