I'm sure that an average vtk user already has seen results like the following more than once.
My question(s): How would you repair such a broken surface? And what is typically the cause for such wholes in the surface?
My particular example was created by using vtkBooleanOperationPolyDataFilter and vtkAppendPolyData, but I've seen such broken, degenerate surfaces also in different occasions.
Many thanks for suggestions.
This is most likely data-related. Suggestions:
Many vtk filters have assumptions about the inputs, and I am guessing your inputs violated some of these assumptions. E.g. vtkBooleanOperationPolyDataFilter expects inputs to be manifolds, otherwise "unexpected results may be obtained". What are you feeding into the boolean filter? Are these inputs manifolds?
Some other filters have much stricter requirements and expect only triangulated surfaces; in the image you posted I think I see quads. Try to run the inputs through vtkTriangleFilter at the beginning of your processing pipeline to split all polys into triangles.
Inspect the second output of vtkBooleanOperationPolyDataFilter which contains the intersection as set of polylines, for any hints on what could be the cause of this.
Try to save the intermediate results into a file and expect them at different stages in your processing pipeline.
If none of this will lead you to the cause of the problem, please post the inputs, the code and vtk version and system that you are running it on, so that we can reproduce your results.
HTH,
Miro
In the case I presented above, the broken surface was caused by problems with the vtkBooleanOperationPolyDataFilter. According to this thread, the algorithm has been improved and is (or will soon be) made available in a newer release of vtk.
I also need to accept the fact that there is no general recipe to recover from such failures in vtk, and, as mirni pointed out, are data-related.
Related
So checking for Sample Ratio Mismatch is good for data quality.
But in Google Optimize i can't influence the sample size or do something against it.
My problem is, out of 15 A/B Tests I only got 2 Experiment with no SRM.
(Used this tool https://www.lukasvermeer.nl/srm/microsite/)
In the other hand the bayesian model deals with things like different sample sizes and I dont need to worry about, but the opinions on this topic are different.
Is SRM really a problem in Google Optimize or can I ignore it?
SRM affects Bayesian experiments just as much as it affects Frequentist. SRM happens when you expect a certain traffic split, but end up with a different one. Google Optimize is a black box, so it's impossible to tell if the uneven sample sizes you are experiencing are intentional or not.
Lots of things can cause a SRM, for example if your variation's javascript code has a bug in some browsers those users may not be tracked properly. Another common cause is if your variation causes page load times to increase, more people will abandon the page and you'll see a smaller sample size than expected.
That lack of statistical rigor and transparency is one of the reasons I built Growth Book, which is an open source A/B testing platform with a Bayesian stats engine and automatic SRM checks for every experiment.
I'm working with the VTK library in C++.
I have a mesh given as an unstructured grid and certain data given on integration points of gaussian quadrature on the cells (which was created by an external solver). For the sake of simplicity, let's assume that we talk about scalar data.
I also have a tool which displays VTK data graphically. What I want is to display the mentioned data with that tool, simply as interpolated/extrapolated scalar data on the whole grid.
My question is, is there something native to VTK with which I can give the mesh the scalar data at the integration points in some way and VTK handles the interpolation and extrapolation?
I mean, I could write an algorithm that processes the data, creates a new grid in which the cells do not share nodes (as the extrapolated values might not be continuous there), extrapolate the scalars to those nodes for each cell and then display that. However, by this I take away from the native possibilities of the VTK library (which seems to be quite strong in most other regards) and I don't want to reinvent the wheel anyway.
From https://vtk.org/Wiki/images/7/78/VTK-Quadrature-Point-Design-Doc.pdf, I am aware that there is the vtkQuadratureSchemeDefinition class and I think I know how to handle it, and I noticed vtkQuadraturePointInterpolator, which seems to do the opposite of what I'm searching for - interpolation to the integration points rather than extrapolating from them.
The newest entry in the VTK wiki otherwise seems to be https://vtk.org/Wiki/VTK/VTK_integration_point_support, which seems to be quite old, given that it pleads for the existence of some sort of quadrature point support in general, which currently already exists.
Also there is a question in the VTK mailing list which looks just like my question here:
https://public.kitware.com/pipermail/vtkusers/2013-January/078077.html, which seems to be without an answer.
Likewise, the issue https://gitlab.kitware.com/vtk/vtk/issues/17124 also seems to be about what I want to do, and it might hint at it currently not being possible, but it existing as an issue does not imply that it is not already solved (especially with no asignee to the issue).
I try to do some nodejs profiling using Linux perf_events as described by Brendan Gregg here.
Workflow is following:
run node >0.11.13 with --perf-basic-prof, which creates /tmp/perf-(PID).map file where JavaScript symbol mapping are written.
Capture stacks using perf record -F 99 -p `pgrep -n node` -g -- sleep 30
Fold stacks using stackcollapse-perf.pl script from this repository
Generate svg flame graph using flamegraph.pl script
I get following result (which look really nice at the beginning):
Problem is that there are a lot of [unknown] elements, which I suppose should be my nodejs function calls. I assume that whole process fails somwhere at point 3, where perf data should be folded using mappings generated by node/v8 executed with --perf-basic-prof. /tmp/perf-PID.map file is created and some mapping are written to it during node execution.
How to solve this problem?
I am using CentOS 6.5 x64, and already tried this with node 0.11.13, 0.11.14 (both prebuild, and compiled as well) with no success.
FIrst of all, what "[unknown]" means is the sampler couldn't figure out the name of the function, because it's a system or library function.
If so, that's OK - you don't care, because you're looking for things responsible for time in your code, not system code.
Actually, I'm suggesting this is one of those XY questions.
Even if you get a direct answer to what you asked, it is likely to be of little use.
Here are the reasons why:
1. CPU Profiling is of little use in an I/O bound program
The two towers on the left in your flame graph are doing I/O, so they probably take a lot more wall-time than the big pile on the right.
If this flame graph were derived from wall-time samples, rather than CPU-time samples, it could look more like the second graph below, which tells you where time actually goes:
What was a big juicy-looking pile on the right has shrunk, so it is nowhere near as significant.
On the other hand, the I/O towers are very wide.
Any one of those wide orange stripes, if it's in your code, represents a chance to save a lot of time, if some of the I/O could be avoided.
2. Whether the program is CPU- or I/O-bound, speedup opportunities can easily hide from flame graphs
Suppose there is some function Foo that really is doing something wasteful, that if you knew about it, you could fix.
Suppose in the flame graph, it is a dark red color.
Suppose it is called from numerous places in the code, so it's not all collected in one spot in the flame graph.
Rather it appears in multiple small places shown here by black outlines:
Notice, if all those rectangles were collected, you could see that it accounts for 11% of time, meaning it is worth looking at.
If you could cut its time in half, you could save 5.5% overall.
If what it's doing could actually be avoided entirely, you could save 11% overall.
Each of those little rectangles would shrink down to nothing, and pull the rest of the graph, to its right, with it.
Now I'll show you the method I use. I take a moderate number of random stack samples and examine each one for routines that might be speeded up.
That corresponds to taking samples in the flame graph like so:
The slender vertical lines represent twenty random-time stack samples.
As you can see, three of them are marked with an X.
Those are the ones that go through Foo.
That's about the right number, because 11% times 20 is 2.2.
(Confused? OK, here's a little probability for you. If you flip a coin 20 times, and it has a 11% chance of coming up heads, how many heads would you get? Technically it's a binomial distribution. The most likely number you would get is 2, the next most likely numbers are 1 and 3. (If you only get 1 you keep going until you get 2.) Here's the distribution:)
(The average number of samples you have to take to see Foo twice is 2/0.11 = 18.2 samples.)
Looking at those 20 samples might seem a bit daunting, because they run between 20 and 50 levels deep.
However, you can basically ignore all the code that isn't yours.
Just examine them for your code.
You'll see precisely how you are spending time,
and you'll have a very rough measurement of how much.
Deep stacks are both bad news and good news -
they mean the code may well have lots of room for speedups, and they show you what those are.
Anything you see that you could speed up, if you see it on more than one sample, will give you a healthy speedup, guaranteed.
The reason you need to see it on more than one sample is, if you only see it on one sample, you only know its time isn't zero. If you see it on more than one sample, you still don't know how much time it takes, but you do know it's not small.
Here are the statistics.
Generally speaking it is a bad idea to disagree with a subject matter expert but (with the greatest respect) here we go!
SO urges the answer to do the following:
"Please be sure to answer the question. Provide details and share your research!"
So the question was, at least my interpretation of it is, why are there [unknown] frames in the perf script output (and how do I turn these [unknown] frames in to meaningful names)?
This question could be about "how to improve the performance of my system?" but I don't see it that way in this particular case. There is a genuine problem here about how the perf record data has been post processed.
The answer to the question is that although the prerequisite set up is correct: the correct node version, the correct argument was present to generate the function names (--perf-basic-prof), the generated perf map file must be owned by root for perf script to produce the expected output.
That's it!
Writing some new scripts today I hit apon this directing me to this SO question.
Here's a couple of additional references:
https://yunong.io/2015/11/23/generating-node-js-flame-graphs/
https://github.com/jrudolph/perf-map-agent/blob/d8bb58676d3d15eeaaf3ab3f201067e321c77560/bin/create-java-perf-map.sh#L22
[ non-root files can sometimes be forced ] http://www.spinics.net/lists/linux-perf-users/msg02588.html
My application present a (raster) moving map.
I need to be able to show the map rotated base on any given angle.
The program is currently in VC++/MFC but the problem is generic.
I have a source bitmap (CBitmap or HBITMAP) and draw it to the device context (CDC) using StretchBlt.
While this works fast and smooth for angle=0 (and the user can grab the map smoothly with the mouse), this is not the case if I try to rotate the bitmap and then present it (the rotation of the bitmap using SetWorldTransform() or so takes hundreds of miliseconds and this is too slow).
I think that the solution is to be able to relate only to the pixels that currently on the screen and not rotating the original source bitmap - and this is the key.
If someone has experience with similar implementation then it might save me lots of trial and error efforts.
Thanks!
Avi.
It looks like SetWorldTransform is extremely slow:
http://www.codeguru.com/Cpp/G-M/bitmap/specialeffects/article.php/c1743
And while the other options presented in that article are faster, there are of course other better solutions like this:
http://www.codeguru.com/cpp/g-m/gdi/article.php/c3693/ (check the comments for fixes and improvements as well)
Also here are some non-Windows centric fast rotation algorithms:
http://www.ddj.com/windows/184416337?pgno=11
Note that if you guarantee power of 2 dimensions you can get significant speed improvements.
As follow up to my question and provided answer, let me summarize the following:
I used the algorithm mentioned at http://www.codeguru.com/cpp/g-m/gdi/article.php/c3693/.
It works and provide pretty good performance and smooth display.
There were some bugs in it that I needed to fix as well as simplify the formulas and
code in some cases.
I will examine the algorithm mentioned at http://www.ddj.com/windows/184416337?pgno=11 to see if it provides some break through performance that worth adapting it.
My implementation required using a large source bitmap, so I needed to modify the code so I will not rotate the whole bitmap each time but only the relevant portion that will be displayed at the screen (otherwise performance would be unacceptable).
Avi.
Imagine the radio of a car, does the electro magnetic fields through which the car goes through, have interference in the processing? It's easy to understand that a strong field can corrupt data. But what about the data under processment? Can it also be changed?
If so how could you protect your code against this? (without electrial protections just code ones)
For the most robust mission critical systems you use multiple processors and compare results. This is what we did with aircraft auto pilot (autolanding). We had three autopilots, one flying the aircraft and two check that one. If any one of the three disagreed, it was shut down.
You're referring to what Wikipedia calls soft errors. The traditional, industry-accepted work-around for this is through redundancy, as Jim C and fmsf noted.
Several years ago, our repair department's analysis showed an unacceptable number of returned units with single-bit errors in the battery-backed SRAM that held the firmware. Despite our efforts at root-cause analysis, we were unable to explain the source of the problem. At that point a hardware change was out of the question, so we needed a software-only solution to treat the symptom.
We wanted a reliable fix that we could implement simply and quickly, so we generated parity checks on blocks of code in the SRAM. We chose a block size that required very little additional storage for the parity data, yet provided enough redundancy to detect and correct any of the errors we'd seen and then some. It logs the errors it detects and indicates whether it can correct them, so we still know when bit errors occur in the field. So far, so good!
Our product manager did some additional research out of curiosity and convinced himself that the culprit was cosmic radiation. We never proved it unequivocally, but he was satisfied that the number of errors seemed to agree with what would be expected based on the data he found. I'm just glad the returns have stopped.
I doubt you can.
Code that is changed won't run, so likely your program(s) will crash if you have this problem.
This is a hardware problem.