Finding the bounds of an area covered by n of m rectangles - dynamic-programming

I have a set of m non-rotated, integer (pixel) aligned rectangles, each of which may or may not overlap. The rectangles cover thousands of pixels. I need to find the minimum sized bounding box that covers all areas that are covered by n of the m rectangles.
A (dirty) way of doing this is to paint a canvas that covers the area of all the targets. This is O(mk) where m is the number of rectangles and k is the number of pixels per rectangle. However since k is much greater than m I think there is a better solution out there.
This feels like a dynamic programming problem...but I am having trouble figuring out the recursion.
Solution which is better but still not great:
Sort the start and end points of all the rectangles in the X direction O(mlogm), iterate and find the x positions that may have over n rectangles, O(m) loop. For each x position that may have over n rectangles, take the rectangles at that position and sort the starts and stops at that position (O(mlogm)). Find the region of overlap, keep track of the bounds that way. Overall, O(m^2logm).

Hello MadScienceDreams,
Just to clarify, the bounding box is also non-rotated, correct?
If this is the case, then just keep track of the four variables: minX, maxX, minY, maxY–representing left-most, right-most, top-most, and bottom-most pixels–that define the bounding box, loop through each of the rectangles updating the four variables, and defining the new bounding box given those four variables.
EDIT
It looks like you are asking about finding the bounds of some subset of rectangles, not the whole set.
So you have M rectangles, and you choose N rectangles from them, and find the bounds within that.
Even in this situation, looping through the N rectangles and keeping track of their bound would be at most O(m), which isn't bad at all.
I feel that I must be misunderstanding your question since this response isn't what you are probably looking for; is your question actually trying to ask how to precompute the bounds so that given any subset, know the total bounds in constant time?

Is this defines your question? For bounding box => #rect_label >= n

How about we starts with one box and find the next box that has nearest furthest corner from it. Now we have a region with two box. Recursively find the next region, until we have n boxes.
While we need to start on every box, we only need to actively work on the currently smallest regions. The effect is we start from the smallest cluster of boxes and expand out from there.
If n is closer to m than 0, we can reverse the search tree so that we start from the omni-all-enclosing box, chopping off each bordering box to create the next search level. Assuming we only actively work on the smallest remaining region, effect is we chop off the emptiest region first.
Is it too complicated? Sorry I can't remember the name of this search. I'm not good at maths, so I'll skip the O notation. >_<

I propose the following algorithm :
prepareData();
if (findBorder('left')) {
foreach (direction in ['top', 'right', 'bottom']) {
findBorder(direction)
}
} else noIntersectionExists
prepareData (O(mlogm)):
Order vertical bounds and horizontal bounds
Save the result as:
- two arrays that point to the rectangle (arrX and arrY)
- save the index as a property of the rectangle (rectangle.leftIndex, rectangle.topIndex, etc.
findBorder(left): // the other direction are similar
best case O(n), worst case O(2m-n)
arrIntersections = new Array;
//an intersection has a depth (number of rectangles intersected), a top and bottom border and list of rectangles
for(i=0; i < 2*m-n-1; i++){ // or (i = 2*m-1; i > n; i--)
if(isLeftBorder(arrX[i])){
addToIntersections(arrX[i].rectangle, direction);
if(max(intersections.depth) = n) break;
} else {
removeFromIntersections(arrX[i].rectangle, direction);
}
}
addToIntersections(rectangle, direction): // explanations for direction=left
Best case: O(n), worst case: O(m)
hasIntersected = false;
foreach(intersection in intersection){
if(intersect(intersection, rectangle)){
hasIntersected = true
intersections[] = {
depth: intersection.depth,
bottom: min(intersection.bottom, rectangle.bottom),
top: max(...)}
intersection.depth++
intersection.bottom = max(intersection.bottom, rectangle.bottom)
intersection.top = max(...)
}
}
if(!hasIntersected)
intersections[]={depth:1, bottom:rectangle.bottom, top:rectangle.top}
This gives an overall order between O(n^2) and O(m*(m-n/2))
I hope my pseudo code is clear enough

Related

Why do we return the max distance out of the four corners when finding the heuristic for A*?

I was trying to solve the cs188 ai project-1 search. In cornersHeuristic() function we need to return a heuristic, so that we use it when traversing the corners. And we want to use the shortest path so we use A-star search.
Summary of the code to understand it better: We have the locations of the corners in (x,y) format, and we have our characters current location again as (x, y) format.
The heuristic returned is maximum distance of the distances between the location to all the corners. But what is the logic here? Shouldn't we return the minimum distance to the nearest corner? Why do we return the max out of these, how does this help to find the shortest path?
The code is as below:
Some explanations for the code: The corner1, corner2, corner3, corner4, and location variables are all tuples and store x and y values. Such as (1, 1), (4, 5)
def cornersHeuristic(state, problem):
A heuristic for the CornersProblem that you defined.
state: The current search state
(a data structure you chose in your search problem)
problem: The CornersProblem instance for this layout.
This function should always return a number that is a lower bound on the
shortest path from the state to a goal of the problem; i.e. it should be
admissible (as well as consistent).
corners = problem.corners # These are the corner coordinates
walls = problem.walls # These are the walls of the maze, as a Grid (game.py)
location, corner1, corner2, corner3, corner4 = state
corners = [corner1, corner2, corner3, corner4]
heuristic = 0
for x in corners[1]:
heuristic = max(heuristic,(abs(location[0]-x[0]) + abs(location[1] - x[1])))
return max_r
The heuristic is a lower-bound on the actual distance. You want this lower-bound to be as large as possible, while still being guaranteed to be a lower-bound.
So you're correct that min would work. But max is better, because it's a larger lower-bound.
We can guarantee it will still be a lower-bound because the problem statement is that we want to visit all of the corners. The path-length required to do that will always be at least the cost of visiting the furthest corner. If the problem were to visit any one corner, then we'd need to use min

How to make sure strings would not overlap each other in java processing?

I'm having a problem that I need to make the words I took from an external file "NOT" overlap each other. I have over 50 words that have random text sizes and places when you run it but they overlap.
How can I make them "NOT" overlap each other? the result would probably look like a word cloud.
if you think my codes would help here they are
String [] words;
int index = 0;
void setup ()
{
size (500,500);
background (255);
String [] lines = loadStrings ("alice_just_text.txt");
String entireplay = join(lines, " "); //splits it by line
words = splitTokens (entireplay, ",.?!:-;:()03 "); //splits it by word
for (int i = 0; i < 50; i++) {
float x = random(width);
float y = random(height);
int index = int(random(words.length));
textSize (random(60)); //random font size
fill (0);
textAlign (CENTER);
text (words[index], x, y, width/2, height/2);
println(words[index]);
index++ ;
}
}
Stack Overflow isn't really designed for general "how do I do this" type questions. You'll have much better luck if you post a more specific "I tried X, expected Y, but got Z instead" type question. But I'll try to help in a general sense:
You need to break your problem down into smaller pieces and then take on those pieces one at a time.
For example, you can isolate your problem to making sure rectangles don't overlap, which you can break down even further. There are a number of ways to do that:
You could use a grid to lay out your rectangles. Figure out how many squares a line of text takes up, then find a place in your grid where that word will fit. You could use something like a 2D array of boolean values, for example.
Or you could generate a random location, and then check whether there's already a rectangle there. If so, pick a new random location until you find a clear spot.
In any case, you'll probably need to use collision detection (either point-rectangle or rectangle-rectangle) to determine whether your rectangles are overlapping.
Start small. Create a small example program that just shows two rectangles on the screen. Hardcode their positions at first, but make it so they turn red if they're colliding. Work your way up from there. Make it so you can add rectangles using the mouse, but only let the user add them if there is no overlap. Then add the random location choosing. If you get stuck on a specific step, then post a MCVE and we'll go from there. Good luck.

What is the fastest way to find the center of an irregular convex polygon?

I'm interested in a fast way to calculate the rotation-independent center of a simple, convex, (non-intersecting) 2D polygon.
The example below (on the left) shows the mean center (sum of all points divided by the total), and the desired result on the right.
Some options I've already considered.
bound-box center (depends on rotation, and ignores points based on their relation to the axis).
Straight skeleton - too slow to calculate.
I've found a way which works reasonably well, (weight the points by the edge-lengths) - but this means a square-root call for every edge - which I'd like to avoid.(Will post as an answer, even though I'm not entirely satisfied with it).
Note, I'm aware of this questions similarity with:What is the fastest way to find the "visual" center of an irregularly shaped polygon?
However having to handle convex polygons increases the complexity of the problem significantly.
The points of the polygon can be weighted by their edge length which compensates for un-even point distribution.
This works for convex polygons too but in that case the center point isn't guaranteed to be inside the polygon.
Psudo-code:
def poly_center(poly):
sum_center = (0, 0)
sum_weight = 0.0
for point in poly:
weight = ((point - point.next).length +
(point - point.prev).length)
sum_center += point * weight
sum_weight += weight
return sum_center / sum_weight
Note, we can pre-calculate all edge lengths to halve the number of length calculations, or reuse the previous edge-length for half+1 length calculations. This is just written as an example to show the logic.
Including this answer for completeness since its the best method I've found so far.
There is no much better way than the accumulation of coordinates weighted by the edge length, which indeed takes N square roots.
If you accept an approximation, it is possible to skip some of the vertices by curve simplification, as follows:
decide of a deviation tolerance;
start from vertex 0 and jump to vertex M (say M=N/2);
check if the deviation along the polyline from 0 to M exceeds the tolerance (for this, compute the height of the triangle formed by the vertices 0, M/2, M);
if the deviation is exceeded, repeat recursively with 0, M/4, M/2 and M/2, 3M/4, M;
if the deviation is not exceeded, assume that the shape is straight between 0 and M.
continue until the end of the polygon.
Where the points are dense (like the left edge on your example), you should get some speedup.
I think its easiest to do something with the center of masses of the delaunay triangulation of the polygon points. i.e.
def _centroid_poly(poly):
T = spatial.Delaunay(poly).simplices
n = T.shape[0]
W = np.zeros(n)
C = 0
for m in range(n):
sp = poly[T[m,:],:]
W[m] = spatial.ConvexHull(sp).volume
C += W[m] +np.mean(sp, axis = 0)
return C / np.sum(W)
This works well for me!

How to calculate area of intersection of an arbitrary triangle with a square?

So, I've been struggling with a frankly now infuriating problem all day today.
Given a set of verticies of a triangle on a plane (just 3 points, 6 free parameters), I need to calculate the area of intersection of this triangle with the unit square defined by {0,0} and {1,1}. (I choose this because any square in 2D can be transformed to this, and the same transformation can move the 3 vertices).
So, now the problem is simplified down to only 6 parameters, 3 points... which I think is short enough that I'd be willing to code up the full solution / find the full solution.
( I would like this to run on a GPU for literally more than 2 million triangles every <0.5 seconds, if possible. as for the need for simplification / no data structures / libraries)
In terms of my attempt at the solution, I've... got a list of ways I've come up with, none of which seem fast or ... specific to the nice case (too general).
Option 1: Find the enclosed polygon, it can be anything from a triangle up to a 6-gon. Do this by use of some intersection of convex polygon in O(n) time algorithms that I found. Then I would sort these intersection points (new vertices, up to 7 of them O(n log n) ), in either CW or CCw order, so that I can run a simple area algorithm on the points (based on Greens function) (O(n) again). This is the fastest i can come with for an arbitrary convex n-gon intersecting with another m-gon. However... my problem is definitely not that complex, its a special case, so it should have a better solution...
Option 2:
Since I know its a triangle and unit square, i can simply find the list of intersection points in a more brute force way (rather than using some algorithm that is ... frankly a little frustrating to implement, as listed above)
There are only 19 points to check. 4 points are corners of square inside of triangle. 3 points are triangle inside square. And then for each line of the triangle, each will intersect 4 lines from the square (eg. y=0, y=1, x=0, x=1 lines). that is another 12 points. so, 12+3+4 = 19 points to check.
Once I have the, at most 6, at fewest 3, points that do this intersection, i can then follow up with one of two methods that I can think of.
2a: Sort them by increasing x value, and simply decompose the shape into its sub triangle / 4-gon shapes, each with an easy formula based on the limiting top and bottom lines. sum up the areas.
or 2b: Again sort the intersection points in some cyclic way, and then calculate the area based on greens function.
Unfortunately, this still ends up being just as complex as far as I can tell. I can start breaking up all the cases a little more, for finding the intersection points, since i know its just 0s and 1s for the square, which makes the math drop out some terms.. but it's not necessarily simple.
Option 3: Start separating the problem based on various conditions. Eg. 0, 1, 2, or 3 points of triangle inside square. And then for each case, run through all possible number of intersections, and then for each of those cases of polygon shapes, write down the area solution uniquely.
Option 4: some formula with heaviside step functions. This is the one I want the most probably, I suspect it'll be a little... big, but maybe I'm optimistic that it is possible, and that it would be the fastest computationally run time once I have the formula.
--- Overall, I know that it can be solved using some high level library (clipper for instance). I also realize that writing general solutions isn't so hard when using data structures of various kinds (linked list, followed by sorting it). And all those cases would be okay, if I just needed to do this a few times. But, since I need to run it as an image processing step, on the order of >9 * 1024*1024 times per image, and I'm taking images at .. lets say 1 fps (technically I will want to push this speed up as fast as possible, but lower bound is 1 second to calculate 9 million of these triangle intersection area problems). This might not be possible on a CPU, which is fine, I'll probably end up implementing it in Cuda anyways, but I do want to push the limit of speed on this problem.
Edit: So, I ended up going with Option 2b. Since there are only 19 intersections possible, of which at most 6 will define the shape, I first find those 3 to 6 verticies. Then i sort them in a cyclic (CCW) order. And then I find the area by calculating the area of that polygon.
Here is my test code I wrote to do that (it's for Igor, but should be readable as pseudocode) Unfortunately it's a little long winded, but.. I think other than my crappy sorting algorithm (shouldn't be more than 20 swaps though, so not so much overhead for writing better sorting)... other than that sorting, I don't think I can make it any faster. Though, I am open to any suggestions or oversights I might have had in chosing this option.
function calculateAreaUnitSquare(xPos, yPos)
wave xPos
wave yPos
// First, make array of destination. Only 7 possible results at most for this geometry.
Make/o/N=(7) outputVertexX = NaN
Make/o/N=(7) outputVertexY = NaN
variable pointsfound = 0
// Check 4 corners of square
// Do this by checking each corner against the parameterized plane described by basis vectors p2-p0 and p1-p0.
// (eg. project onto point - p0 onto p2-p0 and onto p1-p0. Using appropriate parameterization scaling (not unit).
// Once we have the parameterizations, then it's possible to check if it is inside the triangle, by checking that u and v are bounded by u>0, v>0 1-u-v > 0
variable denom = yPos[0]*xPos[1]-xPos[0]*yPos[1]-yPos[0]*xPos[2]+yPos[1]*xPos[2]+xPos[0]*yPos[2]-xPos[1]*yPos[2]
//variable u00 = yPos[0]*xPos[1]-xPos[0]*yPos[1]-yPos[0]*Xx+yPos[1]*Xx+xPos[0]*Yx-xPos[1]*Yx
//variable v00 = -yPos[2]*Xx+yPos[0]*(Xx-xPos[2])+xPos[0]*(yPos[2]-Yx)+yPos[2]*Yx
variable u00 = (yPos[0]*xPos[1]-xPos[0]*yPos[1])/denom
variable v00 = (yPos[0]*(-xPos[2])+xPos[0]*(yPos[2]))/denom
variable u01 =(yPos[0]*xPos[1]-xPos[0]*yPos[1]+xPos[0]-xPos[1])/denom
variable v01 =(yPos[0]*(-xPos[2])+xPos[0]*(yPos[2]-1)+xPos[2])/denom
variable u11 = (yPos[0]*xPos[1]-xPos[0]*yPos[1]-yPos[0]+yPos[1]+xPos[0]-xPos[1])/denom
variable v11 = (-yPos[2]+yPos[0]*(1-xPos[2])+xPos[0]*(yPos[2]-1)+xPos[2])/denom
variable u10 = (yPos[0]*xPos[1]-xPos[0]*yPos[1]-yPos[0]+yPos[1])/denom
variable v10 = (-yPos[2]+yPos[0]*(1-xPos[2])+xPos[0]*(yPos[2]))/denom
if(u00 >= 0 && v00 >=0 && (1-u00-v00) >=0)
outputVertexX[pointsfound] = 0
outputVertexY[pointsfound] = 0
pointsfound+=1
endif
if(u01 >= 0 && v01 >=0 && (1-u01-v01) >=0)
outputVertexX[pointsfound] = 0
outputVertexY[pointsfound] = 1
pointsfound+=1
endif
if(u10 >= 0 && v10 >=0 && (1-u10-v10) >=0)
outputVertexX[pointsfound] = 1
outputVertexY[pointsfound] = 0
pointsfound+=1
endif
if(u11 >= 0 && v11 >=0 && (1-u11-v11) >=0)
outputVertexX[pointsfound] = 1
outputVertexY[pointsfound] = 1
pointsfound+=1
endif
// Check 3 points for triangle. This is easy, just see if its bounded in the unit square. if it is, add it.
variable i = 0
for(i=0; i<3; i+=1)
if(xPos[i] >= 0 && xPos[i] <= 1 )
if(yPos[i] >=0 && yPos[i] <=1)
if(!((xPos[i] == 0 || xPos[i] == 1) && (yPos[i] == 0 || yPos[i] == 1) ))
outputVertexX[pointsfound] = xPos[i]
outputVertexY[pointsfound] = yPos[i]
pointsfound+=1
endif
endif
endif
endfor
// Check intersections.
// Procedure is: loop over 3 lines of triangle.
// For each line
// Check if vertical
// If not vertical, find y intercept with x=0 and x=1 lines.
// if y intercept is between 0 and 1, then add the point
// Check if horizontal
// if not horizontal, find x intercept with y=0 and y=1 lines
// if x intercept is between 0 and 1, then add the point
for(i=0; i<3; i+=1)
variable iN = mod(i+1,3)
if(xPos[i] != xPos[iN])
variable tx0 = xPos[i]/(xPos[i] - xPos[iN])
variable tx1 = (xPos[i]-1)/(xPos[i] - xPos[iN])
if(tx0 >0 && tx0 < 1)
variable yInt = (yPos[iN]-yPos[i])*tx0+yPos[i]
if(yInt > 0 && yInt <1)
outputVertexX[pointsfound] = 0
outputVertexY[pointsfound] = yInt
pointsfound+=1
endif
endif
if(tx1 >0 && tx1 < 1)
yInt = (yPos[iN]-yPos[i])*tx1+yPos[i]
if(yInt > 0 && yInt <1)
outputVertexX[pointsfound] = 1
outputVertexY[pointsfound] = yInt
pointsfound+=1
endif
endif
endif
if(yPos[i] != yPos[iN])
variable ty0 = yPos[i]/(yPos[i] - yPos[iN])
variable ty1 = (yPos[i]-1)/(yPos[i] - yPos[iN])
if(ty0 >0 && ty0 < 1)
variable xInt = (xPos[iN]-xPos[i])*ty0+xPos[i]
if(xInt > 0 && xInt <1)
outputVertexX[pointsfound] = xInt
outputVertexY[pointsfound] = 0
pointsfound+=1
endif
endif
if(ty1 >0 && ty1 < 1)
xInt = (xPos[iN]-xPos[i])*ty1+xPos[i]
if(xInt > 0 && xInt <1)
outputVertexX[pointsfound] = xInt
outputVertexY[pointsfound] = 1
pointsfound+=1
endif
endif
endif
endfor
// Now we have all 6 verticies that we need. Next step: find the lowest y point of the verticies
// if there are multiple with same low y point, find lowest X of these.
// swap this vertex to be first vertex.
variable lowY = 1
variable lowX = 1
variable m = 0;
for (i=0; i<pointsfound ; i+=1)
if (outputVertexY[i] < lowY)
m=i
lowY = outputVertexY[i]
lowX = outputVertexX[i]
elseif(outputVertexY[i] == lowY)
if(outputVertexX[i] < lowX)
m=i
lowY = outputVertexY[i]
lowX = outputVertexX[i]
endif
endif
endfor
outputVertexX[m] = outputVertexX[0]
outputVertexY[m] = outputVertexY[0]
outputVertexX[0] = lowX
outputVertexY[0] = lowY
// now we have the bottom left corner point, (bottom prefered).
// calculate the cos(theta) of unit x hat vector to the other verticies
make/o/N=(pointsfound) angles = (p!=0)?( (outputVertexX[p]-lowX) / sqrt( (outputVertexX[p]-lowX)^2+(outputVertexY[p]-lowY)^2) ) : 0
// Now sort the remaining verticies based on this angle offset. This will orient the points for a convex polygon in its maximal size / ccw orientation
// (This sort is crappy, but there will be in theory, at most 25 swaps. Which in the grand sceme of operations, isn't so bad.
variable j
for(i=1; i<pointsfound; i+=1)
for(j=i+1; j<pointsfound; j+=1)
if( angles[j] > angles[i] )
variable tempX = outputVertexX[j]
variable tempY = outputVertexY[j]
outputVertexX[j] = outputVertexX[i]
outputVertexY[j] =outputVertexY[i]
outputVertexX[i] = tempX
outputVertexY[i] = tempY
variable tempA = angles[j]
angles[j] = angles[i]
angles[i] = tempA
endif
endfor
endfor
// Now the list is ordered!
// now calculate the area given a list of CCW oriented points on a convex polygon.
// has a simple and easy math formula : http://www.mathwords.com/a/area_convex_polygon.htm
variable totA = 0
for(i = 0; i<pointsfound; i+=1)
totA += outputVertexX[i]*outputVertexY[mod(i+1,pointsfound)] - outputVertexY[i]*outputVertexX[mod(i+1,pointsfound)]
endfor
totA /= 2
return totA
end
I think the Cohen-Sutherland line-clipping algorithm is your friend here.
First off check the bounding box of the triangle against the square to catch the trivial cases (triangle inside square, triangle outside square).
Next check for the case where the square lies completely within the triangle.
Next consider your triangle vertices A, B and C in clockwise order. Clip the line segments AB, BC and CA against the square. They will either be altered such that they lie within the square or are found to lie outside, in which case they can be ignored.
You now have an ordered list of up to three line segments that define the some of the edges intersection polygon. It is easy to work out how to traverse from one edge to the next to find the other edges of the intersection polygon. Consider the endpoint of one line segment (e) against the start of the next (s)
If e is coincident with s, as would be the case when a triangle vertex lies within the square, then no traversal is required.
If e and s differ, then we need to traverse clockwise around the boundary of the square.
Note that this traversal will be in clockwise order, so there is no need to compute the vertices of the intersection shape, sort them into order and then compute the area. The area can be computed as you go without having to store the vertices.
Consider the following examples:
In the first case:
We clip the lines AB, BC and CA against the square, producing the line segments ab>ba and ca>ac
ab>ba forms the first edge of the intersection polygon
To traverse from ba to ca: ba lies on y=1, while ca does not, so the next edge is ca>(1,1)
(1,1) and ca both lie on x=1, so the next edge is (1,1)>ca
The next edge is a line segment we already have, ca>ac
ac and ab are coincident, so no traversal is needed (you might be as well just computing the area for a degenerate edge and avoiding the branch in these cases)
In the second case, clipping the triangle edges against the square gives us ab>ba, bc>cb and ca>ac. Traversal between these segments is trivial as the start and end points lie on the same square edges.
In the third case the traversal from ba to ca goes through two square vertices, but it is still a simple matter of comparing the square edges on which they lie:
ba lies on y=1, ca does not, so next vertex is (1,1)
(1,1) lies on x=1, ca does not, so next vertex is (1,0)
(1,0) lies on y=0, as does ca, so next vertex is ca.
Given the large number of triangles I would recommend scanline algorithm: sort all the points 1st by X and 2nd by Y, then proceed in X direction with a "scan line" that keeps a heap of Y-sorted intersections of all lines with that line. This approach has been widely used for Boolean operations on large collections of polygons: operations such as AND, OR, XOR, INSIDE, OUTSIDE, etc. all take O(n*log(n)).
It should be fairly straightforward to augment Boolean AND operation, implemented with the scanline algorithm to find the areas you need. The complexity will remain O(n*log(n)) on the number of triangles. The algorithm would also apply to intersections with arbitrary collections of arbitrary polygons, in case you would need to extend to that.
On the 2nd thought, if you don't need anything other than the triangle areas, you could do that in O(n), and scanline may be an overkill.
I came to this question late, but I think I've come up with a more fully flushed out solution along the lines of ryanm's answer. I'll give an outline of for others trying to do this problem at least somewhat efficiently.
First you have two trivial cases to check:
1) Triangle lies entirely within the square
2) Square lies entirely within the triangle (Just check if all corners are inside the triangle)
If neither is true, then things get interesting.
First, use either the Cohen-Sutherland or Liang-Barsky algorithm to clip each edge of the triangle to the square. (The linked article contains a nice bit of code that you can essentially just copy-paste if you're using C).
Given a triangle edge, these algorithms will output either a clipped edge or a flag denoting that the edge lies entirely outside the square. If all edges lie outsize the square, then the triangle and the square are disjoint.
Otherwise, we know that the endpoints of the clipped edges constitute at least some of the vertices of the polygon representing the intersection.
We can avoid a tedious case-wise treatment by making a simple observation. All other vertices of the intersection polygon, if any, will be corners of the square that lie inside the triangle.
Simply put, the vertices of the intersection polygon will be the (unique) endpoints of the clipped triangle edges in addition to the corners of the square inside the triangle.
We'll assume that we want to order these vertices in a counter-clockwise fashion. Since the intersection polygon will always be convex, we can compute its centroid (the mean over all vertex positions) which will lie inside the polygon.
Then to each vertex, we can assign an angle using the atan2 function where the inputs are the y- and x- coordinates of the vector obtained by subtracting the centroid from the position of the vertex (i.e. the vector from the centroid to the vertex).
Finally, the vertices can be sorted in ascending order based on the values of the assigned angles, which constitutes a counter-clockwise ordering. Successive pairs of vertices correspond to the polygon edges.

What is the complexity of this code?

I have a the above model represented in a Face Table List where the F1, F2,...F_n are the faces of the model and their face number is the index of the list array. Each list element is another array of 3 vertices. And each vertex is an array of 3 integers representing its x,y,z coordinates.
I want to find out all the neighbouring faces of the vertex with coordinates (x2, y2, z2). I came out with this code that I believe would do the task:
List faceList; //say the faceList is the table in the picture above.
int[] targetVertex = {x2, y2, z2}; //say this is the vertex I want to find with coordinates (x2, y2, z2)
List faceIndexFoundList; //This is the result, which is a list of index of the neighbouring faces of the targetVertex
for(int i=0; i<faceList.length; i++) {
bool vertexMatched = true;
for(int j=0; j<faceList[i].length; j++) {
if(faceList[i][j][0] != targetVertex[0] && faceList[i][j][1] != targetVertex[1] && faceList[i][j][2] != targetVertex[2]) {
vertexMatched = false;
break;
}
}
if(vertexMatched == true) {
faceIndexFoundList.add(i);
}
}
I was told that the complexity to do the task is O(N^2). But with the code that I have, it looks like only O(N). The length of targetVertex is 3 since there is only 3 vertices per polygon. So, the second inner loop is merely a constant. Then, I left only with the outer for loop, which is then O(N) only.
What is the complexity of the code that I have above? What could I have done wrong?
The complexity is (aproximatly) faceList.length * faceList[i].length, these are independent, but can both grow very large, and as they grow they will each approch infinity at which point (conceptually) they will converge on n, resulting in the complexity being O(n^2)
If the vertex list is explicitly limited to 3, then the complexity becomes faceList[i].length * 3, which is O(n)
It's pretty obvious that in the worst case you must look at each vertex of each polygon.
This is just O(size of the table) in your post, which in turn is the sum of all row lengths or the sum of all polygon vertex counts, whichever you prefer.
If you say polygons have no more than m vertices and there are n polygons, then the algorithm is O(mn).
FWIW it's possible to get the answer with no searching at all with a more sophisticated data structure. See for example the winged edge data structure and others. In this case, you just go to the vertex you're interested in and traverse the links that connect all adjacent polygons. Cost is constant for each polygon in the output.
These fancier data structures for polygonal meshes support lots of frequently used operations with wonderful efficiency.
From Wikipedia:
Big O notation is used to describe the limiting behavior of a function when the argument tends towards a particular value or infinity.
In this single case you might only be running the one for loop. But what happens when the number of vertices of the polygon approaches infinity? Do the majority of the cases cause the second for loop to run, or to break? This will determine whether your function is O(n) or O(n^2).

Resources