I have a large dataset of points and to stop from showing them all at the same time i'm using nested regions. These regions have network links in them to a server which generates kml files of points and more network links. I'll show x points at the outermost region. That region is made up of smaller regions each showing x points etc. I don't want to show each point more than once. Each point is eligible under many different regions. How can i determine if i've already shown a point?
Ideas:
Use a deterministic algorithm to pick out the points for a level so each region can reverse calculate what points would already have been selected. Problem is that as soon as a point gets added to the database, the algorithm is no longer going to work as the parameters have changed. Also sounds sloooow.
store in the db upon population what region a point should be shown at based on some algorithm. Problem is have to decide where to put a point every time you add one. (not that bad initially assign regions for all points though)
Edit: Or instead of using accumulative regions i could set the maximum LOD for each region.
Con is lots of points could be loaded multiple times. Pro, should be pretty simple / updating DB when already in google earth won't hurt things
Edit Edit: Tried using exclusive regions (https://developers.google.com/kml/documentation/images/LodRange3.gif) Ran into some problems. Namely, if you set there is slight overlap where one region ends (max LOD) and another starts (min LOD) could make it so there's a gap, but it will never be perfect. Also this is okay if you start fully zoomed out so the outermost region is 'active', then as you zoom in somewhere the regions will become active. Problem is if you start zoomed in, then your your outer region is never loaded, so it never loads any of it's subregions and the placemarks won't be displayed. Generating all the possible regions from the start (just not displaying them) is not an option as there would be millions of files
Related
I am working on a project that requires fast performing proximity queries on a database with location data.
In my database I want to store locations with additional information. Idea is that user opens a map on a certain location and my program only fetches the markers visible to the user. If I plan on having millions of values, fetching markers from NYC when I'm zoomed in on London would make the map activity work extremely slow and the data I send back from the db would be HUGE.
That's why when the user opens the map I want to fetch all the markers that are for example in 10km distance from the center of the map. (I'm okay with fetching markers outside of the visible area. I just don't want to fetch markers that are 100km away)
After a thorough research I chose the S2 Geometry Library approach with Hilbert's space filling curve.
The idea of mapping a 2D value to one integer value, where the longer a shared prefix between two indexes is, the spatially closer they are together, was a big selling point.
I need my database to be able to perform this SELECT query lightning fast and I expect to have A LOT of data in the future so operating on only one column is a big plus.
Also the thing that intrigued me the most was the ability to perform fast proximity searches because of the fact that two numbers that are close to each other on the map will have 1D indexes also close to each other.
The idea looks very simple (If I don't miss anything).
The thing I'm having problems with is how to (If it's even possible) pick the min value and max value on the 1D plane to be sure I'm scanning the whole visible area.
Most of the answers and tutorials I find on the internet propose a solution where you take a bounding area full of smaller S2 index "boxes" and then scan every index in the database to see if it's contained in one of the "boxes" from the array. This is easy to do but when you have 50 milion records it's not possible to go through every single one of them to see if it's in on of the "boxes".
What I have in mind is a solution where you take the minimum value of the area and the maximum value of the area you're searching in and you perform something in the lines of SELECT (...) WHERE s2cellid BETWEEN min AND max
For example I'm in a location 47194c and want to fetch all markers in 10km distance so I take a value that's x to the left of the indeks and a value that's x to the right of the index and perform a BETWEEN 47194c-x AND 47194c+x query
Is something like that possible with the S2 library?
If no then what approach should I take to make my queries as quick as possible?
Thanks in advance :)
[I plan on using PostgreSQL]
I've spent hours searching for an answer to this, but in most cases either the
question is about plots/charts (rather than graphs as in "control flow graph"),
or the answer "just use graphviz" is a valid answer.
However I have some constraints and requirements that make "just use graphviz" a
non-answer.
The full graph is large enough that it's not possible to generate a graphviz
for all of it.
Nodes and edges will be dynamically added and removed.
Nodes have lots of information that will be hidden by default and will be
expanded on request (imagine every node as a table with expandable rows/cols)
I want to be able to show only a subset of the graph on request, e.g. for
features like "only show reachable part of the graph from this node" or "show
all simple paths from this node to this node".
Basically I want to be able to start drawing nodes and edges on a 2D plane, and
add new nodes and edges dynamically. It's fine if nodes/edges move around as new
stuff is added. While I don't yet have hard requirements for this, it'd be good
if it looked "nice" -- for example if a node has lots of incoming edges (this is
a directed graph) ideally it'd be in a central place on the plane with all other
nodes around it etc.
Anything that gets me going would be helpful. Thanks.
(I don't know what label to add to this, adding "graph-theory" because I don't know what else to add)
I'm using the Overpass API to query Open Street Maps for nearby road segments. I am pretty sure that my query is returning all of the nodes of the nearby way... but I only want nearby nodes of the nearby way.
In the documentation it references this problem:
In general, you will be rather interested in complete data than just
elements of a single type. First, there are several valid definitions
of what "complete map data" means. The first unclear topic is what to
do with nodes outside the bounding box which are members of ways that
lie partly inside the bounding box.
The same question repeats for relations. If you wait for a turn
restriction, you may prefer to get all elements of the relation
included. If your bounding box hits for example the border of Russia,
you likely don't want to download ten thousands kilometers of boundary
around half the world.
But I looked at the subsequent examples and didn't see the solution.
Basically, in their example, how would I restrict the elements returned to those strictly in the bounding box (rather than returning the whole boundary of Russia)?
My current query is
way (around:100,50.746,7.154) [highway~"^(secondary|tertiary)$"];
>;
out ids geom;
I'm thinking maybe I need to change it to node (around:...) and then recurse upwards to the way to query for the highway tag but I'm not sure if I am even on the right track.
Actually, it's even a bit more complicated, as you need the set intersection of all nodes in a 100m distance and those nodes belonging to one of the relevant ways. Here's how your query should look like: Adjust distance, tags for ways as needed.
Note that depending on the tagging, there's no guarantee that you will find a node in a certain distance, especially if roads tend to be rather straight and longish. This for sure will impact your results, so a bit experimenting with a suitable radius is probably needed.
// Find nodes up to 100m around center point
// (center is overpass turbo specific for center point lat/lon in current map view)
node(around:100,{{center}})->.aroundnodes;
// recurse up to ways with highway = secondary/tertiary
way(bn.aroundnodes)[highway~"^(secondary|tertiary)$"]->.allways;
// determine nodes belonging to found ways
node(w.allways)->.waynodes;
(
// determine intersection of all ways' nodes and nodes around center point
node.waynodes.aroundnodes;
// and return ways (intersection is just a workaround for a bug)
way.allways.allways;
);
out;
check it out in overpass turbo: http://overpass-turbo.eu/s/hPV
I'm trying to add multiple overlays of about 1500 on map view. I'm getting the locations from database and adding them on map view. The time to get data from database is very low but the time it takes to draw them on map is very high which is about 30 sec and I want to add overlays based on zoom levels, like level<4 1000 overlays, >=4 2000 overlays, redrawing these overlays screwed me. Please show me the solution to add them in a less amount of time.
I've had another problem with multiple overlays, it is causing memory issues on an actual device (Not the sim). The solution to this was creating one overlay from all. This might also be the solution to your problem as drawing the "combined-overlay" should be a lot faster:
The credits go to this answer and the code provided on the Apple Dev-forum
You then should be able to create one overlay from all and then draw that one overlay on the map.
Basically you create a class that handles the multiple overlays and draws them together onto on OverlayView
My algorithm is processing DEMs. a DEM (Digital Elevation Model) is a representations of ground topography where elevation is known at grid nodes.
My problem can be summarized as follows:
Q is a queue containing nodes to visit.
at start, the boundary of the grid is pushed in Q.
while Q is not empty, do
remove Node N from the top of Q
if N was never visited then do
consider the 8 neighbors of N
among them select the unvisited ones
among them select those with a higher elevation than N's
push these at Q's tail
mark N as visited
done
done
As described, the algorithm will mark as 'visited' every node that can be reached from the border by a continuously ascendant path. It is worth noticing that the order of processing the nodes in the queue is unimportant. Note also that some points may request a tortuous ascendant path to be reached from the border. Think for example to a cone with a furrow spiraling around it. The ridge of the furrow is such a unique and tortuous path capable of reaching the top of the cone without never descending into the furrow.
Anyway, I want to mutithread this algorithm. I am still in the first step of wondering which is the best organization of data and threads in order to have as least pain as possible at debugging the beast when it is written.
My first thought is to divide the grid into tiles and split the Queue in as many tiles as there is in the grid. The tiles are piled in a work-list. A few threads are parsing the work-list and grab any tile where something can be done at the moment.
Working on a specific tile will firstly need that the tile's queue is not empty. I may also need that the neighboring tiles can be locked if the walker's tile has to visit a node at the edge of the tile.
I am thinking that when a walker cannot lock a neighboring tile while it needs to, then it can skip to the next node in the local queue, or even the thread itself can release the tile to the work-list and seek for another tile to work on.
My actual experience of multi-thread programming is good enough to understand that this lovely description is very likely to turn into a nightmare when I will debug it. However I am not experienced enough to evaluate the various possibilities of programming the algorithm and make a good decision, having in mind that I will not be given a month to debug a spaghetti dish.
Thanks for reading :)