Search engine by distance - geospatial

I am looking to make an option of my serach engine on my site so that users can search for items within a set distance, e.g. search items within 10 miles, or 20 miles etc. I was wondering how this could be done?
The user would have to enter thier postcode, while i also have the postcode of the item's location and once they hit search there needs to be a away to work the distance between the two locations in miles and then display the results in order by distance; as in the closest item is the first result. Is there a google api for this as in use the maps 'get directions' option to work out the distance in miles? Or something i can add to my database?

The Google Geocoding API provides zip code lookup and can provide the country, city, lat/lon given even just a zip code as the address. Once you have the lat/lon then you can easily calculate the distance and sort the results.
http://code.google.com/apis/maps/documentation/geocoding/#GeocodingRequests
Note: you can only use the Geocoding API in conjunction with a Google map; geocoding results without displaying them on a map is prohibited. For complete details on allowed usage, consult the link.
So if for example if you request lookup for zip code 94043 you call following URL:
http://maps.googleapis.com/maps/api/geocode/json?address=94043&sensor=false
Which would result with JSON such as following:
{
"results" : [
{
"address_components" : [
{
"long_name" : "94043",
"short_name" : "94043",
"types" : [ "postal_code" ]
},
...
"location" : {
"lat" : 37.4284340,
"lng" : -122.07238160
},
"location_type" : "APPROXIMATE",
...
"status" : "OK"
}
If you cannot use the Google API for some reason then here is list of non-Google Geocoder APIs and services:
http://code.google.com/p/gmaps-api-issues/wiki/NonGoogleGeocoders
http://webgis.usc.edu/Services/Geocode/About/GeocoderList.aspx

Related

CosmosDb Mongo - collection with shardkey, slow query by shardkey?

I have a CosmosDb collection with Mongodb.
This is a customer database, and the ShardKey is actually CustomerId.
My collection has 200000 records, and has an combined index of both e-mail and customerid.
An example of a customer:
{
"CustomerId" : "6a0f4360-d722-4926-9751-9c7fe6a97cb3",
"FirstName" : "This is my company first name",
"LastName" : "This is my company last name",
"Email" : "6a0f4360-d722-4926-9751-9c7fe6a97cb3#somemail.com",
"Addresses" : [
{
"AddressId" : "54e34da9-55fb-4d60-8411-107985c7382e",
"Door" : "11111",
"Floor" : "99",
"Side" : "B",
"ZipCode" : "8888",
}
]
}
What I find strange is if I query by Email it spends 7000RUs (which is too much - at least is what data explorer tells me...) but if I query by CustomerId, it spends more or less the same RUs...
My questions are:
Shoudn't both operations spend less RUs than this, specially by CustomerId?
An example of a query by E-mail:
{ "Email" : { $eq: "3f7da6c3-81bd-4b1d-bfa9-d325388079ab#somemail.com" } }
An example of a query by CustomerId:
{ "CustomerId" : { $eq: "3f7da6c3-81bd-4b1d-bfa9-d325388079ab" } }
Another question, my index contains both Email and CustomerId. Is there any way for me to query by e-mail and return only CustomerId, for example?
Shoudn't both operations spend less RUs than this, specially by CustomerId?
CustomerId is your shard key (aka partition key) which helps in grouping documents with same value of CustomerId to be stored in the same logical partition. This grouping is used during pin-point GET/SET calls to Cosmos but not during querying. So, you would need an index on CustomerId explicitly.
Furthermore, since the index that you have is a composite index on CustomerId and Email, doing a query on only one of these fields at a time will lead to a scan being performed in order to get back the result. Hence the high RU charge and the similar amount of RU charge on each of these queries.
Another question, my index contains both Email and CustomerId. Is there any way for me to query by e-mail and return only CustomerId, for example?
Firstly, in order to query optimally on Email, you would need to create an index on Email separately. Thereafter, you may use the project feature of Mongo to include only certain fields in the response.
Something like this-
find({ "Email" : { $eq: "3f7da6c3-81bd-4b1d-bfa9-d325388079ab#somemail.com" } }, { "CustomerId":1 })

How to optimize api response in nodejs?

I have list of articles. they contain category and sub-category.
const articles = [ { name, ..., category: { name, … , subCategories: [{ name, ... }] } } ]
I have api endpoint that I send all my articles with category and subCategories (it is mandatory).
The problem is category and subcategories are repeats (and it can be big data to send), the same category and subcategory is in article1 and article90 for example.
I think about to exclude category from the article, just replace with an id
and create another property category, and send them back to the client.
res.json({ categories: [{ name: "1", ... , subCategories }], articles: [{ name , …, category: "1" }] });
How to that in easy way and generic for all my objects that repeat data in nodejs?
I think depends on the what you application need to display. But a common solution is using pagination (e.g., getting chunks of 10 unities) in the API and, then, handling the pagination in the frontend.
Other solution is having some kind of consolidation. For example, if you would list all reads of a device in a interval, that sends data to server every 5 seconds, you could run a background service to consolidate data and save a mean of all values. Then, the frontend would get the mean at the interval, not actually all reads .

Query mongo array to count how many times a specific position appears [duplicate]

I want to run a query where an item at a specific position in an array needs to be compared.
For example, consider the GeoJSON format for storing location data.
//sample document from collection user
{
name: "Some name",
location : {
type: "Point",
coordinates : [<Longitude>, <Latitude>]
}
}
How would I query users located at a specific longitude?
I cant seem to find anything in the documentation which can help me do the same.
Queries I have tried:
db.users.find({"location.coordinates[0]" : -73.04303})
Change your query to the following
db.users.find({"location.coordinates.0" : -73.04303})

How much time azure search take for index data

How much time does azure search take to index data?
Suppose I am putting a single record at a time in azure search:
POST https://[service name].search.windows.net/datasources?api-version=2015-02-28-Preview
Content-Type: application/json
api-key: [admin key]
{
"name" : "blob-datasource",
"type" : "azureblob",
"credentials" : { "connectionString" : "<my storage connection string>" },
"container" : { "name" : "my-container", "query" : "<optional-virtual-directory-name>" }
}
So how much time it will take so I can read this data back from REST API?
Your example shows creating a data source, not indexing documents. But, assuming you use indexing API (POST https://[service name].search.windows.net/indexes/index/docs), the delay before the just-indexed documents show up in search results ranges from instant to a few seconds, depending on the service topology and load.

Marklogic|NodeJS API - Query on a specific categorie "properties"

I have a json document in my DB that looks like this :
{
"uri" : "/me/myself/and/bd1e0f91656bfc713eb6560eeaad7ad1.json",
"category" : "content",
"format" : "json",
"versionId" : "14697362595356370",
"contentType" : "application/json",
"contentLength" : "1938",
"collections" : ["http://me.myself.com/collectionA"],
"properties" : {
"relatives" : ["/me/myself/and/B.json", "/me/myself/and/A.json"]
},
"content":{}
}
I'm trying to get all documents that have a specific relative in the properties:
qb.where(
qb.scope(
qb.property('relatives'),
qb.word("/me/myself/and/B.json"),
qb.fragmentScope('properties')
))
But i keep getting a large set of document that doesn't fit the query.
Any idea how to do this using the Marklogic NodeJS API?
I see two things that look like they might be problems. The first is qb.fragmentScope('properties'). This tells MarkLogic to look in the document's properties, rather than the document's content. That doesn't look like what you meant, given your sample JSON document.
The second problem is the word query -- "/me/myself/and/B.json" is likely being broken up into its constituent words (me, myself, and, B, json), which are then matching in other documents. You want to match exactly what's there, so try a value query:
qb.where(
qb.scope(
qb.properties('properties'),
qb.value('relatives', '/me/myself/and/B.json')
)
)
Note that the qb.scope and the qb.properties are to restrict the search to just match the value when it appears in relatives under a properties JSON property. This is different from the JSON property-versus-content point made above.
qb.where(
qb.propertiesFragment(
qb.term('/me/myself/and/B.json')
)
)
This worked for me.

Resources