I have a use case where there in an API in which multiple values are required to be used for searching for an indexed attribute. In this case, how would we make the API paginated? DynamoDB allows only one value to be searched for an indexed attribute in a single query, so how can we do it?
Related
We have an index set up in Azure cognitive search that has two string fields (hash1 & hash2) containing separate hashes. We would like to query the index for documents where the two hashes within a document aren't equal.
I tried applying the filter: $filter=hash1 ne hash2, expecting the query to return all documents with mismatched hashes. Instead, I was greeted with the following error message:
"Invalid expression: Comparison must be between a field, range variable or function call and a literal value.\r\nParameter name: $filter"
From what I can gather there seems to be some kind of limitation preventing comparisons between fields. Would it be possible to perform this type of query in Azure cognitive search using a different technique?
I would use content enrichment in this case. Even if comparing two hashes with a query was supported, it would be inefficient compared to pre-calculating the value using a content enrichment technique.
Introduce a new boolean property called something like HasEqualHashes
Populate that property with an appropriate boolean value
Use a $filter to filter your content as you wish
search=whatever&$filter=HasEqualHashes
Note that two different scenarios determine how you can enrich your content.
CONTENT SUBMITTED VIA SDK
When you use the SDK to submit content, you can enrich your items any way you want using regular code. Populating your HasEqualHashes property is a trivial one-liner in C#.
CONTENT SUBMITTED USING BUILT-IN INDEXERS
If you use one of the built-in indexers, you have to learn and understand the concept of skillsets.
https://learn.microsoft.com/en-us/azure/search/cognitive-search-working-with-skillsets
The data I have is of the form
{"event": {"custom": {"dimensions": [{"Id": ....}, {},...{}]}, ...},...}
The key that I need to index by is in the list. However, Cognitive Search does not seem to let me access the value within the list. Azure Cog. Search also fails to access any content from the list while trying to index.
Are there any solutions you can think?
Not sure how you're trying, but Azure Cognitive Search supports Complex types. Take a look in the following link:
https://learn.microsoft.com/en-us/azure/search/search-howto-complex-data-types
As an Alternative, you can project the internal dimensions (assuming they have a fixed number of dimensions) to fields in your index.
When using Indexers to import the data, key fields are limited to what can be expressed in a field mapping which has some support for mapping functions but wont allow you to select a value of an object in a collection. Your only options are to pre-process and transform the data (such as a query if this is coming from Cosmos DB, or azure function trigger if coming from blobs) or use a different field as the id and put the dimension id in another field that is queryable.
To make the data queryable you can use complex types or if the dimensions are always in the same ordinal you can use output field mappings to map it to a field by collection ordinal such as /document/event/custom/dimensions/1.
I am using azure cosmos db for saving and editing my session information. Currently i am not using ID in my document, instead i have another unique field with all docs. How can i update my query to get documents?
You can use whatever property you want, for your custom key (just make sure you don't remove its index). By default, all properties are indexed unless you explicitly set up a custom index policy that removes certain properties from being indexed.
You cannot eliminate the built-in id property though; if you don't set it explicitly, it will just be set to a guid.
If you are doing queries, this really shouldn't matter, functionality-wise. Just search on whatever properties you want. However: If you are doing point-reads (a read is more efficient, RU-wise, than a query, when retrieving a single document) you can only perform a point-read by specifying the id property, not your custom property. If you must use a custom property and you need to do point-reads, you can consider storing your custom property as id as well (as long as it's guaranteed to be unique per document).
Trying to do a where filter in findOne and in Node api returns empty array when filtering by ids in loopback
https://url/api/Model1/findOne?filter={"where":{"attrs":"id"}}
where id is exactly 24 digits long
It is possible that you simple don't have an object with that ID in that particular collection in the database. In such a case you should get an empty array.
Or you may need to find by id and not by attr - it depends on how the relevant field in your database is named.
See the docs, there are good examples there: https://loopback.io/doc/en/lb2/Where-filter.html
E.g. this:
http://localhost:3000/api/Books?filter={"where":{"or":[{"id":1},{"id":2}]}}
Note that the id is used and not attr. See what is your field in your case.
I'm considering using CouchBase for a very read heavy and write heavy application. I'll also need to support searching based on different attributes of the documents as well as range queries.
CouchBase has views to allow searching beyond key value searches but it seems like this is mainly to get documents within a certain range, eg. get all documents indexed between two specified keys, rather than "give me all documents that have the genre attribute to 'adventure'" or "give me all documents that have creation date between 1/1/1 and 2/1/1"
Is there a way to achieve what I want without an external index?
You can definitely do both of what you describe there. You'd do both with views in Couchbase Server 2.0.
For example, a common technique when needing to search a date range is to emit a JSON array from your map function in the view. This would give you something like:
[2012, 5, 11, 16, 27, 41]
Since when you query a view, a JSON array is a valid place for start key and end key, you can specify that range.
Similarly, extracting all of the attributes you'd emit each one of them from your map function with the doc _id. Then using one of the Couchbase SDKs, you can set the include docs option when querying and the doc will be automatically fetched.