Pagination with per-row access rights - pagination

Hi I am using CouchDB and assuming I have an articles document with the field users, containing an array of user IDs that are allowed to view this article.
Example scenario, there will be a paginated table view showing 10 articles per page, my controller will retrieve the first 10 articles from CouchDB then perform the access rights check one by one on the returned articles. But the current user may only have view access rights on say, 8 of them, therefore the table will only show 8 articles instead of 10.
What are the best practice of handling such situation besides implementing the access rights logic on the CouchDB layer?

To accomplish this, I would simply use a view keyed on the users field:
function (doc) {
doc.users.forEach(function (user) {
emit([ user ]);
});
}
I emitted an array with just 1 item in this case. I presume you'd also emit something like doc.created in order to have your articles sorted, you would simply add them after user in that array.
The view results would look something like:
{
"rows": [
{ "id": "<article-1>", "key": [ "<user-1>", "<created>" ] },
{ "id": "<article-2>", "key": [ "<user-1>", "<created>" ] },
{ "id": "<article-3>", "key": [ "<user-1>", "<created>" ] },
{ "id": "<article-1>", "key": [ "<user-2>", "<created>" ] },
{ "id": "<article-1>", "key": [ "<user-3>", "<created>" ] }
]
}
You can simply paginate like you normally would with CouchDB. You simply use start_key=["<user-1>"]&end_key=["<user-1>","\ufff0"] in addition to the usual paging limit=10&skip=0 for page 1, limit=10&skip=10 for page 2, etc.

Related

Available Fields for Sharepoint Search Query on Microsoft Graph API

I'm using the Graph API to make a global search in my sharepoint website, and I need to retrieve some specific fields. I didn't find any documentation that specifies the available fields that I can use for the fields property on my payload, only a documentation for specific document library search.
I have to use the global search because my search needs to access all the document libraries on my sharepoint web site.
The field that I wanted to get from the request is the version of the document in the list. I could add this field in sharepoint, and my view is displaying the version values, but the request does not take this value. I'm using this request below:
Endpoint: https://graph.microsoft.com/v1.0/search/query
Payload:
"requests": [
{
"entityTypes": [
"listItem"
],
"query": {
"queryString": ""
},
"fields": [
"title",
"_UIVersionString"
]
}
]
}
Response:
{
"value": [
{
"searchTerms": [],
"hitsContainers": [
{
"hits": [
{
"hitId": "83C63693-C621-4CFE-B4F7-A36B68AEB421",
"rank": 1,
"summary": "...",
"resource": {
"#odata.type": "#microsoft.graph.listItem",
"fields": {
"title": "Calc.22090615231879"
}
}
},
],
"total": 1,
"moreResultsAvailable": false
}
]
}
],
"#odata.context": "https://graph.microsoft.com/v1.0/$metadata#Collection(microsoft.graph.searchResponse)"
}
The name of the field I'm using in the payload that corresponds to the version is _UIVersionString, where I got it from the specific list search request, using the https://graph.microsoft.com/v1.0/sites/{site-id}/lists/{list-id}/items?$expand=fields endpoint. But sadly, the version is not appearing on my search result.
Is there some documentation I could use to see a list of available fields for this request? I'm trying to find it in MS GraphAPI documentation, but it looks be a big real encyclopedia.
Do you know the name of the field that corresponds to the version?
Thanks a lot!
Other information:
Sharepoint Version: Sharepoint Web (Online)
Type of the lists: Document Library
Lists version configuration:
- Require content approval for submitted items?: No
- Create a version each time you edit a file in this document library?: Create Major Versions
- Require documents to be checked out before they can be edited?: No
Since Graph is using there managed properties from sharepoint you can try "UIVersionStringOWSTEXT".
{
"requests": [
{
"entityTypes": [
"listItem"
],
"query": {
"queryString": "test"
},
"fields": [
"title",
"UIVersionStringOWSTEXT"
]
}
]
}
The results look like
{
"value": [
{
"searchTerms": [
"test"
],
"hitsContainers": [
{
"hits": [
{
"hitId": "GUID",
"rank": 1,
"summary": "test",
"resource": {
"#odata.type": "#microsoft.graph.listItem",
"fields": {
"title": "test",
"uiVersionStringOWSTEXT": "1.0"
}
}
}
],
"total": 1,
"moreResultsAvailable": false
}
]
}
],
"#odata.context": "https://graph.microsoft.com/v1.0/$metadata#Collection(microsoft.graph.searchResponse)"
}
In my tenant this property was already created. I guess it is by default. If your search schema does not have this managed property you can create it and name it like you wish. Map it to the crawled property "ows_q_TEXT__UIVersionString" and make sure your managed property is set to "retrievable".

Mango index "does not contain a valid index for this query" even when specified manually

I'm trying to efficiently query data via Mango (as that seems to be the only option given my requirements Searching for sub-objects with a date range containing the queried date value), but I can't even get a very simple index/query pair to work: although I specify my index manually for the query, I'm told that my index "was not used because it does not contain a valid index for this query. No matching index found, create an index to optimize query time."
(I'm doing all of this via Fauxton on CouchDB v. 3.0.0)
Let's say my documents look like this:
{
"tenant": "TNNT_a",
"$doctype": "JobOpening",
// a bunch of other fields
}
All documents with a $doctype of "JobOpening" are guaranteed to have a tenant property. The searches I wish to perform will only ever be for documents with $doctype of "JobOpening" and a tenant selector will always be provided when querying.
Here's the test index I've configured:
{
"index": {
"fields": [
"tenant",
"$doctype"
],
"partial_filter_selector": {
"\\$doctype": {
"$eq": "JobOpening"
}
}
},
"ddoc": "job-openings-doctype-index",
"type": "json"
}
And here's the query
{
"selector": {
"tenant": "TNNT_a",
"\\$doctype": "JobOpening"
},
"use_index": "job-openings-doctype-index"
}
Why isn't the index being used for the query?
I've tried not using a partial index, and I think the $doctype escaping is done properly in the requisite places, but nothing seems to keep CouchDB from performing a full scan.
The index isn't being used because the $doctype field is not being recognized by the query planner as expected.
Changing the fields declaration from $doctype to \\$doctype in the design document solves the issue.
{
"index": {
"fields": [
"tenant",
"\\$doctype"
],
"partial_filter_selector": {
"\\$doctype": {
"$eq": "JobOpening"
}
}
},
"ddoc": "job-openings-doctype-index",
"type": "json"
}
After that small refactor, the query
{
"selector": {
"tenant": "TNNT_a",
"\\$doctype": "JobOpening"
},
"use_index": "job-openings-doctype-index"
}
Returns the expected result, and produces an "explain" which confirms the job-openings-doctype-index was queried:
{
"dbname": "stack",
"index": {
"ddoc": "_design/job-openings-doctype-index",
"name": "7f5c5cea5acd90f11fffca3e3355b6a03677ad53",
"type": "json",
"def": {
"fields": [
{
"tenant": "asc"
},
{
"\\$doctype": "asc"
}
],
"partial_filter_selector": {
"\\$doctype": {
"$eq": "JobOpening"
}
}
}
},
// etc etc etc
Whether this change is intuitive or not is unclear, however it is consistent - and perhaps reveals leading field names with a "special" character may not be desirable.
Regarding the indexing of the filtered field, as per the documentation regarding partial_filter_selector
Technically, we don’t need to include the filter on the "status" [e.g.
$doctype here] field in the query selector ‐ the partial index
ensures this is always true - but including it makes the intent of the
selector clearer and will make it easier to take advantage of future
improvements to query planning (e.g. automatic selection of partial
indexes).
Despite that, I would not choose to index a field whose value is constant.

limit in _source in elasticsearch

This is my source from ES:
"_source": {
"queryHash": "query412236215",
"id": "query412236215",
"content": {
"columns": [
{
"name": "Catalog",
"type": "varchar(10)",
"typeSignature": {
"rawType": "varchar",
"typeArguments": [],
"literalArguments": [],
"arguments": [
{
"kind": "LONG_LITERAL",
"value": 10
}
]
}
}
],
"data": [
[
"apm"
],
[
"postgresql"
],
[
"rest"
],
[
"system"
],
[
"tpch"
]
],
"query_string": "show catalogs",
"execution_time": 1979
},
"createdOn": "1514269074289"
}
How can i get the n records inside _source.data?
Lets say _source.data have 100 records , I want only 10 at a time ,also is it possible to assign offset for next 10 records?
Thanks
Take a look at scripting. As far as I know there isn't any built-in solution because Elasticsearch is primarily built for searching and filtering with a document store only as a secondary concern.
First, the order in _source is stable, so it's not totally impossible:
When you get a document back from Elasticsearch, any arrays will be in
the same order as when you indexed the document. The _source field
that you get back contains exactly the same JSON document that you
indexed.
However, arrays are indexed—made searchable—as multivalue fields,
which are unordered. At search time, you can’t refer to "the first
element" or "the last element." Rather, think of an array as a bag of
values.
However, source filtering doesn't cover this, so you're out of luck with arrays.
Also inner hits won't help you. They do have options for sort, size, and from, but those will only return the matched subdocuments and I assume you want to page freely through all of them.
So your final hope is scripting, where you can build whatever you want. But this is probably not what you want:
Do you really need paging here? Results are transferred in a compressed fashion, so the overhead of paging is probably much larger than transferring the data in one go.
If you do need paging, because your array is huge, you probably want to restructure your documents.

Using Azure CosmosDB DocumentDB API with Graph API

What I would like to be able to do:
Save schemaless JSON to documents
Connect those documents arbitrarily
Get recursive tree of documents based on aforementioned connections, like for example:
{
"name": "Document 1",
"includes": [
{
"name": "Document 2.1"
"includes": [
{
"name": "Document 3",
"includes": []
}
]
},
{
"name": "Document 2.2",
"includes": []
}
]
}
Current status of my setup:
CosmosDB instance configured with Graph (Gremlin) API
Possible to create (JSON) documents through DocumentDB API
Possible to created edges to documents through Graph API
Using Node.js SDKs
Questions:
Is it possible to save JSON objects as vertices through Graph API? It allows creating vertices with g.addV('person').property('id', 'thomas').property('firstName', 'Thomas').property('age', 44).property('userid', 1) but something like g.addV({ firstName: 'Thomas' }) does not seem to work.
If I add documents through DocumentDB API and edges between them through Graph API and traverse through the graph, results only include IDs of the documents, not other properties. Is it possible to populate the documents somehow?
Example traversal query:
g.V('03e0576f-2ff7-6109-5ed5-237b43191354').repeat(out('includes')).until(not(out('includes'))).simplePath().dedup().tree().by('id')
Result from this query:
[{
"03e0576f-2ff7-6109-5ed5-237b43191354": {
"key": "03e0576f-2ff7-6109-5ed5-237b43191354",
"value": {
"7fab4007-c80c-ba21-f5d3-8eb353ea3279": {
"key": "7fab4007-c80c-ba21-f5d3-8eb353ea3279",
"value": {
"eec55fbd-6900-130d-247f-fb437b093711": {
"key": "eec55fbd-6900-130d-247f-fb437b093711",
"value": {}
},
"cfd14a8c-1ac3-6cc3-e2a4-ac3f250478e1": {
"key": "cfd14a8c-1ac3-6cc3-e2a4-ac3f250478e1",
"value": {
"acac136a-3bd4-831c-df6e-e5b95e593b9a": {
"key": "acac136a-3bd4-831c-df6e-e5b95e593b9a",
"value": {}
}
}
}
}
}
}
}
}]
Yes, it is possible to insert documents both through the Graph API and through the Document API. However, Cosmos expects a specific GraphSON format for the documents in order for all of their properties to be picked up during graph traversal.
I'd recommend taking a look at both Vertex Properties and GraphSON from the Tinkerpop documentation to start to get a better idea about these topics.
When adding a document through Gremlin the syntax is a name value comma separated for all properties you want to add. Try this:
g.addV('label', 'human', 'name', 'jesse', 'age', 27)
Now if you go to the Azure portal and execute a SQL query SELECT * FROM c you'll be able to see the format that Cosmos has translated your document into.

Query data where userID in multiples ID

I try to make a query and i don't know the right way to do this.
The mongo collection structure contains multiples user ID (uid) and i want to make a query that get all datas ("Albums") where the User ID match one of the uid.
I do not know if the structure of the collection is good for that and I would like to know if I should do otherwise.
{
"_id": ObjectId("55814a9799677ba44e7826d1"),
"album": "album1",
"pictures": [
"1434536659272.jpg",
"1434552570177.jpg",
"1434552756857.jpg",
"1434552795100.jpg"
],
"uid": [
"12814a8546677ba44e745d85",
"e745d677ba4412814e745d7b",
"28114a85466e745d677d85qs"
],
"__v": 0
}
I just searched on internet and found this documentation http://docs.mongodb.org/manual/reference/operator/query/in/ but I'm not certain that this is the right way.
In short, I need to know: if I use the right method for the stucture of the collection and the operator "$in" is the right solution (knowing that it may have a lot of "User ID": between 2 and 2000 maximum).
You don't need $in unless you are matching for more than one possible value in a field, and that field does not have to be an array. $in is in fact shorthand for $or.
You just need a simple query here:
Model.find({ "uid": "12814a8546677ba44e745d85" },function(err,results) {
})
If you want "multiple" user id's then you can use $in:
Model.find(
{ "uid": { "$in": [
"12814a8546677ba44e745d85",
"e745d677ba4412814e745d7b",
] } },
function(err,results) {
}
)
Which is short for $or in this way:
Model.find(
{
"$or": [
{ "uid": "12814a8546677ba44e745d85" },
{ "uid": "e745d677ba4412814e745d7b" }
]
},
function(err,results) {
}
)
Just to answer your question, you can use the below query to get the desired result.
db.mycollection.find( {uid : {$in : ["28114a85466e745d677d85qs"] } } )
However, you need to revisit your data structure, looks like its a Many-to-Many problem and you might need to think about introducing a mid collection for that.

Resources