I have a requirement; where I need to load a list of payments that were updated between a specified date/time range. And for each invoice; I need to get a list of internal id (aka invoice id) that payment has been applied to (since a payment can be applied to one or more invoices).
I tried to achieve this using a search query like this:
var paymentSearch = search.create({
type: search.Type.CUSTOMER_PAYMENT,
filters: [
['lastmodifieddate', 'within', from_datetime, to_datetime],
'and',
['appliedToTransaction.tranid', search.Operator.ISNOTEMPTY, '#NONE']
],
columns: [
'entity',
'tranid',
search.createColumn({
name: 'internalid',
join: 'appliedToTransaction'
})
]
});
for (var i = 0; i < paymentsPagedData.pageRanges.length; i++) {
var currentPage = paymentsPagedData.fetch(i);
currentPage.data.forEach(function(result) {
// TEST
var appliedToInvoiceIds = result.getValue({ name: 'internalid', join: 'appliedToTransaction' });
}
}
When I inspected the appliedToInvoiceIds, it only appears to be returning a single value, the first invoice this payment has been applied to. How can I get all of the invoice ids that the payment has been applied to?
I tried inspecting the result object (inside the forEach loop) to see what was inside and this is what I saw:
{
"recordType": "customerpayment",
"id": "25911",
"values": {
"entity": [
{
"value": "761",
"text": "COMPANY NAME INC"
}
],
"tranid": "722",
"appliedToTransaction.internalid": [
{
"value": "2676",
"text": "2676"
},
{
"value": "2658",
"text": "2658"
}
]
}
}
As you can see on the result object of the payment; it has appliedToTransaction.internalid which is an array and has two items in it. How can I retrieve these?
Is there an alternate version of result.getValue(...) available for retrieving array of items/field values via a join?
You need to return true at the end of your forEach. Otherwise it will only return one result.
Related
Using this structure as an example, saved in a Cosmos database (course-database) in a collection (course-collection):
{
"courseId": "courseId",
"sessions": [
{
"sessionId": "sessionId1",
"venues": [
{
"id": "venueId1"
},
{
"id": "venueId2"
}
]
},
{
"sessionId": "sessionId2",
"venues": [
{
"id": "venueId3"
},
{
"id": "venueId4"
}
]
}
]
}
How do you create the SQL:
Count the total number of courses, where a course has at least one session, which has at least one venue, which has an ID equals to e.g. venueId3
I've got this so far, but it restricts to the first item of the list, as opposed to just any:
SELECT COUNT(c.id) FROM c WHERE c.sessions[0].venues[0].id = "id"
The answer was join:
SELECT COUNT(c.id)
FROM c
JOIN s in c.sessions
JOIN v in s.venues
WHERE CONTAINS(v.id,"venueId3")
You would then add a new join the deeper in the JSON you would want to go e.g. if venues had an array of contacts:
SELECT COUNT(c.id)
FROM c
JOIN s in c.sessions
JOIN v in s.venues
JOIN co in v.contacts
WHERE CONTAINS(co.id,"contactId")
I have the following documents:
{
"_id": "doc1"
"binds": {
"subject": {
"Test1": ["something"]
},
"object": {
"Test2": ["something"]
}
},
},
{
"_id": "doc2"
"binds": {
"subject": {
"Test1": ["something"]
},
"object": {
"Test3": ["something"]
}
},
}
I need a Mango selector that retrieves documents where any field inside binds (subject, object etc) has an object with key equals to any values from an array passed as parameter. That is, if keys of binds contains any values of some array it should returns that document.
For instance, consider the array ["Test2"] my selector should retrieve doc1 since binds["subject"]["Test1"] exists; the array ["Test1"] should retrieve doc1 and doc2 and the array ["Test2", "Test3"] should also retrieve doc1 and doc2.
F.Y.I. I am using Node.js with nano lib to access CouchDB API.
I am providing this answer because the luxury of altering document "schema" is not always an option.
With the given document structure this cannot be done with Mango in any reasonable manner. Yes, it can be done, but only when employing very brittle and inefficient practices.
Mango does not provide an efficient means of querying documents for dynamic properties; it does support searching within property values e.g. arrays1.
Using worst practices, this selector will find docs with binds properties subject and object having properties named Test2 and Test3
{
"selector": {
"$or": [
{
"binds.subject.Test2": {
"$exists": true
}
},
{
"binds.object.Test2": {
"$exists": true
}
},
{
"binds.subject.Test3": {
"$exists": true
}
},
{
"binds.object.Test3": {
"$exists": true
}
}
]
}
}
Yuk.
The problems
The queried property names vary so a Mango index cannot be leveraged (Test37 anyone?)
Because of (1) a full index scan (_all_docs) occurs every query
Requires programmatic generation of the $or clause
Requires a knowledge of the set of property names to query (Test37 anyone?)
The given document structure is a show stopper for a Mango index and query.
This is where map/reduce shines
Consider a view with the map function
function (doc) {
for(var prop in doc.binds) {
if(doc.binds.hasOwnProperty(prop)) {
// prop = subject, object, foo, bar, etc
var obj = doc.binds[prop];
for(var objProp in obj) {
if(obj.hasOwnProperty(objProp)) {
// objProp = Test1, Test2, Test37, Fubar, etc
emit(objProp,prop)
}
}
}
}
}
So the map function creates a view for any docs with a binds property with two nested properties, e.g. binds.subject.Test1, binds.foo.bar.
Given the two documents in the question, this would be the basic view index
id
key
value
doc1
Test1
subject
doc2
Test1
subject
doc1
Test2
object
doc2
Test3
object
And since view queries provide the keys parameter, this query would provide your specific solution using JSON
{
include_docs: true,
reduce: false,
keys: ["Test2","Test3"]
}
Querying that index with cUrl
$ curl -G http://{view endpoint} -d 'include_docs=false' -d
'reduce=false' -d 'keys=["Test2","Test3"]'
would return
{
"total_rows": 4,
"offset": 2,
"rows": [
{
"id": "doc1",
"key": "Test2",
"value": "object"
},
{
"id": "doc2",
"key": "Test3",
"value": "object"
}
]
}
Of course there are options to expand the form and function of such a view by leveraging collation and complex keys, and there's the handy reduce feature.
I've seen commentary that Mango is great for those new to CouchDB due to it's "ease" in creating indexes and the query options, and that map/reduce if for the more seasoned. I believe such comments are well intentioned but misguided; Mango is alluring but has its pitfalls1. Views do require considerable thought, but hey, that's we're supposed to be doing anyway.
1) $elemMatch for example require in memory scanning which can be very costly.
I have two resolver.
The one is Company resolve that return the company details like id, name and list of documents ids, like this example:
{
"data": {
"companyOne": {
"name": "twitter",
"documents": [
"5c6c0213f0fa854bd7d4a38c",
"5c6c02948e0001a16529a1a1",
"5c6c02ee7e76c12075850119",
"5c6ef2ddd16e19889ffaffd0",
"5c72fb723ebf7b2881679ced",
"5c753d1c2e080fa4a2f86c87",
...
]
}
}
}
And the another resolver gets me all the details of documents like this example:
{
"data": {
"documentsMany": [{
"name": "doc1",
"_id": 5c6c0213f0fa854bd7d4a38c,
}, {
"name": "doc2",
"_id": 5c6c02948e0001a16529a1a1,
},
...
]
}
}
How to match every data.companyOne.documents[id] to data.documentsMany[..]._id? in the query level? is it possible to do this graphql?
The expect results should be when I run the companyOne query (without change the code - just in the query level) it's should return with documents as object instead of array of string ids.
maybe something like?
query {
companyOne {
name,
documents on documentsMany where _id is ___???
}
}
In doClient.scan() is there any way to set the count of elements stored in a particular node?
As per the below example i need to add a new node "questionCount" to the below result which should contain the total number of reference_id in node questionList. is there any way other than iterating the result and adding a new node?
Expected Output
{
"status": 1,
"data": [
{
"questionList": [
{
"reference_id": "0df55215-90de-407a-b077-7017924556d3"
},
{
"reference_id": "0df55215-90de-407a-b077-7017924556d3"
},
{
"reference_id": "0df55215-90de-407a-b077-701997924556d3"
}
],
"testName": "sample test231s",
"testId": "2e97e40c-82cb-4126-9f47-b6a93687a59c",
"questionCount": 3
}
]
}
As you add questions to questionList with list_append, you could also increment the questionCount attribute with an UpdateExpression SET data.questionList = list_append(data.questionList, :newQuestion), ADD questionCount :one and ExpressionAttributeValues {":newQuestion": {"reference_id": "blah"}, ":one": 1}.
I'm trying to get only the person's membership info i.e. ID, name and committee memberships in a SELECT query. This is my object:
{
"id": 123,
"name": "John Smith",
"memberships": [
{
"id": 789,
"name": "U.S. Congress",
"yearElected": 2012,
"state": "California",
"committees": [
{
"id": 444,
"name": "Appropriations Comittee",
"position": "Member"
},
{
"id": 555,
"name": "Armed Services Comittee",
"position": "Chairman"
},
{
"id": 678,
"name": "Veterans' Affairs Comittee",
"position": "Member"
}
]
}
]
}
In this example, John Smith is a member of the U.S. Congress and three committees in it.
The result that I'm trying to get should look like this. Again, this is the "DESIRED RESULT":
{
"id": 789,
"name": "U.S. Congress",
"committees": [
{
"id": 444,
"name": "Appropriations Committee",
"position": "Member"
},
{
"id": 555,
"name": "Armed Services Committee",
"position": "Chairman"
},
{
"id": 678,
"name": "Veterans' Affairs Committee",
"position": "Member"
}
]
}
Here's my SQL query:
SELECT m.id, m.name,
[
{
"id": c.id,
"name": c.name,
"position": c.position
}
] AS committees
FROM a
JOIN m IN a.memberships
JOIN c IN m.committees
WHERE a.id = "123"
I'm getting the following results which is correct but the shape is not right. I'm getting the same membership 3 times. Here's what I'm getting which is NOT the desired result:
[
{
"id": 789,
"name": "U.S. Congress",
"committees":[
{
"id": 444,
"name": "Appropriations Committee",
"position": "Member"
}
]
},
{
"id": 789,
"name": "U.S. Congress",
"committees":[
{
"id": 555,
"name": "Armed Services Committee",
"position": "Chairman"
}
]
},
{
"id": 789,
"name": "U.S. Congress",
"committees":[
{
"id": 678,
"name": "Veterans' Affairs Committee",
"position": "Member"
}
]
}
]
As you can see here, the "U.S. Congress" membership is repeated 3 times.
The following SQL query gets me exactly what I want in Azure Query Explorer but when I pass it as the query in my code -- using DocumentDb SDK -- I don't get any of the details for the committees. I simply get blank results for committee ID, name and position. I do, however, get the membership data i.e. "U.S. Congress", etc. Here's that SQL query:
SELECT m.id, m.name, m.committees AS committees
FROM c
JOIN m IN c.memberhips
WHERE c.id = 123
I'm including the code that makes the DocumentDb call. I'm including the code with our internal comments to help clarify their purpose:
First the ReadQuery function that we call whenever we need to read something from DocumentDb:
public async Task<IEnumerable<T>> ReadQuery<T>(string collectionId, string sql, Dictionary<string, object> parameterNameValueCollection)
{
// Prepare collection self link
var collectionLink = UriFactory.CreateDocumentCollectionUri(_dbName, collectionId);
// Prepare query
var query = getQuery(sql, parameterNameValueCollection);
// Creates the query and returns IQueryable object that will be executed by the calling function
var result = _client.CreateDocumentQuery<T>(collectionLink, query, null);
return await result.QueryAsync();
}
The following function prepares the query -- with any parameters:
protected SqlQuerySpec getQuery(string sql, Dictionary<string, object> parameterNameValueCollection)
{
// Declare query object
SqlQuerySpec query = new SqlQuerySpec();
// Set query text
query.QueryText = sql;
// Convert parameters received in a collection to DocumentDb paramters
if (parameterNameValueCollection != null && parameterNameValueCollection.Count > 0)
{
// Go through each item in the parameters collection and process it
foreach (var item in parameterNameValueCollection)
{
query.Parameters.Add(new SqlParameter($"#{item.Key}", item.Value));
}
}
return query;
}
This function makes async call to DocumentDb:
public async static Task<IEnumerable<T>> QueryAsync<T>(this IQueryable<T> query)
{
var docQuery = query.AsDocumentQuery();
// Batches gives us the ability to read data in chunks in an asyc fashion.
// If we use the ToList<T>() LINQ method to read ALL the data, the call will synchronous which is why we prefer the batches approach.
var batches = new List<IEnumerable<T>>();
do
{
// Actual call is made to the backend DocumentDb database
var batch = await docQuery.ExecuteNextAsync<T>();
batches.Add(batch);
}
while (docQuery.HasMoreResults);
// Because batches are collections of collections, we use the following line to merge all into a single collection.
var docs = batches.SelectMany(b => b);
// Return data
return docs;
}
I just write a demo to test with your query and I can get the expected result, check the snapshot below. So I think that query is correct, you've mentioned that you don't seem to get any data when you make the call in my code, would you mind share your code? Perhaps there are some mistakes in you code. Anyway, here is my test just for your reference and hope it helps.
Query used:
SELECT m.id AS membershipId, m.name AS membershipNameName, m.committees AS committees
FROM c
JOIN m IN c.memberships
WHERE c.id = "123"
Code here is very simple, sp_db.innerText represents a span which I used to show the result in my test page:
var docs = client.CreateDocumentQuery("dbs/" + databaseId + "/colls/" + collectionId,
"SELECT m.id AS membershipId, m.name AS membershipName, m.committees AS committees " +
"FROM c " +
"JOIN m IN c.memberships " +
"WHERE c.id = \"123\"");
foreach (var doc in docs)
{
sp_db.InnerText += doc;
}
I think maybe there are some typos in the query you specified in client.CreateDocumentQuery() which makes the result to be none, it's better to provide the code for us, then we can help check it.
Updates:
Just tried your code and still I can get the expected result. One thing I found is that when I specified the where clause like "where c.id = \"123\"", it gets the result:
However, if you didn't make the escape and just use "where c.id = 123", this time you get nothing. I think this could be a reason. You can verify whether you have ran into this scenario.
Just updated my original post. All the code provided in the question is correct and works. I was having a problem because I was using aliases in the SELECT query and as a result some properties were not binding to my domain object.
The code provided in the question is correct.