Loopback "Include" Filter Fails with hasManyThrough Relation - node.js

I have 3 models. 2 resource models, account(id, name) and widget(id, name), and 1 mapping model to map between the two widget_to_account(id, account_id, widget_id), to tell what widgets an account has access to, so to speak.
When stating the relationship between the models in their JSONs, using the guide in http://loopback.io/doc/en/lb3/HasManyThrough-relations.html, RESTful requests like "get widgets of account id=1" for example, works perfectly.
GET /accounts/1/widgets yields the widgets that account 1 has, yielding a widgets array:
[
{
"id": 1,
"name": "wg_user_mgr"
},
{
"id": 2,
"name": "wg_desc"
}
]
That's all good.
However, say I wanted to append this widgets array result along with the account object returned by a GET to the account model?. Loopback documentation suggests that this is done using the include keyword with the request, like so:
GET /accounts/1?filter[include]=widgets, returning hopefully an account model with it's allowed widgets:
{
"id": 1,
"name": "Account1Name",
"widgets": [
{
"id": 1,
"name": "wg_user_mgr",
"display_name": "User Manager"
},
{
"id": 2,
"name": "wg_desc",
"display_name": "Description"
}
]
}
However, what is actually returned by loopback with that request, is:
{
"id": 1,
"name": "Account1Name",
"widgets": []
}
Empty widgets array! When I look at the loopback SQL debugs, I see that it does go to the widget_to_account table and selects the entries of account_id=1, but interestingly it stop there and just returns an empty widgets array.
Any clues? The hasManyThrough loopback docs doesn't actually show any examples of using include like this to bridge two models that are connected via a mapping model.
My guess is they just forgot to code it in ¯\_(ツ)_/¯
UPDATE:
Doing some more digging around, I found the answer at https://groups.google.com/forum/#!topic/loopbackjs/sH7bKoqzU5c.
Where you define the relationships in the 2 resource models, you have to specifically define the "keyThrough" value.
NOT THIS:
"relations": {
"widgets": {
"type": "hasMany",
"model": "widget",
"foreignKey": "account_id",
"through": "widget_to_account"
}
}
BUT THIS:
"relations": {
"widgets": {
"type": "hasMany",
"model": "widget",
"foreignKey": "account_id",
"through": "widget_to_account",
"keyThrough": "account_id"
}
}
This is not made super clear, and is even stated incorrectly in the loopback api docs -.-

UPDATE:
Doing some more digging around, I found the answer at https://groups.google.com/forum/#!topic/loopbackjs/sH7bKoqzU5c.
Where you define the relationships in the 2 resource models, you have to specifically define the "keyThrough" value.
NOT THIS:
"relations": {
"widgets": {
"type": "hasMany",
"model": "widget",
"foreignKey": "account_id",
"through": "widget_to_account"
}
}
BUT THIS:
"relations": {
"widgets": {
"type": "hasMany",
"model": "widget",
"foreignKey": "account_id",
"through": "widget_to_account",
"keyThrough": "widget_id"
}
}
This is not made super clear, and is even stated incorrectly in the loopback api docs. I wish they'de stop this "auto-naming" paradigm they've been pushing around. Looking at loopback SO and the wider community, it's generally caused so much pain with models being named incorrectly, keys like this being set to totally arbitary names -.-

Related

Azure Search match against two properties of the same object

I would like to do a query matches against two properties of the same item in a sub-collection.
Example:
[
{
"name": "Person 1",
"contacts": [
{ "type": "email", "value": "person.1#xpto.org" },
{ "type": "phone", "value": "555-12345" },
]
}
]
I would like to be able to search by emails than contain xpto.org but,
doing something like the following doesn't work:
search.ismatchscoring('email','contacts/type,','full','all') and search.ismatchscoring('/.*xpto.org/','contacts/value,','full','all')
instead, it will consider the condition in the context of the main object and objects like the following will also match:
[
{
"name": "Person 1",
"contacts": [
{ "type": "email", "value": "555-12345" },
{ "type": "phone", "value": "person.1#xpto.org" },
]
}
]
Is there any way around this without having an additional field that concatenates type and value?
Just saw the official doc. At this moment, there's no support for correlated search:
This happens because each clause applies to all values of its field in
the entire document, so there's no concept of a "current sub-document
https://learn.microsoft.com/en-us/azure/search/search-howto-complex-data-types
and https://learn.microsoft.com/en-us/azure/search/search-query-understand-collection-filters
The solution I've implemented was creating different collections per contact type.
This way I'm able to search directly in, lets say, the email collection without the need for correlated search. It might not be the solution for all cases but it works well in this case.

How to query by array of objects in Contentful

I have an content type entry in Contentful that has fields like this:
"fields": {
"title": "How It Works",
"slug": "how-it-works",
"countries": [
{
"sys": {
"type": "Link",
"linkType": "Entry",
"id": "3S5dbLRGjS2k8QSWqsKK86"
}
},
{
"sys": {
"type": "Link",
"linkType": "Entry",
"id": "wHfipcJS6WUSaKae0uOw8"
}
}
],
"content": [
{
"sys": {
"type": "Link",
"linkType": "Entry",
"id": "72R0oUMi3uUGMEa80kkSSA"
}
}
]
}
I'd like to run a query that would only return entries if they contain a particular country.
I played around with this query:
https://cdn.contentful.com/spaces/aoeuaoeuao/entries?content_type=contentPage&fields.countries=3S5dbLRGjS2k8QSWqsKK86
However get this error:
The equals operator cannot be used on fields.countries.en-AU because it has type Object.
I'm playing around with postman, but will be using the .NET API.
Is it possible to search for entities, and filter on arrays that contain Objects?
Still learning the API, so I'm guessing it should be pretty straight forward.
Update:
I looked at the request the Contentful Web CMS makes, as this functionality is possible there. They use query params like this:
filters.0.key=fields.countries.sys.id&filters.0.val=3S5dbLRGjS2k8QSWqsKK86
However, this did not work in the delivery API, and might only be an internal query format.
Figured this out. I used the following URL:
https://cdn.contentful.com/spaces/aoeuaoeua/entries?content_type=contentPage&fields.countries.sys.id=wHfipcJS6WUSaKae0uOw8
Note the query parameter fields.countries.sys.id

Filtering Contentful Query on Linked Objects

I'm attempting to utilize Contentful on a current project of mine and I'm trying to understand how to filter my query results based on a field in a linked object.
My top level object contains a Link defined as such:
"name": "Service_Description",
"fields": [
{
"name": "Header",
"id": "header",
"type": "Link",
"linkType": "Entry",
"required": true,
"validations": [
{
"linkContentType": [
"offerGeneral"
]
}
],
"localized": false,
"disabled": false,
"omitted": false
},
This "header" field links to another content type that has this definition:
"fields": [
{
"name": "General",
"id": "general",
"type": "Link",
"linkType": "Entry",
"required": true,
"validations": [
{
"linkContentType": [
"genericGeneral"
]
}
],
"localized": false,
"disabled": false,
"omitted": false
},
which then links to the lowest level:
"fields": [{
"name": "TagList",
"id": "tagList",
"type": "Array",
"items": {
"type": "Link",
"linkType": "Entry",
"validations": [
{
"linkContentType": [
"tag"
]
}
]
},
"validations": []
}
where tagList is an array of tags this piece of content may have.
I want to be able to run a query from the top level object that says get me X number of these "Service_Description" content entries where it contains a tag from a supplied list of tags.
In PostMan, I've been running with this:
https://cdn.contentful.com/spaces/{SPACE_ID}/entries?access_token={ACCESS_TOKEN}&content_type=serviceDescription&include=3
I'm trying to add a filter something like so:
fields.header.fields.general.fields.tagList.sys.id%5Bin%5D={TAG_SYS_ID}
This is clearly incorrect, but I've been struggling with how to walk this relationship to achieve my goal. Perusing the documentation this seems to have something to do with includes, but I'm unsure of how to rectify the problem.
Any direction on how to achieve my goal or if this is possible?
This is now possible, something I believe was solved for in the API based on requests for this functionality. You can see the thread here.
This gist of it is that you have to query on the entries that have linked entries and then include the contentType for those linked entries in the query like so:
contentfulClient.getEntries({
'content_type': 'location',
'fields.market.fields.marketName': 'New York',
'fields.market.sys.contentType.sys.id': 'marketRegion'
})
Unfortunately what you are requesting is not currently possible in Contentful.
We were facing a very similar issue with nested/referenced content types and support said it wasn't possible.
We ended up writing a very complicated system that allowed us to do what you want. Essentially doing a full text search for the referenced content and then querying all of the parents entries. We then matched the relationships by iterating over the parents to find the relationship.
Sorry it couldn't be easier. Hopefully the devs work on something that improve this complication. We have brought this to their attention.

Loopback API include filters not working as expected

I have used different models to simplify what I am trying to achieve and to remove clutter, although the theory should be the same as my actual project.
Use the following assumptions: A system can only have one type of speaker and one type of amp.
Say I have the following models:
*** System ***
- id
- name
- speakerId
- ampId
*** Speaker ***
- id
- name
*** Amp ***
- id
- name
I have added the following to my System.json model file (which i think is correct):
"relations": {
"speakers": {
"type": "hasOne",
"model": "Speaker",
"foreignKey": "speakerId"
},
"amps": {
"type": "hasOne",
"model": "Amp",
"foreignKey": "ampId"
}
},
When I start my app and open the API Explorer and go to create a new instance of Speaker or Amp it expects the following:
*** Speaker ***
- id
- name
- speakerId *** Why is this here??? ***
If I change the System.json model file to look like this (which I think is the wrong way to do it):
"relations": {
"speakers": {
"type": "hasOne",
"model": "Speaker",
"foreignKey": "id"
},
"amps": {
"type": "hasOne",
"model": "Amp",
"foreignKey": "id"
}
},
Everything looks right in the API explorer. So I add a few speakers and amps, then add a system.
When I add an include filter to the System GET request:
{"include":["speakers","amps"]}
It includes the speakers and amps but using the System.id as the index for the Speakers and Amps Models instead of System.speakerId and System.ampId. So to ellaborate if the:
System.id = 5
System.speakerId = 1
System.ampId = 3
It will include the speaker with an id of 5 and the amp with the id of 5 instead of including the speaker with an id of 1 and the Amp with an id of 3.
I want to be able to list all Systems including their related Speakers and Amps. How can this be done with Loopback. At the minute I am using the above models with in memory DB instead of my usual ones just to test. Any help would be appreciated as I am now at the point of tearing my hair out!
Regards,
James
Your relation between models is not correct. You need to tell loopbackjs to use hasMany through relation to get your data.
Your system.json should be changed to
"relations": {
"speaker": {
"type": "belongsTo",
"model": "Speaker",
"foreignKey": "speakerId"
},
"amp": {
"type": "belongsTo",
"model": "Amp",
"foreignKey": "ampId"
}},
And your speakers should be
"relations": {
"amps": {
"type": "hasMany",
"model": "Amp",
"foreignKey": "speakerId",
"through": "System"
},
amp.json
"relations": {
"speakers": {
"type": "hasMany",
"model": "Speaker",
"foreignKey": "ampId",
"through": "System"
},

How to search through data with arbitrary amount of fields?

I have the web-form builder for science events. The event moderator creates registration form with arbitrary amount of boolean, integer, enum and text fields.
Created form is used for:
register a new member to event;
search through registered members.
What is the best search tool for second task (to search memebers of event)? Is ElasticSearch well for this task?
I wrote a post about how to index arbitrary data into Elasticsearch and then to search it by specific fields and values. All this, without blowing up your index mapping.
The post is here: http://smnh.me/indexing-and-searching-arbitrary-json-data-using-elasticsearch/
In short, you will need to do the following steps to get what you want:
Create a special index described in the post.
Flatten the data you want to index using the flattenData function:
https://gist.github.com/smnh/30f96028511e1440b7b02ea559858af4.
Create a document with the original and flattened data and index it into Elasticsearch:
{
"data": { ... },
"flatData": [ ... ]
}
Optional: use Elasticsearch aggregations to find which fields and types have been indexed.
Execute queries on the flatData object to find what you need.
Example
Basing on your original question, let's assume that the first event moderator created a form with following fields to register members for the science event:
name string
age long
sex long - 0 for male, 1 for female
In addition to this data, the related event probably has some sort of id, let's call it eventId. So the final document could look like this:
{
"eventId": "2T73ZT1R463DJNWE36IA8FEN",
"name": "Bob",
"age": 22,
"sex": 0
}
Now, before we index this document, we will flatten it using the flattenData function:
flattenData(document);
This will produce the following array:
[
{
"key": "eventId",
"type": "string",
"key_type": "eventId.string",
"value_string": "2T73ZT1R463DJNWE36IA8FEN"
},
{
"key": "name",
"type": "string",
"key_type": "name.string",
"value_string": "Bob"
},
{
"key": "age",
"type": "long",
"key_type": "age.long",
"value_long": 22
},
{
"key": "sex",
"type": "long",
"key_type": "sex.long",
"value_long": 0
}
]
Then we will wrap this data in a document as I've showed before and index it.
Then, the second event moderator, creates another form having a new field, field with same name and type, and also a field with same name but with different type:
name string
city string
sex string - "male" or "female"
This event moderator decided that instead of having 0 and 1 for male and female, his form will allow choosing between two strings - "male" and "female".
Let's try to flatten the data submitted by this form:
flattenData({
"eventId": "F1BU9GGK5IX3ZWOLGCE3I5ML",
"name": "Alice",
"city": "New York",
"sex": "female"
});
This will produce the following data:
[
{
"key": "eventId",
"type": "string",
"key_type": "eventId.string",
"value_string": "F1BU9GGK5IX3ZWOLGCE3I5ML"
},
{
"key": "name",
"type": "string",
"key_type": "name.string",
"value_string": "Alice"
},
{
"key": "city",
"type": "string",
"key_type": "city.string",
"value_string": "New York"
},
{
"key": "sex",
"type": "string",
"key_type": "sex.string",
"value_string": "female"
}
]
Then, after wrapping the flattened data in a document and indexing it into Elasticsearch we can execute complicated queries.
For example, to find members named "Bob" registered for the event with ID 2T73ZT1R463DJNWE36IA8FEN we can execute the following query:
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "flatData",
"query": {
"bool": {
"must": [
{"term": {"flatData.key": "eventId"}},
{"match": {"flatData.value_string.keyword": "2T73ZT1R463DJNWE36IA8FEN"}}
]
}
}
}
},
{
"nested": {
"path": "flatData",
"query": {
"bool": {
"must": [
{"term": {"flatData.key": "name"}},
{"match": {"flatData.value_string": "bob"}}
]
}
}
}
}
]
}
}
}
ElasticSearch automatically detects the field content in order to index it correctly, even if the mapping hasn't been defined previously. So, yes : ElasticSearch suits well these cases.
However, you may want to fine tune this behavior, or maybe the default mapping applied by ElasticSearch doesn't correspond to what you need : in this case, take a look at the default mapping or, for even further control, the dynamic templates feature.
If you let your end users decide the keys you store things in, you'll have an ever-growing mapping and cluster state, which is problematic.
This case and a suggested solution is covered in this article on common problems with Elasticsearch.
Essentially, you want to have everything that can possibly be user-defined as a value. Using nested documents, you can have a key-field and differently mapped value fields to achieve pretty much the same.

Resources