Graphql query way too slow with nested fields - node.js

So I'm using apollo-graphql,
I have a subgraph named Course, in this subgraph I have a type Course which looks like this:
type Course {
_id: ID
title: String
}
Now I have a second subgraph named progress, in this one I extend the Course type and add a field:
extend type Course #key(fields: "_id") {
_id: ID #external
totalProgression: Int
}
And now I want to query it like this:
Courses {
_id
title
totalProgression
}
(Courses is a query which returns all courses)
This query works, but the problem is, it's way too slow. And that's because for each course, it is going to make a db request to get the totalProgression. If I add another nested field from another subgraph, it is going to take even longer.
My question here is, is there any way I could make this faster? For example could I make one db request to get all the totalProgression and then putting them in the courses? Without having to merge my subgraphs.
My only solution for now is to get the totalProgressions inside the Courses resolvers, but that would kill the point of making another subgraph. Yes I could juste merge all my subgraphes, but that's not how graphql is supposed to work.

Related

Convert JavaScript array element from string to object

In React, I am trying to create the order/sort array to send to Nodejs for a situation where there we need to sort by an included table column. So, in React, I have:
sort = `[[Contact, "phone1", "asc"]]`
That is variable depending on which column header they click on in a screen, and whether descending or ascending.
In Nodejs on the backend, this shows up as a string (not an array):
sort = [ [ Contact, 'phone', 'asc' ] ]
I need it to look like this (an array AND with Contact without quotes, so that Sequelize will accept it):
sort = [ [ Contact, "phone", "asc" ] ]
so that it can be passed to Sequelize, such as:
Table.findAll({
where: {
...
},
include [ { model: Contact, ... } ]
order: sort
})
In React I can make the "Contact" have quotes around it, so that in Node I can use JSON.parse, but then Contact has quotes around it, which doesn't work when passing it to Sequelize's sort, as it thinks it is part of the original table that we are querying.
How can this be done?
Thank you very much!
Contact is without quotes because the Contact is reference to the actual class definition which is being used in the frontend. When you send your request to the backend then this class information is lost.
Even if there is a class with the same name on the backend, it cannot be converted to that class definition just by the same name. You should include for example a middleware (if you are using express for example) that replace your stringified "Contact" with an actual definition that Sequelize can use.
Something like this on the server:
const Contact = required('./your-path-to-contact')
// ...some other stuff happening...
sort.forEach(s => {
if(s === 'Contact')
s = Contact
})
This solution is a little bit hacky but can work. For a good solution I would refrain from sending class to the server. I would rather have some kind of query builder which the server understands and can select the appropriate class to use when the time comes.
We solved this by not passing an array from the frontend to the backend, but instead passing an object, with key-value pairs such as table, column, & direction, and then converting to an order array using sequelize.col & literals, such as...
order: [
[sequelize.col(`${table}.${column}`), direction]
]
It's a bit more complicated than just this one line given all of the different possibilities in our app, but that's a good summary.
Some notes:
table = the included table
column = the column in the included table
to sort on
direction = 'asc' or 'desc'
With sequelize.col, you really don't even need to have the included table name, if there are no other column names that are the same.

GraphQl filter in list field

I'm working on GraphQL API, and I want to filter my data "Products" by sellerId knowing that a product can be sold by several sellers, which means the sellers' field is an array.
Here is the query:
query GetProducts($filterObject:ProductWhereInput!){
products(where:$filterObject){
id
name
description
sku
price
sellers(where:$filterObject.sellers){
id
firstname
lastname
}
images{
url
fileName
}
}
}
Filter variable is defined like that
{
"filter":{
"sellerId":"ckzia0llkfngz0d09mrppd7kh"
}
}
and when I execute this query I get the error
"message": "unknown field 'filterObject.sellers' in variables"
I'm not sure if that's the correct method to apply the filter, it worked for me when I use it for single-value fields, but not with arrays.
If someone could help me, I'll be thankful.
Here you defined $filterObject as a query variable
query GetProducts($filterObject:ProductWhereInput!)
But the way this line is written, it "uses" a variable that is not defined:
sellers(where:$filterObject.sellers)
It's looking for a variable named: "$filterObject.sellers", not a property "sellers" of $filterObject.
Possible solution 1 - server side:
You can change the sellers field definition to use the whole $filterObject and then the resolver function on the server can extract the field it needs.
sellers(where:$filterObject)
This solution makes sense if you have control over the server side.
Possible solution 2 - client side:
If you cannot change the code on the server side, you can define a separate variable and use it instead:
query GetProducts($filterObject:ProductWhereInput!, $filterSellersObject:ProductSellersWhereInput!){
//...
sellers(where:$filterSellersObject){
//...
(assuming ProductSellersWhereInput is a defined type)

Is there a way to define a type definition for an object with changing property names in GraphQL? [duplicate]

Let's say my graphql server wants to fetch the following data as JSON where person3 and person5 are some id's:
"persons": {
"person3": {
"id": "person3",
"name": "Mike"
},
"person5": {
"id": "person5",
"name": "Lisa"
}
}
Question: How to create the schema type definition with apollo?
The keys person3 and person5 here are dynamically generated depending on my query (i.e. the area used in the query). So at another time I might get person1, person2, person3 returned.
As you see persons is not an Iterable, so the following won't work as a graphql type definition I did with apollo:
type Person {
id: String
name: String
}
type Query {
persons(area: String): [Person]
}
The keys in the persons object may always be different.
One solution of course would be to transform the incoming JSON data to use an array for persons, but is there no way to work with the data as such?
GraphQL relies on both the server and the client knowing ahead of time what fields are available available for each type. In some cases, the client can discover those fields (via introspection), but for the server, they always need to be known ahead of time. So to somehow dynamically generate those fields based on the returned data is not really possible.
You could utilize a custom JSON scalar (graphql-type-json module) and return that for your query:
type Query {
persons(area: String): JSON
}
By utilizing JSON, you bypass the requirement for the returned data to fit any specific structure, so you can send back whatever you want as long it's properly formatted JSON.
Of course, there's significant disadvantages in doing this. For example, you lose the safety net provided by the type(s) you would have previously used (literally any structure could be returned, and if you're returning the wrong one, you won't find out about it until the client tries to use it and fails). You also lose the ability to use resolvers for any fields within the returned data.
But... your funeral :)
As an aside, I would consider flattening out the data into an array (like you suggested in your question) before sending it back to the client. If you're writing the client code, and working with a dynamically-sized list of customers, chances are an array will be much easier to work with rather than an object keyed by id. If you're using React, for example, and displaying a component for each customer, you'll end up converting that object to an array to map it anyway. In designing your API, I would make client usability a higher consideration than avoiding additional processing of your data.
You can write your own GraphQLScalarType and precisely describe your object and your dynamic keys, what you allow and what you do not allow or transform.
See https://graphql.org/graphql-js/type/#graphqlscalartype
You can have a look at taion/graphql-type-json where he creates a Scalar that allows and transforms any kind of content:
https://github.com/taion/graphql-type-json/blob/master/src/index.js
I had a similar problem with dynamic keys in a schema, and ended up going with a solution like this:
query lookupPersons {
persons {
personKeys
person3: personValue(key: "person3") {
id
name
}
}
}
returns:
{
data: {
persons: {
personKeys: ["person1", "person2", "person3"]
person3: {
id: "person3"
name: "Mike"
}
}
}
}
by shifting the complexity to the query, it simplifies the response shape.
the advantage compared to the JSON approach is it doesn't need any deserialisation from the client
Additional info for Venryx: a possible schema to fit my query looks like this:
type Person {
id: String
name: String
}
type PersonsResult {
personKeys: [String]
personValue(key: String): Person
}
type Query {
persons(area: String): PersonsResult
}
As an aside, if your data set for persons gets large enough, you're going to probably want pagination on personKeys as well, at which point, you should look into https://relay.dev/graphql/connections.htm

Storing and querying PostgreSQL database entities with multiple related entities?

Designing a PostgreSQL database that will be queried by a Node API using Sequelize. Currently, I have a table called recipes that has columns called ingredients and instructions. Those columns are stored for a given as an array of strings like {Tomatoes, Onions}.
That method of storage worked fine for simply fetching and rendering a recipe on the client side. But it wasn't working well for fuzzy search querying because, using Sequelize all I could do was ingredients: { [Op.contains] : [query] }. So if a user typed tomatoes there was no way to write a "fuzzy" search query that would return a recipe with an ingredient Tomatoes.
And then I read this in the PostgreSQL documentation:
Arrays are not sets; searching for specific array elements can be a sign of database misdesign. Consider using a separate table with a row for each item that would be an array element. This will be easier to search, and is likely to scale better for a large number of elements.
Now I'm considering storing ingredients and instructions as separate tables, but I have a couple of questions.
1) As a recipe can have multiple ingredients related to it, should I just use a foreign key for each ingredient and the Sequelize hasMany relationship? That seems correct to me, except that I'm now potentially duplicating common ingredients each time a new recipe is created that uses that ingredient.
2) What would be the best way to write a fuzzy search query so that a user could search the main columns of the recipes table (e.g. title, description) and additionally apply their query to the instructions and ingredients tables?
Essentially I'd like to end up with a fuzzy search query applied to the three tables that looks something like this...
const recipes = await req.context.models.Recipe.findAll({
where: {
[Op.or]: [
{ title: { [Op.iLike]: '%' + query + '%' } },
{ description: { [Op.iLike]: '%' + query + '%' } },
{ ingredients: { ingredient: { [Op.iLike]: '%' + query + '%' } } },
{ instructions: { instruction: { [Op.iLike]: '%' + query + '%' } } }
]
}
});
Thanks!
I have done this, i happen to use graphql in my node layer with sequelize, and i have filter objects that do this type of thing. You'll just need some include statements in your Recipie.findAll.. after your initial where clause where you evaluate whether you are searching title or description or both type thing. i sent my search params in with prefix's i could strip off that told me what sequelize op's i would want to use on them and just ran my args through a utility method to create my where clause, but i know there are many ways to skin that cat. i just did not want to clutter up my resolvers with tonnes of hardcoded ops and conditional clauses was all.... your include might look something like this
include: [{
model: models.Ingredient,
as: 'Ingredients',
through: { some join table specifying keys where necessary since this
is many to many }
where: {some conditional code around your search param},
}, {
model: models.Instruction,
as: 'Instructions',
where: {some conditional code around your search param},
}],
There is good documentation around multiple includes, or nested includes in the sequelize docs, but from what i see above you have a fairly good understanding of what you need to do. To uncomplicate things a bit, i'd start with just searching on your fields from recipie (title, description) before you add the includes and get that working, then it will be a little clearer how you want to form your where clauses.
alternativley.. you can skip the includes and write associations in your models and call them with getters and pass the where clauses to those... i do that as well and again well documented stuff now.. Sequelize has really upped their game
Recipie.associate = function (models) {
models.Recipie.hasMany(models.Ingredient, { as: 'Ingredients', through: "recipie_ingredient" foreignKey: 'recipie_id'});
};
now you have a getter for Ingredients, and if you declare belongsToMany targetting back at Recipie in the Ingredient model then you'll have a getter there as well, and you can pass your search string to that via where clause and get all recipies that have a given ingredient or ingredient list type thing.... Clear as mud?

How do you search a Model's attributes of type 'array'

I have a Model, Pet, with an attribute favoriteFoods.
// Pet.js
module.exports = {
attributes: {
name: {
type: 'string'
},
favoriteFoods: {
type: 'array'
}
}
};
favoriteFoods is stored as an array, because it's a free-text field for the user.
Now, is there a way to find based on values of that array?
I'm actually interested in querying from the URL, the following example DOES work, but it's not ideal.
/pet?where={"favoriteFoods":{"contains":"Best Food"}}
Is there a better way? A string comparison doesn't seem like the best solution.
For example, if one example entry has favoriteFoods: ["Best Food", "Best Treats"] and another has favoriteFoods: ["Second Best Food", "Best Treats"], then the query above will return 2 results. Our goal is a result w/ an exact match to one of the entries in the array, so this does not give us the intended result.
The simplest solution, although maybe not the most performant is to simply encapsulate in quotes. The following would find "Best Food" and not "Second Best Food"
/pet?where={"favoriteFoods":{"contains":"\"Best Food\""}}
Sub-document queries are being implemented in the next version of waterline. Your other other option would be to go native or query depending on your adapter.
In this instance your adapter would make the biggest difference.

Resources