How to use Mongoose with GraphQL and DataLoader? - node.js

Question

I am using MongoDB as my database and GraphQL. I am using Mongoose for my model. I realised my GraphQL queries are slow because the same documents are being loaded over and over again. I would like to use DataLoader to solve my problem, but I don't know how.
Example
Let's say I have the following schema, describing users with friends :
// mongoose schema
const userSchema = new Schema({
name: String,
friendIds: [String],
})
userSchema.methods.friends = function() {
return User.where("_id").in(this.friendIds)
}
const User = mongoose.model("User", userSchema)
// GraphQL schema
const graphqlSchema = `
type User {
id: ID!
name: String
friends: [User]
}
type Query {
users: [User]
}
`
// GraphQL resolver
const resolver = {
Query: {
users: () => User.find()
}
}
Here is some example data in my database :
[
{ id: 1, name: "Alice", friendIds: [2, 3] },
{ id: 2, name: "Bob", friendIds: [1, 3] },
{ id: 3, name: "Charlie", friendIds: [2, 4, 5] },
{ id: 4, name: "David", friendIds: [1, 5] },
{ id: 5, name: "Ethan", friendIds: [1, 4, 2] },
]
When I do the following GraphQL query :
{
users {
name
friends {
name
}
}
}
each user is loaded many times. I would like each user Mongoose document to be loaded only once.
What doesn't work
Defining a "global" dataloader for fetching friends
If I change the friends method to :
// mongoose schema
const userSchema = new Schema({
name: String,
friendIds: [String]
})
userSchema.methods.friends = function() {
return userLoader.load(this.friendIds)
}
const User = mongoose.model("User", userSchema)
const userLoader = new Dataloader(userIds => {
const users = await User.where("_id").in(userIds)
const usersMap = new Map(users.map(user => [user.id, user]))
return userIds.map(userId => usersMap.get(userId))
})
then my users are cached forever rather than on a per request basis.
Defining the dataloader in the resolver
This seems more reasonable : one caching mechanism per request.
// GraphQL resolver
const resolver = {
Query: {
users: async () => {
const userLoader = new Dataloader(userIds => {
const users = await User.where("_id").in(userIds)
const usersMap = new Map(users.map(user => [user.id, user]))
return userIds.map(userId => usersMap.get(userId))
})
const userIds = await User.find().distinct("_id")
return userLoader.load(userIds)
}
}
}
However, userLoader is now undefined in the friends method in Mongoose schema. Let's move the schema in the resolver then!
// GraphQL resolver
const resolver = {
Query: {
users: async () => {
const userLoader = new Dataloader(userIds => {
const users = await User.where("_id").in(userIds)
const usersMap = new Map(users.map(user => [user.id, user]))
return userIds.map(userId => usersMap.get(userId))
})
const userSchema = new Schema({
name: String,
friendIds: [String]
})
userSchema.methods.friends = function() {
return userLoader.load(this.friendIds)
}
const User = mongoose.model("User", userSchema)
const userIds = await User.find().distinct("_id")
return userLoader.load(userIds)
}
}
}
Mh ... Now Mongoose is complaining on the second request : resolver gets called again, and Mongoose doesn't like 2 models being defined with the same model name.
"Virtual populate" feature are of no use, because I can't even tell Mongoose to fetch models through the dataloader rather than through the database directly.
Question
Has anyone had the same problem? Does anyone have a suggestion on how to use Mongoose and Dataloader in combination? Thanks.
Note: I know since my schema is "relational", I should be using a relational database rather than MongoDB. I was not the one to make that choice. I have to live with it until we can migrate.

Answer 1

Keep your mongoose schema in a separate module. You don't want to create your schema each request -- just the first time the module is imported.
const userSchema = new Schema({
name: String,
friendIds: [String]
})
const User = mongoose.model("User", userSchema)
module.exports = { User }
If you want, you can also export a function that creates your loader in the same module. Note, however, that we do not want to export an instance of a loader, just a function that will return one.
// ...
const getUserLoader = () => new DataLoader((userIds) => {
return User.find({ _id: { $in: userIds } }).execute()
})
module.exports = { User, getUserLoader }
Next, we want to include our loader in the context. How exactly this is done will depend on what library you're using to actually expose your graphql endpoint. In apollo-server, for example, context is passed in as part of your configuration.
new ApolloServer({
typeDefs,
resolvers,
context: ({ req }) => ({
userLoader: getUserLoader()
}),
})
This will ensure that we have a fresh instance of the loader created for each request. Now, your resolvers can just call the loader like this:
const resolvers = {
Query: {
users: async (root, args, { userLoader }) => {
// Our loader can't get all users, so let's use the model directly here
const allUsers = await User.find({})
// then tell the loader about the users we found
for (const user of allUsers) {
userLoader.prime(user.id, user);
}
// and finally return the result
return allUsers
}
},
User: {
friends: async (user, args, { userLoader }) => {
return userLoader.loadMany(user.friendIds)
},
},
}