Let's say my graphql server wants to fetch the following data as JSON where person3 and person5 are some id's:
"persons": {
"person3": {
"id": "person3",
"name": "Mike"
},
"person5": {
"id": "person5",
"name": "Lisa"
}
}
Question: How to create the schema type definition with apollo?
The keys person3 and person5 here are dynamically generated depending on my query (i.e. the area used in the query). So at another time I might get person1, person2, person3 returned.
As you see persons is not an Iterable, so the following won't work as a graphql type definition I did with apollo:
type Person {
id: String
name: String
}
type Query {
persons(area: String): [Person]
}
The keys in the persons object may always be different.
One solution of course would be to transform the incoming JSON data to use an array for persons, but is there no way to work with the data as such?
GraphQL relies on both the server and the client knowing ahead of time what fields are available available for each type. In some cases, the client can discover those fields (via introspection), but for the server, they always need to be known ahead of time. So to somehow dynamically generate those fields based on the returned data is not really possible.
You could utilize a custom JSON scalar (graphql-type-json module) and return that for your query:
type Query {
persons(area: String): JSON
}
By utilizing JSON, you bypass the requirement for the returned data to fit any specific structure, so you can send back whatever you want as long it's properly formatted JSON.
Of course, there's significant disadvantages in doing this. For example, you lose the safety net provided by the type(s) you would have previously used (literally any structure could be returned, and if you're returning the wrong one, you won't find out about it until the client tries to use it and fails). You also lose the ability to use resolvers for any fields within the returned data.
But... your funeral :)
As an aside, I would consider flattening out the data into an array (like you suggested in your question) before sending it back to the client. If you're writing the client code, and working with a dynamically-sized list of customers, chances are an array will be much easier to work with rather than an object keyed by id. If you're using React, for example, and displaying a component for each customer, you'll end up converting that object to an array to map it anyway. In designing your API, I would make client usability a higher consideration than avoiding additional processing of your data.
You can write your own GraphQLScalarType and precisely describe your object and your dynamic keys, what you allow and what you do not allow or transform.
See https://graphql.org/graphql-js/type/#graphqlscalartype
You can have a look at taion/graphql-type-json where he creates a Scalar that allows and transforms any kind of content:
https://github.com/taion/graphql-type-json/blob/master/src/index.js
I had a similar problem with dynamic keys in a schema, and ended up going with a solution like this:
query lookupPersons {
persons {
personKeys
person3: personValue(key: "person3") {
id
name
}
}
}
returns:
{
data: {
persons: {
personKeys: ["person1", "person2", "person3"]
person3: {
id: "person3"
name: "Mike"
}
}
}
}
by shifting the complexity to the query, it simplifies the response shape.
the advantage compared to the JSON approach is it doesn't need any deserialisation from the client
Additional info for Venryx: a possible schema to fit my query looks like this:
type Person {
id: String
name: String
}
type PersonsResult {
personKeys: [String]
personValue(key: String): Person
}
type Query {
persons(area: String): PersonsResult
}
As an aside, if your data set for persons gets large enough, you're going to probably want pagination on personKeys as well, at which point, you should look into https://relay.dev/graphql/connections.htm
I'm using Mongoose in Node.js, and I am wondering if it is possible to refer to the currently selected document using "this" or a similar mechanism. Here is the use case I'm looking for :
Mongoose Schema :
const mySchema = mongoose.Schema({
position: Number,
date: Number,
lastEventDate: Number
});
Let's say that, at some point in time, an event occurs.
For a document selected through its position, I want to update "lastEventDate" to the document's date.
Here is my dream code :
myModel.findOneAndUpdate(
{position: myPosition},
{$set: {
'lastEventDate': THISDOCUMENT.date
}}
);
Note : I'm using $set here because the actual code updates subdocuments...
Is there a built-in "THISDOCUMENT" reference such as the one I'm dreaming of, to do it all in a single query ?
Or do I have to first query the value before updating the document (two queries).
Couldn't find anything on the web, and I'm quite the newbie when it comes to using "this".
Thanks for any kind of help !
[EDIT :] Precisions about the objective :
I am in a situation where I only have the position "myPosition" to identify the correct document, and I want to set "lastEventDate" to the same value as "date" for that document.
My question is about efficiency : is it possible to perform the update in a single upload query ? Or do I have to first download the "date" value before uploading it back to the "lastEventDate" key ?
Gathering all the information provided, I will venture on a possible answer!
You could try something like:
Your schema JS file
const mySchema = mongoose.Schema({
position: Number,
date: Number,
lastEventDate: Number
});
mySchema.methods.doYourThing(){
this.lastEventDate=this.date; //it will set the lastEventDate
}
mongoose.model("myModel", MySchema, "mycollection")
Now, whenever you call doYourThing(), the action wanted will take place, you call it after you have a instance of the mode.
This is from my own code
const token = user.generateJwt(expirationDate); //send a token, it will be stored locally in the browser
it is inside a function that return an instance of user, and in the model User I have done a function called generateJwt like I have showed, and we have something like this:
return jwt.sign(
{
_id: this._id, //this is created automatically by Mongo
email: this.email,
name: this.name,
exp: parseInt(expiry.getTime() / 1000, 10), //Includes exp as UNIX time in seconds
level: this.level,
lastLogin: this.lastLogin,
failedLogin: this.failedLogin
},
process.env.JWT_SECRET
); // DO NOT KEEP YOUR SECRET IN THE CODE!
It returns all the information of the user!
Please, do not hesitate to add comments and feebacks, I am not sure it is what you want, but that is why I have understood your request.
Anothe option is using Virtuals, they also have access to this.
I'm trying to make a messaging system that writes each message to a mongo entry. I'd like the message entry to reflect the user that sends the message, and the actual message content. This is the message schema:
const MessageSchema = new Schema({
id: {
type: String,
required: true
},
messages: {
type: Array,
required: true
},
date: {
type: Date,
default: Date.now
}
});
And this is where I either create a new entry, or append to an existing one:
Message.findOne({ id: chatId }).then(message => {
if(message){
Message.update.push({ messages: { 'name': user.name, 'message': user.message } })
} else {
const newMessage = new Message(
{ id: chatId },
{ push: { messages: { 'name': user.name, 'message': user.message } } }
)
newMessage
.save()
.catch(err => console.log(err))
}
})
I'd like the end result to look something like this:
id: '12345'
messages: [
{name: 'David', message: 'message from David'},
{name: 'Jason', message: 'message from Jason'},
etc.
]
Is something like this possible, and if so, any suggestions on how to get this to work?
This questions contains lots of topics (in my mind at least). I really want to try to break this questions to its core components:
Design
As David noted (first comment) there is a design problem here - an ever-growing array as a sub document is not ideal (please refer to this blog post for more details).
On the over hand - when we imagine how a separate collection of messages will looks like, it will be something like this:
_id: ObjectId('...') // how do I identify the message
channel_id: 'cn247f9' // the message belong to a private chat or a group
user_id: 1234 // which user posted this message
message: 'hello or something' // the message itself
Which is also not that great because we are repeating the channel and user ids as a function of time. This is why the bucket pattern is used
So... what is the "best" approach here?
Concept
The most relevant question right now is - "which features and loads this chat is suppose to support?". I mean, many chats are only support messages display without any further complexity (like searching inside a message). Keeping that in mind, there is a chance that we store in our database an information that is practically irrelevant.
This is (almost) like storing a binary data (such an image) inside our db. we can do this, but with no actual good reason. So, if we are not going to support a full-text search inside our messages, there is no point to store the messages inside our db.. at all
But.. what if we want to support a full-text search? well - who said that we need to give this task to our database? we can easily download messages (using pagination) and make the search operation on the client side itself (while keyword not found, download previous page and search it), taking the loads out of our database!
So.. it seems like that messages are not ideal for storage in database in terms of size, functionality and loads (you may consider this conclusion as a shocking one)
ReDesign
Using a hybrid approach where messages are stored in a separated collection with pagination (the bucket pattern supports this as described here)
Store messages outside your database (since your are using Node.js you may consider using chunk store), keeping only a reference to them in the database itself
Set your page with a size relevant to your application needs and also with calculated fields (for instances: number of current messages in page) to ease database loads as much as possible
Schema
channels:
_id: ObjectId
pageIndex: Int32
isLastPage: Boolean
// The number of items here should not exceed page size
// when it does - a new document will be created with incremental pageIndex value
// suggestion: update previous page isLastPage field to ease querying of next page
messages:
[
{ userId: ObjectID, link: string, timestamp: Date }
]
messagesCount: Int32
Final Conclusion
I know - it seems like a complete overkill for such a "simple" question, but - Dawid Esterhuizen convinced me that designing your database to support your future loads from the very beginning is crucial and always better than simplifying db design too much
The bottom line is that the question "which features and loads this chat is suppose to support?" is still need to be answered if you intend to desgin your db efficiently (e.g. to find the Goldilocks zone where your design suits your application needs in the most optimal way)
Following is my user schema in user.js model -
var userSchema = new mongoose.Schema({
local: {
name: { type: String },
email : { type: String, require: true, unique: true },
password: { type: String, require:true },
},
facebook: {
id : { type: String },
token : { type: String },
email : { type: String },
name : { type: String }
}
});
var User = mongoose.model('User',userSchema);
module.exports = User;
This is how I am using it in my controller -
var user = require('./../models/user.js');
This is how I am saving it in the db -
user({'local.email' : req.body.email, 'local.password' : req.body.password}).save(function(err, result){
if(err)
res.send(err);
else {
console.log(result);
req.session.user = result;
res.send({"code":200,"message":"Record inserted successfully"});
}
});
Error -
{"name":"MongoError","code":11000,"err":"insertDocument :: caused by :: 11000 E11000 duplicate key error index: mydb.users.$email_1 dup key: { : null }"}
I checked the db collection and no such duplicate entry exists, let me know what I am doing wrong ?
FYI - req.body.email and req.body.password are fetching values.
I also checked this post but no help STACK LINK
If I removed completely then it inserts the document, otherwise it throws error "Duplicate" error even I have an entry in the local.email
The error message is saying that there's already a record with null as the email. In other words, you already have a user without an email address.
The relevant documentation for this:
If a document does not have a value for the indexed field in a unique index, the index will store a null value for this document. Because of the unique constraint, MongoDB will only permit one document that lacks the indexed field. If there is more than one document without a value for the indexed field or is missing the indexed field, the index build will fail with a duplicate key error.
You can combine the unique constraint with the sparse index to filter these null values from the unique index and avoid the error.
unique indexes
Sparse indexes only contain entries for documents that have the indexed field, even if the index field contains a null value.
In other words, a sparse index is ok with multiple documents all having null values.
sparse indexes
From comments:
Your error says that the key is named mydb.users.$email_1 which makes me suspect that you have an index on both users.email and users.local.email (The former being old and unused at the moment). Removing a field from a Mongoose model doesn't affect the database. Check with mydb.users.getIndexes() if this is the case and manually remove the unwanted index with mydb.users.dropIndex(<name>).
If you are still in your development environment, I would drop the entire db and start over with your new schema.
From the command line
➜ mongo
use dbName;
db.dropDatabase();
exit
I want to explain the answer/solution to this like I am explaining to a 5-year-old , so everyone can understand .
I have an app.I want people to register with their email,password and phone number .
In my MongoDB database , I want to identify people uniquely based on both their phone numbers and email - so this means that both the phone number and the email must be unique for every person.
However , there is a problem : I have realized that everyone has a phonenumber but not everyone has an email address .
Those that don`t have an email address have promised me that they will have an email address by next week. But I want them registered anyway - so I tell them to proceed registering their phonenumbers as they leave the email-input-field empty .
They do so .
My database NEEDS an unique email address field - but I have a lot of people with 'null' as their email address . So I go to my code and tell my database schema to allow empty/null email address fields which I will later fill in with email unique addresses when the people who promised to add their emails to their profiles next week .
So its now a win-win for everyone (but you ;-] ): the people register, I am happy to have their data ...and my database is happy because it is being used nicely ...but what about you ? I am yet to give you the code that made the schema .
Here is the code :
NOTE : The sparse property in email , is what tells my database to allow null values which will later be filled with unique values .
var userSchema = new mongoose.Schema({
local: {
name: { type: String },
email : { type: String, require: true, index:true, unique:true,sparse:true},
password: { type: String, require:true },
},
facebook: {
id : { type: String },
token : { type: String },
email : { type: String },
name : { type: String }
}
});
var User = mongoose.model('User',userSchema);
module.exports = User;
I hope I have explained it nicely .
Happy NodeJS coding / hacking!
In this situation, log in to Mongo find the index that you are not using anymore (in OP's case 'email'). Then select Drop Index
Check collection indexes.
I had that issue due to outdated indexes in collection for fields, which should be stored by different new path.
Mongoose adds index, when you specify field as unique.
Well basically this error is saying, that you had a unique index on a particular field for example: "email_address", so mongodb expects unique email address value for each document in the collection.
So let's say, earlier in your schema the unique index was not defined, and then you signed up 2 users with the same email address or with no email address (null value).
Later, you saw that there was a mistake. so you try to correct it by adding a unique index to the schema. But your collection already has duplicates, so the error message says that you can't insert a duplicate value again.
You essentially have three options:
Drop the collection
db.users.drop();
Find the document which has that value and delete it. Let's say the value was null, you can delete it using:
db.users.remove({ email_address: null });
Drop the Unique index:
db.users.dropIndex(indexName)
I Hope this helped :)
Edit: This solution still works in 2023 and you don't need to drop your collection or lose any data.
Here's how I solved same issue in September 2020. There is a super-fast and easy way from the mongodb atlas (cloud and desktop). Probably it was not that easy before? That is why I feel like I should write this answer in 2020.
First of all, I read above some suggestions of changing the field "unique" on the mongoose schema. If you came up with this error I assume you already changed your schema, but despite of that you got a 500 as your response, and notice this: specifying duplicated KEY!. If the problem was caused by schema configuration and assuming you have configurated a decent middleware to log mongo errors the response would be a 400.
Why this happens (at least the main reason)
Why is that? In my case was simple, that field on the schema it used to accept only unique values but I just changed it to accept repeated values. Mongodb creates indexes for fields with unique values in order to retrieve the data faster, so on the past mongo created that index for that field, and so even after setting "unique" property as "false" on schema, mongodb was still using that index, and treating it as it had to be unique.
How to solve it
Dropping that index. You can do it in 2 seconds from Mongo Atlas or executing it as a command on mongo shell. For the sack of simplicity I will show the first one for users that are not using mongo shell.
Go to your collection. By default you are on "Find" tab. Just select the next one on the right: "Indexes". You will see how there is still an index given to the same field is causing you trouble. Just click the button "Drop Index". Done.
So don't drop your database everytime this happens
I believe this is a better option than just dropping your entire database or even collection. Basically because this is why it works after dropping the entire collection. Because mongo is not going to set an index for that field if your first entry is using your new schema with "unique: false".
I faced similar issues ,
I Just clear the Indexes of particular fields then its works for me .
https://docs.mongodb.com/v3.2/reference/method/db.collection.dropIndexes/
This is my relavant experience:
In 'User' schema, I set 'name' as unique key and then ran some execution, which I think had set up the database structure.
Then I changed the unique key as 'username', and no longer passed 'name' value when I saved data to database. So the mongodb may automatically set the 'name' value of new record as null which is duplicate key. I tried the set 'name' key as not unique key {name: {unique: false, type: String}} in 'User' schema in order to override original setting. However, it did not work.
At last, I made my own solution:
Just set a random key value that will not likely be duplicate to 'name' key when you save your data record. Simply Math method '' + Math.random() + Math.random() makes a random string.
I had the same issue. Tried debugging different ways couldn't figure out. I tried dropping the collection and it worked fine after that. Although this is not a good solution if your collection has many documents. But if you are in the early state of development try dropping the collection.
db.users.drop();
I have solved my problem by this way.
Just go in your mongoDB account -> Atlast collection then drop your database column. Or go mongoDB compass then drop your database,
It happed sometimes when you have save something null inside database.
This is because there is already a collection with the same name with configuration..Just remove the collection from your mongodb through mongo shell and try again.
db.collectionName.remove()
now run your application it should work
I had a similar problem and I realized that by default mongo only supports one schema per collection. Either store your new schema in a different collection or delete the existing documents with the incompatible schema within the your current collection. Or find a way to have more than one schema per collection.
I got this same issue when I had the following configuration in my config/models.js
module.exports.models = {
connection: 'mongodb',
migrate: 'alter'
}
Changing migrate from 'alter' to 'safe' fixed it for me.
module.exports.models = {
connection: 'mongodb',
migrate: 'safe'
}
same issue after removing properties from a schema after first building some indexes on saving. removing property from schema leads to an null value for a non existing property, that still had an index. dropping index or starting with a new collection from scratch helps here.
note: the error message will lead you in that case. it has a path, that does not exist anymore. im my case the old path was ...$uuid_1 (this is an index!), but the new one is ....*priv.uuid_1
I have also faced this issue and I solved it.
This error shows that email is already present here. So you just need to remove this line from your Model for email attribute.
unique: true
This might be possible that even if it won't work. So just need to delete the collection from your MongoDB and restart your server.
It's not a big issue but beginner level developers as like me, we things what kind of error is this and finally we weast huge time for solve it.
Actually if you delete the db and create the db once again and after try to create the collection then it's will be work properly.
➜ mongo
use dbName;
db.dropDatabase();
exit
Drop you database, then it will work.
You can perform the following steps to drop your database
step 1 : Go to mongodb installation directory, default dir is "C:\Program Files\MongoDB\Server\4.2\bin"
step 2 : Start mongod.exe directly or using command prompt and minimize it.
step 3 : Start mongo.exe directly or using command prompt and run the following command
i) use yourDatabaseName (use show databases if you don't remember database name)
ii) db.dropDatabase()
This will remove your database.
Now you can insert your data, it won't show error, it will automatically add database and collection.
I had the same issue when i tried to modify the schema defined using mangoose. I think the issue is due to the reason that there are some underlying process done when creating a collection like describing the indices which are hidden from the user(at least in my case).So the best solution i found was to drop the entire collection and start again.
If you are in the early stages of development: Eliminate the collection. Otherwise: add this to each attribute that gives you error (Note: my English is not good, but I try to explain it)
index:true,
unique:true,
sparse:true
in my case, i just forgot to return res.status(400) after finding that user with req.email already exists
Go to your database and click on that particular collection and delete all the indexes except id.
I'm learning to use with mongoose and node.js when editing to generate a content that gives me the error: 500 TypeError: Can not read property 'ObjectID' of undefined. Check whether the parameter that I send comes empty and is not.
Remove the option and gives me that error when saving the data did not find the method I use to save some registry update.
If I can help as I have code in https://github.com/boneyking/PruebaNode.git
First of all, please provide code or at least a link to the code. A link to your github repo is not cool !!!
Now back to the question... Add an id to your model and get it like other types:
var SUPER = mongoose.model('Producto',{
nombre: String,
descripcion: String,
precio: Number,
id: mongoose.Schema.Types.ObjectId
});
SUPER.editar = function(newData, callback){
SUPER.findOne({id: newData.id}, function(e,o){
o.nombre = newData.nombre;
o.descripcion = newData.descripcion;
o.precio = newData.precio;
SUPER.save(o);
callback(o);
});
}