Should we send a null or empty string value on request ?
I mean we have an optional value and it had a value currently. If use want to delete value of that optional field, should API understand null or empty is delete value ?
Ex:
{
name: { type: String, required: true },
phone: { type: String, required: false }
}
In database:
{
name: "Alex",
phone: "012-333.222"
}
And now, use want to delete their phone number
Should we define looks like:
PUT /users/user-id-1
{
phone: null
}
Seems it's a bad convention
Should we send a null or empty string value on request ?
REST doesn't care; which is to say that it tells us to use self descriptive messages to transfer documents over a network, but it doesn't tell us what the representations of the documents should be.
Where you want to be looking instead is at message schema definitions, and in particular designing your schema in such a way that it can be extended in backwards compatible ways. The XML community spent a lot of time exploring those ideas; Orchard 2004 might be a good starting point.
In HTTP, the basic mechanism for describing a change to a resource is to use a PUT command with a copy of the new representation. So a request would probably look like:
PUT /users/user-id-1
Content-Type: application/json
{
name: "Alex",
phone: null
}
If your schema is defined in such a way that the phone field is optional, and that optional and null are equivalent (as opposed to some other implied value), then you might equivalently use:
PUT /users/user-id-1
Content-Type: application/json
{
name: "Alex"
}
In cases where the representation is very big, and the changes you are making are small, you might want to support PATCH.
PATCH /users/user-id-1
Content-Type: application/merge-patch+json
{
phone: null
}
Note that the HTTP PATCH specification includes the Allow-Patch which allows clients to discover which patch representations a server supports for a resource.
Though, in front end, usually it depends on whether use the delete button or, they just leave the field empty
phone: '' - means user left field empty
phone: null - means user click on delete field button. You decide whether to delete the field, or just set the document field to null.
I will usually delete the field, since it is now useless.
If you want to update only one property in a document you can use PATCH method instead of PUT method and your code should look like this:
PATCH /users/user-id-1
{
phone: ""
}
Related
Let's say my graphql server wants to fetch the following data as JSON where person3 and person5 are some id's:
"persons": {
"person3": {
"id": "person3",
"name": "Mike"
},
"person5": {
"id": "person5",
"name": "Lisa"
}
}
Question: How to create the schema type definition with apollo?
The keys person3 and person5 here are dynamically generated depending on my query (i.e. the area used in the query). So at another time I might get person1, person2, person3 returned.
As you see persons is not an Iterable, so the following won't work as a graphql type definition I did with apollo:
type Person {
id: String
name: String
}
type Query {
persons(area: String): [Person]
}
The keys in the persons object may always be different.
One solution of course would be to transform the incoming JSON data to use an array for persons, but is there no way to work with the data as such?
GraphQL relies on both the server and the client knowing ahead of time what fields are available available for each type. In some cases, the client can discover those fields (via introspection), but for the server, they always need to be known ahead of time. So to somehow dynamically generate those fields based on the returned data is not really possible.
You could utilize a custom JSON scalar (graphql-type-json module) and return that for your query:
type Query {
persons(area: String): JSON
}
By utilizing JSON, you bypass the requirement for the returned data to fit any specific structure, so you can send back whatever you want as long it's properly formatted JSON.
Of course, there's significant disadvantages in doing this. For example, you lose the safety net provided by the type(s) you would have previously used (literally any structure could be returned, and if you're returning the wrong one, you won't find out about it until the client tries to use it and fails). You also lose the ability to use resolvers for any fields within the returned data.
But... your funeral :)
As an aside, I would consider flattening out the data into an array (like you suggested in your question) before sending it back to the client. If you're writing the client code, and working with a dynamically-sized list of customers, chances are an array will be much easier to work with rather than an object keyed by id. If you're using React, for example, and displaying a component for each customer, you'll end up converting that object to an array to map it anyway. In designing your API, I would make client usability a higher consideration than avoiding additional processing of your data.
You can write your own GraphQLScalarType and precisely describe your object and your dynamic keys, what you allow and what you do not allow or transform.
See https://graphql.org/graphql-js/type/#graphqlscalartype
You can have a look at taion/graphql-type-json where he creates a Scalar that allows and transforms any kind of content:
https://github.com/taion/graphql-type-json/blob/master/src/index.js
I had a similar problem with dynamic keys in a schema, and ended up going with a solution like this:
query lookupPersons {
persons {
personKeys
person3: personValue(key: "person3") {
id
name
}
}
}
returns:
{
data: {
persons: {
personKeys: ["person1", "person2", "person3"]
person3: {
id: "person3"
name: "Mike"
}
}
}
}
by shifting the complexity to the query, it simplifies the response shape.
the advantage compared to the JSON approach is it doesn't need any deserialisation from the client
Additional info for Venryx: a possible schema to fit my query looks like this:
type Person {
id: String
name: String
}
type PersonsResult {
personKeys: [String]
personValue(key: String): Person
}
type Query {
persons(area: String): PersonsResult
}
As an aside, if your data set for persons gets large enough, you're going to probably want pagination on personKeys as well, at which point, you should look into https://relay.dev/graphql/connections.htm
I'm trying to make a messaging system that writes each message to a mongo entry. I'd like the message entry to reflect the user that sends the message, and the actual message content. This is the message schema:
const MessageSchema = new Schema({
id: {
type: String,
required: true
},
messages: {
type: Array,
required: true
},
date: {
type: Date,
default: Date.now
}
});
And this is where I either create a new entry, or append to an existing one:
Message.findOne({ id: chatId }).then(message => {
if(message){
Message.update.push({ messages: { 'name': user.name, 'message': user.message } })
} else {
const newMessage = new Message(
{ id: chatId },
{ push: { messages: { 'name': user.name, 'message': user.message } } }
)
newMessage
.save()
.catch(err => console.log(err))
}
})
I'd like the end result to look something like this:
id: '12345'
messages: [
{name: 'David', message: 'message from David'},
{name: 'Jason', message: 'message from Jason'},
etc.
]
Is something like this possible, and if so, any suggestions on how to get this to work?
This questions contains lots of topics (in my mind at least). I really want to try to break this questions to its core components:
Design
As David noted (first comment) there is a design problem here - an ever-growing array as a sub document is not ideal (please refer to this blog post for more details).
On the over hand - when we imagine how a separate collection of messages will looks like, it will be something like this:
_id: ObjectId('...') // how do I identify the message
channel_id: 'cn247f9' // the message belong to a private chat or a group
user_id: 1234 // which user posted this message
message: 'hello or something' // the message itself
Which is also not that great because we are repeating the channel and user ids as a function of time. This is why the bucket pattern is used
So... what is the "best" approach here?
Concept
The most relevant question right now is - "which features and loads this chat is suppose to support?". I mean, many chats are only support messages display without any further complexity (like searching inside a message). Keeping that in mind, there is a chance that we store in our database an information that is practically irrelevant.
This is (almost) like storing a binary data (such an image) inside our db. we can do this, but with no actual good reason. So, if we are not going to support a full-text search inside our messages, there is no point to store the messages inside our db.. at all
But.. what if we want to support a full-text search? well - who said that we need to give this task to our database? we can easily download messages (using pagination) and make the search operation on the client side itself (while keyword not found, download previous page and search it), taking the loads out of our database!
So.. it seems like that messages are not ideal for storage in database in terms of size, functionality and loads (you may consider this conclusion as a shocking one)
ReDesign
Using a hybrid approach where messages are stored in a separated collection with pagination (the bucket pattern supports this as described here)
Store messages outside your database (since your are using Node.js you may consider using chunk store), keeping only a reference to them in the database itself
Set your page with a size relevant to your application needs and also with calculated fields (for instances: number of current messages in page) to ease database loads as much as possible
Schema
channels:
_id: ObjectId
pageIndex: Int32
isLastPage: Boolean
// The number of items here should not exceed page size
// when it does - a new document will be created with incremental pageIndex value
// suggestion: update previous page isLastPage field to ease querying of next page
messages:
[
{ userId: ObjectID, link: string, timestamp: Date }
]
messagesCount: Int32
Final Conclusion
I know - it seems like a complete overkill for such a "simple" question, but - Dawid Esterhuizen convinced me that designing your database to support your future loads from the very beginning is crucial and always better than simplifying db design too much
The bottom line is that the question "which features and loads this chat is suppose to support?" is still need to be answered if you intend to desgin your db efficiently (e.g. to find the Goldilocks zone where your design suits your application needs in the most optimal way)
I have an index 'user' which has a mapping with field of "first", "last", and "email". The fields with names get indexed at one point, and then the field with the email gets indexed at a separate point. I want these indices to have the same id though, corresponding to one user_id parameter. So something like this:
function indexName(client, id, name) {
return client.update({
index: 'user',
type: 'text',
id: id,
body: {
first: name.first
last: name.last
}
})
}
function indexEmail(client, id, email) {
return client.update({
index: 'user',
type: 'text',
id: id,
body: {
email: email
}
})
}
When running:
indexName(client, "Jon", "Snow").then(indexEmail(client, "jonsnow#gmail.com"))
I get an error message saying that the document has not been created yet. How do I account for a document with a variable number of fields? And how do I create the index if it has not been created and then subsequently update it as I go?
The function you are using, client.update, updates part of a document. What you actually needs is to first create the document using the client.create function.
To create and index, you need the indices.create function.
About the variable number of fields in a document type, it is not a problem because Elastic Search support dynamic mapping. However, it would be advisable to provide a mapping when creating the index, and try to stick to it. Elastic Search default mapping can create you problems later on, e.g. analyzing uuids or email addresses which then become difficult (or impossible) to search and match.
I'm working on an API and should return based on permissions only a subset of the actual object's properties. I'm writing my tests in mocha and chai and would like to test for something like this (given res is the response object from the server and res.body contains the received JSON data):
res.body.should.not.contain.properties.except(['prop1', 'prop2.subprop'])
in which case res.body.prop1 can be any kind of object, and res.body.prop2 is only allowed to contain the property subprop - which again could be any kind of object.
Now, I could write custom functions to test this, but I thought someone else had already a similar problem and there is an extension for chai for it maybe or some other library I could use instead.
Out of the box, I do not think Chai offers a way to build a query like this. However, a JSON Schema is a perfect fit for testing if an object matches a certain format. Luckily, a Chai JSON Schema Plugin exists. Using that, the result looks like this:
chai.use(require('chai-json-schema'));
var bodySchema = {
title: 'response body',
type: 'object',
required: ['prop1', 'prop2'],
additionalProperties: false,
properties: {
prop1: {},
prop2: {
type: 'object',
required: ['subprop'],
additionalProperties: false,
properties: {
subprop: {}
}
}
}
};
res.body.should.be.jsonSchema(bodySchema);
A short explanation:
The required property takes an array of required properties. If prop1 or prop2 are actually optional, remove them from this array (or leave it out alltogether).
The additionalProperties: false ensures no properties other than the ones defined in the properties hash are allowed.
prop2 contains a subschema, which can contain the same fields as the root schema and specifies the format of the sub-property.
Granted, these schema's can grow a bit large, but so would your validation function. Of course you can make the schema a true JSON file to separate it from the rest of your code.
I hope I'm having a big brainfart moment. But here's my situation in a scraping szenario;
I'm wanting to be able to scrape over multiple machines and cores. Per site, I have different Front pages, I scrape (exmpl. for the site stackoverflow I'd have fronts stackoverflow.com/questions/tagged/javascript and stackoverflow.com/questions/tagged/nodejs).
An article could be on every Front and when I discover an article I want to create an Article if the url is unknown, if its known I want to make an Front entry in article.discover if Front is unknown and otherwise insert my FrontDiscovery to the apropriate Front.
Here are my Schemas;
FrontDiscovery = new Schema({
_id :{ type:ObjectId, auto:true },
date :{ type: Date, default:Date.now},
dims :{ type: Object, default:null},
pos :{ type: Object, default:null}
});
Front = new Schema({
_id :{ type:ObjectId, auto:true },
url :{type:String}, //front
found :[ FrontDiscovery ]
});
Article = new Schema({
_id :{ type:ObjectId, auto:true },
url :{ type: String , index: { unique: true } },
site :{ type: String },
discover:[ Front]
});
The Problem I am thinking I will eventually be running into is a race condition. When two job-runners (in parallel) find the same (before unknown) article and create a new one. Yes, I have a unique index on it and could handle it that way - quite inelegantly imho.
But lets go further; When - for what ever reason - my 2 job-runners are scraping the same front at the same time and both notice that for Front there is no entry yet and create a new one adding the FrontDiscovery, I'd end with two entry's for the same Front.
What are your strategies to circumvent such a situation? findByIdAndUpdate with the upsert:true for each document seperately? If so, how can I only push something to the embedded document collection and not overwrite everything else at the same time but still create the defaults if it hasnt been created?
Thank you for any help in directing me in the right direction! I really hope I'm having a massive brainfart..
Update with upsert=true can be used to perform an atomic "insert or update" (http://docs.mongodb.org/manual/core/update/#update-operations-with-the-upsert-flag).
For instance if we want to make sure a document in Front collection with specific url is inserted exactly once, we could run something like:
db.Front.update(
{url: 'http://example.com'},
{$set: {
url: 'http://example.com'},
found: true
}
)
Operations on a single document in MongoDB are always atomic. If you make updates that span over multiple documents, then no atomicity is guaranteed. In such cases, you can ask yourself: do I really need the operations to be atomic? If the answer is no, then you probably will find your way around working with potentially unconsistent data. If the answer is yes and you want to stick with MongoDB, check out the design pattern on Two Phase Commits.