Am using express js on node for my api service ! In which am using sequelize for query handling purposes !
So in some usecase like creating record, or updating record its simply returning "1" or sometimes nothing !
In this case , am just using
res.sendStatus(200);
or sometimes
res.send("success");
Is there any better way or this is the correct way to handle ? Or should in need .end() in order to end the process ??
which is a good way to handle these kind of useless responses which we dont need to send back ?
This is where Status 204 comes in to play: https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#2xx_success
It states: everything is OK (like in 200), but there is simple no need to send a body.
Using express, it's just as simple as: res.sendStatus(204)
Usually a json is send back to the client with different information in it that lets the client-side know that the operation that they request through the api is successfull or failed. for e.g. here, you can define a standard json response like this
//in case of success of the request
{
requestStatus: 1,
...other data if you want to like new Id of the created data/updated data or info about it if you want or depending upon use case
}
or
// in case of failure of the request
{
requestStatus: 0,
...other data if you want to like reason for failure, error message or depending upon your use case
}
Just add this line of code, should be fine:
//For all bad endpoints
app.use('*', (req, res) => {
res.status(404).json({ msg: 'Not a good endpoint, why are you here? Go play FIFA.'})
})
If you want you can generate an error HTML file, but since it's back-end, JSON format is strongly suggested. You can also add success: false for more clarity.
Related
i am trying to stream a response. But i want to be able to read the response (and work with the data) while it is still being sent. I basically want to send multiple messages in one response.
It works internally in node.js, but when i tried to do the same thing in typescript it doesnt work anymore.
My attempt was to do the request via fetch in typescript and the response is coming from a node.js server by writing parts of the response on the response stream.
fetch('...', {
...
}).then((response => {
const reader = response.body.getReader();
reader.read().then(({done, value}) => {
if (done) {
return response;
}
console.log(String.fromCharCode.apply(null, value)); //just for testing purposes
})
}).then(...)...
On the Node.js side it basically looks like this:
// doing stuff with the request
response.write(first_message)
// do some more stuff
response.write(second_message)
// do even more stuff
response.end(last_message)
In Node.js, like i said, i can just read every message once its sent via res.on('data', ...), but the reader.read in typescript only triggers(?) once and that is when the whole response was sent.
Is there a way to make it work like i want, or do i have to look for another way?
I hope it is kinda understandable what i want to do, i noticed while writing this how much i struggled explaining this :D
I found the problem, and as usual it was sitting in front of the pc.
I forgot to write a header first, before writing the response.
I have made an application where the user will send the Program in either C or Python and Input via STDIN, and my API will return back the output of Program in JSON Format.
Nodejs/Expressjs Code :
route.post('/exec', (req, res, next) => {
if(req.body.lang === 'c')
cExec(req, res)
if(req.body.lang === 'py3')
py3Exec(req, res)
})
And after the code execution in py3Exec() and cExec(), I wrote this to send back the output, using Exit Codes:
if(code === 0)
{
if(stdout === req.body.output)
res.send({
"result": "AC",
"output": stdout
})
else
res.send({
"result": "WA",
"output": stdout
})
}
else
{
res.send({
"result": "ERR",
"error": stderr
})
}
https://github.com/vkartik97/Online-IDE-API/blob/master/routes/run.js#L41
So, this API is REST. I wanted to know if this is Perfect way for this Use Case or if RPC should have been used as REST is used only for Resources from Server whereas RPC is used for Function Invocation(which is my case, maybe).
Thanks!
In my opinion it's not really a big deal which way you go, a lot will depend on the rest of the application. If all your application does is consume code, run it, and provides an output then either would most likely be fine.
You obviously can see how this might be an RPC call, with something like /exec, but it could also easily be a REST call, especially if you wanted to add some form of logging. For example: POST /program could be designed to consume some code, create a "program" in a database, run the code, and log the output along side the program in the database. You'd then return the program and it's response to the client.
If you're literally just consume code, running it, and giving an output back across HTTP then I'd be tempted to go with a JSON-RPC sort of endpoint for simplicity and to reveal intent (simply triggering an action).
Much depends on the future of your application and how much bigger it might become, and what other functionality you will have. If you're going to have a 90% RESTful API and a couple endpoints as a JSON-RPC, then I'd just go with a REST API for everything for consistency.
I am trying to send a json response back to the application with a status code. This what I tired but didn't work.
if (validation.fails()) {
console.log(validation.messages())
return response.json(validation.messages(),false,401)
// or return response.json(validation.messages(),401)
// It always sends 200 status code
}
Found the solution. I need to use like this return response.status(401).json(validation.messages())
One better solution is to use descriptive methods: https://adonisjs.com/docs/4.1/response#_descriptive_methods
Example:
response.unauthorized('Login First')
I'm writing a website which implements a usermanagement system and I wonder what best practices regarding form processing I have to consider.
Especially performance, security, SEO and user experience are important to me. When I was working on it I came across a couple questions and I didn't find an complete node/express code snippet where I could figure out all of my below questions.
Use case: Someone is going to update the birthday of his profile. Right now I am doing a POST request to the same URL to process the form on that page and the POST request will respond with a 302 redirect to the same URL.
General questions about form processing:
Should I do a POST request + 302 redirect for form processing or rather something else like an AJAX request?
How should I handle invalid FORM requests (for example invalid login, or email address is already in use during signup)?
Express specific questions about form processing:
I assume before inserting anything into my DB I need to sanitize and validate all form fields on the server side. How would you do that?
I read some things about CSRF but I have never implemented a CSRF protection. I'd be happy to see that in the code snippet too
Do I need to take care of any other possible vulnerabilities when processing forms with Express?
Example HTML/Pug:
form#profile(method='POST', action='/settings/profile')
input#profile-real-name.validate(type='text', name='profileRealName', value=profile.name)
label(for='profile-real-name') Name
textarea#profile-bio.materialize-textarea(placeholder='Tell a little about yourself', name='profileBio')
| #{profile.bio}
label(for='profile-bio') About
input#profile-url.validate(type='url', name='profileUrl', value=profile.bio)
label(for='profile-url') URL
input#profile-location.validate(type='text', name='profileLocation', value=profile.location)
label(for='profile-location') Location
.form-action-buttons.right-align
a.btn.grey(href='' onclick='resetForm()') Reset
button.btn.waves-effect.waves-light(type='submit')
Example Route Handlers:
router.get('/settings/profile', isLoggedIn, profile)
router.post('/settings/profile', isLoggedIn, updateProfile)
function profile(req, res) {
res.render('user/profile', { title: 'Profile', profile: req.user.profile })
}
function updateProfile(req, res) {
var userId = req.user._id
var form = req.body
var profile = {
name: form.profileRealName,
bio: form.profileBio,
url: form.profileUrl,
location: form.profileLocation
}
// Insert into DB
}
Note: A complete code snippet which takes care of all form processing best practices adapted to the given example is highly appreciated. I'm fine with using any publicly available express middleware.
Should I do a POST request + 302 redirect for form processing or rather something else like an AJAX request?
No, best practice for a good user experience since 2004 or so (basically since gmail launched) has been form submission via AJAX and not web 1.0 full-page load form POSTs. In particular, error handling via AJAX is less likely to leave your user at a dead end browser error page and then hit issues with the back button. The AJAX in this case should send an HTTP PATCH request to be most semantically correct but POST or PUT will also get the job done.
How should I handle invalid FORM requests (for example invalid login, or email address is already in use during signup)?
Invalid user input should result in an HTTP 400 Bad Request status code response, with details about the specific error(s) in a JSON response body (the format varies per application but either a general message or field-by-field errors are common themes)
For email already in use I use the HTTP 409 Conflict status code as a more particular flavor of general bad request payload.
I assume before inserting anything into my DB I need to sanitize and validate all form fields on the server side. How would you do that?
Absolutely. There are many tools. I generally define a schema for a valid request in JSON Schema and use a library from npm to validate that such as is-my-json-valid or ajv. In particular, I recommend being as strict as possible: reject incorrect types, or coerce types if you must, remove unexpected properties, use small but reasonable string length limits and strict regular expression patterns for strings when you can, and of course make sure your DB library property prevents injection attacks.
I read some things about CSRF but I have never implemented a CSRF protection.
The OWSAP Node Goat Project CSRF Exercise is a good place to start with a vulnerable app, understand and exploit the vulnerability, then implement the fix (in this case with a straightforward integration of the express.csrf() middleware.
Do I need to take care of any other possible vulnerabilities when processing forms with Express?
Yes generally application developers must understand and actively code securely. There's a lot of material out there on this but particular care must be taken when user input gets involved in database queries, subprocess spawning, or being written back out to HTML. Solid query libraries and template engines will handle most of the work here, you just need to be aware of the mechanics and potential places malicious user input could sneak in (like image filenames, etc).
I am certainly no Express expert but I think I can answer at least #1:
You should follow the Post/Redirect/Get web development pattern in order to prevent duplicate form submissions. I've heard a 303-redirect is the proper http statuscode for redirecting form submissions.
I do process forms using the POST route and once I'm done I trigger a 302-redirect.
As of #3 I recommend looking into express-validator, which is well introduce here: https://developer.mozilla.org/en-US/docs/Learn/Server-side/Express_Nodejs/forms . It's a middleware which allows you to validate and sanitize like this:
req.checkBody('name', 'Invalid name').isAlpha();
req.checkBody('age', 'Invalid age').notEmpty().isInt();
req.sanitizeBody('name').escape();
I wasn't able to comment hence the answer even though it's not a complete answer. Just thought it might help you.
If user experience is something you're thinking about, a page redirection is a strong no. Providing a smooth flow for the people visiting your website is important to prevent drops, and since forms are already not such a pleasure to fill, easing their usage is primary. You don't want to reload their page that might have already took some time to load just to display an error message. Once the form is valid and you created the user cookie, a redirection is fine though, even if you could do things on the client app to prevent it, but that's out-of-scope.
As stated by Levent, you should checkout express-validator, which is the more established solution for this kind of purpose.
req.check('profileRealName', 'Bad name provided').notEmpty().isAlpha()
req.check('profileLocation', 'Invalid location').optional().isAlpha();
req.getValidationResult().then(function (result) {
if (result.isEmpty()) { return null }
var errors = result.array()
// [
// { param: "profileRealName", msg: "Bad name provided", value: ".." },
// { param: "profileLocation", msg: "Invalid location", value: ".." }
// ]
res.status(400).send(errors)
})
.then(function () {
// everything is fine! insert into the DB and respond..
})
From what it looks like, I can assume you are using MongoDB. Given that, I would recommend using an ODM, like Mongoose. It will allow you to define models for your schemas and put restrictions directly on it, letting the model handles these kind of redundant validations for you.
For example, a model for your user could be
var User = new Schema({
name: { type: String, required: [true, 'Name required'] },
bio: { type: String, match: /[a-z]/ },
age: { type: Number, min: 18 }, // I don't know the kind of site you run ;)
})
Using this schema on your route would be looking like
var user = new User({
name: form.profileRealName,
bio: form.profileBio,
url: form.profileUrl,
location: form.profileLocation
})
user.save(function (err) {
// and you could grab the error here if it exists, and react accordingly
});
As you can see it provides a pretty cool api, which you should read about in their docs if you want to know more.
About CRSF, you should install csurf, which has pretty good instructions and example usages on their readme.
After that you're pretty much good to go, there is not much more I can think about apart making sure you stay up to date with your critical dependencies, in case a 0-day occurs, for example the one that happened in 2015 with JWTs, but that's still kinda rare.
Background: In Node, it is common to pass an error object to a callback function in async calls as well explained in Error handling in Node.js by Joyent. A standard error object contains a name, a message, a stack trace and possible additional properties. Passing or throwing strings or plain objects as errors is considered a bad practice. With Socket.io, data can be transmitted from client to server and vice versa by emitting events and having an optional response callback. This opens up multiple ways to pass and handle errors.
Question: If a socket.io event from a client causes an error on the server, what is the best practice to inform the client about this? A plain error object in response? A separate error event and listener? How the error should be structured? Is there any best practices like is the case with Node? What are you using?
Example: Imagine socket.io used for login. A client logs in by emitting a username and a password like below:
socket.emit('login', {user: 'Foo', pass: 'Bar'}, function (response) {
...
});
On a successful login, the response contains a session token. But what if the username or the password do not match? Several approaches come to mind:
Approach 1 - plain error object: The response could contain a property error having a plain error-like object as its value, with standard name and message properties and possible additional ones:
{
error: {
name: 'InvalidUsernameOrPasswordError',
message: 'Username or password was invalid.',
usernameExists: false
}
}
The client thus tests if response contains an error and if not, continues with the login procedure:
socket.emit('login', {user: 'Foo', pass: 'Bar'}, function (response) {
if (response.hasOwnProperty('error')) {
// handle error
}
// do something with response.token
});
Approach 2 - simple string: Similar to Approach 1, but the error property is just a plain string representing the name of the error.
{
error: 'InvalidUsernameOrPasswordError'
}
This approach is light and sufficient in this naïve example but lacks ability to pass additional data about the error.
Approach 3 - separate event: The server could emit and the client listen additional events to handle possible outcomes:
socket.on('loginError', function (error) {
// handle error based on error.name and error.message
});
socket.on('loginSuccess', function (data) {
// handle successful login with data.token
});
socket.emit('login', {user: 'Foo', pass: 'Bar'});
This approach feels the most explicit and pure under the event emitting paradigm but requires multiple event handlers.
Conclusions: There seem to be many possible ways to pass and handle errors. Any experiences, thoughts or feels?
It depends on your "clients". If you have end users like people using an application they really don't care so much about how you send the error. At the end you are going to have a text on the frontend saying they are wrong. So you are the one who have to select which way you prefer for implementing the error handling on the backend.
If your clients are not end users like the case above, imaging you are creating an application or library and you have to say there was an error. You should return as much as information you can in order for people who use your application identify where you error is.
So at the end:
-For customers: You will show your errors on the frontend so that ups to you how you want manage the error.
-For developers: You should show as many error information as you can. Showing the stack trace, error code...