Evaluating Nodejs and trying to see if it will fit our needs. Coming from a rails world, I have a few unanswered questions despite searching for a long time.
Whats the best way to manage assets with Nodejs (Expressjs)? In rails, the static assets are a) fingerprinted for caching forever b) js and css are minified 3) scss is compiled down to css
Whats the best way to handled uploaded images from users such as avatars?
Does grunt help with minifying and gzipping html/css/javascript?
How can I avoid mutliple http requests to the server with Node. I don't want to make multiple http requests for every javascript asset I need. Rails helps here by combining all js and css files.
Also, is Mongodb the preferred solution for most projects? I hear a lot of bad things about Mongodb and good things about it. Having a difficult time deciding if Mongo can help with more reads than writes with probably 400-500 GB data in the long run.
Any help or pointers? Thanks so much for taking the time to point me to the right places.
For each of the point you mentioned I give you a few module examples that might fit your need. Remember that at every point there are much more modules serving the same purpose:
node-static (as a static file server), node-uglify (for minifying JS code), css-clean (same for CSS), merge-js, sqwish, and jake can help you with the building of the website (in this step you could plug-in the previous modules)
node-formidable is pretty famous
check out this question
checkout this question
I am not sure if it is the "preferred". It is the noSQL and the Javascript nature in it that makes it attractive. There are modules already for every type of database. ongo should handle that data. Depends also how large is one document. There are some limitations.
There is this Github Wiki page in the NodeJS project listing and categorizing many of the important modules out there.
The exact choice of modules also depends what framework will you use use to build the app. A pretty established (but certainly not the only one) is express. But you can find more on this topic here.
Related
I've been learning Node.js for a couple of weeks. These days I was looking for advice on best practices and came across the concept of "Separation of concerns". I understood (tell me if I'm wrong) that it consists on split the backend and the frontend into two different servers. I've been using template engines such as ejs and pug so far, but I always have these files in the same directory as the rest of the Express server (in a "views" folder). I think there are many ways to separate the frontend and backend, maybe having 2 folders, 1 for the frontend with just simple HTML, CSS, and js files and make AJAX requests to the API, that will be in the other folder, the backend's one.
I'm pretty new in the backend so I hope this question makes sense haha. I'm a bit confused about how backend and frontend connections should work and how I can make a frontend that is independent of the technology used in the backend, so if you know books or resources in general about this, tell me.
welcome to stack
Best practices are different from developer to another and they are tend to be evolving.. so that is what i’m doing :
1 - Separate Directory for fronted and backendusing node proxy .
2- src Folder that contains ( for backend ) views , routes and controllers … etc .
Separation of concerns is the idea that each module or layer in an application should only be responsible for one thing and should not contain code that deals with other things.
Separating concerns reduces code complexity by breaking a large application down into many smaller units of encapsulated functionality.
i want to create a website based on node js and mysql , but i've read that there is a framework called express for node js , and i'm wondering if i must to use such kind of a framework to create a decent website or it is possible without it and just work with pure node js.
No framework is required. You can write a full-blown web server using only the http module or if you really want to write everything yourself, you can even do it with only the net module.
It's really about what is the most effective use of your time and skill as a developer. Except for academic or pure learning experience reasons, if you're just trying to accomplish a task as efficiently as possible and free, pre-existing, pre-tested code exists that makes your job easier, then that's a better way to go.
For example, if I need to do a file upload from a browser to my back-end and the data is coming in as the multipart/formdata content-type from the browser, I have zero interest in reading and learning the multipart/formdata RFC then writing my own code to parse the multipart/formdata content-type. Pre-existing, already tested code exists to do that for me and I'm adding no value to the goals of my project by re-implementing and then testing it all myself. Therefore, I'd like to use a pre-built module that does all that for me. I can just configure the right library on the right route and out plops my uploaded file in only the amount of time it takes to understand the interface to the 3rd party module and how to use it properly.
This is where Express comes in. Not only does it offer a useful set of features and architecture for configuring routes, installing middleware, sending responses, using template engines, handling errors, etc... but there are also thousands of third party modules that are built to hook into Express and it is easiest to use them if you're using Express as your core framework. Some of these modules could be used outside of Express, some cannot - it really depends upon how they're designed and what Express interfaces they do or don't use.
Also, Express is fairly "un-opinionated" and fairly "lightweight" which means it doesn't force you into a particular methodology. It just offers you easier ways to do things you were already going to have to write code for yourself.
Look at it this way. When you get node.js, there are thousands of APIs that offer lots of already tested things such as a TCP library, a file I/O library, etc... Those are frameworks (in a sense) too. You don't have to use them either. You could rewrite whatever functionality you need from scratch. But, you wouldn't even think about doing that because tested code already exists that solves your problem. So, you happily build on top of things that are already done.
One of the BIG advantages of coding with node.js is getting access to the tens of thousands of pre-built modules on NPM that already solve problems that many people have. Coding in node.js with a mindset that you will never use any outside modules from NPM is throwing away one of the biggest advantages of coding with node.js.
could you tell me what are the Routes used for in frameworks?
A route is a URL that you wish for your web server to respond to. So, if you want http://myserver.com/categories to be URL that your server responds to, then you create a route for /categories so that you can write code for what should happen when that URL is requested. A framework like Express allows you to create that route very simply with just a single statement such as:
app.get('/categories', function(req, res) {
// put code here to handle that request
});
This is just the tip of the iceberg for what Express supports. It allows you to use wildcards in route definitions, identify parameters in urls, create middleware that does prep work on lots of routes (such as check if the user is logged in), etc...
You don't have to use a framework but it is recommended to use one of them since frameworks like Express make your life easier in many ways. Check this: What is Express.js?
Yes you CAN write a Node.js-based backend without any back end implementation framework such as Express. And if you are using Node.js for the first time without any previous experience of asynchronous coding, I'd advise against using Express, KOA or other Node implementation frameworks for your simple learner apps (e.g. those needing things like register/login form processing, logout button, user preference updates to database, etc) because:
(1) Node.js is a core skill for JavaScript back ends.
Stupid analogies between server tasking and restaurant waiters are no use to a real web engineer. You must first know what exactly Node can/cannot do in the server CPU that makes it different to most other back end technologies. Then you have to see how the Node process actually does this. Using Express/KOA/Hapi/etc you are sometimes effectively removing the mental challenges that come with a Node back end. Any time-saving is achieved at the expense of gaining a proper working understanding of what Node is and how it really operates.
(2) Learning Node.js and its asynchronous coding is hard enough without the added complication of coding with an unknown framework like Express/KOA that assumes users' familiarity with JavaScript constructs like callback functions and Promises. It's always better to learn something in isolation so you get the essence of its individual effects, rather than the overall effects if used with other packages/frameworks. So many of these Node.js Express tutorials are the software equivalent of learning to make a cake by watching Momma do it. We can copy it but we don't know how or why it's working. Professional coders can't just be good copycats.
(3) Available learning tutorials using Express often drag in other technologies like MongoDB, Mongoose, Mustache, Handlebars, etc that make learning Node.js even more awkward still.
(4) A share of basic web apps can be written more efficiently with Node.js, custom JS and existing JS modules imported off the npm repository rather than with Express.
(5) Once asynchronous coding and the JavaScript constructs available to assist with it are understood clearly, pure Node.js apps for basic tasks aren't that hard.
(5) After you do get your head around Node.js and can get basic web app functionalities working using server-side JavaScript constructs, you can then judiciously start to explore Express/Hapi/KOA/etc and see what an implementation framework can do for your workflow when doing larger projects needing numerous functionalities. At this point you know what Express code should be doing and why it is done the way it is.
Node.js has become the back-end technology of choice for most small to medium scale web applications over the last 10 years. It is also the major reason why the JavaScript language has evolved from a mere front-end scripting tool with a limited set of Java-aping constructs to the innovative and comprehensive language that it is today. It is also the most popular language in use today. Investing time in understanding the Node server framework, and the latest JavaScript constructs used in Node, is time well spent. Implementation frameworks such as Express, KOA, Hapi, Sails, etc have great benefit when writing more elaborate back ends on the Node.js platform. But all these implementation frameworks are predicated on the behaviour patterns of Node.js. So unless Node itself is understood first, the full utility of Express/KOA/Sails/etc will never be enjoyed.
Try here for the pure Node.js.
I want to stream videos from a server to a web page with an html video tag. I am using node, and plan to stick with it (no nginx).
For the moment, I am using express.static middleware, i.e. serve-static, but I am wondering since it is made for serving assets, html pages, etc ... if it is suitable for streaming big videos!?
I took a peak at the code, and it seems that it does things properly : support for Accept-Ranges header, etc ... but I lack experience and knowledge about this specific topic, so I can't figure out whether things are as optimal as they could be.
Any suggestion of a better express middleware, or node server for this? purpose?
EDIT
I do not need to do anything fancy such adaptative bitrate, etc ... I simply want to make sure that - within the node realm - this setup is optimal to serve a video, since my server is installed on an embedded system with very little RAM available.
The best solution is to use a proper optimized web server, such as Nginx.
express.static is for utility purposes. Node.js as a whole is useful for building your application server. If you want to serve static files, use a web server. Otherwise you have the extra overhead of JavaScript for no benefit.
This goes for any static files, not just video. The size of the static content really has no bearing here on what's best, as all the servers stream large resources from disk.
I am considering using nodejs to make a non-realtime app. For example, a website like a blog, forum, or image boards.
I have read that nodejs is good when used for asynchronous jobs. So I am wondering what the result would be when used to serve a lot of static files, like big images, css & js files, etc.
Is it true that when sending a file (suppose it's 2-3MB), the whole server will be blocked until the transfer is complete? I have also read that it might be possible to use the OS's sendfile() syscall to do this job. In this case, does Express support this?
No it is not true. You can easily send files that are large (much larger than 2-3 MB) without blocking. People who complain about things like this blocking the Node event loop just don't know what they're doing.
You don't necessarily need to use Express to get this behavior.
That being said, if what you want is a file server, there's no reason to use NodeJS. Just point Apache at a directory, and let it fly. Why reinvent the wheel just to use the new sexy technology, when old faithful does just fine?
If you would like to use node as a simple http server, may I recommend the very simple command line module.
https://npmjs.org/package/http-server
I haven't looked at the code of the module, but it is likely not optimized for large files. Let's define large in this case, as files that are not easily cached in memory(whatever this means for your setup). If your use case calls for more optimization (piping "large" files for example) you may still have to write your own module, but this will get you started very quickly, and is an excellent utility to use for general development when you need to serve up a directory real quick.
I have a question - does anyone has any benchmark data re. using Express JS framework vs pure Node.js?
Is this something that even should be considered -- or using Express or similar framework is definitely a MUST if you're building a more or less large web app?
EDIT:
The article I linked to below disappeared, and I couldn't find it after searching around a bit. However, in lieu of that, here are a couple of relevant references with sufficiently scientific benchmarks:
https://github.com/koajs/koa/blob/master/docs/koa-vs-express.md#is-koa-faster-than-express
http://serdardogruyol.com/?p=111
http://qzaidi.github.io/2013/07/20/surprises/
Although this doesn't answer the delta part of your question, there's some absolute statistics about ExpressJS performance here:
Updated Link
http://blog.perfectapi.com/2012/benchmarking-apis-using-perfectapi-vs-express.js-vs-restify.js/
Looks like, on a small Amazon instance, Express averages:
a very respectable 1,600-1,700 requests per second
Express is a convenience wrapper around node.js, and should not add much latency to your server; your code would probably go through many of the same steps anyway. That being said, if your load tests (or hapless customers) find the server is too slow and/or not scalable, then you should profile at that time, and the bottleneck probably won't be Express.
BTW, even though Express calls itself "high performance", they're naughty for not posting measurements.