Full stack asset pipeline for node, connect/express and broccoli - node.js

Firstly I must confess I am a noob at node. I've been using ASP.NET then PHP then Django before. Regardless, I've found node a breath of fresh air. This problem is also not strictly a node problem, but I need a node specific answer.
I have an express server and angular frontend. The server side templates are in swig and currently only serve for error pages and the index page. Mostly the angular templates will make up most of the front matter.
What I'm struggling with, if only only in deciding how to do it, is getting an efficient work flow for the asset pipeline. Server side templates must be able to inject the vanilla of assets during dev and testing. The same for client side templates during testing. Basically, running with express' static middleware should be an option without any configuration (maybe with some helper in server side assets). Thus git clone -> grunt -> viola.
However, during staging and production, I would like the server side files to stay vanilla. The template helpers may parse a manifest file indicating the cache busted links (CDN path maybe too). How to make the link from logical asset bundle name to production ready asset is a mystery for me, while keeping development transparent.
The client side templates may be minified, concatenated, injected or whatever, as it will be saved to some dist folder for uploading. It is important that the whole dependency tree (images, fonts, css, js) must be "exported" to the dist folder.
To deploy would then be: pushing the server side code to the server and running. And pushing client dist folder to some asset host (CDN, nginx, another node, maybe even connect static)
What my question(s) then actually is(are):
Is this workflow possible with tools such as broccoli/gulp/grunt alone?
I've tried connect-assets but I don't want to conform to some predetermined folder path. Also the cli tool didn't produce the other static assets. Perhaps I don't understand the tool.
Am I following the correct approach?
I've added to a discussion on broccoli concerning the manifest file consumption:
https://github.com/mjackson/broccoli-rev/issues/1#issuecomment-49076249
Edit: I forgot to mention that I use bower, so assets should be pulled from arbitrary (URLs too maybe) locations.

I think angular-fullstack is what you want. Even if you don't use it, it does almost all of what you're looking for.
The only thing that it might be missing for you is deployment. It has built in support for Heroku and OpenShift deployment. You could use something like grunt-ssh or grunt-deploy for other deployment scenarios.

Related

How to package server side JavaScript for Node

I have been using webpack to build a server side app using express. This code is harder to debug since I don't have the immediate comfort of a web browser and if I use something like VS Code to debug, it won't accept breakpoints inside request handlers when using source maps. Besides it takes no time to compile if I just stick to Node compatible JS and skip all transpiling and whatnot. Further, if I use treeshaking, I can reduce the size, but what is the point of that when it is running on the server (no client will ever download it).
My point is that I don't see why one would want to create a bundle of server side code if the server don't have any issues with memory or other limiting factors. It's easier to read, takes no time to compile and is easier to debug.
So a question. Is it ok to have the server app as an npm package and deploy that? Is that what is common or what do people do?
The project my team works on is a TypeScript monorepo that includes a web application and server-side daemon processes. We use webpack for producing build artefacts for all targets.
I've read some posts saying there's no point using webpack server-side because it was designed for web (obv), and I can understand that point of view.
However, even if our code was JavaScript and didn't require any transpiling, we'd still use webpack for the server-side. I want a single, minimal set of files and no node modules to deploy to the servers.
Size itself doesn't matter that much for server side (though every bit helps), but running 'yarn install' on servers is out of the question IMO.

Basic Question First Node+express+angular app in production

I am creating my first app node express app with angular 7 on the frontend to be deployed in production. I have below question?
What folder structure is preferred, should I create separate
projects for node and angular or same project(server.js in the root
of angular project and server folder to create express server
files)? What is the preferred one and I have to checkin the project
in one folder of svn.
Should I use babel and create the node server code with es2015 or
continue with old approach?
Its all up to you, what I am doing is I have sepreate directory for Angular and Node projet
project
|
client - Your anguar project
server - Your Apis and server side coding (Only this folder require at productino level)
Then we can create a gulp file and task to gulp that Build my client
project and put that build folder inside the
server -> public
Now only server can be use to production where Build will be render as static.
And next to authentication and autherization process you can follow JWT based permission .
Generally I would say that separating your client and server code into separate projects is preferred so that you do not have to release both your client and server at the same time when you make a change to one or the other. The rest of my answer is based on the assumption that you would separate the two sides into different projects.
As far as structuring your server side Express-based application, check out this link for some guidance on how to handle your situation. See the answer to the first question about different approaches to how to structure your Express application for different deployment scenarios. Also, if you use the latest LTS version of node, you will not need to use a transpiler to convert your files to Javascript because the Node environment will handle that for you.
As far as structuring your client side Angular-based application, check out this link for a very detailed discussion about best practices for structuring your Angular application.
I would prefer following, in case in future you need to separate the API layer with client you can do it with ease,
project
|----client
| ---client-template //All UI code like .css/htmls and node process initiates from here
| ---client-angular // All the directives and controllers goes here
| ---client-service //Service layer, All the API call to server goes here
|----server
| ---server API's // separated by its own module if any
|--- you API modules and so on..
This will help you to have flexibility over client and server integration without any tight coupling. Also easy to maintain and debug.
Answer 1: you should make two separate folder/repository structure for frontend and backend.
let's suppose your application grows fast at that time you want to scale your backend and you want to host your Angular app as static web app using Amazon-S3 so at that time it will be very easy to manage this.
May you want to use CICD, in that case also it will be good if your separate folder so you can create separate CICD jobs for backend and frontend.
May be your company hired some developer which is either expert in frontend or in backend only. in that case your company don't want give them unnecessary code access. so separate repo will be an easy option for this case. (this may be Depends on your team and company's approach for development)
Answer 2: I recommend go for es6 or es6+ features.
latest node.js version is supporting some of the features of es6. for example
- spread operator
- destructing
- classes (you can use OOPs)
- arrow functions
- let, const
- async await and etc
you can use babel if any other feature which is not supported by node.js. there could be may reason for using babel, but i want to know which specific feature do you want to use with babel? so i can explain according to that.
I have used the following approach that bind the Angular Application and the Node server as a single unit.
Steps for creating the project structure is:
Create a new Angular project with the CLI.
Create a server.js file in the root directory of the project and configure it to render the contents of the dist/ folder on the / route.
You can refer the link for the server code: https://github.com/nikhilbaby/node-server
Running the server
I usually run the project with ng build && node server. This will make sure that the angular application is build first and after that node server is started.

What's the best way to lay out a React/webpack project with an API backend?

I'm a little late to the party, but I'm just getting started with React. I have a base project that I'm serving with webpack-dev-server, but now I need to actually get data for it.
My chosen API is Strava. I'm trying to use the recommended API wrapper for node, but I've come to a point where I don't want to expose my API keys and know that I need to create my own API endpoints. I was about to start creating an Express server to quickly create these endpoints, but I feel like there might be something that I'm missing.
Is there a better way to go about this with webpack/webpack-dev-server?
If I want this working on the internet, I know that I'll need at least one node instance running to serve my API endpoints; I just want to make sure there's not some super-simple, possibly webpack-based (to keep everything in the same family) solution that I could then, later, just deploy on a ready-made node server.
My assumption is that I'll just be better off creating two projects:
Static content (currently webpacked and served with webpack-dev-server for development)
Express server for API-wrapping.
If that's the case, is there a simple way to point hook Express up with my static project (everything webpack does is sent out to an /app/dist directory) for production deployment?

Hexo (and other static-site-generators) front-end dependency-management workflow

Are there any recommended workflows for managing front-end dependencies? I've been reading a lot of articles that recommend moving away from Bower, and on to an npm-only solution like Webpack, but Webpack is a whole new paradigm (loading js, scss, fonts, etc through a single js file) that by default, requires js to be running in the browser for css to load. Part of the reason I want a static site is so that js isn't mandatory for an end-user. However, I'm really tired of bower-installing things, and then having to either host everything in bower_components, targeting specific filenames (js, css, img) to include in output, or move their css/img dependencies into my own repo. Not to mention that relying on two registries is less than ideal.
Does Hexo have a recommended way, or does anyone have an opinion on how to do this? Running a Hexo server in a separate terminal from a webpack-dev-server seems painful and awkward, and possibly create some confusion as to which library should be handling which files.
Are other tools more suited for dependency management in a static site generator's dev/build process?

Node.js on Heroku: use middleware on development, but static assets on production?

Some middle languages, such as Stylus, provides two ways to be compiled: through connect middleware or through CLI tool. The later can generate static compiled assets(i.e. .css files).
So I want to use middleware on development mode but static assets on production. I know that I can use app.configure('developmen'...) to ask express (not) to use some middlewares on development mode.
On an IaaS enviroment, like Amazon EC2, I can run a simple shell script to automatically re-compile all my assets. But how about PaaS, specifically Heroku? How can I tell it where my .styl are and where the .css should be generated?
You may want to take a look at https://github.com/adunkman/connect-assets . It caches any built javascript or css files (it has stylus built-in support for stylus) if you pass it build:true .
You can ignore snockets (sprockets-like javascript include system) if you're not interested, although I enjoy using it. #= require_tree app and you include all the js files in that directory. And in development, you get separate script includes for easy debugging.
The biggest downside of serving directly with connect-assets on Heroku is that you need to git push to Heroku for every update to client code, which automatically triggers a restart. I ended up manually building my assets (with stylus and snockets), then uploading to S3. If you don't need to update the client code often, it's not that big of a problem though.
You can take a look at express-cdn, which will upload your assets to S3 on server start.
What I ended up doing was signing up at CloudFlare, and found that it wasn't as fast as using CloudFront, but it was very easy to setup and it works better than serving asset files from my dyno.

Resources