We are trying to implement a grpc service in Node.
We have a common.proto-file where we describe common messages, that we can reuse across different services.
Up until now, we have only implemented services in Go, and consumed them in either Go or PHP. This all works fine.
Now that we want to implement one service in Node, we have generated the pb.js-files, both from common.proto, and lets call it service.proto.
The problem is, that in service_pb.js it generates the following code: require("../common_pb.js")
This is of course not valid, as the path to common_pb.js is node_modules/#company/common-node/common_pb.js, while now it looks for it in node_modules/#company/common-node/common_pb.js.
I still have not been able to figure out how we can make this work correctly for Node, so if anyone has a solution that would be great.
Here is what we came up with as a solution:
The tree of the project looks like this:
root/
├─ common/
│ ├─ common.proto
├─ some-service/
│ ├─ some-service.proto
├─ other-service/
│ ├─ other-service.proto
If we in some-service.proto needed a message definition from common.proto we had import "common.proto"; and when running the project we included the common dir.
We changed this, so that we in the some-service.proto file wrote common/common.proto, and when running protoc just included . which is the root dir as shown in the tree above. That way the includes came out as require('../common/common_pb.js'); instead of require('../common_pb.js');.
Related
I have an application that looks like this one:
https://github.com/heroku-examples/node-workers-example
In short, I have 2 processes:
Server: it server US, handles requests and adds them to Redis
Worker: it pulls requests from Redis and works on them
Should I use only one Docker image for both processes or should I have 2 docker images (one for the server and the second for the worker)? What is the best practice?
I personally think, it's better to have 2 images. In this case, can my project structure be like this one:
Project Folder
-node_modules
-utils
-server.js
-package.json
-Dockerfile
-docker-compose.yml
-/worker
-/worker/index.js
-/worker/Dockerfile
Is there any advice?
Thanks a lot.
Disclaimer: this is very opinionated response, but author is asking for an opinion.
You can do one or two but it all depends on how you wanna schedule it.
If you want to stay flexible in the amount of processes you want for each I would go with two docker images otherwise you'll need to each time spin a fixed amount of each or you'll need to tweak that setting via env variables or via other means...
Hence one for the frontend part and one for the background process.
As you are having two different images, I usually prefer to separate that in two distincts projects but that's a matter of taste. Even though because of how NodeJS manages dependencies (node_modules) it's easier to have 2 distincts folders when the dependencies are very different.
I would go with following:
.
├── docker-compose.yml
├── front
│ ├── Dockerfile
│ ├── node_modules
│ ├── package.json
│ └── src
│ └── main.js
└── worker
├── Dockerfile
├── node_modules
├── package.json
└── src
└── main.js
I'm having trouble adding an image url to a variable in my LESS file.
I am using a project called Guide4You where I would like to add a new LESS file with my own images. This project uses node with webpack.
For my project I use this folder structure:
├── root
│ ├── load_image.png
│ ├── styles
│ │ └── substyles
│ │ ├── load.less
In the less file I have the following code:
#test: url("../../load_image.png");
Whenever I try to compile the code into webpack I get the following error:
[ './root/load_image.png
Module parse failed:
C:\projects\root\\load_image.png
Unexpected character \'�\' (1:0)\nYou may need an appropriate loader to handle this file type.
Is it possible that the less-loader sees my url as a reference to another loader and tries to execute it?
You need an appropriate loader that would match that png of yours. To solve this problem, use either url-loader or file-loader so that it matches your png.
Documentation: https://github.com/webpack-contrib/url-loader, https://github.com/webpack-contrib/file-loader
First of all, I have it wired and working, but I am somewhat discontent with the result and have a feeling it can be improved.
(The current result can be found here - https://github.com/MarkKharitonov/Angular2WebpackNodeExpress/tree/v0.0.1.)
The directory structure is:
C:.
│ .gitignore
│ package.json
│ tsconfig.json
│ tslint.json
│ typings.json
│ webpack.config.js
│
├───dist
│ └───server
│ api.js
│ api.js.map
│ main.js
│ main.js.map
│
└───src
├───client
│ app.component.ts
│ index.html
│ main.ts
│ polyfills.ts
│ tsconfig.json
│ vendor.ts
│
└───server
api.ts
main.ts
tsconfig.json
Right now the dist folder has only the server side files compiled from ./src/server. They are placed there by the IntelliJ IDEA, because ./src/server/tsconfig.json requests compilation on save.
The client side bundling occurs in memory courtesy of webpack-dev-server. The ./src/client/tsconfig.json does not request compilation on save.
The things I dislike about my current setup are described here - https://github.com/MarkKharitonov/Angular2WebpackNodeExpress/tree/v0.0.1#problems, namely:
webpack is going to take care of any plain .js files under ./src/client - they would be bundled and placed under ./dist/client automatically. But what about plain .js files under ./src/server ? Do I need a task runner for that (gulp, grunt, whatever ...) or is there a solution within webpack?
I have three tsconfig.json files - ./src/client/tsconfig.json, ./src/server/tsconfig.json and ./tsconfig.json. The three files share most of the options, but not all. Right now I copy them in each of the three files - not very good.
In addition, because the typings folder is at the root I have to start all the top level TypeScript files (.\src\client\main.ts, .\src\client\polyfills.ts, .\src\client\vendor.ts and .\src\server\main.ts) with /// <reference path="../../typings/index.d.ts" />.
Hence the questions:
Can webpack also handle the server side files, but differently from the client side ones? I.e. transpile - yes, copy to dist - yes, bundle - no? Please, bear in mind I am using webpack-dev-server.
Is it possible to inherit tsconfig.json configuration so that I could avoid duplicating many options across the three files I have?
Is it possible to avoid the inclusion of /// <reference path="../../typings/index.d.ts" /> in the top level TypeScript files when the file layout is similar to mine ?
I know these are three questions instead of one, but I feel they are all closely related and the answer to one could also be the answer to another.
I don't think you need/want webpack to handle server-side files. Transpiling and copying over of the server side files to /dist is handled by Typescript compiler (via outDir config), already. There will be no bundling of server-side files since no server files were indicated as entry points in the webpack config.
Is not currently possible. However, looks like there is an issue to track this: https://github.com/Microsoft/TypeScript/issues/9876
Unsure, related to #3 in a way (but not really). I'd imagine no as long as you want to keep the client and server files truly seperate.
I'm newbie developing MEAN apps, i'm coming from a Laravel structure to develop web applications, what i don't like about Laravel is that is not so modular, everything is separated, especially Models, Views, and Controllers, to navigate through the folders you have to scroll a lot of times when building large apps...
My question is all about Express under the MEAN stack, what i'm looking for is a structure where i create modules for every entity of an app, for example:
I have three modules: users, questions, and answers; each module will contain the routes, the model, and the controller for that specific module, for example:
├── node_modules
├── src
│ ├── client
│ │ └── ... //Frontend things managed by angular (like views, etc...)
│ └── server
│ ├── modules //By module i mean an entity
| | ├── users
| | | ├── users.model.js
| | | ├── users.controller.js
| | | ├── users.routes.js
| | | └── index.js
| | ├── questions
| | └── answers
| ├── config
| └── etc... //Suggestions please...
|
├── package.json
└── server.js
How could i set up that structure?, so far i've found this tutorial about modularizing an Express app, but i would like to extend it, so that i keep it DRY, and make use of the LIFT principle described by John Papa in his Angular Style Guide but under the backend.
Why am i asking this? Simple, i don't like to scroll through large folders to get open a file and then scroll again to open another file that are related, i want to work with a structure easy to maintain, easy to understand, etc...
If possible could someone explain an example of how to setup an application using this structure and upload it to github?
Place one node_modules at the top level.
Give each module a package.json
Now write a script in your bootstrap that finds all unique node_modules required from the package.json.
There, node_modules solved. Make sure it identifies the variances of versions too.
Build a universal run script file that reads a universal configuration file that points to all of the other modules.
Angular 2..4..5 whatever is a great modularized file structure to model after.
I found that Express has an application generator, however the documentation does not explain the purpose of each directory and file. If someone could just give me a short explanation of which files I should be putting where, that would be much appreciated. Here's the generated app structure:
├── app.js
├── bin
│ └── www
├── package.json
├── public
│ ├── images
│ ├── javascripts
│ └── stylesheets
│ └── style.css
├── routes
│ ├── index.js
│ └── users.js
└── views
├── error.jade
├── index.jade
└── layout.jade
7 directories, 9 files
The app.js file is the entry-point of your application.
The package.json file contains all of your dependencies and various details regarding your project.
The bin folder should contain the various configuration startup scripts for your application.
For example, instead of applying all the Express middleware in the app.js file, you module.exports = {} them from their own configuration file and require them in app.js. [additional info LINK]
The views folder contains all of your server-side views.
The public folder contains all of your front-end code.
The routes folder contains all the routes that you have created for your application.
As stated in the official documentation, be aware that this is just one way to organize your code.
You should test it out and see if it fits your project.
This thread gives a deeper answer about the www file specifically: What does "./bin/www" do in Express 4.x?
Basically, running your app from the www file (which calls app.js) allows you to start your app with different configurations. You might have a "www" file representing the way the app should be run when on the web, a "dev" file that you, as the developer, would run, a "test" file you would run when running tests, etc. Read the thread linked above for more detail!
This structure is a standard organization for web-app
public contains all client static files (css, client javascript (ex. jQuery), images, fonts...)
routes contains the main back-end code (server side), which compute data before calling a template engine (see below) or respond to the client (via json of xml).
views contains each page template, see jade template. These files are used by scripts in "route"
app.js contains the express core, such as uri parser, modules, database...
package.json is the project descriptor file (used by npm for dependencies and sharing)
If the application generator provide a full example, don't hesitate to open each file (starting from app.js) to understand the project's organization.