I have a node.js app using a single git repo, that has various packages. I am trying to make things modularized, and separate.
├── common
│ └── formattr
│ ├── node_modules
│ ├── package.json
│ └── formattr.js
└── providers
├── aws
│ ├── node_modules
│ └── package.json
└── google
├── node_modules
└── package.json
The problem I am running into is that when somebody clones the app, they have to recurse into each package direction and manually run npm install. Additionally, there is lots of wasted resources and duplication, since the package aws might use npm install request but so does the google package. Right now, they both store independ versions.
Finally, there is this problem as well:
// Inside of the aws package, I need formattr. This makes me cry inside.
let format = require('../../common/formattr/formattr');
What is the recommended pattern and structure?
Related
The problem is that let's say I want to start multiple services (several npm start) concurrently, it would be inconvenient to run the services as separate projects. I want to have a folder structure similar to the following under one project workspace:
project
├── service1
│ ├── node_modules
│ │ ├── #module1
│ │ └── #module2
│ ├── package-lock.json
│ ├── package.json
│ ├── public
│ └── src
├── service2
│ ├── node_modules
│ │ ├── #module1
│ │ └── #module2
│ ├── package-lock.json
│ ├── package.json
│ ├── public
│ └── src
└── service3
├── node_modules
│ ├── #module1
│ └── #module2
├── package-lock.json
├── package.json
├── public
└── src
What would be a clean way to do so? I need to start multiple services together and obviously debug any in a convenient fashion if a possibility exists.
You may want to look into Lerna or Nx.
Both are tools that manage mono-repo microservices.
There are some subtle differences between them, but essentially both do the same thing.
They offer ways to share dependencies between your microservices.
They offer ways to created shared libraries.
They offer ways to launch
multiple services together.
Lerna
One of the subtle differences, is that Nx will force you to use a single package.json in your root folder, essentially forcing you to use the same dependencies for all microservices. By contrast, Lerna still allows a specific package.json in each individual folder, which seems to resemble your current directory structure better.
In general, I think Lerna is a safe choice. And you can find a good tutorial here.
Nx
On the other hand, even though Lerna has been around for a longer time it has some quirks at times. I believe Nx is probably technically a more robust solution.
However, I must admit that I've mostly seen it being used for mono-repo front-end projects, and less often for back-ends. Technically, it should be able to handle both.
To get you started with Nx, you could follow this tutorial.
Spoiler: Nx has commands like nx run-many that can help you to execute multiple services together. After migrating to nx, you could then put that command in your "start": script of the package.json, so that npm run start and npm start will execute it.
I currently have a Node.js project with this file structure:
strava-descriptions/
├── .DS_Store
├── .env
├── .eslintrc.js
├── .git
├── .gitignore
├── authorizeStrava.js
├── buildDescription.js
├── config.js
├── db.js
├── getActivity.js
├── getSongsPlayedDuringActivity.js
├── getSpotifyAccessToken.js
├── getSpotifyRecentlyPlayed.js
├── getStravaAccessToken.js
├── getWeatherConditions.js
├── handleWebhookEvent.js
├── node_modules
├── package-lock.json
├── package.json
├── server.js
└── updateDescription.js
The parent folder strava-descriptions is its own GitHub repository and is also deployed to Heroku.
What I want to have happen is to put all of these files into a sub-directory called backend, and create a new frontend folder in strava-descriptions to house a React front end. However, I know that using create-react-app will create its own git repo inside the frontend folder.
My question is, how do I accomplish this without totally messing up my Heroku deployment and current GitHub repo? Ideally I would like to have one complete project on GitHub (frontend and backend together), but have them deployed and functioning as two separate apps.
One approach is to use create-react-app to initialize you project and then just paste the files on your old repository, reorganize the files and then you can manage you builds for react with npm
I am currently trying to create a docker container for a node.js project that contains a local dependency. This seems to cause an issue with docker so as a workaround I am trying to just copy the local dependency folders and just ignore their dependency entries in the package.json file. Is there a way to specify dependencies I would like to ignore and have npm install run and skip those enties?
That can be done using devDependencies
The npm modules which you require only to develop, e.g.: unit tests, Coffeescript to Javascript transpilation, minification etc,make the required module a devDependency.
To skip Installation of devDepenencies pass --production flag to npm install,with the --production flag(or NODE_ENV environment variable set to production) npm will not install modules listed in devDependencies."
npm install --production
To make any module to be part of devDependencies pass --dev while installing.
npm install packagename --save-dev
It is a common issue, not only with Docker, but also with some cloud deployments. For example deployment to CloudFoundry using standard Node.js buildpack will cause npm install/yarn to run anyway. So, you'll also need to apply some tricks to work with local modules
If you don't mind to switch from NPM to Yarn for dependency management, you can use workspaces feature.
My package.json looks like this:
{
...
"dependencies": {
"some-module-i-want-to-install": "1.0.0",
"another-module-i-want-to-install": "1.0.0",
"#my/local-dependency-one": "1.0.0",
"#my/local-dependency-two": "1.0.0"
},
"workspaces": ["packages/*"]
}
And my project source layout has the following structure:
.
├── index.js
├── package.json
├── packages
│ ├── local-dependency-one
│ │ ├── index.js
│ │ └── package.json
│ └── local-dependency-two
│ ├── index.js
│ └── package.json
└── yarn.lock
After running yarn, modules I want to install are fetched from NPM registry, and local dependencies are installed from packages directory to node_modules.
.
├── index.js
├── node_modules
│ ├── #my
│ │ ├── local-dependency-one
│ │ │ └── ...
│ │ └── local-dependency-two
│ │ └── ...
│ ├── another-module-i-want-to-install
│ │ └── ...
│ └── some-module-i-want-to-install
│ └── ...
├── package.json
├── packages
│ ├── local-dependency-one
│ │ └── ...
│ └── local-dependency-two
│ └── ...
└── yarn.lock
As you can see, I prefer to define my local packages as scoped (#my/...). It is not mandatory, but a best practice. NPM treats scoped packages as private by default, so I don't need to worry that they will be occasionally published or explicitly mark them as private.
I'm interested in learning how the #shopify/polaris project in github is built and published to npm. My main questions are:
How are the index.es.js and index.js files being generated? Are those being generated programmatically on my computer, or are they published to npm like this?
What mechanism is preventing the files in the github repo from being downloaded when installed? I don't see a .npmignore in the repo.
Below I have the files in the npm package, and the github, and you can see they're different.
Here's what the polaris project looks like when it's installed via NPM / yarn.
.
├── CHANGELOG.md
├── README.md
├── index.es.js
├── index.js
├── package.json
├── src
├── styles
├── styles.css
├── styles.scss
└── types
Here's what the project looks like on github.
.
├── CHANGELOG.md
├── README.md
├── circle.yml
├── config
├── documentation
├── examples
├── package.json
├── scripts
├── src
├── tests
├── tsconfig.json
├── tslint.json
└── yarn.lock
We use Rollup to generate the different entry files for Polaris. You can see our generic config file here: https://github.com/Shopify/polaris/blob/master/config/rollup/index.js. Note that it does all the work of compiling our TypeScript source files (using TypeScript and Babel), as well as kicking off the work required to bundle our CSS and icons as appropriate. This config generator is then run three times, which you can see here: https://github.com/Shopify/polaris/blob/master/scripts/build.js#L36-L38
The main entry file (.js, and spits out the CSS for our CDN)
The module entry file (.es.js)
The embedded entry file
We then copy these into their final locations for the NPM publish.
As for how we control what files are in the NPM package, we use the files key in our package.json; these are the relevant lines.
I am working with node.js/npm.
My current configuration looks like this: npm list -g --depth=0
├── bower#1.7.7
├── cordova#6.0.0
├── grunt-cli#0.1.13
├── grunt-sass#1.1.0
├── ionic#1.7.14
├── ios-deploy#1.8.5
├── ios-sim#5.0.6
├── n#2.1.0
├── to#0.2.9
└── update#0.4.2
As I am newbie in developing mobile apps using Cordova, Ionic, etc. --> what do "to" and "update" stand for?
You're listing top level dependencies for the relative node.js project. to and update are node.js packages that are likely defined in your package.json file and exist in the top level directory of ./node_modules.
To better understand what those two specific packages are, read about them here:
The to package: https://www.npmjs.com/package/to
The update package: https://www.npmjs.com/package/update