I've a monorepo using nx with multiple node/nestjs apps. Some of the apps doesn't require all the packages used in the other apps. Because it's a monorepo, I need to install all packages for every apps during the deployment.
Is there a way generate a package.json on build that would contain only the packages needed for the app that I'm building?
I've tryed to use "generate-package-json-webpack-plugin" to generate the package.json, but it only detect half the dependencies.
I've also tried to build a single js file containing all the apps, but it doesn't seem to work and always require tslib.
After I look at the nx source code I found the answer.
Set generatePackageJson to true in workspace.json where <project-name>/targets/build/options.
This will generate you package.json with the necessary dependencies for your app.
Here example:
"node-api": {
"root": "apps/node-api",
"sourceRoot": "apps/node-api/src",
"projectType": "application",
"prefix": "node-api",
"targets": {
"build": {
"executor": "#nrwl/node:build",
"outputs": ["{options.outputPath}"],
"options": {
"showCircularDependencies": false,
"outputPath": "dist/apps/node-api",
"main": "apps/node-api/src/main.ts",
"tsConfig": "apps/node-api/tsconfig.app.json",
"assets": ["apps/node-api/src/assets"],
"generatePackageJson": true <----------------------
},
....
Nx encourages a single-version policy and has a single package.json.
If the issue is that you are installing all the dependencies every time in CI before building then you might need to rely on the functionality provided by your CI system to cache these between runs - a lot of existing CI systems do provide these:
* Gitlab: https://docs.gitlab.com/ee/ci/caching/
* CircleCI: https://circleci.com/docs/2.0/caching/
* Travis: https://docs.travis-ci.com/user/caching/
However this comes with its own set of issues (e.g. parallel jobs where one or more is changing the dependencies).
We can try to explore having a command in Nx: sort of a "affected:dep-install" that will detect which packages to install as part of the affected command. Please create an issue for it here: https://github.com/nrwl/nx/issues
Related
So I want to run an e2e test using jest. I'm using nx monorepo architecture, and I have all my assets in a library folder and also nestjs microservices for my backend. I have all my proto files for my microservices in the library, and when I want to load them in my microservices, I do it like this :
protoPath: join(__dirname, 'assets-shared/job.proto'),
and in my workspace.json in my build i change the assets-shared like this:
"targets": {
"build": {
"options": {
"assets": [
{
"input": "libs/backend/shared/src/lib/assets",
"glob": "**/*",
"output": "assets-shared"
}
]
},
all is good, but when I run the test and when it wants to import and give value to it, it doesn't change it, and I have this error which is trying in its folder and not the library folder
ENOENT: no such file or directory, open '/home/dev/Project/apps/backend/api/src/modules/product/assets-shared/job.proto'
I tried the moduleNameMapper to give the libs folder to it manually but no avail.
moduleNameMapper: {
'^.+\\.(proto)$':
'<rootDir>/libs/backend/shared/src/lib/assets/$1',
// '^assets-shared(.*)': '/libs/backend/shared/src/lib/assets/$1',
},
non of these two worked
Have you considered publishing the libraries to a private npm repository or something like artifactory e.g. #my-company/assets
The approach you are trying may work locally, but for a ci/cd pipeline it would be much better to have a versioned artifact in npm or artifactory
I'm using PM2+ to manage my NodeJS deployments.
Usually, when I deploy an application with pm2 start src/app.js, I get details about versioning like in the screenshot below. However, when I deploy using an ecosystem file I only get N/A:
PM2 normally extracts this information directly using vizion.
But since it didn't work with the ecosystem file, I specified the GitHub repository directly just like the documentation stated.
This is my current pm2-services.json ecosystem-file:
{
"apps": [
{
"name": "my-node-app",
"cwd": "./my-node-app-repo-folder",
"script": "src/app.js",
"env": {
"NODE_ENV": "production"
},
"repo": "https://github.com/MyUserName/MyNodeAppRepo.git",
"ref": "origin/master"
}
]
}
For the ref field, I also tried putting refs/remotes/origin/master, remotes/origin/master and master.
Sadly none of them worked (I made sure they are correct using git show-ref).
Additional info:
NodeJS Version: v15.11.0
NPM Version: 7.6.3
PM2 Version: 4.5.6 (latest, by the time of writing this)
So, how do I get the Versioning field to display correctly?
Note: This isn't really an issue but rather a minor inconvenience. I just want to know what I'm doing wrong.
TL;DR How can I create an alias for a local yarn workspace dependency?
I've tried yarn workspaces before and never succeeded, and I'm giving it another try.
I've set "workspaces": ["packages/*"] in package.json.
For each package, I decided to use the naming convention #-/package-name to prevent naming conflicts and without worrying about having namespaces for internal packages.
When adding packages as dependencies, I've been following a style where I use an interface name for resolution, but point that towards a concrete implementation. This is what I did before using yarn workspaces:
"dependencies": {
"my-interface-name": "file:some/path/to/packages/some-concrete-implementation"
}
This is basically to allow what I like to call compile-time static dependency injection. And it also means each package can individually name their interface dependencies appropriately to their need and to prevent naming conflicts.
However, I can't figure out how to accomplish this with yarn workspaces. How do I create an alias for my yarn workspaces package #-/some-concrete-implementation called my-interface-name?
What I've already tried with no success:
Defining the dependency like "my-interface-name": "#-/some-concrete-implementation"} - for some reason this causes yarn to look for #-/some-concrete-implementation on the npm registry instead of in the local workspace
I've also tried to use the workspace protocol: "my-interface-name": "workspace:#-/some-concrete-implementation"} but it still looks for the package on the npm registry!
What I haven't yet tried and could work but removes benefits of using yarn workspaces in the first place:
"dependencies": {"my-interface-name": "file:../../node_modules/#-/some-concrete-implementation"}"
Have you seen the resolutions package.json key? Is it what you need?
I've used it for aliasing/overriding external packages but the example in the docs shows it working with local packages.
Resolutions
Allows you to override a version of a particular nested dependency. See the Selective Versions Resolutions RFC for the full spec.
{
"resolutions": {
"transitive-package-1": "0.0.29",
"transitive-package-2": "file:./local-forks/transitive-package-2",
"dependencies-package-1/transitive-package-3": "^2.1.1"
}
}
From the RFC:
"**/a" denotes all the nested dependencies a of the project.
"a" is an alias for **/a (for retro-compatibility, see below, and because if it wasn't such an alias, it wouldn't mean anything as it would represent one of the non-nested project dependencies, which can't be overridden as explained below).
So, I believe the rule you need is:
"**/my-interface-name": "file:some/path/to/packages/some-concrete-implementation"
// OR equivalent
"my-interface-name": "file:some/path/to/packages/some-concrete-implementation"
I believe it works in the package's package.json. Worst case you can hoist it to the workspace root and make the rule specific to the workspace e.g. "a/b".
The workspace: alias protocol (available in pnpm too) seems the direction to take.
I've also tried to use the workspace protocol: "my-interface-name": "workspace:#-/some-concrete-implementation"} but it still looks for the package on the npm registry!
Be sure to have yarn 3 installed, otherwise you'll run into weird issues.
Note that the syntax of "my-interface-name": "workspace:#-/some-concrete-implementation" looks incorrect.
It should be "#xxx/some-concrete-implementation": "workspace:*", assuming the name of linked the package is "name": "#xxx/some-concrete-implementation".
With this in mind you don't even need to create a specific #-/name. With workspace protocol, yarn will ensure it's never downloaded from npm. It becomes an internal workspace dependency.
PS:
Yarn 3 installation
Generally a simple yarn set version 3.0.2 && yarn plugin import workspace-tools) will work.
To avoid pnp current limitation, check the generated config .yarnrc.yml and ensure nmLinker is set to 'node-modules'
# Yarn 2+ supports pnp or regular node_modules installs. Use node-modules one.
nodeLinker: node-modules
plugins:
- path: .yarn/plugins/#yarnpkg/plugin-workspace-tools.cjs
spec: "#yarnpkg/plugin-workspace-tools"
yarnPath: .yarn/releases/yarn-3.0.2.cjs
PS: you might want to add this to .gitignore too
.yarn/*
!.yarn/patches
!.yarn/releases
!.yarn/plugins
!.yarn/sdks
!.yarn/versions
.pnp.*
Run a yarn install just after.
About package.json's
Like you did, the root package.json will define the workspace paths:
{
"name": "monorepo",
"workspaces": [
"packages/*" // Enable package discovery in packages/* directory.
],
// ...
"devDependencies": {
"husky": "7.0.2", // Only what's needed for monorepo management
}
In your app packages/app/package.json
{
"name": "my-app",
"devDependencies": {
"#types/node": "16.10.1",
//...
},
"dependencies": {
// Assuming the name of packages/shared is "#your-org/some-concrete-implementation",
// we explicitly declare the dependency on it through
// workspace: alias (package-manager perspective)
"#your-org/some-concrete-implementation": "workspace:*",
}
}
You consumed package should declare the same name
{
"name": "#your-org/some-concrete-implementation",
}
Bonus: Typescript aliases
If your project is written in ts, you can even replicate your paths through
typescript path mapping. It will allow to include the files just as is (no prior compilation needed).
Following your example, just edit a ./packages/xxx/tsconfig.json in this way
{
"compilerOptions": {
// here baseUrl is set at ./src (good practice), can
// be set to '.'
"baseUrl": "./src",
"paths": {
// Declare deps here (keep them in sync with what
// you defined in the package.json)
// PS: path are relative to baseUrl
"#your-org/some-concrete-implementation/*": ["../../some-concrete-implementation/src/*"],
// if you have a barrel in ui-lib
"#your-org/some-concrete-implementation": ["../../some-concrete-implementation/src/index"],
}
},
}
PS: for non typescript: babel/plugin-module-resolver can be used in a similar manner.
I have a (demo) application hosted on Heroku. I've enabled Heroku's "review app" feature to spin up new instances for pull request reviews. These review instances all get a new MongoDB (on mLab) provisioned for them through Heroku's add-on system. This works great.
In my repository, I've defined some seeder scripts to quickly get a test database up and running. Running yarn seed (or npm run seed) will fill the database with test data. This works great during development, and it would be perfect for review apps as well. I want to execute the seeder command in the postdeploy hook of the Heroku review app, which can be done by specifying it under the environment.review section of the app.json file. Like so:
{
"name": "...",
"addons": [
"mongolab:sandbox"
],
"environments": {
"review": {
"addons": [
"mongolab"
],
"scripts": {
"postdeploy": "npm run seed"
}
}
}
}
The problem is, the seeder script relies on some development-only dependencies (faker, ts-node [this is a TypeScript project], and mongo-seeding) to execute. And these dependencies are not available in the postdeploy phase of an Heroku app.
I also don't think that "compiling" the typescript in the regular build step is the best idea. This seeder script is only used in development (and review apps). Besides, I'm not sure that would resolve the issue with missing dependencies like faker.
How would one go about this? Any tricks I'm missing?
Can I maybe skip Heroku's step where it actively deletes development dependencies? But only for review apps? Or even better, can I "exclude" just the couple of dependencies I need, and only for review apps?
The Heroku docs indicate that when the NODE_ENV variable contains anything but "production", the devDependencies will not be removed after the build step.
To make sure this only happens for Heroku review apps, you can set the NODE_ENV variable under the environments.review section of the app.json file. The following config should do the trick:
{
"name": "...",
"addons": [
"mongolab"
],
"environments": {
"review": {
"addons": [
"mongolab:sandbox"
],
"env": {
"NODE_ENV": "development"
},
"scripts": {
"postdeploy": "npm run seed"
}
}
}
}
As of 1/5/20 all of my builds are failing that are deployed on Now. I was getting the warning that node 8.x was no longer supported so I specified an engine version of 12.x in my package.json, this successfully stopped the warning from popping during deployment but I'm still receiving the following error
Error: No output directory named "build" found.
I thought maybe a recent change I pushed was causing this but going back and redeploying old deployments that built successfully now received this error. I'm wondering if something on the Now platform changed that I wasn't aware of because it does not seem that code I've pushed recently was the catalyst for this error.
It's also very odd since right before this error the deployment log shows the following
Creating an optimized production build...
Compiled successfully.
File sizes after gzip:
207.04 KB build/static/js/2.7d84160a.chunk.js
11.64 KB build/static/js/main.65999b58.chunk.js
1.24 KB build/static/css/main.cacda93c.chunk.css
762 B build/static/js/runtime~main.a8a9905a.js
The project was built assuming it is hosted at the server root.
You can control this with the homepage field in your package.json.
For example, add this to build it for GitHub Pages:
"homepage" : "http://myname.github.io/myapp",
The build folder is ready to be deployed.
So it looks like the build folder is created but now for some reason now can't find it.
My now.json looks like this
{
"version": 2,
"name": "appname",
"public": false,
"builds": [
{ "src": "package.json", "use": "#now/static-build" },
{ "src": "index.js", "use": "#now/node-server" }
]
}
Any idea why my deployments would suddenly start failing in the last day?
Classic case of finding the solution as soon as I post on stackoverflow...
Not sure which tutorial I followed when first spinning this up but in my package.json I had my now-build script set to
react-scripts build && mv build dist
I removed the last bit of that so now my now-build script is simply
react-scripts build
and all is well again.
I couldn't really tell you why I set this, I just blindly followed the tutorial. For the past 4 months, this has worked, not sure why yesterday it started throwing errors, oh well.