I have a library that uses workspaces to house a bunch of plugin modules inside the library repo.
The dir structures looks something like:
package.json
src/app
src/plugins/package-a
src/plugins/package-a/package.json
The workspace modules are specified in the library’s package.json using local paths:
"workspaces": [
"src/plugins/package-a",
…
],
"dependencies": {
"package-a": "src/plugins/package-a"
…
}
It appears to work fine, when I npm install everything installs.
However since this is a library, the way it gets used is by being a dependency of other projects.
When I setup a separate project and add the library as a dependency, and npm install the separate project it throws an error. The error says the plugin doesn’t have a package.json, but it clearly does have a package.json.
Why can’t the project be used inside another project as a dependency?
Related
I can’t find the answer to this seemingly obvious question anywhere.
I have some npm workspaces setup in a project. It’s basically a main project with several workspaces within it. They are in a top level folder called “packages”, each in its own folder with its own package.json.
I need to add the workspaces as dependencies of the main project. I’ve added them to package.json of the main project but npm keeps trying to install them from npm.org, and so it fails.
I’m using this syntax:
“workspaces”:[
“packages/*”
],
“dependencies”: {
“workspace-a”: “^0.0.1”
…
}
How do I specify the workspaces as dependencies in package.json?
[Update: the eventual way the main project is used is that it is a dependency of a totally separate project]
I am using explicit workspaces attribute, in the root package.json
"workspaces": [
"foo-bar",
"another-package"
]
Dependency from another workspace package is referenced using file: prefix.
"name": "#project/foo-bar",
"dependencies": {
"#project/another-package": "file:another-package"
},
were #project is the package scope, name it according to your needs.
I am working on an npm package that includes an example directory to run/test the actual package. In the example directory, I have included back the parent package using "file:..".
This works fine when developing and making frequent changes to the parent package, but if I want to use the example as a stand-alone app, I would need to point to the actual npm package.
Is there a way to have "2 configs" in the same package.json:
one that points to `"file:.." for local development
one that points to the npm package to use as a stand-alone app
This would avoid duplicating the example directory
You could do this with lerna which is a mono-repository CLI tool.
First of all, you would have to define multiple projects within the same repository. Lerna calls these projects "packages" and stores all of them within a /packages folder.
package.json
/packages
/my1stPackage
package.json
/my2ndPackage
package.json
Lerna has all kind of optimizations, which I won't dive in too deep here. But there are a couple of basics:
to initially install all dependencies of all repos, run lerna bootstrap --hoist.
You can still run npm run ... as before, but those refer to your root package.json file. To run npm scripts for specific sub-package you should run lerna run <args> -scope=<packageName>. (e.g. lerna run build --scope=my1stPackage)
You can add shortcuts for this in the root /package.json script section.
"scripts": {
"setup": "lerna bootstrap --hoist",
"build:my1stPackage": "lerna run build --scope=my1stPackage"
}
What will interest you most, is that sibling packages can just reference each other from their specific package.json to include each other as dependencies.
So, let's assume that my1stPackage uses my2ndPackage. Inside the package.json file of my1stPackage there would be something like
"dependencies": {
...
"my2ndPackage": "^0.0.1"
}
The my2ndPackage could actually be a package which is published in npm. However (!) while developing locally, lerna will add a symbolic link inside the /packages/my1stPackage/node_modules/my2ndPackage, which points to the folder of /packages/my2ndPackage. (And that really does work on all relevant operating systems.)
You package.json looks the same for local development as it does for people who download your package through npm. However, it's your lerna setup that fixes this with this symbol link.
I found two potential ways to do this:
npm link : https://docs.npmjs.com/cli/v7/commands/npm-link/
npm workspaces : https://docs.npmjs.com/cli/v7/using-npm/workspaces
But in my specific case, there are packages that can conflict between the parent and child (example) packages.
I couldn't find a robust way to make it work and decided that the simpler approach would be to simply create a separate repository that would contain a stand-alone version of the example directory and a script that can keep it up to date with the "master example" in the original repository. This way development stays fast and the "example copy" is easy to keep up to date without duplicating code.
I have some local modules that I want to include in my package to be sent to a server. Since these local packages are not on the npm registry, they need to be part of the package.
I have tried some ways to get the node_modules folder included in the package using npm pack, however it seems this is not possible at all?
Secondly I tried to list the local modules in the bundledDepencies in the package.json file and use npm pack, however this also does not include the local modules, no matter what;
{
"name": "dev",
"version": "1.0.0",
"main": "main.js",
"dependencies": {
"local-module": "file:../../local-module"
},
"bundledDependencies": [
"local-module"
]
}
How can I get these local modules included in the dev package?
The local module does contain dependencies itself, not sure if that makes things more complicated?
I had a similar issue a while back, and a good and simple solution, is just to put your local modules into private git repos (which are now free on GitHub, thanks Microsoft 🙌)
Then, in your package.json, just add:
"dependencies" : {
"name1" : "git://github.com/user/project.git#commit-ish",
"name2" : "git://github.com/user/project.git#commit-ish"
}
Source, npm docs
I believe Alicia's approach is the cleanest one. However for someone that runs into the same issue as I did, whereby a server requires a tarball, but does not have git installed, I added my local packages to a seperate folder in the project called repo and referenced them in the package.json as;
"dependencies": {
"my-local-package": "file:./repo/my-local-package"
}
There's also yalc, which creates a local store for local packages. After adding a local package to the store, you can use yalc to copy the local package into another package. You can also use it to do updates of local packages.
We're starting to adopt a monorepo setup using yarn workspaces and we'd like to have our firebase functions inside it. The repo structure is something like:
repo
node_modules <- all dependencies
packages
core
commom
functions <- firebase functions
So, I have 2 problems with this setup:
The dependencies of the functions don't live on the same folder as the entry file from functions
The functions depends on other packages such as core and commom that are in the repo so yarn symlinks from node_modules to the packages in the repo.
Is there anyway I can handle this?
With Yarn 2 node_modules aren't fetched and placed into in the respective functions directory (as it would be the case with calling npm i in the functions directory). So when calling firebase deploy --project default --only function the node_modules folder is missing and firebase will complain about this and abort the deployment process with the following error (or similar):
Error parsing triggers: Cannot find module [...]
Try running "npm install" in your functions directory before deploying.
There are two github issues that are tracking this issue at the moment:
Support mono-repos in deployment
Functions deployment fails when firebase-functions has been hoisted by a monorepo manager like yarn/workspaces or lerna
In the two issues above, several clever workarounds are presented by firebase users, e.g. using webpack to create a build that contains all the local packages in the release or using rsync or other tools that rewire the packages before release.
Another solution is not hoisting your project packages, if that is possible. You can do this, be adding the following two directives to your .yarnrc.yml file.
# yarnrc.yml
# disables yarn's plugnplay style and uses node_modules instead
nodeLinker: node-modules
# makes sure the node_modules are not hoisted to the (monorepo) project root
nmHoistingLimits: "dependencies"
The two directives above are explained in the yarnrc configuration docs as follows:
nmHoistingLimits Defines the highest point where packages can be hoisted. One of workspaces (don't hoist packages past the workspace that depends on them), dependencies (packages aren't hoisted past the direct dependencies for each workspace), or none (the default, packages are hoisted as much as possible). This setting can be overriden per-workspace through the installConfig.hoistingLimits field.
nodeLinker Defines what linker should be used for installing Node packages (useful to enable the node-modules plugin), one of: pnp, node-modules.
The solution I found for this is Yarn's nohoist option in your root package.json file.
By default Yarn hoists dependencies to the root directory so they can be shared between your packages. Unfortunately this will not work with Firebase. This means you need to tell Yarn not to hoist the dependencies used by your Firebase functions.
The documentation for nohoist is less than ideal, but here is an official blog post about it here:
https://yarnpkg.com/blog/2018/02/15/nohoist/
You probably want something like this:
{
"workspaces": {
"packages": [
"packages/*"
],
"nohoist": [
"functions/core",
"functions/common",
"functions/**"
]
}
}
Keep in mind that this uses the name field used in the package.json files of each workspace package. So in this example, it is assume that the functions directory has a package.json with "functions" as it's name.
functions/** tells yarn not to hoist any of the dependencies specified in packages/functions/package.json. This doesn't work for your shared yarn packages though, so functions/core and functions/common need to be specified separately.
You also need to include your workspaces as dependencies in your functions project, so add them to your package.json:
{
"name": "functions",
"dependencies": {
"core": "*",
"common": "*",
}
}
Once you have added all that, you should delete your packages/functions/node_modules directory and run yarn install. After doing this, you should see all your dependencies included in packages/functions/node_modules (not symlinks).
I am not sure I understand the question exactly, but I could give you my two cents on yarn workspaces based on whatever I understood from your question and from my experience using it.
Yarn workspaces consolidate all your dependencies into the node_modules present in project root as well as in a single package-lock.json to reduce conflicts and enables yarn to optimize the installation process giving you a faster yarn install. And also another advantage of it is, with a single pass yarn install can install dependencies of all packages under the workspace.
Edit: I think for some reason yarn link is not being called and instead only yarn install is being run, which will search the npm registries and throws the error mentioned in comment since it can't find the mentioned package on npm registry. So for a solution try creating an entry in the firebase's package.json like
"dependencies": {
"a": "file:../dependency-package-name/",
}
I have the following situation:
Frontend's package.json
{
"dependencies": {
"lib" : "1.0.0",
"foo" : "1.0.0"
}
}
lib's package.json
{
"devDependencies": {
"foo" : "1.0.0"
},
peerDependencies": {
"foo" : "1.0.0"
}
}
While I am developing with webpack-dev-server some Frontend I am linking the module foo with a npm link leading into this dependency tree of the Frontend:
├── foo#1.0.0
└─┬ lib#1.0.0
└── foo#1.0.0
Rather than having something like this:
├── foo#1.0.0
└── lib#1.0.0
I have already found out that webpack finds the module foo in my lib in its node_modules folder, because I defined this package as devDependencies for my unit tests. Now I have the package twice in my bundled source which leads into some bugs.
Question: How can I force webpack or npm to use the module foo of my parent (the Frontend) in my lib like the peerDependencies suggests?
I assume you have frontend and lib checked out locally in two different folders and linked lib to frontend using something like cd frontend && npm link lib. So you goal is to work (change code in two editors) on both packages at the same time, but also want to be able to run your compiler and unit tests in frontend and lib separately.
The answer is: you can't (that easy). That's because dependencies are resolved always first to the folder of the package first. That means, if you have foo installed in lib, it will always pick lib:foo, not root:foo.
The simplest solution for that is to not install lib:foo at all, or just remove it. It's using then root:foo, as you want. If you use npm link however, it installs first all lib's dependencies.
So for a one time build in frontend, you can simply remove that lib:foo package and then build. However, from the point of view of the lib package nothing will work as its dependencies got screwed.
The more robust solution is to install lib in root in a away that only installs the actual dependencies of foo that are not in peer/devDependencies
To do so, you can use npm-local-development at https://github.com/marcj/npm-local-development
It basically does the same thing as npm link, but works around the devDependency limitation by setting up a file watcher and syncs file changes automatically in the background, excluding all devDependencies/peerDependencies.
You install npm-local-development: npm i -g npm-local-development
You create file called .links.json in your root package.
You write every package name with its local relative folder path into it like so
{
"#shared/core": "../../my-library-repo/packages/core"
}
Open a console and run npm-local-development in that root package. Let it run in the background.
Disclaimer: I'm the author of this free open-source project.