npm workspaces "WARN: some-package-name in filter set, but no workspace folder present" - node.js

I have a repo with many npm packages inside, using npm workspaces
The top-level package.json contains the line:
"workspaces": [
"*"
]
When I run npm i -ws or other npm commands, I receive the warning:
WARN: some-package-name in filter set, but no workspace folder present
I'm not quite sure what the message means - I think the 'filter set' is the -w option, but the workspace folder some-package-name definitely exists.
One note is that the some-package-name/package.json contains an org prefix, eg:
"name": "#mycompany/some-package-name",
So maybe that's the cause. I can't rename the folder from some-package-name to #mycompany/some-package-name though as I'm concerned a directory starting with # may break things.
What does the warning mean and how can I resolve it?

I ran into the same problem, using the name configured in the workspace's package instead of the folder name fixed the issue for me, even with an #.
npm install --workspace=#mycompany/some-package-name

I have a slightly different answer.
I'm creating a package.json programatically, using Fs.writeFileSync('./path/to/package.json').
I run npm install -w=workspace foo bar immediately after, but it seems that this fails sometimes; NPM somehow hasn't picked up on the file, as if I run the same command twice, the second one always works.
Adding a small delay using setTimeout() also works, though I feel a bit dirty having had to do this; I don't know if this is an issue with Node or my lack of knowledge around Node's process lifecycle.
Anyway, in this case – and in lieu of better solution – it works.

I solved this issue using a file: dependency as "recommended by npm" (read below).
Actually many articles say that npm recommends to use file: to reference a dependency between two workspace packages (and then often they use the asterisk anyway) but honestly I did not found that hint in npm docs, but it works! ... at least it worked for me, using the following
Explicitly list the workspaces
So instead of using an asterisk I have, for example
"workspaces": [
"packages/models",
"packages/database",
"webapp"
]
I also have a scoped package name
for instance
"name": "#project/models",
"private": "true"
I am using "file:" dependencies
Instead of
"dependencies": {
"#project/models": "*"
}
I use file:WORKSPACE_PATH
"dependencies": {
"#project/models": "file:packages/models"
}

Related

Fix "kexec" error (failure, problem, broken) that prevents (unable to) "npm install" WebPack etc "node_modules" dir after node upgrade (node 12,13,14)

I upgraded "node" to "node 14", and ran an "npm install", but the install failed because of an error while attempting to compile "kexec" (a.k.a. "node-kexec"); which is required by "WebPack".
Error messages include:
npm ERR! path /Users/my_self/my_project/node_modules/kexec
...
npm ERR! In file included from ../src/kexec.cc:14:
npm ERR! In file included from ../../nan/nan.h:2884:
...etc...
npm ERR! ../src/kexec.cc:19:11: error: no member named 'Handle' in namespace 'v8'
...etc...
npm ERR! 1 warning and 9 errors generated.
npm ERR! make: *** [Release/obj.target/kexec/src/kexec.o] Error 1
npm ERR! gyp ERR! build error
...etc...
npm ERR! gyp ERR! not ok
This issue only shows up for projects that have a dependency upon "kexec". The actual purpose of "kexec" is to launch programs by passing a request to the OS, either directly or through a shell like "bash".
As it happens, WebPack depends upon "kexec". So to get WebPack working with "node" versions 12, 13, 14, etc, "kexec" has to compile.
This problem has already been documented and fixed on GitHub. In fact, there is a radically new version of "kexec" available, waiting for a "pull" to the master branch. The problem is that the "pull" has languished.
How can use the fixed version of "kexec" within my "npm install" to rebuild the "node_modules" directory as required by "WebPack"?
I was able to successfully get a revised version of "kexec" (a.k.a. "node-kexec") to compile via "npm install" despite an upgrade to "node" (node 14), the result being well built "node_modules" fulfilling requirements for "WebPack".
If using "npm" version "8.3.0" or higher, try using "overrides". See "Candidate Method 3", below. In my case, I didn't explore using "overrides" becasue I am using a version of "npm" that does not have the capability. Instead, I did some trickery.
The required version of "kexec" is 3.0.0, but that is old code which breaks when "node" is upgraded to or beyond "node" version 12.
An all new "kexec" 4.0.0 has been written. The re-write was available back in Oct 2020, but as of Jan 26, 2022, the revision is still languishing. The new code is awaiting a "pull" into the "master" branch. This v4.0.0 fixes the problem, except for the fact that "npm" can't find it.
"WebPack" requirements call for "kexec" v3.0.0; which is the broken version. It can't find v4.0.0 because it's not in the master branch. The obvious solution would be to convince "npm" to look elsewhere for "kexec". Some transparent override methods are described here. These methods involve modifying the "package.json" file; (hence, transparency). Later in this answer, I also describe a non-transparent method; this "trickery" is what worked for me.
Method 1 with package.json: Use "yarn" and "resolutions"
As per "tamlyn", Jul 15, 2020, "if you're using Yarn...[add] the following to your package.json:
"resolutions": {
"kexec": "meteor/node-kexec#update-for-node-12"
}
But this does not fit my situation, since I am using "npm", and not "yarn".
Method 2 with package.json: Use "npm" and "npm-force-resolutions"
There is a module "npm-force-resolutions" that attempts to make the above mentioned "resolutions" idea work with "npm". However, the word is that it works only with older versions of "npm".
Method 3 with package.json: Use "npm" and "overrides"
For "npm" version 8.3.0, there is a built in feature called "overrides" that is supposed to be the equivalent of "resolutions". Here is the documentation for it:
https://docs.npmjs.com/cli/v8/configuring-npm/package-json#overrides
Presumably, the syntax for use would be like that shown above for "yarn", but with the word "overrrides" replacing "resolutions". If you are using a version of "npm" that is v8.3.0 or higher, you should try this. I did not explore this.
Non-Transparent Method
If possible, start by upgrading to the latest version of "npm".
$ sudo npm install -g npm#latest
And then check the "npm" and "node" versions. In my case, because several machines need to be coordinated, I did not have the absolute latest. Here is what I was working with:
$ npm --version
v7.12.0
$ node --version
v14.18.3
Having sufficiently upgraded "npm", you are ready to actually start fixing the "kexec" problem. If you have "npm" version 8.3.0 or more, try "Method 3", mentioned above. If that doesn't work, proceed as follows.
In this documentation, I will use the word "my_project_dir" to represent the directory in which you are trying to create your "node_modules" sub-directory. Start by going to that location.
$ cd my_project_dir
Move aside the existing "node_modules" directory (if any exists).
$ mv node_modules node-modules-old
You should now attempt a fresh install. Of course, this install will look at "package.json" or "package-lock.json", and know what is needed. However, presuming your development environment requires "kexec", and because the old "kexec" has trouble compiling within the new "node" environment, the install hits a wall; as shown here
$ npm install
...
make: *** [Release/obj.target/kexec/src/kexec.o] Error 1
... etc,
... etc, etc, more "build error" messages
In my case, I use "WebPack", which has a dependency on "kexec v3.0.0", which breaks when used with upgraded "node".
The totally new version of "kexec", which is the creation of a group of developers at "Meteor", is available on "github", directly through one of the developers, who goes by the user-name "StorytellerCZ". (See also list of URLs at end of this doc).
Use "git" to download the updated code. Actually, what I did was a "git clone" of the entire "node-kexec" module repository, as presented by "StorytellerCZ". This download has more than is needed, but that's ok.
$ pwd
/home/myself/my_project_dir
A "node_modules" sub-directory exists; it was created by the attempted "npm install". Descend into the sub-directory.
$ cd node_modules
$ git clone https://github.com/StorytellerCZ/node-kexec.git
This creates a sub-directory named "node-kexec"; which is a "git" repository. Note that the desired directory name is actually "kexec", but, for now, having two different names ("kexec" versus "node-kexec") will actually keeps things cleaner.
The next step depends upon whether a "kexec" directory already exists. Check for it.
$ ls -ld kexec
drwxr-xr-x 16 myself staff 512 Apr 30 2021 kexec
The "kexec" subdiretory might not exist; this depends upon what was left over from the failed "install"; (the "install" that got you looking for a fix in the first place). On my Mac OS X (BSD) system with "npm --version" of "7.12.0", the "kexec" remnant was preserved. On an AWS Linux system with "npm --version" of "6.14.15", the "kexec" directory was destroyed at every failed "npm install".
If the "kexec" directory exists, you just need to cherry-pick one file from the git clone. First move the old file asside; the following command moves the old file to a new name where-in "-2016" is a reminder of the year of the most recent "pull" of the source code for "kexec".
$ mv kexec/src/kexec.cc kexec/src/kexec-2016.cc
Now replace that with the desired version
$ cp -p node-kexec/src/kexec.cc kexec/src
And remove the clone, so that it doesn't cause problems at some later date.
$ rm -rf node-kexec
On the other hand, if "kexec" is not present as a subdirectory, then you must convert the "node-kexec" repository into a directory that is named "kexec".
$ mv node-kexec kexec
However, you probably need to do one more thing; a bit of subterfuge. The package dependencies for your project might include a requirement for exactly version "3.0.0" of "kexec"; this is the case for my project, which I assume is because of some dependancy of "WebPack". The "kexec" from "StorytellerCZ" is labeled as version "4.0.0"; (and indeed, it is an entirely new body of code). This is problematic, because, on some systems, "npm" will actually delete the subdirectory you just created, and replace it with the same old, bad "3.0.0" code. Therefore, some subterfuge is necessary.
To trick the "npm" system into accepting the new directory, edit the "package.json" file, and modify the value of the "version" parameter to "3.0.0". Yes, this really stinks, but it solves the immediate problem. (The real solution will be for the "4.0.0" changes to finally get integrated into the "master" branch.) The following steps make the change, cautiously.
Descend into pre-existing "kexec" directory, under "node_modules".
$ cd kexec
Make a copy of the file you are about to change. This leaves a clue for you, in the future, when you are wondering how the version number got the way it is.
$ cp -p package.json package.json-original
Edit the "package.json" file. For example, if your editor is "vim", you could invoke the editor like this:
$ vim package.json
Change the "version" value; make it "3.0.0". When that's done, save the edit and quit the editor.
Ascend out of the "kexec" sub-directory, back to "node_modules".
$ cd ..
So, at this point you have prepared a "kexec" module that should compile. You did it one of two ways: either (1) you had a pre-existing "kexec" directory and you simply copied over the updated "kexec.cc" file, or (2) you renamed "node-kexec" to "kexec" and you fiddled with the "version" number.
To re-try your install, first ascend out of the "node_modules" sub-directory, back to your project directory.
$ cd ..
Prove to yourself that this is the right place.
$ pwd
/home/myself/my-project-dir
Do the install.
$ npm install
The "kexec" error should be gone.
If you are still getting errors, check whether the very first line of warning message says the following:
npm WARN read-shrinkwrap This version of npm is compatible with lockfileVersion#1, but package-lock.json was generated for lockfileVersion#2. I'll try to do my best with it!
This warning should go away when you upgrade the "npm" version to 7.12.0 or better. Do so!
If "npm" protests about the presence of "git" information, that should go away once you upgrade the "npm" version to 7.12.0 or better. If the warning persists, then you can just get rid of the "git" info, as follows.
Descend into the repository dir
$ cd kexec # formerly named "node-kexec"
Confirm, just for fun, that the "git" structure is actually here. Note that "ls *" won't show hidden files, hence use this command:
$ ls -a | grep git
.git
.gitignore
Strip out the git repository information.
$ rm -rf .git
$ rm .gitignore
Ascend out of the "kexec" sub-directory, back to "node_modules".
$ cd ..
$ pwd
/home/myself/my-project-dir/node_modules
Retry the "npm install".
Appendix I
Here are some URLs to relevant github archives:
[https://github.com/jprichardson/node-kexec/issues/36][1]
[https://github.com/StorytellerCZ/node-kexec.git][1]
[https://github.com/meteor/node-kexec/tree/update-for-node-12][1]
Appendix II
On OS-X, the intermediate binary object file created during the build is:
node_modules/kexec/build/Release/obj.target/kexec/src/kexec.o
On OS-X, the executable appears to be:
node_modules/kexec/build/Release/kexec.node
On AWS Linux, I did not find either of those. I didn't bother to
seek an explanation, since WebPack had become functional, and
therefore my goal was achieved.

I need to add dependency in my package.json and load module , after taking user input while running npm install

I need to add a dependency in my package.json and load module, by taking user input in command prompt while running npm install.
Is it possible to do so.
As you haven't provided much information about what exactly you try to accomplish, it's difficult to advise you or give a concrete example. Anyhow, in package.json, you can override the default behavior of npm scripts with the scripts field:
"scripts": {
"install": "./scripts/install.sh",
}
As someone else has noted before, you cannot call npm install in your custom script, though, as this would lead to infinite recursion. Thus, rather provide a preinstall script, which npm runs before install.
For use cases other than adding dependencies based on user input, I recommend to check how to set the environment variables and configurations, which the user then could override with npm config set project-name:config-name config-value.

NPM: How to link 2 package with same name but different version?

I wanna develop 2 versions of package with same name, but different versions:
first
// package.json
{
"name":"mypackage",
"version": "1.0.0-base"
}
second
// package.json
{
"name":"mypackage",
"version": "2.0.0-next"
}
Now I wanna in two different projects do:
npm link <path to mypackage#1.0.0-base>
or
npm link <path to mypackage#2.0.0-next>
For now , when I do that, last linked package "replace"
the every linked module with name mypackage in previous linked projects.
I know that npm has "global space" for linking... but is there any way to get around this ?
https://github.com/ORESoftware/npm-link-up may be the solution for you. It looks like it uses actual symlinks, rather than the NPM global space. I have not used it yet (honestly I was looking into the same problem but haven't gotten around to trying this as a solution yet).
Let me know if this solves your problem!
(https://github.com/ORESoftware/npm-link-up/blob/dev/docs/in-detail.md describes it as not using global space)

How to fix broken Typescript definitions when definitions files are updated

I have a project that uses Typescript, using the newer #types/foo style of installing typings packages.
When my build server installs all npm modules, sometimes I get a complete failure when compiling the typescript as some dependent definitions are no longer matching up.
For instance, I now have a problem with #types/gulp. In its package.json, dependencies are listed as:
"dependencies": {
"#types/node": "*",
"#types/orchestrator": "*",
"#types/vinyl": "*"
},
But now #types/orchestrator has updated, and it now breaks the version of #types/gulp that I have defined in my apps package.json.
How am I supposed to lock down version of dependencies like this so I no longer get this problem, or is there another workaround?
Unfortunately, I suddenly get these issues which sets development back by hours trying to sort it out. This makes using Typescript in a fast moving environment difficult.
How am I supposed to lock down version of dependencies like this so I no longer get this problem
Run npm shrinkwrap or just specify an exact version:
"#types/vinyl": "6.3.12"

npm installs dependencies with complete readme in a package.json -- invalid json results

Today I upgraded npm and node. I'm at 1.4.9 and 0.10.28, respectively. (OSX 10.9.2.)
Since then, it seems that npm install is writing "bad" package.json files for all of my dependencies.
Specifically, it is putting sevaral fields into the files that do not seem to be present in the source repo's package.json.
Like this example. I have about 40 dev dependencies and they all have this junk (not to pick on gulp, I just happened to grab it for this example):
"readme": "<p align=\"center\">\n <a href=\"http...",
"readmeFilename": "README.md",
"bugs": {
"url": "https://github.com/gulpjs/gulp/issues"
},
"_id": "gulp#3.6.2",
"_shasum": "ea6b33d768db4a22294fa6339afb61842f5e6fb5",
"_from": "gulp#~3.6.2",
"_resolved": "https://registry.npmjs.org/gulp/-/gulp-3.6.2.tgz"
These are on the root of the package.json object. Note, I've abbreviated the readme -- it's an entire html-ized version of the readme, and in many of the files it's not properly escaped and thus makes package.json invalid and unusable.
Obviously this leaves me dead in the water. My questions are "what did I do"? And how do I undo it? Did I wind up with a version of NPM that I shouldn't have? Could there be something else about my project, my node install, or the rest of my environment that might cause this? I do have nave and a version of node in the 0.11 family, but that's not active and hasn't been in recent history -- could npm update -g npm have discovered that I have 0.11 in a different directory and jumped me into territory to which I should not have gone? If so, how to I go backward? I've tried npm installing npm back down into earlier versions but npm still reports the higher version number as if my install didn't happen, and I can't quite figure out what version I'm supposed to have, anyway.
What I've tried so far -- reinstalling node and npm, wiping out my node_modules for the project, and npm cache clean. Nothing seems to have any effect. I'd be happy to answer any questions that I can about my environment that might help me resolve this.
TIA!
I just poked my way somewhat blindly into a solution.
(although I suspect it didn't matter, I upgraded to the master branch code of npm)
I believe that I fixed this by experimenting with setting long = false in my .npmrc file. No more readmes in package.json files! :)
Aside: the package.json content that I see is quite foreign to me in its arrangement, but I guess that may be a function of being on a newer version.
At the very least, the package.json is now valid json and life is better again. I'm not going to accept this, my own answer, just yet because I hope someone might be able to tell me why I needed to do this. This is an override in my .npmrc file as far as I understand it, so it seems bizarre that out of the blue I needed to add this override by hand based on no documentation....
I'll accept an answer that points me to an explanation of what happened or of what might have happened, if there is such an answer! If none are forthcoming, I'll just accept my own answer so that the SO question loop is closed. Thanks!

Resources