I have an npm package for common components hosted on an internal git server. For some reason when I call npm install in another project I want to consume this in it will not run the prepare hook. Obviously, this does not work since the npm package needs a /dist folder in node_modules to be able to consume the package.
I have already tried things such as using the deprecated prepublish hook and even that does not get called. I also tried to do postinstall to see if I could build after install, while that hook did get called it failed because the devDependencies were not installed
package.json
{
"name": "common-components",
"version": "0.1.0",
"scripts": {
"prepare": "npm run build",
"build": "ng build",
...
},
"private": true,
"dependencies": {
...
},
"devDependencies": {
...
},
}
command being used for install
npm install --save git+ssh://{URL-to-common-components-repo}}
I have read through the npm-scripts documentation https://docs.npmjs.com/misc/scripts thoroughly and it seems like they insist that prepare hook should always be called for this exact use-case
Updated 5/6/2019
Just as a note I found this bug on NPM community https://npm.community/t/using-npm-ci-does-not-run-prepare-script-for-git-modules/632/4.
I am using npm 6.4.1 which should work according to the bug
One thing to check that hit me on a package recently - if there is a .gitignore and not .npmignore npm may be ignoring your /dist folder. Adding an empty .npmignore worked in this case.
"If there’s no .npmignore file, but there is a .gitignore file, then
npm will ignore the stuff matched by the .gitignore file. If you want
to include something that is excluded by your .gitignore file, you can
create an empty .npmignore file to override it."
from https://docs.npmjs.com/misc/developers
For those that are wondering the status of this. I was unable to ever get it to work. What I ended up doing was hosting the components on a private npm registry and that works fine since the npm publish command will do the build and only publish the dist folder
If adding an empty .npmignore does not help, you can try specifying all files in dist explicitly in package.json#files. If this works you might want to consider using a wildcard pattern that matches the files in dist to simplify the package.json.
package.json
...
"files": [
"source",
"dist/cjs/main.js",
"dist/es/main.js"
]
}
see this comment to a similar issue in the npm/cli repository https://github.com/npm/cli/issues/1287#issuecomment-635021757
It's very likely that your dist/ folder is in your .gitignore file. According to this response from an npm-cli maintainer:
In order to be able to properly prepare a git repo npm will run the
extracted files through npm-packlist in order to get the expected
files that are going to be placed in your node_modules folder.
Further checking the documentation of npm-packlist, we find that npm-packlist will respect the .gitignore file if it has nothing else to go off of:
If there's no package.json with a files list, and there's no
.npmignore file, but there is a .gitignore file, then ignore all the
files in the .gitignore file.
This article further expands on the idea.
It seems to me that the best fix is to explicitly declare the files that your package needs (including dist/) in the files section of your package.json file. Then you have complete control over what's included and the package size is minimized.
If you are using root user to npm install the package then the prepare script might not be triggered. The reason was the prepare child process has no permission to run (user account was set to default of 'nobody' when using npm with root). You can read more here: https://github.com/npm/npm/issues/17346
To fix this, in the lib package, create an .npmrc file and add:
unsafe-perm: true
Adding main in the package.json fixed this issue for me.
"main": "./dist/index.js",
"scripts": {
"build": "babel src --out-dir dist",
"prepare": "npm run build",
"lint": "eslint ."
},
node v14.15.4 npm 6.14.11
I need some help to setup NodeJS on a Windows 10 machine.
I installed the 10.15.3 LTS version from the official website which has done the following:
Node installed at C:\Program Files\nodejs\
Added to the Path in the user's variables: C:\Users\rsantos\AppData\Roaming\npm
Added to the Path in the environment variables: C:\Program Files\nodejs\
Created an empty npm folder at: C:\Users\rsantos\AppData\Roaming
Then with npm init I initialized the package file which was created at C:\Users\rsantos.
Followed by npm install supervisor which:
Created a npm-cache folder at C:\Users\rsantos\AppData\Roaming\npm-cache
Created a node_modules folder (with the supervisor) at C:\Users\rsantos\node_modules
Created a package-lock.json file at C:\Users\rsantos with:
{
"name": "rsantos",
"version": "1.0.0",
"lockfileVersion": 1,
"requires": true,
"dependencies": {
"supervisor": {
"version": "0.12.0",
"resolved": "https://registry.npmjs.org/supervisor/-/supervisor-0.12.0.tgz",
"integrity": "sha1-3n5jNwFbKRhRwQ81OMSn8EkX7ME="
}
}
}
Updated the package.json file to:
{
"name": "rsantos",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "",
"license": "ISC",
"dependencies": {
"supervisor": "^0.12.0"
}
}
Then when I open the command line and type:
C:\Users\rsantos\Desktop>supervisor --watch C:\NodeJS_Projects\Proj1\ -e js C:\NodeJS_Projects\Proj1\Test.js
I get:
'supervisor' is not recognized as an internal or external command, operable program or batch file.
After that I tried to install supervisor globally and it started to work.
The next error was 'require' not being recognized after installed locally.
I followed the same approach which made supervisor work and installed require globally. My code is requiring the module with require('request'). But even installed globally, it fails with this error:
Error: Cannot find module 'request' at Function.Module._resolveFilename (internal/modules/cjs/loader.js:582:15)
Notes:
I uninstalled everything and reinstalled, the result was always the same.
Also played with the environment variables but still no good.
npm list -g request outputs this: C:\Users\rsantos\AppData\Roaming\npm -- request#2.88.0
At this point my C:\Users\rsantos\node_modules folder has all the modules, including supervisor and request.
The package.json has supervisor and request as dependencies.
The package-lock.json seems to have all the modules in the node_modules folder, including supervisor and request.
I still get the Cannot find module 'request' error.
Can someone help please?
when you want to use a NPM package as tool (e.g. >supervisor --watch PATH), you can install the package --global. When you want to load the package in Node with require() I would install it locally.
Does the following work for you?
Install the tooling globally:
npm install --global supervisor
Set up your project in local folder:
mkdir project
cd project
npm init -y
npm install --save request # install to project/node_modules
node index.js # start your app
How to save the installation of global package using package.json?
I couldn't find any good solutions but I did this like below:
In package.json I added the following:
"scripts": {
"preinstall": "npm install babel babel-cli -g"
},
It would run and install the above packages globally before installing all the dependencies and devDependencies in package.json.
But the problem would be that I would not be able to find out, are those packages installed globally in any of the machines.
Please help, if anyone have any better solution of this.
It's a bad practice to force global install of a module. You can put babel and babel-cli in your devDependencies and then use them in your npm scripts :
{
"build" : "babel src -d build"
},
"devDependencies": {
"babel-cli": "^6.18.0"
}
If you are looking to run your package via CLI, you need to set up using bin approach.
Here's one of my libraries for example, fol. It is meant to be ran via command line only.
At the most minimal setup, you would wire both main and bin to the same JS file, for example, adding the following in package.json:
"main": "./bin/fol.js",
"bin": {
"fol": "./bin/fol.js"
},
Then, put the files into bin folder for the same of consistency, everybody will recognise that's CLI app stuff if it's in /bin/.
So I need to read the package.json before installing a new package via npm.
Why reading package.json in the first place?
I am using npm for CSS components that are individually versioned and can have inter dependencies. (No javascript is delivered)
Looking for version conflicts for a bunch of dependencies I need to detect when package A requires package C#1.0.0 and package B requires package C#2.0.0 and deal with it.
Npm (as of version 3) deals with these issues by nesting a conflicting module deeper inside the tree. You now end up with both versions of the same module. CSS has a global namespace and a mixin (in Sasss case) would then overwrite each other and break your CSS.
This flat dependency issue is perfectly outlined in the npm blog: http://blog.npmjs.org/post/101775448305/npm-and-front-end-packaging
Even not considering our specific use case it strikes me as odd that you don't have access to the package.json in preinstall and postinstall scripts. They seem to be just for that use case.
What I tried
My package.json of the package I'm installing looks like this:
{
"name": "testmodule",
"version": "0.3.6",
"description": "TODO",
"scripts": {
"preinstall": "npm i some-script && some-script",
},
"author": "TODO",
"license": "MIT"
}
Inside that some-script package I run:
console.log( process.cwd() );
console.log( __dirname );
I then run:
~/path/to/folder $ npm i testmodule
This will result in:
$ npm i testmodule
> testmodule#0.3.6 preinstall /path/to/folder/node_modules/.staging/testmodule-5cc9d333
> some-script
/path/to/folder/node_modules/.staging/test-module-5cc9d333
/path/to/folder/node_modules/.staging/test-module-5cc9d333/node_modules/some-script
Now I totally get that I can't really access the root of where npm i was ran because my script was run by a subprocess of npm and has an entirely different root.
I then thought npm root should keep track where the actual root was for me and passed that as a parameter to my script from inside the testmodule package.json:
{
"name": "testmodule",
"version": "0.3.6",
"description": "TODO",
"scripts": {
"preinstall": "npm i some-script && some-script \"$(npm root)\"",
},
"author": "TODO",
"license": "MIT"
}
Unfortunately that also defaults back to a staging path:
/path/to/folder/node_modules/.staging/testmodule-5cc9d333/node_modules
I filed an issue with the registry but not holding my hopes up for them to get to that in time. Also my script needs to work on older npm installations.
In the meantime I came up with something like that inside my some-script:
let pgkPath = process.cwd().split('/node_modules/')[0];
That will return /path/to/folder/ which is correct but it makes the assumption no-one runs an npm i inside a folder incidentally named node_modules... Seems hacky.
Question
How can I access the path to the package.json from inside an npm script that is run via preinstall? To me that seems like something not too outrageous to ask for?
I don't understand your use-case entirely, but to answer your specific question of finding a parent package.json from a preinstall script:
Pass $(cd .. && npm prefix) as an argument to your script, then load ./package.json.
npm prefix will return the closest parent directory to contain a package.json file, which when invoked from the .. directory, should return the parent npm package's path.
{
"name": "testmodule",
"version": "0.3.6",
"description": "TODO",
"scripts": {
"preinstall": "npm i some-script && some-script \"$(cd .. && npm prefix)\"",
},
"author": "TODO",
"license": "MIT"
}
I want to do something like this, so npm install also installs the package.json of ../somelocallib or more importantly its dependencies.
"dependencies": {
"express": "*",
"../somelocallib": "*"
}
npm >= 2.0.0
This feature was implemented in the version 2.0.0 of npm. Local paths can be saved using npm install -S or npm install --save, using any of these forms:
../foo/bar
~/foo/bar
./foo/bar
/foo/bar
Example package.json:
{
"name": "baz",
"dependencies": {
"bar": "file:../foo/bar"
}
}
npm ls:
app#0.0.1 /private/tmp/app
└── somelocallib#0.0.1 -> /private/tmp/somelocallib
npm < 2.0.0
Put somelocallib as dependency in your package.json as normal:
"dependencies": {
"somelocallib": "0.0.x"
}
Then run npm link ../somelocallib and npm will install the version you're working on as a symlink.
Reference: link(1)
It is now possible to specify local Node module installation paths in your package.json directly. From the docs:
Local Paths
As of version 2.0.0 you can provide a path to a local directory that contains a package. Local paths can be saved using npm install -S or npm install --save, using any of these forms:
../foo/bar
~/foo/bar
./foo/bar
/foo/bar
in which case they will be normalized to a relative path and added to your package.json. For example:
{
"name": "baz",
"dependencies": {
"bar": "file:../foo/bar"
}
}
This feature is helpful for local offline development and creating tests that require npm installing where you don't want to hit an external server, but should not be used when publishing packages to the public registry.
This works for me.
Place the following in your package.json file
"scripts": {
"preinstall": "npm install ../my-own-module/"
}
This is how you will add local dependencies:
npm install file:src/assets/js/FILE_NAME
Add it to package.json from NPM:
npm install --save file:src/assets/js/FILE_NAME
Directly add to package.json like this:
....
"angular2-autosize": "1.0.1",
"angular2-text-mask": "8.0.2",
"animate.css": "3.5.2",
"LIBRARY_NAME": "file:src/assets/js/FILE_NAME"
....
If you want to further automate this, because you are checking your module into version control, and don't want to rely upon devs remembering to npm link, you can add this to your package.json "scripts" section:
"scripts": {
"postinstall": "npm link ../somelocallib",
"postupdate": "npm link ../somelocallib"
}
This feels beyond hacky, but it seems to "work". Got the tip from this npm issue:
https://github.com/npm/npm/issues/1558#issuecomment-12444454
Master project
Here is the package.json you will use for the master project:
"dependencies": {
"express": "*",
"somelocallib": "file:./somelocallib"
}
There, ./somelocallib is the reference to the library folder as relative to the master project package.json.
Reference: https://docs.npmjs.com/cli/v7/configuring-npm/package-json#local-paths
Sub project
Handle your library dependencies.
In addition to running npm install, you will need to run (cd node_modules/somelocallib && npm install).
This is a known bug with NPM.
Reference: https://github.com/npm/npm/issues/1341 (seeking a more up-to-date reference)
Notes for Docker
Check in your master package.lock and your somelocallib/package.lock into your source code manager.
Then in your Dockerfile use:
FROM node:10
WORKDIR /app
# ...
COPY ./package.json ./package-lock.json ./
COPY somelocallib somelocallib
RUN npm ci
RUN (cd node_modules/zkp-utils/ && npm ci)
# ...
I use parenthesis in my (cd A && B) constructs to make the operation idempotent.
Two steps for a complete local development:
Provide the path to the local directory that contains the package.
{
"name": "baz",
"dependencies": {
"bar": "file:../foo/bar"
}
}
Symlink the package folder
cd ~/projects/node-redis # go into the package directory
npm link # creates global link
cd ~/projects/node-bloggy # go into some other package directory.
npm link redis # link-install the package
Here in 2020, working on a Windows 10, I tried with
"dependencies": {
"some-local-lib": "file:../../folderY/some-local-lib"
...
}
Then doing a npm install. The result is that a shortcut to the folder is created in node-modules.
This doesn't work. You need a hard link - which windows support, but
you have to do something extra in windows to create a hard symlink.
Since I don't really want a hard link, I tried using an url instead:
"dependencies": {
"some-local-lib": "file:///D:\\folderX\\folderY\\some-local-lib.tar"
....
}
And this works nicely.
The tar (you have to tar the stuff in the library's build / dist folder) gets extracted to a real folder in node-modules, and you can import like everything else.
Obviously the tar part is a bit annoying, but since 'some-local-lib' is a library (which has to be build anyway), I prefer this solution to creating a hard link or installing a local npm.
I know that npm install ../somelocallib works.
However, I don't know whether or not the syntax you show in the question will work from package.json...
Unfortunately, doc seems to only mention URL as a dependency.
Try file:///.../...tar.gz, pointing to a zipped local lib... and tell us if it works.
With yarn it can be done as
yarn add file:../somelocallib
Curious.....at least on Windows (my npm is 3.something) I needed to do:
"dependencies": {
"body-parser": "^1.17.1",
"module1": "../module1",
"module2": "../module2",
When I did an npm install ../module1 --save it resulted in absolute paths and not relative per the documentation.
I messed around a little more and determined that ../xxx was sufficient.
Specifically, I have the local node modules checked out to say d:\build\module1, d:\build\module2 and my node project (application) in d:\build\nodeApp.
To 'install', I:
d:\build\module1> rmdir "./node_modules" /q /s && npm install
d:\build\module2> rmdir "./node_modules" /q /s && npm install
d:\build\nodeApp> rmdir "./node_modules" /q /s && npm install
module1's package.json has a dependency of "module2": "../module2"; module2 has no local dependency; nodeApp has dependencies "module1": "../module1" and "module2": "../module2".
Not sure if this only works for me since all 3 folders (module1, module2 and nodeApp) sit on that same level.......
This worked for me: first, make sure the npm directories have the right user
sudo chown -R myuser ~/.npm
sudo chown -R myuser /usr/local/lib/node_modules
Then your in your package.json link the directory
"scripts": {
"preinstall": "npm ln mylib ../../path/to/mylib"
},
"dependencies": {
"mylib" : "*"
}
Actually, as of npm 2.0, there is support now local paths (see here).
There is great yalc that helps to manage local packages. It helped me with local lib that I later deploy. Just pack project with .yalc directory (with or without /node_modules). So just do:
npm install -g yalc
in directory lib/$ yalc publish
in project:
project/$ yalc add lib
project/$ npm install
that's it.
When You want to update stuff:
lib/$ yalc push //this will updated all projects that use your "lib"
project/$ npm install
Pack and deploy with Docker
tar -czvf <compresedFile> <directories and files...>
tar -czvf app.tar .yalc/ build/ src/ package.json package-lock.json
Note: Remember to add .yalc directory.
inDocker:
FROM node:lts-alpine3.9
ADD app.tar /app
WORKDIR /app
RUN npm install
CMD [ "node", "src/index.js" ]
I wanted to use a set of local dependencies written in TypeScript, and none of the answers here worked for me. npm install would simply refuse to build the dependencies.
I had to resort to using tsconfig.json to add the packages to my project without marking them as dependencies. My usecase is further complicated by the fact that some dependencies depend on each other, and I wanted all of them to come from the local folder.
Here is my solution:
// tsconfig.json
{
"compilerOptions": {
"baseUrl": "./",
"paths": {
"#tiptap/*": [
"tiptap/packages/*/src"
]
}
}
}
In the example above, I have a local project subfolder named tiptap/ and there are many packages in tiptap/packages/*. The "paths" option will rewrite all #tiptap/foo imports into ./tiptap/packages/foo/src, across both my own files and the files in tiptap/.
It's not a good solution, but it is the only thing that worked for me.
use workspaces
the disadvantage using the file:../path/to/your-library is that you either have to npm install or using npm link in order to to the changes to take effect in the packages that import your package.
if you using pnpm: a better solution is using workspace: protocol: workspace:../path/to/your-library. it will symlink the directory to your node_modules rather than copying it, so any changes at the source immediately take effect.
for example:
...
"dependencies": {
...
"my-package": "workspace:../../dist"
},
note: this solution is intended to be used in a workspace, so you may need to create pnpm-workspace.yaml (even an empty one) file in the root of your project.
In 2021 you need to use it like:
npm i my-pkg#file:./path-to-my-pkg.js
# To remove it later
npm un my-pkg
Use .js in the end if its file OR path to folder if its complete package with package.json.
Usage
const myPkg = require('my-pkg')
That works like charm!
Complete local development guide for yarn users:
First add dependency to your main project:
cd main-project
yarn add file:../path/to/your-library
Next, if you want to avoid re-building this dependency every time you change it's source:
cd your-library
yarn link
This will register a link to your-library. Next, use the link you just created in your main project.
cd main-project
yarn link your-library
Now every time you change code in your-library, you don't need to rebuild it and it will automatically be included in your main-project. Yarn link works by creating symlinks in your node_modules folder, read more about it here: https://classic.yarnpkg.com/lang/en/docs/cli/link/
Using Module Alias
Install the module-alias package:
npm i --save module-alias
Add paths to your package.json like this:
{ "_moduleAliases": { "#lib": "app/lib", "#models": "app/models" } }
In your entry-point file, before any require() calls:
require('module-alias/register')
You can now require files like this:
const Article = require('#models/article');