Is there a pip freeze equivalent for Node and NPM? - node.js

This is idiomatic in Python:
pip freeze > requirements.txt
pip install -r requirements.txt
The first command saves a list of requirements to a file. Then later you can use the command to install the requirements into your environment.
Node has npm install, but I don't get how to I'm supposed to dump the state of my dependencies to a package.json. I Googled and found this:
npm ls | grep -E "^(├|└)─" | cut -d" " -f2 | awk '{FS = "#"; print "\""$1"\"", ":", "\""$2"\""}'
but as the author of this pipeline suggests, there's got to be a better way? What am I missing here?
I just want to dump my current deps into a package.json. As https://npmjs.org/doc/shrinkwrap.html says,
The "package.json" file is still required if you want to use "npm install".
I've skimmed the info on shrinkwrap, but I'm not seeing how to simply accomplish this task with shrinkwrap.

This is the closest I got
npm freezelol
npm ls | grep -E "^(├|└)─" | cut -d" " -f2 | awk -v quote='"' 'BEGIN { FS = "#" } ; { print quote $1 quote,":",quote $2 quote"," }' | sed -e 's/ :/:/g'
Output is like
"bower": "1.3.12",
"chai": "2.1.2",
"cucumber": "0.4.8",
Still needs to trim the final trailing comma but it's pretty close!

You can create a package.json out of the currently installed package using npm init. Then you can easily move the package.json and simply do npm install to install the packages wherever you want.

This will probably work for you. npm shrinkwrap
npm shrinkwrap

Here is alternative shorter version of command which parses npm ls:
npm ls | grep -o "\S\+#\S\+$" | tr # ' ' | awk -v q='"' '{print q$1q": "q"^"$2q","}'
And here is an alias which is worth to add to your shell rc file:
alias npm-freeze='npm ls | grep -o "\S\+#\S\+$" | tr # " " | awk -v q='\''"'\'' '\''{print q$1q": "q"^"$2q","}'\'''
and run it as:
npm-freeze
The output is like:
"backbone": "^1.3.2",
"underscore": "^1.8.3",
"bootstrap": "^3.3.6",
"bootstrap-sass": "^3.3.6",
"grunt": "^0.4.5",
To filter, just pipe it into grep, e.g.
$ npm-freeze | grep grunt
"grunt": "^0.4.5",
"grunt-cli": "^0.1.13",
"grunt-contrib-watch": "^0.6.1",
"grunt-sass": "^1.1.0",
"load-grunt-tasks": "^3.4.1",
Here is the example saving the output into package.json and running npm install:
printf "{\n"\""name"\"": "\""npm-freeze"\"",\n"\""dependencies"\"": {\n$(npm-freeze | grep grunt | head -c -2)\n}\n}" | tee package.json && npm install

It appears to be built into NPM versions 6+ via the package-lock.json
https://docs.npmjs.com/cli/v7/configuring-npm/package-lock-json

Related

How to install all existing global NPM packages into local directory?

This is not similar nor does it answer question: copy npm global installed package to local
Basically I want to tell NPM, look at what I have installed globally, and install that locally.
I'm not interested in the symlinks, but if above is not possible, could I use a single symlink to node_modules instead of a symlink for each package?
You can parse the output of node ls -g --depth 0 and npm install the resulting list of packages.
npm i -S -dd $(npm ls -g --depth 0 | tail -n +2 | cut -c 5- | tr '\n' ' ')
Run this command in the directory of the package that you wish to install global packages to.
Explanation:
npm i -S -dd Shorthand for npm install --save --verbose. The -S is unnecessary on recent npm versions that save installed packages to package.json by default.
npm ls -g --depth 0 List first-level global packages.
tail -n +2 Remove the first line from the output.
cut -c 5- Remove the first four characters from each line in the output.
tr '\n' ' ' Combine each line to place all packages on one line, separated by spaces.

How to check for only high vulnerabilities when using "npm audit"?

When you I execute npm install using new npm 6
i got a messages that tell me I have some vulnerabilities :
[!] 75 vulnerabilities found [4867 packages audited]
Severity: 66 Low | 4 Moderate | 5 High
Run npm audit for more detail
I ran npm audit but got a truncated list of vulnerabilities.
How I can check for only High vulnerabilities list ?
Thanks
Not the answer you are looking for, but it will do the same:
npm audit | grep -B 1 -A 10 High
This one worked for me:
Show High Only
npm audit | grep -E "(High)" -B3 -A10
Show both Critical and High Issues
npm audit | grep -E "(High | Critical)" -B3 -A10
Look at the issue discussion where this solution is proposed.
If your are looking to do it in Powershell, just use the following command (Adapted from #stayingcool's answer):
Show High Only
npm audit | Select-String -Pattern "High" -Context 0,10
Show both High and Critical
npm audit | Select-String -Pattern "(High | Critical)" -Context 0,10
Edit: I recommend this (better) answer: https://stackoverflow.com/a/58056454/88111
It's not as pretty, but you can do:
npm audit --parseable | grep high
With one additional downside being any package/issue metadata containing "high" will also be printed.
The --audit-level=high flag doesn't change the output of npm audit.
I'm sending this to html for reporting purposes, so looking to clean it up further:
npm audit | grep -E "(High | Critical)" -B3 -A11 --color=always | grep -E '┌|│|├|└' --color=never
But this will lose the title, and the 'found vulnerabilities' at the bottom. I found it simplest to just run npm audit a couple times and get the bits I need appended to a file.
Ended up going with something like this:
npm audit | grep '===' --color=never > temp.txt
npm audit | grep -E "(High | Critical)" -B3 -A11 --color=never | grep -E '┌|│|├|└' --color=never >> temp.txt
npm audit | grep -E "(found|scanned packages)" --color=never >> temp.txt
cat temp.txt
Or as a catchy one liner (lol) that also removes the temp.txt file:
npm audit | grep '=== npm audit' --color=never > temp.txt; npm audit | grep -E "(High | Critical)" -B3 -A11 --color=never | grep -E '┌|│|├|└' --color=never >> temp.txt; npm audit | grep -E "(found|scanned packages)" --color=never >> temp.txt; cat temp.txt; rm temp.txt;
The line is ugly but is working well across a bunch of different repos, provided you only need the output in the terminal.
When outputting to a file, npm audit includes ansi color codes, that can't be turned off. And this is a problem for my reports! Sed can be used to remove them:
sed -i '' $'s,\x1b\\[[0-9;]*[a-zA-Z],,g' temp.txt
Just to count the High(s):
npm audit | grep 'High' | wc -l | rev
Put this line into your audit scripts:
"audit": "level=$(npm audit --parseable | grep -E 'high|critical' | wc -l | rev); [ $level == 0 ] && exit 0"
This code does check the output of npm audit. If there are no high or critical vulnerabilities the process will not exit with error.
This package might be what you are looking for:
https://www.npmjs.com/package/audit-filter
It lets you filter by advisory number, which is better than filtering by level.
$ cat .nsprc
{
"exceptions": [
"https://npmjs.com/advisories/532",
"https://npmjs.com/advisories/577"
]
}
Couple that with npm config for audit level and you're golden.

Can I tell which global node dependencies my app is using?

I'm deploying my first app to an AWS Elastic Beanstalk and it's failing to run as some of the dependencies on my dev machine have been installed locally and so are missing from my app'#s node_module folder than I have uploaded to AWS.
Is there a way to tell which dependencies my app is using that are not in package.json or node_modules?
Thanks
Try running this command on your terminal npm list -g --depth=0. It will return all globally installed modules and then you can manually check which ones you require on your project.
As a rough first round info gathering you can find all modules you require in your project with a simple grep:
grep -R --exclude-dir node_modules --include '*.js' require .
To extract just the module names and remove duplicates you can pipe the result through cut and sort:
grep -R --exclude-dir node_modules --include '*.js' require . |
cut -d '(' -f2 |
cut -d "'" -f2 |
sort -u
You can then filter the result and only print what's globally installed by comparing it with the output of npm list -g. The comm command comes in handy:
grep -R --exclude-dir node_modules --include '*.js' require . |
cut -d '(' -f2 |
cut -d "'" -f2 |
sort -u > required.txt
npm list -g --depth=0 2> /dev/null |
grep # |
cut -d ' ' -f2 |
cut -d# -f1 |
sort -u > global_install.txt
comm -1 -2 required.txt global_install.txt

How to remove all packages from node_modules that are listed in package.json

Basically I am looking for the "opposite" of npm prune or this SO question.
More specifically:
I am looking to clean up node_modules folder from all packages that are listed in my root package.json file. Sort of a fresh start before npm install.
The reason I don not want to simply rm -rf node_modules/ is because I have some local modules that I don't want to get deleted.
it isnt possible to remove at once all in your package.json you could write shell script to loop through. or you can check npm ls and then remove npm rm <name> or npm uninstall each manually. and to update it in package.json simultaneously npm rm <name> --save
A better approach would be to have your permanent (local) modules in a directory of a higher level:
-node_modules (local)
-my_project
|-node_modules (npm)
That way, when you wipe the node_modules directory, the outer local modules remain.
As pointed out by others, there's no native npm command to do that.
I've taken this answer and modified the command:
$ npm ls | grep -v 'npm#' | awk '/#/ {print $2}' | awk -F# '{print $1}' | xargs npm rm
This lists all of your local packages and then npm rm them one by one.
For convenience, if you want, you can add the following line to your package.json:
"scripts": {
"uninstall": "npm ls | grep -v 'npm#' | awk '/#/ {print $2}' | awk -F# '{print $1}' | xargs npm rm"
}
and then run
$ npm run uninstall

Command to remove all npm modules globally

Is there a command to remove all global npm modules? If not, what do you suggest?
The following command removes all global npm modules. Note: this does not work on Windows. For a working Windows version, see Ollie Bennett's Answer.
npm ls -gp --depth=0 | awk -F/ '/node_modules/ && !/\/npm$/ {print $NF}' | xargs npm -g rm
Here is how it works:
npm ls -gp --depth=0 lists all global top level modules (see the cli documentation for ls)
awk -F/ '/node_modules/ && !/\/npm$/ {print $NF}' prints all modules that are not actually npm itself (does not end with /npm)
xargs npm -g rm removes all modules globally that come over the previous pipe
For those using Windows, the easiest way to remove all globally installed npm packages is to delete the contents of:
C:\Users\username\AppData\Roaming\npm
You can get there quickly by typing %appdata%/npm in either the explorer, run prompt, or from the start menu.
I tried Kai Sternad's solution but it seemed imperfect to me. There was a lot of special symbols left after the last awk from the deps tree itself.
So, I came up with my own modification of Kai Sternad's solution (with a little help from cashmere's idea):
npm ls -gp --depth=0 | awk -F/node_modules/ '{print $2}' | grep -vE '^(npm|)$' | xargs -r npm -g rm
npm ls -gp --depth=0 lists all globally-installed npm modules in parsable format:
/home/leonid/local/lib
/home/leonid/local/lib/node_modules/bower
/home/leonid/local/lib/node_modules/coffee-script
...
awk -F/node_modules/ '{print $2}' extracts module names from paths, forming the list of all globally-installed modules.
grep -vE '^(npm|)$' removes npm itself and blank lines.
xargs -r npm -g rm calls npm -g rm for each module in the list.
Like Kai Sternad's solution, it'll only work under *nix.
sudo npm list -g --depth=0. | awk -F ' ' '{print $2}' | awk -F '#' '{print $1}' | sudo xargs npm remove -g
worked for me
sudo npm list -g --depth=0. lists all top level installed
awk -F ' ' '{print $2}' gets rid of ├──
awk -F '#' '{print $1}' gets the part before '#'
sudo xargs npm remove -g removes the package globally
For those using Powershell:
npm -gp ls --depth=0 | ForEach-Object { Get-Item $_ } | Where { $_.Name -ne 'npm' } | ForEach-Object { npm rm -g $_.Name }
To clear the cache:
npm cache clear
Just switch into your %appdata%/npm directory and run the following...
for package in `ls node_modules`; do npm uninstall $package; done;
EDIT: This command breaks with npm 3.3.6 (Node 5.0). I'm now using the following Bash command, which I've mapped to npm_uninstall_all in my .bashrc file:
npm uninstall `ls -1 node_modules | tr '/\n' ' '`
Added bonus? it's way faster!
https://github.com/npm/npm/issues/10187
How do you uninstall all dependencies listed in package.json (NPM)?
If you would like to remove all the packages that you have installed, you can use the npm -g ls command to find them, and then npm -g rm to remove them.
in windows go to
"C:\Users{username}\AppData\Roaming" directory and manually remove npm folder
If you have jq installed, you can go even without grep/awk/sed:
npm ls -g --json --depth=0 |
jq -r '.dependencies|keys-["npm"]|join("\n")' |
xargs npm rm -g
On Debian and derived you can install jq with:
sudo apt-get install jq
OS not specified by OP. For Windows, this script can be used to nuke the local and the user's global modules and cache.
I noticed on linux that the global root is truly global to the system instead of the given user. So deleting the global root might not be a good idea for a shared system. That aside, I can port the script to bash if interested.
For Windows, save to a cmd file to run.
#ECHO OFF
SETLOCAL EnableDelayedExpansion
SETLOCAL EnableExtensions
SET /A ecode=0
:: verify
SET /P conf="About to delete all global and local npm modules and clear the npm cache. Continue (y/[n])?
IF /I NOT "%conf%"=="y" (
ECHO operation aborted
SET /A ecode=!ecode!+1
GOTO END
)
:: wipe global and local npm root
FOR %%a IN ("" "-g") DO (
:: get root path into var
SET cmd=npm root %%~a
FOR /f "usebackq tokens=*" %%r IN (`!cmd!`) DO (SET npm_root=%%r)
:: paranoid
ECHO validating module path "!npm_root!"
IF "!npm_root:~-12!"=="node_modules" (
IF NOT EXIST "!npm_root!" (
ECHO npm root does not exist "!npm_root!"
) ELSE (
ECHO deleting "!npm_root!" ...
:: delete
RMDIR /S /Q "!npm_root!"
)
) ELSE (
ECHO suspicious npm root, ignoring "!npm_root!"
)
)
:: clear the cache
ECHO clearing the npm cache ...
call npm cache clean
:: done
ECHO done
:END
ENDLOCAL & EXIT /b %ecode%
All you done good job. This is combined suggestions in to one line code.
npm rm -g `npm ls -gp --depth=0 | awk -F/node_modules/ '{print $2}' | tr '/\n' ' '`
What is different? Uninstall will be done in single command like: npm rm -g *** *** ***
For yarn global
nano ~/.config/yarn/global/package.json
<Manually remove all packages from package.json>
yarn global add
Or, if you don't care about what is actually inside package.json
echo {} > ~/.config/yarn/global/package.json && yarn global add
This should apply to NPM too, but I am not exactly sure where NPM global is stored.
You can locate your all installed npm packages at the location:
C:\Users\username\AppData\Roaming\npm
and delete the content of npm which you want to remove.
If AppData is not showing, it means it is hidden and you can go to View in file explorer and checked the Hidden items then there you can see all the hidden folders.
For a more manual approach that doesn't involve an file explorers, doesn't care where the installation is, is very unlikely to break at a later date, and is 100% cross-platform compatible, and feels a lot safer because of the extra steps, use this one.
npm ls -g --depth=0
Copy output
Paste into favorite code editor (I use vsCode. Great multi-cursor editing)
Check for any packages you'd like to keep (nodemon, yarn, to name a few) Remove those lines
Remove every instance of +-- or other line decorators
Remove all the version information (eg '#2.11.4')
Put all items on same line, space separated
Add npm uninstall -g to beginning of that one line.
Mine looks like npm uninstall -g #angular/cli #vue/cli express-generator jest mocha typescript bindings nan nodemon yarn, but I didn't install many packages globally on this machine.
Copy line
Paste in terminal, hit enter if not already added from the copy/paste
Look for any errors in the terminal.
Check npm ls -g to make sure it's complete. If something got reinstalled, rinse and repeat
The other cli-only approaches are great for computer administrators doing something for 100 near-identical computers at once from the same ssh, or maybe a Puppet thing. But if you're only doing this once, or even 5 times over the course of a year, this is much easier.
For Windows:
rmdir /s /q "%appdata%/npm"
Well if you are on windows, and want to remove/uninstall all node_modules then you need to do following steps.
Go to windows command prompt
Navigate to node_modules directory (Not inside node_modules folder)
Type below command and give it for 1-2 minutes it will uninstall all directories inside node_module
rmdir /s /q node_modules
Hope this will help some one on windows
if you have Intellij Webstorm you can use its built-in graphical package manager.
open it as root and create an emtpy project. go to
File > Settings > Language and Frameworks > Node.js and NPM
there you will see all the installed packages. Uninstalling is easy, you can select and deselect any package you want to uninstall, Ctrl+a woks as well.
Simply use below for MAC,
sudo rm -rf
/usr/local/{bin/{node,npm},lib/node_modules/npm,lib/node,share/man//node.}
npm ls -gp | awk -F/ '/node_modules/&&!/node_modules.*node_modules/&&!/npm/{print $NF}' | xargs npm rm -g
Use this code to uninstall any package:
npm rm -g <package_name>
Since this is the top answer in search I'm posting this here as it was the solution I used in the past to clean the computer switching laptops.
cd ~/Documents # or where you keep your projects
find . -name "node_modules" -exec rm -rf '{}' +
source: https://winsmarts.com/delete-all-node-modules-folders-recursively-on-windows-edcc9a9c079e
Here is a more elegant solution that I tried where I let npm do all the work for me.
# On Linux Mint 19.1 Cinnamon
# First navigate to where your global packages are installed.
$ npm root # returns /where/your/node_modules/folder/is
$ cd /where/your/node_modules/folder/is # i.e for me it was cd /home/user/.npm-packages/lib/node_modules
Then if you do npm uninstall or npm remove these modules will be treated as if they were normal dependencies of a project. It even generates a package-lock.json file when it is done:
$ npm remove <package-name> # you may need sudo if it was installed using sudo
If you have have MSYS for Windows:
rm -rf ${APPDATA//\\/\/}/npm
The npm README.md states:
If you would like to remove all the packages that you have installed,
then you can use the npm ls command to find them, and then npm rm to
remove them.
To remove cruft left behind by npm 0.x, you can use the included
clean-old.sh script file. You can run it conveniently like this:
npm explore npm -g -- sh scripts/clean-old.sh
In macOS, I believe you can simply delete the .npm-global folder in your User directory.
.npm and .npm-global folders in macOS User directory:
npm list -g
will show you the location of globally installed packages.
If you want to output them to a file:
npm list -g > ~/Desktop/npmoutputs.txt
npm rm -g
will remove them
sudo npm uninstall npm -g
Or, if that fails, get the npm source code, and do:
sudo make uninstall
To remove everything npm-related manually:
rm -rf /usr/local/{lib/node{,/.npm,_modules},bin,share/man}/npm*
sed solution
npm -gp ls | sed -r '/npm$|(node_modules.*){2,}/d; s:.*/([^/]+)$:\1:g' | xargs npm rm -g
Just put in your console:
sudo npm list -g --depth=0. | awk -F ' ' '{print $2}' | awk -F '#' '{print $1}' | sudo xargs npm remove -g
Its work for me...

Resources