pip install -r all dependencies from the file except one - python-3.x

Is there a way with single command to install all dependencies in the file except one specified in the command?
I am looking for something like this:
pip install -r req_file.txt --except unwanted_package

Just do with a shell trick:
pip install -r <(grep -v unwanted_package requirements.txt)
This is process substitution.

In my case the process substitution didn't work inside a jenkins container.
Alternative to process substitution could be
grep -v "unwanted_package_1|unwanted_package_2" requirements.txt | xargs pip install

Related

Trying to use dpkg within a folder only on files with the keyword 'mono' in the title

I'm currently trying to install mono using dpkg, and all the other files within the same folder using apt-get, I know I need to use some form of this:
sudo grep 'mono' | dpkg -R --install >/dev/null
however there are too many unknowns for me to complete it and fill in whatever blanks there may be, any help would be greatly appreciated!
Try this:
ls | grep "mono" | sudo xargs dpkg -R --install >/dev/null
The ls will only give files from the current directory. You could also use ls -d *mono* instead of the ls and grep, but I think the ls and grep is easier to understand
The grep is as you had, but now has input from the ls to grep on. You can try ls | grep "mono" to see what files it selects.
Then the sudo is moved to the dpkg part of the script to make dpkg run as root. The way you had it grep runs as root and dpkg as your user
The xargs will take whatever input you had, and put it after the next command. It will take command line length limits in account and execute multiple dpkg commands if the command line gets too big. Note that if your files have spaces in their names, xargs will see the space as start of a new file and you will have problems. There are solutions to that, but really, the easiest solution is to have no files with spaces.
In this example, lets say there are 2 files from the grep "mono1.deb and "mono2.deb" the command executed will be dpkg -R --install mono1.deb mono2.deb. If for some reason you want only one deb per dpkg execution you can change it to ...xargs -n1 dpkg... and it will run dpkg -R --install mono1.deb and also dpkg -R --install mono2.deb
The >/dev/null make sure you won't get any output. Note that you will still get the errors though!

How to install all existing global NPM packages into local directory?

This is not similar nor does it answer question: copy npm global installed package to local
Basically I want to tell NPM, look at what I have installed globally, and install that locally.
I'm not interested in the symlinks, but if above is not possible, could I use a single symlink to node_modules instead of a symlink for each package?
You can parse the output of node ls -g --depth 0 and npm install the resulting list of packages.
npm i -S -dd $(npm ls -g --depth 0 | tail -n +2 | cut -c 5- | tr '\n' ' ')
Run this command in the directory of the package that you wish to install global packages to.
Explanation:
npm i -S -dd Shorthand for npm install --save --verbose. The -S is unnecessary on recent npm versions that save installed packages to package.json by default.
npm ls -g --depth 0 List first-level global packages.
tail -n +2 Remove the first line from the output.
cut -c 5- Remove the first four characters from each line in the output.
tr '\n' ' ' Combine each line to place all packages on one line, separated by spaces.

Syntax Error: Multiline bash command in dockerfile RUN statement

I have this Dockerfile, to which I am echoing the following line:
echo $"RUN cat file | while read pkg \
do\
sudo apt-get install -qy $pkg \
done" >> Dockerfile
Now, when docker executes this line, I get the following error:
/bin/sh: -c: line 1: syntax error: unexpected end of file
The command '/bin/sh -c cat autobuild/buildenv_packages | while read pkg do sudo apt-get install -qy done' returned a non-zero code: 1
I know there is something small and syntactical I am missing, but I am unable to figure it out. Notice that the $pkg variable in the apt-get install statement isn't in the error.
Any assistance would be appreciated!
The backslashes in your code only prevent the string from containing literal newlines; they are not written to the Dockerfile. Without a newline (or a semicolon) before do, the while condition never ends; you just have a giant list of arguments for the command read.
echo 'RUN while read pkg; do sudo apt-get install -qy "$pkg"; done < file' >> Dockerfile

Install npm package without dependencies

I am looking for best solution how to install npm package without it's dependencies described in it's package.json file.
The goal is to change dependencies versions before install package. I can do it manually for one package by downloading source, but if you have many nested dependencies it becomes a problem.
Here's a shell script that seems to get you the extracted files you need.
#!/bin/bash
package="$1"
version=$(npm show ${package} version)
archive="${package}-${version}.tgz"
curl --silent --remote-name \
"https://registry.npmjs.org/${package}/-/${archive}"
mkdir "${package}"
tar xzf "${archive}" --strip-components 1 -C "${package}"
rm "${archive}"
Save it as npm_download.sh and run it with the name of the package you want:
./npm_download.sh pathval
Please check simialr quesition on stackexchange: https://unix.stackexchange.com/questions/168034/is-there-an-option-to-install-an-npm-package-without-dependencies
My solution was to rename package.json to package.bak before the install, then reverting rename afterwards:
RENAME package.json package.bak
npm install <package_name> --no-save
RENAME package.bak package.json
I supplemented the above script to allow the specification of multiple packages, avoid the temporary downloaded file and to install the packages straight to node_modules:
#!/bin/sh
# filename suggestion: `npm-i-no-deps`
while [ $# -gt 0 ]
do
package="$1"
version=$(npm show ${package} version)
mkdir -p "node_modules/${package}"
echo "Installing ${package}-${version}"
curl --silent "https://registry.npmjs.org/${package}/-/${package}-${version}.tgz" | tar xz --strip-components 1 -C "node_modules/${package}"
shift
done
I found this script useful for situations where dependencies are too strict with their dependency requirements, you can install deps you know work ok without adding them to your package.json until the upstream dependency is updated.
I adapted the script from mxcl to handle scoped packages (#user/package)
#!/bin/sh
# filename suggestion: `npm-i-no-deps`
while [ $# -gt 0 ]
do
package="$1"
version=$(npm show ${package} version)
mkdir -p "node_modules/${package}"
echo "Installing ${package}-${version}"
packagename=$(echo $package | sed 's/#.*\///g')
curl --silent "https://registry.npmjs.org/${package}/-/${packagename}-${version}.tgz" | tar xz --strip-components 1 -C "node_modules/${package}"
shift
done
Warning: mac has a weird version of sed, it should work but I can't guarantee.

Command to remove all npm modules globally

Is there a command to remove all global npm modules? If not, what do you suggest?
The following command removes all global npm modules. Note: this does not work on Windows. For a working Windows version, see Ollie Bennett's Answer.
npm ls -gp --depth=0 | awk -F/ '/node_modules/ && !/\/npm$/ {print $NF}' | xargs npm -g rm
Here is how it works:
npm ls -gp --depth=0 lists all global top level modules (see the cli documentation for ls)
awk -F/ '/node_modules/ && !/\/npm$/ {print $NF}' prints all modules that are not actually npm itself (does not end with /npm)
xargs npm -g rm removes all modules globally that come over the previous pipe
For those using Windows, the easiest way to remove all globally installed npm packages is to delete the contents of:
C:\Users\username\AppData\Roaming\npm
You can get there quickly by typing %appdata%/npm in either the explorer, run prompt, or from the start menu.
I tried Kai Sternad's solution but it seemed imperfect to me. There was a lot of special symbols left after the last awk from the deps tree itself.
So, I came up with my own modification of Kai Sternad's solution (with a little help from cashmere's idea):
npm ls -gp --depth=0 | awk -F/node_modules/ '{print $2}' | grep -vE '^(npm|)$' | xargs -r npm -g rm
npm ls -gp --depth=0 lists all globally-installed npm modules in parsable format:
/home/leonid/local/lib
/home/leonid/local/lib/node_modules/bower
/home/leonid/local/lib/node_modules/coffee-script
...
awk -F/node_modules/ '{print $2}' extracts module names from paths, forming the list of all globally-installed modules.
grep -vE '^(npm|)$' removes npm itself and blank lines.
xargs -r npm -g rm calls npm -g rm for each module in the list.
Like Kai Sternad's solution, it'll only work under *nix.
sudo npm list -g --depth=0. | awk -F ' ' '{print $2}' | awk -F '#' '{print $1}' | sudo xargs npm remove -g
worked for me
sudo npm list -g --depth=0. lists all top level installed
awk -F ' ' '{print $2}' gets rid of ├──
awk -F '#' '{print $1}' gets the part before '#'
sudo xargs npm remove -g removes the package globally
For those using Powershell:
npm -gp ls --depth=0 | ForEach-Object { Get-Item $_ } | Where { $_.Name -ne 'npm' } | ForEach-Object { npm rm -g $_.Name }
To clear the cache:
npm cache clear
Just switch into your %appdata%/npm directory and run the following...
for package in `ls node_modules`; do npm uninstall $package; done;
EDIT: This command breaks with npm 3.3.6 (Node 5.0). I'm now using the following Bash command, which I've mapped to npm_uninstall_all in my .bashrc file:
npm uninstall `ls -1 node_modules | tr '/\n' ' '`
Added bonus? it's way faster!
https://github.com/npm/npm/issues/10187
How do you uninstall all dependencies listed in package.json (NPM)?
If you would like to remove all the packages that you have installed, you can use the npm -g ls command to find them, and then npm -g rm to remove them.
in windows go to
"C:\Users{username}\AppData\Roaming" directory and manually remove npm folder
If you have jq installed, you can go even without grep/awk/sed:
npm ls -g --json --depth=0 |
jq -r '.dependencies|keys-["npm"]|join("\n")' |
xargs npm rm -g
On Debian and derived you can install jq with:
sudo apt-get install jq
OS not specified by OP. For Windows, this script can be used to nuke the local and the user's global modules and cache.
I noticed on linux that the global root is truly global to the system instead of the given user. So deleting the global root might not be a good idea for a shared system. That aside, I can port the script to bash if interested.
For Windows, save to a cmd file to run.
#ECHO OFF
SETLOCAL EnableDelayedExpansion
SETLOCAL EnableExtensions
SET /A ecode=0
:: verify
SET /P conf="About to delete all global and local npm modules and clear the npm cache. Continue (y/[n])?
IF /I NOT "%conf%"=="y" (
ECHO operation aborted
SET /A ecode=!ecode!+1
GOTO END
)
:: wipe global and local npm root
FOR %%a IN ("" "-g") DO (
:: get root path into var
SET cmd=npm root %%~a
FOR /f "usebackq tokens=*" %%r IN (`!cmd!`) DO (SET npm_root=%%r)
:: paranoid
ECHO validating module path "!npm_root!"
IF "!npm_root:~-12!"=="node_modules" (
IF NOT EXIST "!npm_root!" (
ECHO npm root does not exist "!npm_root!"
) ELSE (
ECHO deleting "!npm_root!" ...
:: delete
RMDIR /S /Q "!npm_root!"
)
) ELSE (
ECHO suspicious npm root, ignoring "!npm_root!"
)
)
:: clear the cache
ECHO clearing the npm cache ...
call npm cache clean
:: done
ECHO done
:END
ENDLOCAL & EXIT /b %ecode%
All you done good job. This is combined suggestions in to one line code.
npm rm -g `npm ls -gp --depth=0 | awk -F/node_modules/ '{print $2}' | tr '/\n' ' '`
What is different? Uninstall will be done in single command like: npm rm -g *** *** ***
For yarn global
nano ~/.config/yarn/global/package.json
<Manually remove all packages from package.json>
yarn global add
Or, if you don't care about what is actually inside package.json
echo {} > ~/.config/yarn/global/package.json && yarn global add
This should apply to NPM too, but I am not exactly sure where NPM global is stored.
You can locate your all installed npm packages at the location:
C:\Users\username\AppData\Roaming\npm
and delete the content of npm which you want to remove.
If AppData is not showing, it means it is hidden and you can go to View in file explorer and checked the Hidden items then there you can see all the hidden folders.
For a more manual approach that doesn't involve an file explorers, doesn't care where the installation is, is very unlikely to break at a later date, and is 100% cross-platform compatible, and feels a lot safer because of the extra steps, use this one.
npm ls -g --depth=0
Copy output
Paste into favorite code editor (I use vsCode. Great multi-cursor editing)
Check for any packages you'd like to keep (nodemon, yarn, to name a few) Remove those lines
Remove every instance of +-- or other line decorators
Remove all the version information (eg '#2.11.4')
Put all items on same line, space separated
Add npm uninstall -g to beginning of that one line.
Mine looks like npm uninstall -g #angular/cli #vue/cli express-generator jest mocha typescript bindings nan nodemon yarn, but I didn't install many packages globally on this machine.
Copy line
Paste in terminal, hit enter if not already added from the copy/paste
Look for any errors in the terminal.
Check npm ls -g to make sure it's complete. If something got reinstalled, rinse and repeat
The other cli-only approaches are great for computer administrators doing something for 100 near-identical computers at once from the same ssh, or maybe a Puppet thing. But if you're only doing this once, or even 5 times over the course of a year, this is much easier.
For Windows:
rmdir /s /q "%appdata%/npm"
Well if you are on windows, and want to remove/uninstall all node_modules then you need to do following steps.
Go to windows command prompt
Navigate to node_modules directory (Not inside node_modules folder)
Type below command and give it for 1-2 minutes it will uninstall all directories inside node_module
rmdir /s /q node_modules
Hope this will help some one on windows
if you have Intellij Webstorm you can use its built-in graphical package manager.
open it as root and create an emtpy project. go to
File > Settings > Language and Frameworks > Node.js and NPM
there you will see all the installed packages. Uninstalling is easy, you can select and deselect any package you want to uninstall, Ctrl+a woks as well.
Simply use below for MAC,
sudo rm -rf
/usr/local/{bin/{node,npm},lib/node_modules/npm,lib/node,share/man//node.}
npm ls -gp | awk -F/ '/node_modules/&&!/node_modules.*node_modules/&&!/npm/{print $NF}' | xargs npm rm -g
Use this code to uninstall any package:
npm rm -g <package_name>
Since this is the top answer in search I'm posting this here as it was the solution I used in the past to clean the computer switching laptops.
cd ~/Documents # or where you keep your projects
find . -name "node_modules" -exec rm -rf '{}' +
source: https://winsmarts.com/delete-all-node-modules-folders-recursively-on-windows-edcc9a9c079e
Here is a more elegant solution that I tried where I let npm do all the work for me.
# On Linux Mint 19.1 Cinnamon
# First navigate to where your global packages are installed.
$ npm root # returns /where/your/node_modules/folder/is
$ cd /where/your/node_modules/folder/is # i.e for me it was cd /home/user/.npm-packages/lib/node_modules
Then if you do npm uninstall or npm remove these modules will be treated as if they were normal dependencies of a project. It even generates a package-lock.json file when it is done:
$ npm remove <package-name> # you may need sudo if it was installed using sudo
If you have have MSYS for Windows:
rm -rf ${APPDATA//\\/\/}/npm
The npm README.md states:
If you would like to remove all the packages that you have installed,
then you can use the npm ls command to find them, and then npm rm to
remove them.
To remove cruft left behind by npm 0.x, you can use the included
clean-old.sh script file. You can run it conveniently like this:
npm explore npm -g -- sh scripts/clean-old.sh
In macOS, I believe you can simply delete the .npm-global folder in your User directory.
.npm and .npm-global folders in macOS User directory:
npm list -g
will show you the location of globally installed packages.
If you want to output them to a file:
npm list -g > ~/Desktop/npmoutputs.txt
npm rm -g
will remove them
sudo npm uninstall npm -g
Or, if that fails, get the npm source code, and do:
sudo make uninstall
To remove everything npm-related manually:
rm -rf /usr/local/{lib/node{,/.npm,_modules},bin,share/man}/npm*
sed solution
npm -gp ls | sed -r '/npm$|(node_modules.*){2,}/d; s:.*/([^/]+)$:\1:g' | xargs npm rm -g
Just put in your console:
sudo npm list -g --depth=0. | awk -F ' ' '{print $2}' | awk -F '#' '{print $1}' | sudo xargs npm remove -g
Its work for me...

Resources