installing requirements with pip inside a python script - python-3.x

I need to install packages using Python that is supposed to perform the same as the following command line:
pip install -r requirements.txt
note: I need to run this command on a virtual environment
I didn't succeed to do that with os.system.

import subprocess
import sys
def install(package):
subprocess.check_call([sys.executable, "-m", "pip", "install", package])
install('xlrd') #example - this will install xlrd package
Use sys.executable to ensure that you will call the same pip associated with the current runtime.

Related

I get "syntax error" while installing numpy [duplicate]

I'm trying to use pip to install a package. I try to run pip install from the Python shell, but I get a SyntaxError. Why do I get this error? How do I use pip to install the package?
>>> pip install selenium
^
SyntaxError: invalid syntax
pip is run from the command line, not the Python interpreter. It is a program that installs modules, so you can use them from Python. Once you have installed the module, then you can open the Python shell and do import selenium.
The Python shell is not a command line, it is an interactive interpreter. You type Python code into it, not commands.
Use the command line, not the Python shell (DOS, PowerShell in Windows).
C:\Program Files\Python2.7\Scripts> pip install XYZ
If you installed Python into your PATH using the latest installers, you don't need to be in that folder to run pip
Terminal in Mac or Linux
$ pip install XYZ
As #sinoroc suggested correct way of installing a package via pip is using separate process since pip may cause closing a thread or may require a restart of interpreter to load new installed package so this is the right way of using the API: subprocess.check_call([sys.executable, '-m', 'pip', 'install', 'SomeProject']) but since Python allows to access internal API and you know what you're using the API for you may want to use internal API anyway eg. if you're building own GUI package manager with alternative resourcess like https://www.lfd.uci.edu/~gohlke/pythonlibs/
Following soulution is OUT OF DATE, instead of downvoting suggest updates. see https://github.com/pypa/pip/issues/7498 for reference.
UPDATE: Since pip version 10.x there is no more get_installed_distributions() or main method under import pip instead use import pip._internal as pip.
UPDATE ca. v.18 get_installed_distributions() has been removed. Instead you may use generator freeze like this:
from pip._internal.operations.freeze import freeze
print([package for package in freeze()])
# eg output ['pip==19.0.3']
If you want to use pip inside the Python interpreter, try this:
import pip
package_names=['selenium', 'requests'] #packages to install
pip.main(['install'] + package_names + ['--upgrade'])
# --upgrade to install or update existing packages
If you need to update every installed package, use following:
import pip
for i in pip.get_installed_distributions():
pip.main(['install', i.key, '--upgrade'])
If you want to stop installing other packages if any installation fails, use it in one single pip.main([]) call:
import pip
package_names = [i.key for i in pip.get_installed_distributions()]
pip.main(['install'] + package_names + ['--upgrade'])
Note: When you install from list in file with -r / --requirement parameter you do NOT need open() function.
pip.main(['install', '-r', 'filename'])
Warning: Some parameters as simple --help may cause python interpreter to stop.
Curiosity: By using pip.exe you actually use python interpreter and pip module anyway. If you unpack pip.exe or pip3.exe regardless it's python 2.x or 3.x, inside is the SAME single file __main__.py:
# -*- coding: utf-8 -*-
import re
import sys
from pip import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
sys.exit(main())
To run pip in Python 3.x, just follow the instructions on Python's page: Installing Python Modules.
python -m pip install SomePackage
Note that this is run from the command line and not the python shell (the reason for syntax error in the original question).
I installed python and when I run pip command it used to throw me an error like shown in pic below.
Make Sure pip path is added in environmental variables. For me, the python and pip installation path is::
Python: C:\Users\fhhz\AppData\Local\Programs\Python\Python38\
pip: C:\Users\fhhz\AppData\Local\Programs\Python\Python38\Scripts
Both these paths were added to path in environmental variables.
Now Open a new cmd window and type pip, you should be seeing a screen as below.
Now type pip install <<package-name>>. Here I'm installing package spyder so my command line statement will be as pip install spyder and here goes my running screen..
and I hope we are done with this!!
you need to type it in cmd not in the IDLE. becuse IDLE is not an command prompt if you want to install something from IDLE type this
>>>from pip.__main__ import _main as main
>>>main(#args splitted by space in list example:['install', 'requests'])
this is calling pip like pip <commands> in terminal. The commands will be seperated by spaces that you are doing there to.
If you are doing it from command line,
try -
python -m pip install selenium
or (for Python3 and above)
python3 -m pip install selenium

Can import a module inside python shell but cannot import the same module inside a python script

I run the following commands:
conda create -n my-environment python=3.7
conda activate my-environment
conda install pytorch-gpu
Then, I type python3 and run the following code inside the python3 shell:
import torch
The above command runs successfully. Then, I create a file "myfile.py" with the following content:
import torch
In the linux command line I run
python3 myfile.py
I receive the following error:
ModuleNotFoundError: PyTorch isn't installed. To install PyTorch, run 'pip install torch'

Why do I get error ModuleNotFoundError: No module named 'azure.storage' during execution of my azure python function?

I am currently deploying using Python 3.9 to install the depencies and setup the azure function to also use Python 3.9
Here is the requirements file I am currently using
msrest==0.6.16
azure-core==1.6.0
azure-functions
azure-storage-blob==12.5.0
pandas
numpy
pyodbc
requests==2.23.0
snowflake-connector-python==2.4.0
azure.identity
azure.keyvault.secrets==4.1.0
azure.servicebus==0.50.3
pyarrow==3.0.0
stopit==1.1.2
The bash script to install the required dependencies during the build definition
python3.9 -m venv worker_venv
source worker_venv/bin/activate
pip3.9 install setuptools
pip3.9 install --upgrade pip
pip3.9 install -r requirements.txt
My python scripts are using the following imports
import logging
from azure.storage.blob import *
import datetime
import azure.functions as func
import json
The most helpful article I could find was
https://learn.microsoft.com/en-us/azure/azure-functions/recover-python-functions?tabs=coretools
As a work-around I tried the remote build option using command:
func azure functionapp publish . Interestingly enough when I use that command the error disappears during execution and the function works as expected. I would like to enable the automatic build and deploy process again which did work until I needed to include the pyarrow library.
Any suggestions on what I am doing incorrectly?
I was able to download the content which was generated by the remote build. I then discovered it has a .python_packages folder. I now updated my install dependencies bash script to the example below which mimics how the remote build creates the the .python_packages .In essence I am copying the downloaded packages from worker_venv/lib64/python3.9/site-packages to .python_packages/lib/site-packages. My function is now executing without any errors anymore.
python3.9 -m venv worker_venv
source worker_venv/bin/activate
pip3.9 install setuptools
pip3.9 install --upgrade pip
pip3.9 install -r requirements.txt
mkdir .python_packages
cd .python_packages
mkdir lib
cd lib
mv ../../worker_venv/lib64/python3.9/site-packages .

No module named tqdm after a pip install

I'm trying to use tqdm as following:
from tqdm import tqdm
it tells me: " No module named tqdm "
i installed it with:
pip install tqdm
It has been installed successfully but i still have the same message: " No Module named tqdm ".
I see that you've got this tagged as python 3. Could be your system's default installation of python is version 2.
Try sudo pip3 install tqdm.
Make sure you're running your script in version 3. python3 ./example.py
make sure you are not executing your .py file in a virtual environment.
If you are, all you have to do is activate the virtual environment and install it there.
change directory to where you have installed your virtual environment
cd .../bin
# this activates virtual environment
source activate
then
#install through pip
pip install tqdm

Pip packages not importing

So I am unable to use the package PyMySQL from pip.
I've tried installing it with pip, pip3, and easy install.
I've tried running the file in python2 and python3.
I've tried using
python -m pip install ...
I'm on mac.
My ~/.bash_profile has the following
export PATH="/usr/local/sbin:$PATH"
export PATH="/Users/taylorcochran/anaconda2/bin:$PATH"
export PATH="/Library/Frameworks/Python.framework/Versions/3.6/bin:${PATH}"
export PATH="/usr/local/cellar/openvpn:$Path"
export PATH="/Users/taylorcochran/anaconda3/bin:$PATH"
export PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/X11/bin"
export PYTHONPATH="/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages:$PATH"
I've looked at similar problem across StackOverflow but none of the suggested solutions produce any results.
Additionally, when I try to add # PATH="/usr/local/lib"
to my path it breaks the built in commands like ls and cd
Edit:
I'm running the script using python3 Test.py which simply contains the line import PyMySQL
Which outputs: ModuleNotFoundError: No module named 'PyMySQL'
Tried:
pip3 install --install-option="--prefix=/Library/Frameworks/Python.framewor k/Versions/3.6" PyMySQL
Outputs:
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/commands/install.py:194: UserWarning: Disabling all use of wheels due to the use of --build-options / --global-options / --install-options. cmdoptions.check_install_build_global(options) Requirement already satisfied: PyMySQL in /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages
And still doesn't let my import the module
You have Python 3.6 in the PYTHONPATH so you should go with pip3. Try to use the following command:
pip install --install-option="--prefix=$PREFIX_PATH" package_name
where $PREFIX_PATH in your case should be /Library/Frameworks/Python.framework/Versions/3.6
When you use Python located in /Library/Frameworks/Python.framework/Versions/3.6/bin the module library should be available.

Resources