I am an extreme beginner with Python and its libraries and installation in general. I want to make an extremely simple google search web scraping tool. I was told to use Requests and BeautifulSoup. I have installed python3 on my Mac by using brew install python3 and I am wondering how to get those two libraries
I googled around and many results said that by doing brew install python3 it will automatically install pip so I can use something like pip install requests but it says pip: command not found.
by running python3 --version it says Python 3.7.4
Since you're running with Python3, not Python (which usually refers to 2.7), you should try using pip3.
pip on the other hand, is the package installer for Python, not Python3.
Related
libdoc2testbench is a tool of Robot Framework which supports importing test results to imbus testbench. Due to Robot Framework documentation, it is to be installed by
pip install robotframework-libdoc2testbench
I want to install it on Ubuntu 18.04; there I get the error:
Could not find a version that satisfies the requirement robotframework-libdoc2testbench (from versions: )
No matching distribution found for robotframework-libdoc2testbench
Best regards
Gerhard
You are probably calling pip from Python 2.7.
Make sure you use Python 3, for example with:
python3 -m pip install robotframework-libdoc2testbench
EDIT: You can download the .wheel or source tar from pypi.org and install with pip pointing to it (and then if needed, download other required packages).
However the solution to your problem is the Python version. From the project page, we see it needs Python 3.7.
I've been looking around and I can't seem to find an answer on how to run scrapyd in Python 3 and above. When I run it it keeps defaulting to python 2.7, though I recall reading in the docs or elsewhere that scrapyd supports python3:
https://scrapyd.readthedocs.io/en/stable/news.html#id1
Running on an AWS Ubuntu 18.04 server. What am I doing wrong? How can I change to Python 3.4 and above?
I'm having problems because I'm using abstract inheritance in my spiders which python 2.7 doesn't support.
Edit: I'm able to run Scrapyd in Python 3 using a virtual environment, but how would you do it without one?
Simplest solution is use a virtual enviroment.
Since you are using Ubuntu and you can make it work using a venv I assume you installed scrapy using pip instead of pip3. Since Py2.7 is the stardard reference to python in Ubuntu, when running pip it will execute pip for py2 instead of py3.
Another way would be uninstalling pip uninstall scrapy and installing it again using pip3 pip3 install scrapy. This should fix the reference to scrapy. If by any chance you also use scrapy in py2.7 and therefore can't uninstall, then you have another reason to use venv.
Some things you should know before I ask my question:
I am utterly new to both Linux & Python, and have a hard time understanding official documentation and technical answers (but have a burning desire to deeply understand both)
I am running elementary OS 0.4.1 Loki
My Python 3 version is 3.5.2. When I search the online documentation on the venv module for python 3.5.2, I get the documentation for the 3.5.6 version. I don't understand why there is no documentation for the .2 version.
So, here's my problem. I was trying to create a virtual environment using venv and proceeded thusly:
According to Python's 3.5.6 venv module documentation, a virtual environment is created using the command pyvenv /path/to/new/virtual/environment. I tried that command and got:
The program 'pyvenv' is currently not installed. You can install it by typing: sudo apt install python3-venv
I then searched documentation for newer Python versions and tried the new venv command python3 -m venv /path/to/new/virtual/environment and got the following result:
The virtual environment was not created successfully because ensurepip is not available. On Debian/Ubuntu systems, you need to install the python3-venv package using the following command. apt-get install python3-venv
In both cases, the solution seems to be to install python3-venv. My question is: What exactly am I installing by installing python3-venv: Isn't venv already part of the Standard Library? Furthermore, why do I have to install it via apt-get if it is a Python module? It is my understanding that standard library modules are imported, not installed; and that modules external to the standard library are installed via pip. Related to this, why is ensurepip not available?
Second part of my question: if installing python3-venv is the way to go, what is the proper way to create a virtual environment using venv in Python 3.5.2: pyvenv my_virtual_environment or python3 -m venv my_virtual_environment?
Don't worry about the documentation not matching the micro version number – increments in that place are only for bugfixes, so the documentation stays the same.
Your question is interesting since venv is indeed not an optional module. My guess is that the Python version shipped with your OS (or that you installed yourself) seems to come with a stripped down or no standard library. For instance, the python3.5-minimal package doesn't appear to have it. Does your Python have the other modules in the standard library?
Edit: See also this question.
Installation can be described as "putting files onto your computer, in the right place". Importing a module, however, means that you tell Python to make available some functionality. To import a module, it must be installed (e.g. in /usr/lib/python3.5 for Python 3 on my computer), and one method for installing additional modules is via apt.
The python3 -m venv my_virtual_environment method should work in 3.5 as well and is the future-proof version, so you should probably go with that.
I am trying to run some code using ortools on a python environment. I did not have troubles on a windows machine but I am having problems on mac (10.12.6). if in my virtual environment I run
pip freeze
or
conda list
ortools appears in my list of installed packages. But if I try to use it
ipython
from ortools.linear_solver import pywrapplp
I get an error saying that there's no module named ortools. If I go to
mac/anaconda3/envs/nameenv/lib/python3.6/site-packages I do have a folder called ortools with some python files including pywrapplp. Do you know what I am doing wrong ?
EDIT
following request from coments:
import os
os.getcwd()
returns '/Users/imac'
which ipython
/anaconda3/bin/ipython
Installing ortools is a bit of a headache. It was some days ago, I think I finally made it with
easy_install ortools
I think it is a problem with the path. I guess because I did not install it with conda it does not find the package. I got around writting:
sys.path.append('/anaconda3/envs/env_name/lib/python3.6/site-packages/')
at the begining of my ipynb. That way I can run ortools.
You could have several python interpreter installed (python2 and python3)
So if you want to use it with ipython, which seems to be bind on python3 in your case.
First check if you have pypi package ortools installed
ipython -m pip show ortools
If you got an error it means the package is not installed.
so you can easily install it using:
ipython -m pip install --user ortools
note: We provide Pypi ortools package (64 bits) for Manylinux, Windows and MacOS.
You can also rebuild it from source https://developers.google.com/optimization/introduction/installing/source
It bugs me theese simple things:
I notice that my installed lxml can't be found from my python3.1 shell.
There is no problem in python2.6 shell
So my question now is - how can I install lxml to python3.1? sudo pip install lxml just tells me that lxml is already installed.
I know I could properbly use virtualenv to do this - but quite honestely I am just trying to learn python in the most simple manner posible. I therefore don't think I need "package lockdown" the way I (mis)understands virtualenv provides. Anything that can keep the confusion at bay and let me focus on the programming :-)
I could of course just uninstall python2.6 but I guess all hell would break loose on my debian box then.
Thanks in advance
Each version of Python is independent of the others. This goes for everything including pip commands. You need to install pip once per each version, and you then get one pip command per version.
For a longer explanation, see my blog post on the topic.