Install apache airflow in python 3.7.3 - python-3.x

I have python2.7 and 3.7 both installed on my Mac machine. I have set python3 default in .bash_profile file using
alias python=python3
alias pip=pip3
when i hit the below command
pip install apache-airflow
it says
Requirement already satisfied: apache-airflow in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (1.10.3)
But when i hit the command "airflow version", it says command not found.
So how to install apache airflow for python3

You might need setup adjust the path environment variable accordingly.
My way to solve such environment case is to run applications in docker containers as micro service.

Related

Apache Superset : bad interpreter: No such file or directory

I created a new environment to run a superset dashboard, super_dash, I get an error bad interpreter: No such file or directory when I run it.Not able to do any activities.
I tried
To reinstall `pip install apache-superset', but the results seems to be same.
Install `pip install apache-superset' in a different environment, the result is still the same.
However when I run superset in my base environment there is no issue.
Please find the environment details
(super_dash) hashi-MacBook-Air:~ hashi$ which python
/Users/hashi/DG/env/super_dash/bin/python
(super_dash) hashi-MacBook-Air:~ hashi$ which python3
/Users/hashi/DG/env/super_dash/bin/python3
(super_dash) hashi-MacBook-Air:~ hashi$ which pip
/Users/hashi/DG/env/super_dash/bin/pip
Base version
(base) hashi-MacBook-Air:~ hashi$ which python
/Library/Frameworks/Python.framework/Versions/2.7/bin/python
(base) hashi-MacBook-Air:~ hashi$ which python3
/Library/Frameworks/Python.framework/Versions/3.8/bin/python3
(base) hashi-MacBook-Air:~ hashi$ which pip
/Library/Frameworks/Python.framework/Versions/2.7/bin/pip
Mac : macOS Catalina 10.15.7
Several issues available in the net
Wrong pointing of Python version
Issue with improper pip version / linkage
I basically understand from all these is that issue with the way python was installed.
Tried many solutions with no result.
Can anyone give me more clarity on this ?
I finally resolved the issue.
Root Cause : Multiple Python Versions
Solution :
install : 'python3 -m pip install apache-superset'
instead of superset run -p 8088
run python3 superset run -p 8088

Error while trying to run snakemake in a venv on a protected server

For a project I made a virtual environment (venv) using Python3. I installed all the necessary dependencies using a simple bash script (see picture below) after I activated my venv. (I verified the installed packages using: pip3 list and concluded that every dependency was installed succesfully.)
My project uses snakemake, so I ran this snakemake commando:
snakemake --snakefile Snakefile.py all
I get this error:
I know it has to do something with the venv, because without the venv snakemake runs perfectly. I have read the Snakemake installation documents and it says I have to install conda and make & activate a conda venv. But, I do not have the sudo privileges to download and install conda (I work on a protected server).
What is happening and does someone know a fix?
One possible reason could be the difference in Python versions. What version of Python does the pip3 prepare environment for?
As I can see from the picture provided, the invalid syntax may be because of the version of Python doesn't support f-strings.
Imagine the following two scenarios: when you run Snakemake manually, you use the latest Python3 (e.g. 3.9). But if the pip3 is configured for an older version (e.g. 3.5), you can configure a very different environment for Python3.5 that doesn't support f-strings.

Spyder Kernels "module not found" on Ubuntu VM

I am trying to connect a spyder kernel with my remote Ubuntu machine and was following this guide. Installing spyder kernels using sudo pip install spyder-kernels worked fine.
However, when trying to run:
python -m spyder_kernels.console — matplotlib=’inline’ — ip=my.vm.ip.address. -f=./remotemachine.json
I receive the following error on my VM:
Error while finding module specification for 'spyder_kernels.console' (ModuleNotFoundError: No module named 'spyder_kernels')
My remote VM uses Python 3.7.7 and is hosted by Google's cloud platform.
Any help much appreciated.
The issue is that running sudo pip install, you're using the root user's Python installation, and not the user installation you eventually run your command from.
Try using either python -m pip install spyder-kernels or sudo python -m spyder_kernels.console .... Preferably you would not run either command as root.

Pip install installs libraries into different locations

I have just started using python and I have set up python3 to be installed in C:\Python37 dir. I have added python3 path to environment variables. When I run python3 -m pip install [package_name] it installs it in C:\Python37\Lib\site-packages. But when I try to install pylint with python3 -m pip install pylint it prints:
Requirement already satisfied: pylint in c:\users\radio\appdata\roaming\python\python37\site-packages (2.3.1)
It's installing it in above mentioned completely different location, and then VS Code complains how pylint is not installed. Why doesn't pip install it in C:\Python37\Lib\site-packages where it installs all the other packages?
It seems you might have multiple python installations on the computer or you are not running CMD with administrator priviledges when using pip.
I would refer you to this thread: windows pip installing libraries in wrong directory which seems to deal with a problem similar to the one you are having.
I would also try checking the environment variables to see the path set for Python if that doesn't work.

sphinx-quickstart ImportError no module named sphinx.quickstart

I am trying to run python sphinx on a CentOS-7 VM, using a virtual python 3.6.1 environment.
Steps:
Create python3 virtual environment
$ pyvenv myapp
$ source myapp/bin/activate
Install required packages for myproj
$ pip install -r requirements.txt
Install sphinx
$ pip install sphinx sphinx-autobuild
Try and run the quickstart:
$ sphinx-quickstart
File "/bin/sphinx-quickstart", line 7 in <module>
from sphinx.quickstart import main
ImportError: No module named sphinx.quickstart
pip freeze indicates that I am running the following versions:
Sphinx==1.6.3
sphinx-autobuild==0.7.1
pyvenv has been deprecated. See note in the Python library docs:
Note: The pyvenv script has been deprecated as of Python 3.6 in favor of using python3 -m venv to help prevent any potential confusion as to which Python interpreter a virtual environment will be based on.
Instead first make sure you have not already activated a virtual environment. Then issue the command:
python3 -m venv myapp
The rest of your steps look OK.
I solved the problem. I am new to python and pyvenv, and apparently it was finding a "universally" installed version of sphinx outside of the virtual environment, and that was preempting the version that was installed inside the virtualenv. I solved the problem by specifying the full path:
/full-path-to-myapp/myapp/bin/sphinx-quickstart

Resources