Unable to launch Airflow Webserver in fresh install - python-3.x

I'm trying to launch $ airflow websever (on newly installed instance) but the web server does not start (the browser states: This site can’t be reached).
In the terminal I got following error message
Running the Gunicorn Server with:
Workers: 4 sync
Host: 0.0.0.0:8080
Timeout: 120 Logfiles:
> Error: cannot import name 'models'
here are the steps I took, as per Airflow Quick Start :
just made an install of pip install apache-airflow in fresh virtualenv (py3.6)
set the AIRFLOW_HOME env variable: export AIRFLOW_HOME=$(pwd)
Initialize Airflow DB.. airflow initdb
using PyCharm, macOS Mojave 10.14.1
Thanks a lot for taking a look at it.
UPDATE: a simple statement from airflow import models is throwing an error ImportError: cannot import name 'models'. Howevre, when I try in Python Console, the module seems to import successfully

RESOLVED my own python file was named airflow.py that was causing the name clash.. PY interpreter wa looking models module in my own airflow.py instead from airflow package's...
restrain from naming your own PY module / package as airflow.
hope it helps others

Related

uwsgi plugin "python3" works with Python 3.6 and not with Python3.8

I have the following uwsgi config:
[uwsgi]
callable=app
mount = /labs=/home/vinzee/projectname/app/wsgi.py
manage-script-name = true
touch-reload=/home/vinzee/projectname/app/wsgi.py
socket=/tmp/labshome.sock
processes=5
threads=1
venv=/home/vinzee/.virtualenvs/projectnamevenv38
chdir=/home/vinzee/projectname/app/
plugin=logfile,python3
logger=delete file:/dev/null/c.log
log-route = delete (\] POST \/save => generated )
pythonpath=/home/vinzee/projectname/
The virtual environment specified in venv uses Python3.8. And when reloading uwsgi, I get the classic error:
ModuleNotFoundError: No module named 'flask'
The same error happens when uwsgi's plugin is python and the project uses Python >= 3, as can be seen in this answer.
When I switch to Python3.6, the ModuleNotFoundError disappears and everything works properly.
I already have the package uwsgi-plugin-python3 to the latest version.
How can I get rid of this error for Python3.8? What package do I need to install?
It seems that this is a problem of creating a virtual environment.
virtualenv flask
cd flask
source bin/activate (for activate env )
also you can try to add your python path in wsgi file config
sys.path.append('/home/deployer/anaconda3/lib/python3.5/site-packages')
or
export PYTHONPATH=/root/environments/my_env/lib/python3.6/site-packages/
finally.. run :
python -m flask run

ModuleNotFoundError when accessing python Flask within jupyter notebook in a virtual env

I have created an virtual env gcloudenv on my nividia nano running ubuntu. I was able to successfully install flask and required libraries and able to deploy my appengine into GCP from this virtual env. All my work is in python and I was using nano as my editor to get my code up and running. No issues so far.
my virtual env gcloudenv already has all the required packages for flask, jinga etc and I can see them when I run pip freeze.
Then I tried to work on Jupyter notebook as my code was getting little complicated and I didn't want to write full code and then run.
I already had jupyter notebook installed before creating the virtual env. I also installed jupyter within in virtual env as well.
So I followed the instruction to create a new kernel by running the following command:-
(gcloudenv) sunny#my-nano:~gcloudenv/MyApp/mainfolder$ pip install ipykernel
(gcloudenv) sunny#my-nano:~gcloudenv/MyApp/mainfolder$ ipython kernel install --user --
name=gcloudenv
Now, I ran the notebook as:
(gcloudenv) sunny#my-nano:~gcloudenv/MyApp/mainfolder$
/home/gcloudenv/bin/jupyter notebook
When trying to import the flask I get the following error:
ModuleNotFoundError: No module named 'flask'
Note sure what is going on as I getting blanked out.
Add
!pip install flask
in the beginning of your Jupyter notebook.
Finally I managed to solve my problem. Thanks to a wonderful post https://jakevdp.github.io/blog/2017/12/05/installing-python-packages-from-jupyter/.
In essence, I had 2 problems:-
1. I did not have a jupyter-notebook within my virtual env. Originally I thought I had it installed but that was incorrect. so whenever I tried to launch one, it was picking the first jupyter notebook it could find in the path.
The good way to find out which one is it pointing to , is to run which command
(gcloudenv) sunny#my-nano:~/gcloudenv$ which jupyter-notebook
For me, that was at:
/home/sunny/archiconda3/bin/jupyter-notebook
I had in fact 3 copies of jupyter-notebook on my system. One was probably installed using sudo pip and therefore went into the root folder. Probably not a good thing to do.
So I installed a fresh jupyter-notebook with the following command:-
(gcloudenv) $ pip install jupyter notebook
2.Next is to check list of Jupyter kernels available by running the following from the jupyter notebook ( or from command line):
!jupyter kernelspec list (OR (gcloudenv) $jupyter kernelspec list
My jupyter notebook was not able to import flask libraries because it was pointing to a wrong kernel config outside of my virtualenv gcloudenv.
Available kernels:
gcloudenv /home/sunny/.local/share/jupyter/kernels/gcloudenv ( correct one)
python3 /home/sunny/gcloudenv/share/jupyter/kernels/python3
You can determine which python version it is picking by doing a 'more' on the file:-
(gcloudenv) $
/more/home/sunny/.local/share/jupyter/kernels/gcloudenv/kernel.json
Once I changed my kernel to point to python3 from within the notebook, it picked the correct path and all the relevant libraries I needed.
In summary when you hit the problem as mentioned above, do the following:-
check the path of the python ( whereis python or which python)
check if you are running the 'right' notebook. This is determined by the path and if you have sourced your virtualenv.
Install jupyter notebook using pip from within your virtualenv( do not use sudo)
Check the Jupyter kernel. This may be particularly relevant if you have a common jupyter notebook and you want to work with multiple virtualenv.

Unable to use gitpython ImportError

Tried to run a script I wrote yesterday again today and ran into this:
ImportError: cannot import name 'Repo' from 'git' (/usr/local/lib/python3.7/site-packages/git/__init__.py)
I am at a complete loss. On a new computer so the only thing different install wise would have been that I installed pycharm. I am currently trying to run through bash shell on a mac. The exact code was running as excepted earlier, no code changes.
Things I have tried:
uninstalling/reinstalling python
uninstalling/reinstalling pip
uninstalling/reinstalling gitpython
Running on:
mac catalina
python version 3.7.6
pip version 20.0.1
As a side note, the script works as intended until the automated git push. Wondering if I should just make the os calls myself and not worry about this?
I really dont understand what I did/am doing wrong here.
EDIT:
Again, sorry as this is my first mac computer. I did a brew uninstall of python3 and reinstalled through the app store to 3.8.
ImportError: cannot import name 'is_cygwin_git' from partially initialized module 'git.util' (most likely due to a circular import) (/Library/Frameworks/Python.framework/
I know cygwin is for windows, but I figured I'd play along and trying a pip install pycygwin.
Install threw an error asking for cython, so I did another pip install and tried again. The pycygwin then complained gcc was missing so I did a brew install of gcc. With gcc installed and correctly on path, it still says it cant find it and exits with
build/cygwin/_cygwin.c:611:10: fatal error: 'sys/cygwin.h' file not found
#include <sys/cygwin.h>
^~~~~~~~~~~~~~
1 error generated.
error: command 'gcc' failed with exit status 1
Thinking I might just try a different package manager? Currently, an attempt to rerun the script yeilds
ImportError: cannot import name 'Repo' from partially initialized module 'git' (most likely due to a circular import)
To which I investigated but I shouldn't have any overlapping dependancies.
On the script throwing the error I'm using:
import csv
import yaml
import os
from git import Repo
and on the wrapper I made and imported, I'm using:
import subprocess
import re
Will update if I get any further on this, would love some suggestions.
EDIT:
Importing using just import git works throws a different error, like the python is trying to get itself?
ImportError: cannot import name '<file name>' from '<file name>'
If I change the file name and try to run it, it comes backs with:
ImportError: cannot import name '<old file name>' from '<old file name>'
***FIXED****
Uninstall of python through homebrew
Reinstall of python through mac app store
Uninstall/Reinstall of modules through pip
Saving the file under a new name and deleting the old one
Still have absolutely no idea why/how this happened but the above worked for me. If anyone knows why something like this can happen, I would love to know. Cheers.

ModuleNotFoundError: No module named 'flask_wtf' in python

I'm new to python so please be patient if i ask a silly question.
I am trying to use flask wtf class as seen in a tutorial.
My first line of the program is
from flask_wtf import FlaskForm
It gives me the error ModuleNotFoundError: No module named 'flask_wtf'
I have followed the following steps to make sure i have activated the virtual environment and flask wtf was installed in the virtual environment and not global.
Step 1: Went to the virtual environment and activated the environment C:\Users\Shraddha\PycharmProjects\FlaskProject\venv\scripts activate
Step 2: Installed flask wtf
C:\Users\Shraddha\PycharmProjects\FlaskProject\venv\scripts pip install flask-wtf
I see that flask-wtf is installed correctly in the folder C:\Users\Shraddha\PycharmProjects\FlaskProject\venv\Lib\site-packages
However when i try to run it using command prompt it does not give me anything and when i try to run it using sublime text it gives me ModuleNotFoundError: No module named 'flask_wtf' error.
I know this is a flask_wtf import error because other programs are executing as expected. Any help ?
If you start your virtual environment in the same dir as the file you are running doesn't that give access to the libraries in the install (ie everything shown in 'pip freeze' for the env)?
I faced same issue and tried all the approaches given on stackoverflow ImportError: No module named flask_wtf but didn't work for me.
I am using ubuntu 20.04 LTS operating system.
Note: To follow the given step you must activate your virtual environment.
I used:
pip3 freeze
which listed following directories
libries in my virtual environment
Then i tried the below command in the same directory
git clone git://github.com/lepture/flask-wtf.git
cd flask-wtf/
python3 setup.py install
Through the above commands i successfully installed flask_wtf as show below
installed flask_wtf
but then i faced the below error as i used email_validator in my forms.py file:
Exception: Install 'email_validator' for email validation support
Then I installed email_validator using following command
pip3 install email_validator
Then i successfully run my flask application. I am sure it will work for you too.

set executable PATH in Jupyter Notebook on google cloud cluster Python3

I opened jupyter notebook on my google cloud cluster with these steps: https://cloud.google.com/dataproc/docs/tutorials/jupyter-notebook
Now I get an error on this piece of code:
import selenium
from contextlib import closing
from selenium.webdriver import PhantomJS
with closing(PhantomJS()) as browser:
#some further code
I get the following error message:
WebDriverException: Message: 'phantomjs' executable needs to be in PATH.
Now in my own environment when I got this error, I fixed it by adding the path to my phantomjs.exe in my system variables.
But now while I am on the google cloud cluster environment, I am looking for another way to add the phantomjs.exe path. Any other solution would be appreciated aswell.
I have no experience with Selenium or PhantomJS. However since Dataproc runs on Debian 8 Jessie and not Windows, you probably want to run sudo apt-get install phantomjs instead of using an exe. You could either install it manually after SSHing or in an initialization action.

Resources