Trying to compile DAG from
Link to AirFlow official documentation
Got Airflow dashboard error:
Broken DAG: [/ua/sv/airflow/dags/example_mysql.py] Traceback (most recent call last):
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/ua/sv/airflow/dags/example_mysql.py", line 25, in <module>
from airflow.providers.mysql.operators.mysql import MySqlOperator
ModuleNotFoundError: No module named 'airflow.providers.mysql'
What I've already tried:
pip3 install apache-airflow-providers-mysql
sudo apt install default-libmysqlclient-dev
My local environment:
Ubuntu 20.04 on Windows via wsl
Apache Airflow v2.0.1.0.5.716
Related
I am trying to install few packages and started getting an error. Then used multiple commands in ubuntu to update few things but errors are similar
pip install -U pip setuptools
or
python3 -m pip install --upgrade pip
or
sudo -H pip3 install --upgrade pip
Following is the error sample
user#machine:~$ pip install cryptography
Traceback (most recent call last):
File "/usr/bin/pip", line 11, in <module>
load_entry_point('pip==20.0.2', 'console_scripts', 'pip')()
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 490, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2854, in load_entry_point
return ep.load()
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2445, in load
return self.resolve()
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2451, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/usr/lib/python3/dist-packages/pip/_internal/cli/main.py", line 10, in <module>
from pip._internal.cli.autocompletion import autocomplete
File "/usr/lib/python3/dist-packages/pip/_internal/cli/autocompletion.py", line 9, in <module>
from pip._internal.cli.main_parser import create_main_parser
File "/usr/lib/python3/dist-packages/pip/_internal/cli/main_parser.py", line 7, in <module>
from pip._internal.cli import cmdoptions
File "/usr/lib/python3/dist-packages/pip/_internal/cli/cmdoptions.py", line 24, in <module>
from pip._internal.exceptions import CommandError
File "/usr/lib/python3/dist-packages/pip/_internal/exceptions.py", line 10, in <module>
from pip._vendor.six import iteritems
File "/usr/lib/python3/dist-packages/pip/_vendor/__init__.py", line 65, in <module>
vendored("cachecontrol")
File "/usr/lib/python3/dist-packages/pip/_vendor/__init__.py", line 36, in vendored
__import__(modulename, globals(), locals(), level=0)
.
.
.
.
File "<frozen zipimport>", line 259, in load_module
File "/usr/share/python-wheels/urllib3-1.25.8-py2.py3-none-any.whl/urllib3/contrib/pyopenssl.py", line 46, in <module>
File "/home/dhome/.local/lib/python3.8/site-packages/OpenSSL/__init__.py", line 8, in <module>
from OpenSSL import SSL, crypto
File "/home/dhome/.local/lib/python3.8/site-packages/OpenSSL/SSL.py", line 19, in <module>
from OpenSSL.crypto import (
File "/home/dhome/.local/lib/python3.8/site-packages/OpenSSL/crypto.py", line 3224, in <module>
utils.deprecated(
TypeError: deprecated() got an unexpected keyword argument 'name'
I have updated the system with apt-get install libffi-dev python-dev python3-dev and
apt-get install build-essential libssl-dev already as suggested in here
Something got broken down in OpenSSL and no command was working with pip afterwards. I was even unable uninstall pip.
I removed installation files manually (most likely not a recommended approach) with
sudo rm -rf /usr/local/lib/python3.8/dist-packages/OpenSSL
sudo rm -rf /usr/local/lib/python3.8/dist-packages/pyOpenSSL-22.1.0.dist-info/
and reinstalled using pip3 install pyOpenSSL==22.0.0. The other version was having some issue as described here.
For me even uninstall of pyOpenSSL was encountering same error TypeError: deprecated() got an unexpected keyword argument 'name'
so first I upgraded pip to 23.0 ,uninstall PyOpenSSL,manually deleted openSSL folder from /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages then installed PyOpenSSL 20.0.0, finally issue resolved.
I am trying to install pyinstaller inside of my github actions workflow.
Using pip install pyinstaller was fine on ubuntu-latest but on windows-latest it was not however. It returns the following log.
Requirement already satisfied: future in c:\hostedtoolcache\windows\python\3.9.12\x64\lib\site-packages (from pefile>=2017.8.1->pyinstaller) (0.18.2)
Using legacy 'setup.py install' for pefile, since package 'wheel' is not installed.
Installing collected packages: pywin32-ctypes, altgraph, pyinstaller-hooks-contrib, pefile, pyinstaller
Running setup.py install for pefile: started
ERROR: Operation cancelled by user
Running setup.py install for pefile: finished with status 'canceled'
Error: The operation was canceled.
build:
name: Build Executables
needs: create_release
runs-on: ${{ matrix.config.os }}
strategy:
matrix:
config:
- os: ubuntu-latest
- os: macos-latest
- os: windows-latest
steps:
- uses: actions/checkout#v3
- name: Setup Python
uses: actions/setup-python#v3
with:
python-version: '3.9.12'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install pyinstaller
- name: Build with pyinstaller for ${{ matrix.config.os }}
run: pyinstaller game.spec
Direct link to github action file: https://github.com/daanbreur/SwishandFrick/blob/main/.github/workflows/build.yml
Trying to install the missing dependencies manually doesnt resolve the issue however
- name: Install dependencies
run: |
python -m pip install --upgrade pip wheel setuptools
pip install -r requirements.txt
pip install pywin32 pefile pyinstaller
Direct link to github action file: https://github.com/daanbreur/SwishandFrick/blob/main/.github/workflows/build.yml
ubuntu-latest fails to locate the pywin32 package
Installing collected packages: PyTMX, pygame, future, pyscroll
Successfully installed PyTMX-3.31 future-0.18.2 pygame-2.1.2 pyscroll-2.29
ERROR: Could not find a version that satisfies the requirement pywin32 (from versions: none)
ERROR: No matching distribution found for pywin32
Error: Process completed with exit code 1.
And windows-latest crashes entirely
Run python -m pip install --upgrade pip wheel setuptools
pip install -r requirements.txt
pip install pywin32 pefile pyinstaller
Traceback (most recent call last):
File "C:\hostedtoolcache\windows\Python\3.9.12\x64\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\hostedtoolcache\windows\Python\3.9.12\x64\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\hostedtoolcache\windows\Python\3.9.12\x64\lib\site-packages\pip\__main__.py", line 29, in <module>
from pip._internal.cli.main import main as _main
File "C:\hostedtoolcache\windows\Python\3.9.12\x64\lib\site-packages\pip\_internal\cli\main.py", line 9, in <module>
from pip._internal.cli.autocompletion import autocomplete
File "C:\hostedtoolcache\windows\Python\3.9.12\x64\lib\site-packages\pip\_internal\cli\autocompletion.py", line 10, in <module>
from pip._internal.cli.main_parser import create_main_parser
File "C:\hostedtoolcache\windows\Python\3.9.12\x64\lib\site-packages\pip\_internal\cli\main_parser.py", line 8, in <module>
from pip._internal.cli import cmdoptions
File "C:\hostedtoolcache\windows\Python\3.9.12\x64\lib\site-packages\pip\_internal\cli\cmdoptions.py", line 23, in <module>
from pip._internal.cli.parser import ConfigOptionParser
File "C:\hostedtoolcache\windows\Python\3.9.12\x64\lib\site-packages\pip\_internal\cli\parser.py", line 12, in <module>
from pip._internal.configuration import Configuration, ConfigurationError
File "C:\hostedtoolcache\windows\Python\3.9.12\x64\lib\site-packages\pip\_internal\configuration.py", line 20, in <module>
from pip._internal.exceptions import (
File "C:\hostedtoolcache\windows\Python\3.9.12\x64\lib\site-packages\pip\_internal\exceptions.py", line 13, in <module>
from pip._vendor.requests.models import Request, Response
File "C:\hostedtoolcache\windows\Python\3.9.12\x64\lib\site-packages\pip\_vendor\requests\__init__.py", line 45, in <module>
from .exceptions import RequestsDependencyWarning
File "C:\hostedtoolcache\windows\Python\3.9.12\x64\lib\site-packages\pip\_vendor\requests\exceptions.py", line 11, in <module>
from .compat import JSONDecodeError as CompatJSONDecodeError
File "C:\hostedtoolcache\windows\Python\3.9.12\x64\lib\site-packages\pip\_vendor\requests\compat.py", line 11, in <module>
from pip._vendor import chardet
File "C:\hostedtoolcache\windows\Python\3.9.12\x64\lib\site-packages\pip\_vendor\chardet\__init__.py", line 19, in <module>
from .universaldetector import UniversalDetector
File "C:\hostedtoolcache\windows\Python\3.9.12\x64\lib\site-packages\pip\_vendor\chardet\universaldetector.py", line 46, in <module>
from .latin1prober import Latin1Prober
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 846, in exec_module
File "<frozen importlib._bootstrap_external>", line 941, in get_code
File "<frozen importlib._bootstrap_external>", line 1040, in get_data
KeyboardInterrupt
What am I missing for windows and how can I let ubuntu skip the step for windows dependencies.
Big oops on my part.
Windows didn't fail to install the dependencies, Ubuntu failed to zip the archive and that lead to it erroring.
By default github-actions is configured to automatically cancel all other jobs in the matrix. Setting fail-fast inside the strategy to false stops it from canceling other jobs. See docs: jobs.<job_id>.strategy.fail-fast
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
index a00be16..34b9e82 100644
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
## -38,6 +38,7 ## jobs:
needs: create_release
runs-on: ${{ matrix.config.os }}
strategy:
+ fail-fast: false
matrix:
config:
- os: ubuntu-latest
## -49,11 +50,11 ## jobs:
uses: actions/setup-python#v3
with:
python-version: '3.9.12'
- - name: Install dependencies
+ - name: Install dependencies for ${{ matrix.config.os }}
run: |
python -m pip install --upgrade pip wheel setuptools
pip install -r requirements.txt
- pip install pywin32 pefile pyinstaller
+ pip install pyinstaller
- name: Build with pyinstaller for ${{ matrix.config.os }}
run: pyinstaller game.spec
- name: Archive Release
Installing python package failed with error.
I want to install 'tsfresh' package in Notebook VM. But it failed with error from both jupyter terminal and jupyter notebook cell.
from Notebook jupyter notebook cell
!pip install --upgrade tsfresh
error message
Exception:
Traceback (most recent call last):
File "/anaconda/envs/azureml_py36/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/anaconda/envs/azureml_py36/lib/python3.6/site-packages/pip/commands/install.py", line 342, in run
prefix=options.prefix_path,
File "/anaconda/envs/azureml_py36/lib/python3.6/site-packages/pip/req/req_set.py", line 784, in install
**kwargs
File "/anaconda/envs/azureml_py36/lib/python3.6/site-packages/pip/req/req_install.py", line 851, in install
self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
File "/anaconda/envs/azureml_py36/lib/python3.6/site-packages/pip/req/req_install.py", line 1064, in move_wheel_files
isolated=self.isolated,
File "/anaconda/envs/azureml_py36/lib/python3.6/site-packages/pip/wheel.py", line 345, in move_wheel_files
clobber(source, lib_dir, True)
File "/anaconda/envs/azureml_py36/lib/python3.6/site-packages/pip/wheel.py", line 329, in clobber
os.utime(destfile, (st.st_atime, st.st_mtime))
PermissionError: [Errno 1] Operation not permitted
You are using pip version 9.0.1, however version 19.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
In my case following resolved the issue you described:
!pip install --upgrade pip
!pip install --upgrade tsfresh
I'm trying to install Tweepy on an Ubuntu VM and I'm running into way more issues than I should be. My first step was to successfully get pip installed. After that I tried:
pip install tweepy
which returns:
Traceback (most recent call last):
File "/usr/local/bin/pip", line 5, in <module>
from pkg_resources import load_entry_point
File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 3020, in <module>
File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 616, in _build_master
File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 629, in _build_from_requirements
File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 807, in resolve
pkg_resources.DistributionNotFound: pip==1.5.4
The most that I can gather from that error message is that it can't find pip version 1.5.4. However, if I ask it whereis pip it returns:
pip: /usr/local/bin/pip2.7 /usr/local/bin/pip /usr/share/man/man1/pip.1.gz
So I know pip is installed. I decided to try the other method listed and do
git clone https://github.com/tweepy/tweepy.git
cd tweepy
python setup.py install
But that only seems to partially install with error messages returning. Trying whereis tweepy after that method returns nothing.
Can anyone make sense of why this simple install isn't working for me?
This issue was resolved by using another fresh Ubuntu VM install and once again trying the manual installation by using
git clone https://github.com/tweepy/tweepy.git
cd tweepy
python setup.py install
For whatever reason, it now works. The answer is officially voodoo.
I'm getting the following error when trying to set up a new virtualenv with python3.4 (I'm running this command: virtualenv –p /usr/bin/python /tmp/djangodev):
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site- packages/pkg_resources.py", line 2697, in <module>
working_set.require(__requires__)
File "/System/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/pkg_resources.py", line 669, in require
needed = self.resolve(parse_requirements(requirements))
File "/System/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/pkg_resources.py", line 576, in resolve
raise VersionConflict(dist,req) # XXX put more info here
pkg_resources.VersionConflict: (virtualenv 1.11.6 (/System/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages),
Requirement.parse('virtualenv==1.7.2'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/virtualenv", line 5, in <module>
from pkg_resources import load_entry_point
File "/System/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/pkg_resources.py", line 2701, in <module>
parse_requirements(__requires__), Environment()
File "/System/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/pkg_resources.py", line 572, in resolve
raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: virtualenv==1.7.2
I first downloaded virtualenv1.7.2 and after getting this error updated it by running: pip install virtualenv. Here is the successful installation message from this:
Downloading/unpacking virtualenv
Downloading virtualenv-1.11.6-py2.py3-none-any.whl (1.6MB): 1.6MB downloaded
Installing collected packages: virtualenv
Successfully installed virtualenv
Cleaning up...
Can someone please help me resolve the above error?
Thanks #MattDMo for cluing me in.
I indeed have multiple virtualenvs installed so I ran sudo pip uninstall virtualenv and then sudo pip install virtualenv and this resolved the above error.