cannot import s3fs in pyspark - apache-spark

When i try importing the s3fs library in pyspark using the following code:
import s3fs
I get the following error:
An error was encountered: cannot import name 'maybe_sync' from
'fsspec.asyn' (/usr/local/lib/python3.7/site-packages/fsspec/asyn.py)
Traceback (most recent call last): File
"/usr/local/lib/python3.7/site-packages/s3fs/init.py", line 1, in
from .core import S3FileSystem, S3File File "/usr/local/lib/python3.7/site-packages/s3fs/core.py", line 12, in
from fsspec.asyn import AsyncFileSystem, sync, sync_wrapper, maybe_sync ImportError: cannot import name 'maybe_sync' from
'fsspec.asyn' (/usr/local/lib/python3.7/site-packages/fsspec/asyn.py)
The fsspec package has been installed in my notebook. And I actually had been using it fine for a long time, when this suddenly happened.
I tried googling, but could not find this specific error.
Has anyone come across this before? And if so, do you know how to solve it?

Glad to hear this wasn't just me. It looks like if you pip install versions s3fs==0.5.1 and fsspec==0.8.3, that should fix it.

Related

the correct way to use nornir plugins

I am a beginner with nornir, I am trying to run nornir plugins but I am having these errors:
Traceback (most recent call last):
File "nornirpy/start.py", line 2, in <module>
from nornir.plugins.tasks.commands import commands
ModuleNotFoundError: No module named 'nornir.plugins.tasks.commands'
My start.py file is very basic:
from nornir import InitNornir
from nornir.plugins.tasks.commands import commands
from nornir.core.inventory import Host
from path.to import InventoryPlugin
import json
nr = InitNornir("nornirpy/config.yml")
print(json.dumps(Host.schema(), indent=4))
What is the issue? I am using poetry to organize my project, is there a step I forgot? its not clear to me how can I use plugins in my application.
Hello Somayyah Mohammed;
First, check the version of nornir you are using. If version > 2.4.0 then module nornir.plugins.tasks.commands is not there. You can remove the import command it should work.

Python cv2 throws strange symbol error when importing

typically I can find the solution to any errors I have online, but I just cannot find the solution to this particular problem anywhere.
So I've installed OpenCV4 on my ubuntu 20.04 using pip3, but upon typing the following commands:
$ python3
import cv2
I get this error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/jordan/.local/lib/python3.7/site-packages/cv2/__init__.py", line 5, in <module>
from .cv2 import *
ImportError: /usr/lib/x86_64-linux-gnu/librsvg-2.so.2: undefined symbol: cairo_tag_end
I am not entirely sure what the error means. My guess is that it's something to do with the environment variables, but I don't know how to fix that anyways. I really hope someone can help me fix this.

Jupyter Notebook in Anaconda Not Loading

Whenever I try to open Jupyter Notebook from the Anaconda GUI (or conda terminal), I get the following error:
Traceback (most recent call last):
File "C:\Users\loops\AppData\Local\Continuum\anaconda3\lib\site-packages\notebook\services\sessions\sessionmanager.py", line 10, in
import sqlite3
File "C:\Users\loops\AppData\Local\Continuum\anaconda3\lib\sqlite3\__init__.py", line 23, in
from sqlite3.dbapi2 import *
File "C:\Users\loops\AppData\Local\Continuum\anaconda3\lib\sqlite3\dbapi2.py", line 27, in
from _sqlite3 import *
ImportError: DLL load failed: The specified procedure could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\loops\AppData\Local\Continuum\anaconda3\Scripts\jupyter-notebook-script.py", line 6, in
from notebook.notebookapp import main
File "C:\Users\loops\AppData\Local\Continuum\anaconda3\lib\site-packages\notebook\notebookapp.py", line 86, in
from .services.sessions.sessionmanager import SessionManager
File "C:\Users\loops\AppData\Local\Continuum\anaconda3\lib\site-packages\notebook\services\sessions\sessionmanager.py", line 13, in
from pysqlite2 import dbapi2 as sqlite3
ModuleNotFoundError: No module named 'pysqlite2'
I've gone so far as to uninstall Python, uninstall Anaconda, remove all python related path variables, and reinstall Anaconda with Python. This still has not worked. Is there any way to resolve this error? I've looked on many different sites but I haven't found much help.
EDIT: To add more context, it broke randomly one day and I don't have the slightest idea why. Anaconda had worked for a year or so prior. I checked both scripts that are referenced in the error prompt and they both exist in the directory.
I had the same problem.
The issue seems to arise from the missing of sqlite3.dll in the path ".\Anaconda\Dlls".(if using an env, you should put it under the env directory like ".\your env\DLLS") I solved it by simply copying that .dll file from others and put it under the path mentioned above.
You can download the sqlite3.dll from this link: sqlite3.dll

How can I import bs4 from Beautiful Soup in Python3 on OSX 10.12.5?

I'm trying to fix what seems like a common problem with importing a module in Python 3. I'm running OS X 10.12.5 and have Python 3 installed on my MacBook Air and am using Sublime Text to edit and run my code.
When I try this import:
from bs4 import BeautifulSoup
...I get this error:
Traceback (most recent call last):
File "/Users/<myname>/Python/code-python3/Pgm#001", line 5, in <module>
from bs4 import BeautifulSoup
ImportError: No module named 'bs4'
I successfully installed with this PIP and on every re-install I see this:
$ pip install beautifulsoup4
Requirement already satisfied: beautifulsoup4 in ./anaconda/lib/python3.5/site-packages
I've tried qualifying the location with things like:
from ./anaconda/lib/python3.5/site-packages/bs4 import BeautifulSoup
...but invariably get a variety of syntax errors; usually on the first '/'.
I am not using a virtual environment but do plan to read up on that approach as these kind of configuration and setup errors are big time wasters.
When I try to run this right in python3 I get slightly different errors:
>>> from bs4 import BeautifulSoup
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/<myname>/anaconda/lib/python3.5/site-packages/bs4/__init__.py", line 30, in <module>
from .builder import builder_registry, ParserRejectedMarkup
File "/Users/<myname>/anaconda/lib/python3.5/site-packages/bs4/builder/__init__.py", line 314, in <module>
from . import _html5lib
File "/Users/<myname>/anaconda/lib/python3.5/site-packages/bs4/builder/_html5lib.py", line 70, in <module>
class TreeBuilderForHtml5lib(html5lib.treebuilders._base.TreeBuilder):
AttributeError: module 'html5lib.treebuilders' has no attribute '_base'
Any tips about where the obvious answer exists that I am still missing would be greatly appreciated. I've seen a lot of things about PATH and PYTHONPATH in similar questions but had no success with any of these solutions either.
Yes, I'm not a fan of anaconda but looks like the path is being pointed to:
"/Users//anaconda/lib/python3.5/site-packages"
... and for the sake of keeping things as clean as possible (this being a good example/reason why you'll see people say to use env for your python projects)
Lets google here! Here's how to remove anaconda:
conda install anaconda-clean #first the configs
anaconda-clean --yes
rm -rf ~/anaconda.* #then conda, the ".*" is regex to cover any version
The check your bash_profile and/or bashrc; and here you can set your Python path. Make sure it not pointing to anaconda
if you run
ls -lh `which python # orpython3
You'll get the path of which is the one you'll be setting as the path.

Scipy install help windows 10

I am trying to get scipy running on my computer and I have been fighting for the last few hours with it. Currently, im running python 3.6 and I am on windows 10. I first tried to install it using pip which failed no matter what I did. Then I went and downloaded the wheel file and installed it manually using pip. Then I used pip to get the rest of the dependencies like the website stated pip install --user numpy scipy matplotlib ipython jupyter pandas sympy nose. It installed the rest of the packages just fine.
If I say in IDEL
import scipy
It will give me an error of
Traceback (most recent call last):
File "<pyshell#2>", line 1, in <module>
from scipy import special, optimize
File "C:\Users\Kevin\AppData\Local\Programs\Python\Python36-32\lib\site-packages\scipy\__init__.py", line 116, in <module>
from scipy._lib._ccallback import LowLevelCallable
File "C:\Users\Kevin\AppData\Local\Programs\Python\Python36-32\lib\site-packages\scipy\_lib\_ccallback.py", line 1, in <module>
from . import _ccallback_c
ImportError: cannot import name '_ccallback_c'
if I go in that file to comment out that line, it will import but it after that I get a lot more errors anytime I try and do anything.
Also, I tried installing anaconda as a dependency manager and that seemed to work fine for installing, but if I tried to used Atom or IDLE as my text editor it can't seem to find scipy when its compiling and throws and error. I am kinda at a loss and any advice or help on this would be much appreciated. Thank you in advance.

Resources