I created a python package and wrapped it in whl file - let's say x.whl
Now I want to use this python package in a Pyspark notebook. I added it to the notebook configuration under "spark.submit.pyFiles"
"spark.submit.pyFiles": "x_package_location/x.whl"
x is a whl file that is made of several python files (let's call it x1,x2,x3) that were in a folder named y.
what do I do next?
I tried several things and nothing worked
(e.g:
I tried reading it like a did before when I wrapped the python files into a zip file and not whl, but with no success:
sys.path.append('./x.whl')
from y import x1, x2, x3
got this error -
No module named 'y'
Traceback (most recent call last):
ModuleNotFoundError: No module named 'y' )
I would appreciate a solution using pyspark (and not cmd), if exists
Related
I am trying to load a file which is a pickle compatible lists of dictionaries. My code is below:
with open('data/task_2/b.obj', 'rb') as file:
f = pickle.load(file)
The error I am getting is:
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-133-5fce33f1537b> in <module>
1 with open('data/task_2/b.obj', 'rb') as file:
----> 2 f = pickle.load(file)
ModuleNotFoundError: No module named 'genpy.rostime'
I am using windows 10. Python 3.8 and anaconda. First time I tried this piece of code only genpy was missing and installed that using pip. Now I am stuck with this.
Any hints, suggestions are appreciated.
solution_1:
install genpy
solution_2:
You should find that where is 'genpy.rostime'.
Then add its root path to your python script.
If the "genpy" is in your current script directory, then the following will solve this problem.
sys.path.append(os.path.join(os.path.dirname(__file__), "."))
I am using ROS-melodic with the following environment variables:
ROS_PYTHON_VERSION=2
ROS_PACKAGE_PATH=/home/florian/git/software_integration/src:/opt/ros/melodic/share
For a project I want to get a obstacle detection and implement trajectory prediction using a LIDAR sensor. To achieve this, I installed the following github repo according to the given instructions:
https://github.com/kostaskonkk/datmo
The repo works so far, I can start the examples and the topics are published. Now I want to implement a new listener, with the following MWE:
#!/usr/bin/env python
import rospy
from datmo.msg import TrackArray
def callbackTrackArray(data):
print(type(data.date))
def getInput():
rospy.init_node('obstacleComputer', anonymous= True)
tracks = rospy.Subscriber("datmo/box_kf", TrackArray, callbackTrackArray)
rospy.spin()
if __name__ == '__main__':
getInput()
But if I run it, it yells "No module named 'datmo'". But PyCharm even suggests me "datmo" if I type "import da", so he knows it.
I am using a Python 3.8 venv in PyCharm with the following Interpreter Paths:
Also, my project structure looks like this (and there the datmo package is located):
Full Error Stack:
Traceback (most recent call last):
File "/home/florian/git/software_integration/src/trajectory_prediction/src/ObstacleComputer.py", line 7, in <module>
from datmo.msg import TrackArray
ModuleNotFoundError: No module named 'datmo'
I had to add the ROS_PACKAGE_PATH to the interpreter settings --> environment variables
Out of the blue I got the error listed below.
I am using Eclipse IDE and I have Python 2.7 and 3.6 installed (both WinPython)
I don't know when this started because I have worked for a while in 2.7. I just tried some code which I am writing in Jupyter and it is not working and I wanted to debug it easier in Eclipse. When I press the debug button I get the below
Traceback (most recent call last):
File "C:\Utils\PortableApps\PortableApps\Eclipse 4.6\plugins\org.python.pydev.core_7.0.3.201811082356\pysrc\pydevd.py", line 20, in <module>
from _pydevd_bundle.pydevd_constants import IS_JYTH_LESS25, IS_PYCHARM, get_thread_id, get_current_thread_id, \
ModuleNotFoundError: No module named '_pydevd_bundle'
It's absolutely due to that your files contain a dir named code! The name has conflicted with the system package. Just modify the dir name code to any name else. It should work well!
I have installed Python 3.7 on my system and I am running one simple python file in which imports a .pyd file. When running the script I got error like:
Traceback (most recent call last):
File "demoTemp\run.py", line 1, in
from pyddemo import fun
ModuleNotFoundError: No module named 'pyddemo'
pyddemo is pyd file.
Is there any dependency for pyd file?
Thanks
I found solution of my problem its just mismatch of python version in which I have created pyd and python version in which I am using it.
I have tried to create a package using python 3.5.1 , but got the error when i import the package.
Traceback (most recent call last):
File "Pack.py", line 2, in
import Com
File "C:\Users\admin\Document\Python\packages\Com__init__.py",
line 2, in
from Algebra import *
ImportError: No module named 'Algebra'
To import a package you created named Algebra, your folder structure should look similar to this:
C:\Users\admin\Document\Python\packages\
Com__init__.py
Algebra\
__init__.py
.
.
.
Alternatively, you can put the Algebra packages parent directory on python's sys.path.
You'll likely run into another issue once you resolve this, which is you cannot import * from the Algebra package without specifying the all trait in Algebra's init.py file. Specifying this done like the following, but replacing the values of the list with your module names:
__all__=['add', 'subtract', 'multiply']
Source: https://docs.python.org/3/tutorial/modules.html#packages