for some days I got in touch with the IBM federated learning library from github. After finishing the setup, I wanted to run the files for the aggregator and parties in my jupyter notebook and ran into a ModuleNotFoundError as you can see on the picture attached.
What can not be the cause:
a) Jupyter uses different Python as pip does. Both are using 3.8
b) Jupyter tries to load from a wrong directory: As you can see on the code we are loading generate.data from the directory /home/jovyan/FL_MNIST/federated-learning-lib/federated-learning-lib/examples, and the file generate.data is in the directory /home/jovyan/FL_MNIST/federated-learning-lib/federated-learning-lib/examples.
Did anyone ran into this also using the IBM library and could solve it? or do you have a well educated guess what can be the cause?
Best regards,
Solaris
Related
I am building a desktop app using Python and PySimpleGUI. So far, everything works just fine. Whilst I was working at the project, I realized I need to find a way to get the duration of some mp3 files and to display it in a certain way. I discovered mutagen module that is supposed to help me in this sense, I installed, and here the problem arise:
It throws me ModuleNotFoundError: No module named 'mutagen'.
Seeing this, I started to look for the problem, but I couldn't not understand why my interpretor did not find the module even though I Installed it CORRECTLY. (as PyCharm told me)
I have tried the following:
I am using a local virtual environment that has installed the dependecies for the project(and some extra) and I uninstalled and installed the package 3-4 times
I deleted the local virtual environment and I created another one. I installed the packages again and same issue.
I installed a random module (scipy) and I tried to import it somewhere in the project and it thrown me same error, but this time for scipy module
My guess is that I did not configured properly my interpreter, but to be honest, I have no idea what I am doing wrong, because I followed the same steps I've been using for creating a venv with its according interpreter and for other projects, it worked just fine.
Further details:
Using python3.9 base .exe
I installed the packages in two ways: one using the pycharm IDE, and one by running pip3 install mutagen
You may be using a different pip that is not the one that affects the Python you are using. Instead of using
pip install mutagen
Consider using pip as a module of the Python you are using:
python -m pip install mutagen
This way you'll be sure you are working on the same Python.
If you want to continue using plain pip, try which python and which pip to make sure they are referencing the same environment.
I am having the error while running jupyter notebook. Error replication:
python --version
Python 3.7.9
python
import win32api
The error was: ImportError: DLL load failed
After doing a some stackoverflow, I got to know that there are 2 dll files missing namely: pythoncom37.dll and pywintypes37.dll
I also got to know that I can run this post command: pywin32_postinstall.py in the Scripts folder. I ran this script. Restarted my pc. I manually download these two dll and copied it to my system32. After manually downloading, I started to have a different type of error: ImportError: DLL load failed: %1 is not a valid Win32 application
I used anaconda and ran my jupyter notebook which was my main aim but can you please make me aware as what is going wrong?
I've run into this recently, but with a different version of the DLLs. What solved it for me was using a different version of pywin32.
My solution (conda env, python 3.8.5):
pip install pywin32==300
or try 225, 227, 228. The latest pywin32 (301 as of this post) seems to be having dll search issues (I wouldn't be surprised if whatever version you were using is also having dll search issues). 301 was released after your issue started, but you may have a similar problem nonetheless.
There is currently an issue on pywin32 DLL loading failing: https://github.com/mhammond/pywin32/issues/1709
Factors involved (in my experience) include your PATH variable (if you're using conda). I haven't tested it myself, but I'd be curious to see if this issue occurs without conda. This issue stops happening for me if the first dlls found are those for 301. In my case, that means putting them in my C:\Windows\System32 folder (yeah I'm on Windows; joy).
So a possible solution #2 would be to run the pywin32 post install script which should be located under your venv/Scripts/pywin32_postinstall.py
To try that solution, open an ADMIN command prompt (very important that it's admin), navigate to your venv, and run:
ppython.exe Scripts\pywin32_postinstall.py --install
You shouldn't HAVE to do this, but if you just need a one-off solution and it works, great!
pip install --upgrade pywin32 ==225 worked for me. Tried version 300 and was unsuccessful.
How do I setup a Python virtual environment with the FreeCAD library embedded as to enable import as a module into scripts?
I would like to avoid using the FreeCAD GUI as well as being dependent on having FreeCAD installed on the system, when working with Python scripts that use FreeCAD libraries to create and modify 3D geometry. I hope a Python virtual environment can make that possible.
I am working in PyCharm 2021.1.1 with Python 3.8 in virtualenv on Debian 10.
I started out with FreeCAD documentation for embedding in scripts as a basis:
https://wiki.freecadweb.org/Embedding_FreeCAD
In one attempt, I downloaded and unpacked .deb packages from Debian, taking care to get the correct versions required by each dependency. In another attempt, I copied the contents of a FreeCAD flatpak install, as it should contain all the libraries that FreeCAD depends on.
After placing the libraries to be imported in the virtual maching folder, I have pointed to them with sys.path.append() as well as PyCharm's Project Structure tool in various attempts. In both cases the virtual environment detects where FreeCAD.so is located, but fails to find any of its dependencies, even when located in the same folder. When importing these dependencies explicitly, each of them have the same issue. This leads to a dead end when an import fails because it does not define a module export function according to Python:
ImportError: dynamic module does not define module export function (PyInit_libnghttp2)
I seem to be looking at a very long chain of broken dependencies, even though I make the required libraries available and inform Python where they are located.
I would appreciate either straight up instructions for how to do this or pointers to documentation that describes importing FreeCAD libraries in Python virtual environments, as I have not come across anything that specific yet.
I came across a few prior questions which seemed to have similar intent, but no answers:
Embedding FreeCAD in python script
Is it possible to embed Blender/Freecad in a python program?
Similar questions for Conda focus on importing libraries from the host system rather than embedding them in the virtual environment:
Incude FreeCAD in system path for just one conda virtual environment
Other people's questions on FreeCAD forums went unanswered:
https://forum.freecadweb.org/viewtopic.php?t=27929
EDIT:
Figuring this out was a great learning experience. The problem with piecing dependencies together is that for that approach to work out, everything from the FreeCAD and its dependencies to the Python interpreter and its dependencies seems to need to be built on the same versions of the libraries that they depend on to avoid causing segmentation faults that brings everything to a crashing halt. This means that the idea of grabbing FreeCAD modules and libraries it depends on from a Flatpak installation is in theory not horrible, as all parts are built together using the same library versions. I just couldn't make it work out, presumably due to how the included libraries are located and difficulty identifying an executable for the included Python interpreter. In the end, I looked into the contents of the FreeCAD AppImage, and that turned out to have everything needed in a folder structure that appears to be very friendly to what PyCharm and Python expects from modules and libraries.
This is what I did to get FreeCAD to work with PyCharm and virtualenv:
Download FreeCAD AppImage
https://www.freecadweb.org/downloads.php
Make AppImage executable
chmod -v +x ~/Downloads/FreeCAD_*.AppImage
Create folder for extracting AppImage
mkdir -v ~/Documents/freecad_appimage
Extract AppImage from folder (note: this expands to close to 30000 files requiring in excess of 2 GB disk space)
cd ~/Documents/freecad_appimage
~/Downloads/./FreeCAD_*.AppImage --appimage-extract
Create folder for PyCharm project
mkdir -v ~/Documents/pycharm_freecad_project
Create pycharm project using Python interpreter from extracted AppImage
Location: ~/Documents/pycharm_freecad_project
New environment using: Virtualenv
Location: ~/Documents/pycharm_freecad_project/venv
Base interpreter: ~/Documents/freecad_appimage/squashfs-root/usr/bin/python
Inherit global site-packages: False
Make available to all projects: False
Add folder containing FreeCAD.so library as Content Root to PyCharm Project Structure and mark as Sources (by doing so, you shouldn't have to set PYTHONPATH or sys.path values, as PyCharm provides module location information to the interpreter)
File: Settings: Project: Project Structure: Add Content Root
~/Documents/freecad_appimage/squashfs-root/usr/lib
After this PyCharm is busy indexing files for a while.
Open Python Console in PyCharm and run command to check basic functioning
import FreeCAD
Create python script with example functionality
import FreeCAD
vec = FreeCAD.Base.Vector(0, 0, 0)
print(vec)
Run script
Debug script
All FreeCAD functionality I have used in my scripts so far has worked. However, one kink seems to be that the FreeCAD module needs to be imported before the Path module. Otherwise the Python interpreter exits with code 139 (interrupted by signal 11: SIGSEGV).
There are a couple of issues with PyCharm: It is showing a red squiggly line under the import name and claiming that an error has happened because "No module named 'FreeCAD'", even though the script is running perfectly. Also, PyCharm fails to provide code completion in the code view, even though it manages to do so in it's Python Console. I am creating new questions to address those issues and will update info here if I find a solution.
I took to heart the message that support for anaconda2 was dropped. I installed anaconda3 and used 2to3 to convert my *.py codes. Everything seemed to work as expected. I have several 2 line bat codes that cd to a particular directory and then call python to execute a particular *.py code in that directory. By loading Anaconda Prompt(Anaconda3) and invoking the bat file I could run pythons in that directory as expected.
Then, a couple of days ago, I made such a run and got an error message saying the *.py file could not be found. I immediately looked in the directory and the python file was there as expected. I then did a dir command in Anaconda Prompt(Anaconda3) and found that almost no files in the directory were listed. This was happening on my windows 10 laptop.
On my desktop, everything continues to work as expected. Are there any suggestions as to what could cause such a problem and how to fix it.
Thanks,
Mack Elrod
Responding to the request for additional information, I have a bin directory that is in path. In bin\MackData.bat is
c:
cd %USERPROFILE%\Documents\Medev\MackData
copy sugarhist.png sugarhist.old.png
python MackData.py
When I open Anaconda Prompt(Anaconda3) and enter MackData I get
(base) C:\Users\Mack>MackData
(base) C:\Users\Mack>c:
(base) C:\Users\Mack>cd C:\Users\Mack\Documents\Medev\MackData
(base) C:\Users\Mack\Documents\Medev\MackData>copy sugarhist.png sugarhist.old.png
1 file(s) copied.
(base) C:\Users\Mack\Documents\Medev\MackData>python MackData.py
python: can't open file 'MackData.py': [Errno 2] No such file or directory
(base) C:\Users\Mack\Documents\Medev\MackData>
But MackData.py is a python file in the directory MackData.
This code worked on this laptop until a few days ago and continues to work on my desktop computer. I can, of course, publish MackData.py but that seems irrelevant. The point is that Anaconda can't find it.
Thanks,
Mack
I must submit a sincere apology to this community. I have determined my problem and Anaconda is not responsible in any way. I thank all of you for you concerns and comments.
For any who might be interested I will give a brief outline of what happened. First, I installed Anaconda3. Several days later my directory, MackData, was moved to a new location. I conjecture that happened by me inadvertently letting my hand drag on the touch pad. Then, not knowing the location had changed I updated my laptop from my desktop. My update program not finding MackData where expected recreated it and put only the most recently changed files from the desktop into the new MackData. New but in the correct location. Then of course running my script in the Anaconda3 prompt failed. Doing a dir in the Anaconda3 prompt showed only a few files were there. I then went to Windows File Explore and looked at MackData and saw all of the expected files. I used the quick access feature which unknown to me pointed to the old directory in its new location.
That is not intended to be an excuse but only as an explanation. Again, I do apologize to the community.
Mack Elrod
I'm pretty sure newer anaconda launchers users different environment paths, and not even windows ones.
I suggest either checking the environment path within the anaconda launcher and configurations or, a better solution in my opinion, run these scrips using the regular python engine, without using anaconda (simpler, more determinant and works within your standard environment)
As per the documentation - https://learn.microsoft.com/en-us/azure/hdinsight/spark/apache-spark-python-package-installation
we had installed several external python modules through new anaconda env 'py35_data_prof'. However as soon as we invoke any rdd action calls like rdd.count() or rdd.avg() in our python code, spark2 throws -
Cannot run program "/usr/bin/anaconda/envs/py35_data_prof/bin/python": error=2, No such file or directory
enter image description here
FYI, The python indicated in error path - '/usr/bin/anaconda/envs/py35_data_prof/bin/python' is actually a symlink rather than python dir.
I have been looking up the HDInsight docs but can't seem to find the fix. Please let us know if there is a way around it.
The error message “Cannot run program "/usr/bin/anaconda/envs/py35_data_prof/bin/python": error=2, No such file or directory” clearly says the unable to find/locate the package installed. Make sure the package is installed with the all the requirements mentioned below.
• Create Python virtual environment using conda.
• Install external Python packages in the created virtual environment if needed.
• Change Spark and Livy configs and point to the created virtual environment.
I would request you to follow the each and every step mentioned here: “Safely install external Python packages”.
Hope this helps.