Why are these import errors occurring when running python scripts from cmd or windows task scheduler, but not anaconda? - python-3.x

I am encoutering import errors, but only when running my python scripts from cmd or windows task scheduler (effectively the same issue I assume). I have researched answers already and attempted various solutions (detailed below), but nothing has worked yet. I need to understand the problem in any case so that I can manage anything like it in the future.
Here is the issue:
Windows 10. Anaconda Python 3.9.7. Virtual enviromnent.
I have a script that works fine if I open an anaconda prompt, activate the virtual environment and run it.
However, this is where the fun starts. If I try to run the script from the non-anaconda cmd prompt deploying the commands: "C:\Users\user\anaconda3\envs\venv\python.exe" "C:\Users\user\scripts\script.py" if get the following error:
ImportError: DLL load failed while importing etree: The specified module could not be found.
Traceback includes:
"C:\Users\user\anaconda3\envs\venv\lib\site-packages\lxml\html\__init__.py", line 53, in <module>
from ..import etree
This is not as simple as one specific module not being installed, because of course running the script from within the anaconda prompt and the virtual environment works. Similar also happens when I run other scripts. Other errors I have seen include, for example:
ImportError: DLL load failed while importing _imaging: The specified module could not be found.
Traceback includes:
"C:\Users\user\anaconda3\envs\venv\lib\site-packages\PIL\Image.py", line 114, in <module>
from . import _imaging as core
Also, I think this may be somehow related. Importing numpy (1.22.3) from within the python interpreter in the virtual environment works fine, but when I try to run a test script that imports numpy it fails both from anaconda and the cmd with the following error:
ImportError: cannot import name SystemRandom
The oveall issue was noted originally when trying to run various scripts from Windows Task Scheduler with the path to python "C:\Users\user\anaconda3\envs\venv\python.exe" entered as the Program/script and the script "script.py" entered as an argument. The above errors were produced, then reproduced by running the scripts from a non-anaconda cmd.
I am looking to understand what is happening here and for a solution that can get the scripts running from the virtual enviroment from Windows Task Scheduler effectively.
Update:
I have uninstalled and reinstalled numpy (and pandas) using conda. This has left the venv with numpy==1.20.3 (and pandas=1.4.2). On attempting to re-run one of the scripts, it runs fine from within the venv in anaconda, but produces the following error when attempting to run from cmd or from within Windows Task Scheduler as above:
ImportError: Unable to import required dependencies:
numpy:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions faled. This error can happen for many reasons, often due to issues with your setup or how NumPy was installed.
We have complied some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.9 from "C:\Users\user\anaconda3\envs\venv\python.exe"
* The NumPy version is "1.20.3"
and make sure that they are the versions you expect.
Please carefull study the documentation linked above for further help.
Original error was: DLL load failed while importing _multiarray_umath: The specified module could not be found.
I have looked into the solutions suggested, but am still completely at a loss, especially as to why the script runs from the venv in one place, but NOT the other.

Related

Unable to use gitpython ImportError

Tried to run a script I wrote yesterday again today and ran into this:
ImportError: cannot import name 'Repo' from 'git' (/usr/local/lib/python3.7/site-packages/git/__init__.py)
I am at a complete loss. On a new computer so the only thing different install wise would have been that I installed pycharm. I am currently trying to run through bash shell on a mac. The exact code was running as excepted earlier, no code changes.
Things I have tried:
uninstalling/reinstalling python
uninstalling/reinstalling pip
uninstalling/reinstalling gitpython
Running on:
mac catalina
python version 3.7.6
pip version 20.0.1
As a side note, the script works as intended until the automated git push. Wondering if I should just make the os calls myself and not worry about this?
I really dont understand what I did/am doing wrong here.
EDIT:
Again, sorry as this is my first mac computer. I did a brew uninstall of python3 and reinstalled through the app store to 3.8.
ImportError: cannot import name 'is_cygwin_git' from partially initialized module 'git.util' (most likely due to a circular import) (/Library/Frameworks/Python.framework/
I know cygwin is for windows, but I figured I'd play along and trying a pip install pycygwin.
Install threw an error asking for cython, so I did another pip install and tried again. The pycygwin then complained gcc was missing so I did a brew install of gcc. With gcc installed and correctly on path, it still says it cant find it and exits with
build/cygwin/_cygwin.c:611:10: fatal error: 'sys/cygwin.h' file not found
#include <sys/cygwin.h>
^~~~~~~~~~~~~~
1 error generated.
error: command 'gcc' failed with exit status 1
Thinking I might just try a different package manager? Currently, an attempt to rerun the script yeilds
ImportError: cannot import name 'Repo' from partially initialized module 'git' (most likely due to a circular import)
To which I investigated but I shouldn't have any overlapping dependancies.
On the script throwing the error I'm using:
import csv
import yaml
import os
from git import Repo
and on the wrapper I made and imported, I'm using:
import subprocess
import re
Will update if I get any further on this, would love some suggestions.
EDIT:
Importing using just import git works throws a different error, like the python is trying to get itself?
ImportError: cannot import name '<file name>' from '<file name>'
If I change the file name and try to run it, it comes backs with:
ImportError: cannot import name '<old file name>' from '<old file name>'
***FIXED****
Uninstall of python through homebrew
Reinstall of python through mac app store
Uninstall/Reinstall of modules through pip
Saving the file under a new name and deleting the old one
Still have absolutely no idea why/how this happened but the above worked for me. If anyone knows why something like this can happen, I would love to know. Cheers.

Running python scripts from command line using different python versions defined in anaconda envs

I have 2 python tools that I have to run via the windows cmd line. One is written in python2.7 while the other requires python3.6.
I have installed the newest Anaconda python3.7 version and created two new environments in 'C:\ProgramData\Anaconda3\envs' called 'python27' and 'python36'. For some reason I had to manually install numpy and scipy using conda install -n env_name numpy scipy for each of the new environments.
The reason I have to run both tools using the windows cmd line is that I have integrated them into a workflow environment (RCE by the DLR in case this is relevant), which executes integrated tools in this way. Which means I cannot simply use the Anaconda Prompt instead.
I cannot simply add the python installation to the PATH environment variable because of each tool requiring a different python version (and the file being called 'python.exe' in all versions), so I tried to create aliases for the cmd prompt as suggested by "roryhewitt" in this thread Aliases in Windows command prompt.
my 'python27.bat' file:
#echo off
echo.
C:\ProgramData\Anaconda3\envs\python27\python.exe %*
The problem with this approach is that python encounters an error when trying to import numpy:
>>> import numpy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\ProgramData\Anaconda3\envs\python27\lib\site-packages\numpy\__init__.py", line 142, in <module>
from . import core
File "C:\ProgramData\Anaconda3\envs\python27\lib\site-packages\numpy\core\__init__.py", line 71, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the multiarray numpy extension module failed. Most
likely you are trying to import a failed build of numpy.
Here is how to proceed:
- If you're working with a numpy git repository, try `git clean -xdf`
(removes all files not under version control) and rebuild numpy.
- If you are simply trying to use the numpy version that you have installed:
your installation is broken - please reinstall numpy.
- If you have already reinstalled and that did not fix the problem, then:
1. Check that you are using the Python you expect (you're using C:\ProgramData\Anaconda3\envs\python27\python.exe),
and that you have no directories in your PATH or PYTHONPATH that can
interfere with the Python and numpy versions you're trying to use.
2. If (1) looks fine, you can open a new issue at
https://github.com/numpy/numpy/issues. Please include details on:
- how you installed Python
- how you installed numpy
- your operating system
- whether or not you have multiple versions of Python installed
- if you built from source, your compiler versions and ideally a build log
Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.
Original error was: DLL load failed: The specified module could not be found.
Does anyone know a better way to run python scripts with a specific python environment via the windows cmd line, or what is causing the import error when I use the alias?
TLDR: I have 2 python tools that require python2.7 and python3.6 respectively. I have to run these tools using the windows cmd line and using aliases to the 'python.exe' file in the specific anaconda environments results in an import error of numpy. Is there a better way to handle two python environments via the cmd line or an easy fix for the import error?
It seems that the ImportError when using python2.7 via the alias in the cmd line was due to the anaconda environment not being activated. This caused the module to be unable to load properly.
I have added the conda functionality to be used in the cmd line as instructed in the post by "Simba" here: Conda command is not recognized on Windows 10. This has fixed the numpy ImportError.
I can now execute the desired tools from the cmd line using the correct python version by activating the correct anaconda environment first, and then calling the alias
Example:
C:\Users\user>conda activate python27
(python27) C:\Users\user>python27 main.py

Kivy Installation Guide for Windows 10

I've been trying to follow online youTube videos to install kivy on my Windows 10 computer (python-3.7.5-amd64, kivy 1.11.1). Aside from the fact that they seem to have different variations on how they approach the topic, I am unable to get a solution that operates satisfactorily.
These are the steps I am following:
I install python (python-3.7.5-amd64.exe) to C:\Python37
I modify the path to include to include the following: C:\Python37\Scripts;C:\Python37;C:\Python37\Libs;C:\Python37\DLLs;C:\Python37\Lib\site-packages;
I added the following environment variable PYTHONPATH = C:\Python37\DLLs;C:\Python37\Libs;C:\Python37;C:\Python37\Scripts;C:\Python37\Lib\site-packages;
I open a command window and type in the following commands (taken from kivy.org)
python -m pip install --upgrade pip wheel setuptools virtualenv
python -m pip install docutils pygments pypiwin32 kivy_deps.sdl2==0.1.* kivy_deps.glew==0.1.*
python -m pip install kivy_deps.gstreamer==0.1.*
python -m pip install kivy_deps.angle==0.1.*
python -m pip install kivy==1.11.1
python -m pip install kivy_examples==1.11.1
I try to run a simple program. From within Windows Explorer I right click the code file (label.py) and from the shortcut menu select python.
A windows pops up for an instant and a directory called __pycache__ gets created with kivy.cpython-37.pyc. Double clicking that causes the program to run.
Is it possible to have a easier solution in which the source code, once compiled executes?
If I open a command prompt and attempt to execute the source code using the command python label.py I get the following:
Traceback (most recent call last):
File "label.py", line 1, in <module>
from kivy.app import App
File "C:\Users\chrib\Google Drive\_Software\Python_Kivy\kivy.py", line 1, in <module>
from kivy.base import runTouchApp
ModuleNotFoundError: No module named 'kivy.base'; 'kivy' is not a package
Why should this happen?
Also is it possible to have a cleaner development environment. I am used to Visual Studio IDE and it would be great if I can use this environment.
Thanks
Code for label.py
from kivy.app import App
from kivy.uix.label import Label
class MyApp(App):
def build(self):
return Label(text='Hello world!');
if __name__=='__main__':
MyApp().run();
I've been trying to follow online youTube videos to install kivy on my Windows 10 computer
Have you tried simply following the instructions on kivy.org? There's no need to use youtube videos, the installation is largely a normal python module install.
I try to run a simple program. From within Windows Explorer I right click the code file (label.py) and from the shortcut menu select python.
Don't do this, run the file by opening a command prompt and typing python yourfilename.py. That way you will see the full traceback for any errors that occur.
A windows pops up for an instant and a directory called pycache gets created with kivy.cpython-37.pyc. Double clicking that causes the program to run.
It sounds likely that the first run is crashing. As above, you want to get the information about why.
Is it possible to have a easier solution in which the source code, once compiled executes?
When you run the code it does execute. As above, it's probably crashing.
ModuleNotFoundError: No module named 'kivy.base'; 'kivy' is not a package
Have you made a file named kivy.py? It looks likely that you have, and that this file is being imported in preference to the installed kivy module.
Also is it possible to have a cleaner development environment. I am used to Visual Studio IDE and it would be great if I can use this environment.
I'm not sure what you consider unclean about your development environment, but you should think in terms of python environments and their installed packages. Kivy is just a python module that you install into a python environment. When you use an IDE, it may integrate with one or more python environments (with options to switch between them). There's nothing special about using Visual Studio with Kivy, just do whatever you normally do to use it with Python.
I figured it out. I had a program in the code directory called kivy.py. I renamed that and it worked.

How to fix Python script started in CMD get's DLL Error, but runs in Pycharm and Anaconda Comand Promt

I want to run a python script in cmd via a batch file. The script fails with dll loading error for numpy imports. The same python script works in pycharm (2019 Anaconda Version) and in the anaconda command prompt without errors.
For all I use the same anaconda enviorment.
For Error Reproduction
Install Anaconda, don't set any path variables
Create an enviorment stored not in the default folder
conda create --prefix
conda activate
conda install numpy flask
6.Code for testing. test-numpy.py
import numpy as np
print(np.abs([1,2-4,6]))
create batch script
run batch script
Running in cmd a python script that has only system imports or no imports works fine.
Setting the pythonpath Variable in Windows is no option.
What I did so far:
Checking if there is an PATH error:
I printed the sys.path, both via Pycharm and the same python script run via CMD gave the same results: it points to my anaconda enviorment with the folders:
...\ (the enviorment itself)
...\python37.zip
...\DLLs
...\lib
...\lib\site-packages
Edit
Tested in Visual Studio Code, gives same Error as running via cmd
Found out Visual Studio Coda until now, can't handle conda enviorments, that where installed with -p or -prefix and are not stored in the default path
Uninstalled and reinstalled numpy
Tested to import another module: Flask also fails with DDL load error but works fine while run in Pycharm.
Tested on another PC with differnt anaconda version.
Another person could reproduce the same error I get.
My guess is my code fails in cmd, becaust anacona enviorments needs somehow to be activated before hand and doesn't work as a stand alone.
The Error Messeag I get running my script in CMD (... points to my anaconda enviormant path):
...\lib\site-packages\numpy\core__init__.py", line 40, in
from . import multiarray ...\lib\site-packages\numpy\core\multiarray.py", line 12, in
from . import overrides ...\lib\site-packages\numpy\core\overrides.py", line 6, in
from numpy.core._multiarray_umath import ( ImportError: DLL load failed: Das angegebene Modul wurde nicht gefunden.
During handling of the above exception, another exception occurred:
Traceback (most recent call last): "TestNumpy.py", line 14, in
import numpy as np ...\lib\site-packages\numpy__init__.py", line 142, in
from . import core ...\lib\site-packages\n umpy\core__init__.py", line 71, in
raise ImportError(msg) ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the multiarray numpy extension module failed. Mostlikely
you are trying to import a failed build of numpy. Here is how to
proceed:
- If you're working with a numpy git repository, try git clean -xdf (removes all files not under version control) and rebuild numpy.
- If you are simply trying to use the numpy version that you have installed: your installation is broken - please reinstall numpy.
- If you have already reinstalled and that did not fix the problem, then:
1. Check that you are using the Python you expect
and that you have no directories in your PATH or PYTHONPATH that can
interfere with the Python and numpy versions you're trying to use.
2. If (1) looks fine, you can open a new issue at
https://github.com/numpy/numpy/issues. Please include details on:
- how you installed Python
- how you installed numpy
- your operating system
- whether or not you have multiple versions of Python installed
- if you built from source, your compiler versions and ideally a build log
Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.
Original error was: DLL load failed: Das angegebene Modul wurde nicht
gefunden.
I found not other solution it seems like it is needed to really activate the conda enviorment before calling the script via conda:
call <file_path>/Anaconda/Scripts/activate.bat <file_path/Anaconda_enviorment> && python <file_path/pythonscript.py>

PyCharm Import Error: Claims 'matplotlib' is not a package, but works successfully in IDLE

Happy October everyone,
I've successfully downloaded modules before using either the pycharm installer or pip through the command screen, but for some reason when installing matplotlib pycharm cannot recognize it. I've uninstalled and reinstalled, I've installed through both methods, I've followed past similar questions asked on this site which make sure that you have the same interpreter and that it was installed in the right folder (pycharm error while importing, even though it works in the terminal).
So, here's the whole problem. Here's is the simple code, submitted into both pycharm and IDLE:
import matplotlib.pyplot as plt
plt.plot([1,2,3],[2,1,3])
plt.show()
When submitted into IDLE, my plot appears. When submitted into pycharm, the following error appears:
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/PythonProject/matplotlib.py", line 1, in <module>
import matplotlib.pyplot as plt
File "C:\PythonProject\matplotlib.py", line 1, in <module>
import matplotlib.pyplot as plt
ImportError: No module named 'matplotlib.pyplot'; 'matplotlib' is not a package
I am currently running Python 3.4, PyCharm 2016.2.3, and my matplotlib folders are indeed in my site-packages folder inside my Python34 folder. Also for further verification:
PyCharm installation
Please help I've become frustrated since this is the only module I've run into trouble with. I've scoured StackOverflow and related websites to help, I've made sure I have all the requirements, etc.
I guess if you named your current writing python module as matplotlib.py.That cause the python load your current writing module instead of the actual matplotlib.py, which triggers an error.
I recommend you to use virtualenv. Is not strictly necessary but is good for dividing your project environments.
This is how I tested matplotlib on my Windows 10 installation, hope it helps.
Be sure that you have the python 3 installation folder listed in your Windows PATH environment variable, should be already listed if you checked "Add Python 3.5 to PATH":
You need also to set the Scripts folder in your PATH environment variable usually should be this path:
C:\Users\<your username>\AppData\Local\Programs\Python\Python35\Scripts
If you don't do that you have to prepend python -m to every command below like this: python -m <command>, so the command below would be python -m pip install virtualenv. I prefer the first solution.
To test matplotlib on Pycharm I've used virtualenv, here is how; first install virtualenv:
pip install virtualenv
Then you create your virtual environment in a folder of your choice, in my case I used python_3_env_00:
virtualenv python_3_env_00
After that you can activate you python 3 virtual environment:
python_3_env_00/Scripts/activate.bat
Now you should see in your command line the active virtual environment (python_3_venv_00), like this:
Now you can install matplotlib:
pip install matplotlib
Fire up PyCharm and add your virtual environment as you project interpreter, go to File->Settings search for Project Interpreter click on the gear icon and Add Local and set the path of your virtual environment, should look like something like this:
Test it:
import sys
print(sys.path)
run this code in where the import worked, and run it in the Pycharm project. Compare the lists. Find out which path that is not present in Pycharm sys.path.
Before importing pyplot, append the missing path to sys.path.
import sys
sys.path.append("the path")
import matplotlib.pyplot as plt
Does this work?
Please follow below steps if you are still getting an error:
If you are using PyCharm, it automatically create virtualenv.
Ensure Scripts path is set into PATH
C:\Users\<Username>\AppData\Local\Programs\Python\Python37-32
Then open PyCharm and go to File-> settings. Search for Project Interpreter. You will see window like this
sample image
Click on setting icon -> Existing Environment -> click on ... give below path
C:\Users\Krunal\AppData\Local\Programs\Python\Python37-32\python.exe
Click on Apply -> ok and you are good to go.
After installing matplotlib When I was trying to use matplotlib.pyplot it was giving error module not found.
I browsed some white papers and found out that we also need to install scipy library to use the matplotlib so I used the below in my command prompt
python -mpip install scipy
Restarted my kernel session.
It worked!!!
I was also facing issue while importing matplotlib but it got resolved and now I am able to use it from pycharm as well.
Please make sure you should have visual c++ 14 installed in your system.
2.If you have more than two python version installed on your system then please install matplotlib from both the version.
Eg. pip install matplotlib
pip3 install matplotlib
If matplotlib is working from python idle then please check whether you are using correct interpreter in pycharm or not and try to choose pythonw.exe path from your installed location.
Hope this will help, Please do let me know if you are still facing issue.
I had similar issue but I solved it very easily on pycharm 2019.3.2. In case anyone looking for an easier solution:
I just opened the terminal window on pycharm and typed pip install matplotlib and it was all good to go. Every project has its own virtual environment. Opening terminal window of IDE cds to project directory by default. So the installing command was enough.

Resources