I know this question has been asked thousands of times, but I still don't get the correct answer and stumble upon the same problem always.
I have the following file:
application/app/__init__.py
Here, I have some variable, let it be x.
Then, I have another file here:
application/tests/test_1.py
In test_1.py I do this:
from app import x
And, being in the application folder, I run:
python3 tests/test_1.py
But the following error happens:
ModuleNotFoundError: No module named 'app'
One solution that works is, inside test_1.py, add:
import sys
sys.path.append('../')
from app import db
Then if I do:
cd tests
python3 test_1.py
It works perfectly, but I don't want that sys.path.append and also I would like to do
python3 tests/test_1.py
Another solution is to explicitly modify the PYTHONPATH environment variable, but again, I wouldn't like to do this. Isn't there a clean solution to this problem?
Do not manually "hack" the sys.path. This is a bad practice "quick fix" that you've probably learned from stackoverflow answers.
Python looks for imports in either the site-packages folder or in the PYTHONPATH. The easiest way to get your local folder into the site-packages is using
pip install --editable path/to/myfolder
You only need a setup.py in your program "myfolder" with the following contents
from setuptools import setup
setup(name='myfolder")
The name "myfolder" is only important in case you want to uninstall it. But eventually this will create a symlink of your myfolder in the environments site-packages, and you can continue editing as before. But your program works "as if installed".
This is way more flexible than changing the PYTHONPATH.
As for the other comment:
When `application' is linked to site-packages and thereby recognized via sys.path,
from app import x
is an absolute import, since 'app' is a folder inside the top level folder 'application'.
Related
I have an app that uses PySimpleGUI, Matplotlib and some others (I will refer to them jointly as the 'Modules').
(1) I would like to use better than everage protection (VMProtect is in the list with some others) for my executable (app.exe). To achieve this I use Nuitka to convert the my app.py code into C++ and mingw64 to compile it into an executable:
python -m nuitka --standalone --nofollow-imports --plugin-enable=tk-inter --plugin-enable=numpy --plugin-enable=pylint-warnings --mingw64 --output-dir=X: app.py
(2) Also, I want to convert the Modules (=Modules.py) into *.pyd files (=Modules.pyd). I also use Nuitka to conver Modules.py into Modules.pyd. For example:
python -m nuitka --module "c:\Program Files\Python\Lib\site-packages\PySimpleGUI\PySimpleGUI.py"
(3) app.exe shall refer to Modules.pyd which will be distributed together.
My problem is that when I am trying to refer to Modules.pyd in app.py code, which is:
import PySimpleGUI.pyd as sg # PySimpleGUI.pyd is located in the same folder as app.py
I receve the following error:
ModuleNotFoundError: No module named 'PySimpleGUI.pyd'; 'PySimpleGUI' is not a package
I was unable to find a solution to this problem.
In practical terms I would like to achieve one of the following:
(i) cause app.py (which also means app.exe) to recognise Modules.pyd (I already have the latter)
(ii) find some other way to receive app.exe (NB: ready for protection) and Modules.pyd through any other conversion / compilation process.
What I do NOT want to have is app.exe and dozens of Modules.py in the same directory.
Any advice on how to achieve either (i) or (ii) will be most appreciated.
P.S.: all software and modules are up-to-date, all PATHS are set.
UPDATE (2021-09-20):
After further reserach I was able to find what seems to be a [partial] solution -- adding the following lines above imports in app.py taught app.exe (after it was converted via Nuitka) to recognise the Modules.pyd:
import sys
sys.path.append("./DLL/") # where ./DLL/ is a sub-folder with Modules.pyd within the folder where app.exe is located.
However: now I cannot conver dateutil.relativedelta into a proper *.pyd because no such file exists.
This is turning to be a question of how to configure Nuitka to create *.pyd files from module.class.
If anyone has any solution to this, please resond.
I create a virtual environment; let's say test_venv, and I activate it. All successful.
HOWEVER, the path of the Python Interpreter doesn't not change. I have illustrated the situation below.
For clarification, the python path SHOULD BE ~/Desktop/test_venv/bin/python.
>>> python3 -m venv Desktop/test_venv
>>> source Desktop/test_venv/bin/activate
(test_venv) >>> which python
/usr/bin/python
Please make sure to read Note #2.
This is what you should do if you don't want to create a new virtual environment:
In venv/bin folder there are 3 files that store your venv path explicitly
and if the path is wrong they take the normal python path so you should change the path there to your new path.
change: set -gx VIRTUAL_ENV "what/ever/path/you/need" in activate.fish
change: VIRTUAL_ENV="what/ever/path/you/need" in activate
change: setenv VIRTUAL_ENV "what/ever/path/you/need" in activate.csh
Note #1:
the path is to /venv and not to /venv/bin
Note #2:
If you reached this page it means that you are probably not following Python's best practice for a project structure.
If you were, the process of creating a new virtual environment was just a matter of one command line.
Please consider using one of the following methods:
add a requirements.txt to your project - not recommended.
implement an setup.py script - for real projects.
use a tool like Poetry - just like the latter though somewhat user-friendlier.
Thanks to Khalaimov Dmitrii, I didn't thought it was because I moved the folder.
It is not an answer specifically to your question, but it corresponds the title of the question. I faced similar problem and couldn't find solution on Internet. Maybe someone use my experience.
I created virtual environment for my python project. Some time later my python interpreter also stopped changing after virtual environment activation. Similar to how you described.
My problem was that I moved the project folder to a different directory some time ago. And if I return the folder to its original directory, then everything starts working again.
There is following problem resolution. You save all package requirements (for example, using 'pip freeze' or 'poetry') and remove 'venv'-folder (or in your case 'test_venv'-folder). After that we create virtual environment again, activate it and install all requirements.
This approach resolved my problem.
Check the value of VIRTUAL_ENV in /venv/bin/activate . If you renamed your project directory or moved it, then the value may still be the old value. PyCharm doesn't update your venv files if you used PyCharm to rename the project. You can delete the venv and recreate a new one if the path is wrong, or try the answer that talks about where to change it.
I have been trying to run the Mask R-CNN demo from matter plot (https://github.com/matterport/Mask_RCNN). It works with tensorflow version 1.13.1 and keras 2.1.0 as suggested by someone here (https://github.com/matterport/Mask_RCNN/issues/1797). I get the error
ModuleNotFoundError: No module named 'astor'
The thing is astor 0.8.0 is installed in my virtual env but when trying to import it, it say it does not exist. I have made sure to install it normally as well as sudo. If you think it does exist on $PYTHONPATH, how can I do it. I am out of my depth here, so please be considerate.
EDIT: I am using virtualenv in pyCharm. If I look through my interpreter paths, I get
file:///home/$SUER/anaconda3/lib/python3.6
file:///home/$USER/anaconda3/lib/python3.6/lib-dynload
file:///home/$USER/project/Mask_RCNN/venv/lib/python3.6/site-packages
file:///home/$USER/anaconda3/lib/python3.6/site-packages
I have replace my actual user name with $USER in the above output.
Thanks to #bigbounty, but his questions made me understand a bit more about PYTHONPATH. Maybe something that I already should have known but you make mistakes and learn.
SOLUTION: in the venv interpreter paths I added
/home/$USER/anaconda3/pkgs/astor-0.8.0-py36_0/lib/python3.6/site-packages
and it worked. I do not know why the package was installed in this folder where apparently it was not included.
i solve copyng the astor folder from site-package (in env) to my project folder and set is as source( in pycharm)
The Flask official website says that we can run a Flask application by
$ export FLASK_APP=hello.py
$ flask run
The second command doesn't work for me.
$ flask run
Command 'flask' not found, but can be installed with:
sudo apt install python3-flask
Instead this works
python3 -m flask run
How can I make the second command works? If I run sudo apt install python3-flask, will I get two installations of flask?
Can the two commands be combined into one command without using environment variable?
Bear with me as I will try to explain the different pieces and how they all interconnect. export FLASK_APP=hello.py is setting an operating system environment variable called FLASK_APP and is simply pointing to the entry file to start your flask application. This is no different than setting any other environment variable on your operating system. Now the flask team has provided everyone with a command called flask run which can be used to start up your flask application and this command will use the value set within your FLASK_APP environment variable when it attempts to start your flask server. So the reason why your python3 -m flask run command works is because you're telling your operating system's install of python to run the flask run command as a script, which is how this command is intended to be invoked.
For reference:
-m mod : run library module as a script (terminates option list)
Additionally, python attempts to resolve modules from it's sys.path environment variable and it looks in the following order of directories to resolve the requested module:
The current directory where the script has been invoked. This is why you can always import modules contained in the same directory as one another.
The value of your PYTHONPATH environment variable
The standard library directory on your path
Lastly, the site packages directory, i.e. your third party packages like flask
Now the reason your flask run command didn't initially work is because python couldn't find flask within any of the four locations listed above. However, once you gave the -m python knew to look in your site-packages directory for flask and was able to find said module.
For reference you can see where python is looking to resolve modules by printing out the sys.path variable to the console:
import sys
print(sys.path)
Ok so that answers the first part of your first question, now as for the second part of your first question:
"If I run sudo apt install python3-flask, will I get two installations of flask?"
Yes, this would install flask globally on your system and I would highly advise against this as you can mess up your system pretty badly if you're not careful. So how do I avoid messing with my system level python configurations?
Virtualenv to the rescue, Virtual environments allow you to have a sandboxed area to play around with libraries. With the worst case scenario being you blow them away and start fresh if you screwed something up, without affecting your Operating System's install of python. You should have a one to one relationship between each python project and virtual environment. If you use virtualenv I highly suggest looking into Virtualenvwrapper which wraps virtualenv with easier to remember commands. Although I think all the cool kids are using pipenv now so you may want to look into that as well, I will leave that decision up to you. What's nice is once you've activated your virtual environment and are developing you can just use flask run since your virtual environment will be on your python path.
As for your second question: "Can the two commands be combined into one command without using environment variable?"
No you would still need to set the FLASK_APP environment variable to use flask run since it looks for the value of that environment variable to start your flask server. Perhaps you could try something like:
FLASK_APP=hello.py flask run
on the command line and see if that helps you, but you're still setting the FLASK_APP environment variable. Alternatively, you could just start the entry file for your flask server directly, with a:
python hello.py
I know that was a lot, but hopefully that helps clarify things for you!
I created a venv using python3.6 on my mac os in this folder
/Users/kim/Documents/Apps/PythonApps/python36-miros-a3
I ran a pip install pylint after I activated the virtual env
My workspace is in /Users/kim/Documents/Apps/WebApps/miros-a3
Inside my vscode workspace, I have the following Workspace settings
{
"folders": [
{
"path": "."
}
],
"settings": {
"python.pythonPath": "/Users/kim/Documents/Apps/PythonApps/python36-miros-a3/bin/python3.6",
"python.venvPath": "/Users/kim/Documents/Apps/PythonApps"
}
}
I have tried setting a custom path for the pylint and also changing the venvpath.
The pylint kept complaining about the import statement saying it does not exist.
As you can see, they are in the same folder, and I can definitely execute my python files.
What can I do to avoid these kind of false positive import errors?
I have also tried the following:
go to commandline turn on the virtual env and then type code to activate the vscode as recommended here https://code.visualstudio.com/docs/setup/mac
also tried this https://donjayamanne.github.io/pythonVSCodeDocs/docs/troubleshooting_linting/
Pylint has some quirks. In this case it doesn't know where to find your module because it's in subdirectory of your venv path. To solve this:
Put this setting in your workspace or folder settings:
"python.linting.pylintArgs": [
"--init-hook",
"import sys; sys.path.append('<path to folder your module is in>')"
]
or, maybe better
Generate .pylintrc file. From integrated terminal with venv activated run:
pylint --generate-rcfile > .pylintrc
then open the generated file and uncomment the init-hook= part to be:
init-hook='import sys; sys.path.append("<path to folder you module is in>")'
Read the .pylintrc and tweak settings if you wish. In both cases path should point to your 'database' folder.
After learning about pylint settings, do it the right way:
from database.database_dispatcher import ...
See this answer by Anthony Sottile.
To me, pylint is correct in flagging this error here
the top level module is database (it contains an __init__.py file)
Your import should look like (fully absolute)
from database.database_dispatcher import ...
or (explicit relative) (yes! the . before the module name is intentional)
from .database_dispatcher import ...
My follow-up guess is that you're currently invoking your script as python ./database/main.py ... which is putting ./database at the beginning of sys.path so it would appear that your imports are working correctly -- this is side-stepping your module structure however. You should be invoking your script using python -m database.main ... instead.
Note that implicit relative imports were removed in python 3.x -- though this (imo) wart of script sys.path insertion remains.
Just my $0.02 on how I fixed it in my situation.
My problem was totally related to having pylint installed globally, and coding in a venv. vscode was trying to use the globally installed pylint which simply was not aware of dependencies I installed in my Python venv. This answer solved my problem. It points here which explained how to configure vscode to run using the venv for my project. Once i did that vscode immediately threw a warning saying I had no linting tool installed and prompted me to install one. Once that was done my linting false-positives went away.
Expanding on Gatmando's answer, you need to tell vscode to use the pylint in your .env instead of the global pylint:
In your workspace settings file: .vscode/settings.json, add python.linting.pylintPath and point it to pylint package in your virtualenv:
{
"python.pythonPath": ".env/bin/python",
"python.linting.pylintPath": ".env/bin/pylint"
}
Another option is to create a .env file in the vscode project root dir:
.env file:
PYTHONPATH=.
Some caveats of using the .env file:
It will be used for anything that the vscode ide runs (e.g. pylint, pytest), so the PYTHONPATH may clash between different tools.
You could specify multiple dirs to PYTHONPATH to accommodate multiple tools by using separators. (Example: PYTHONPATH=src:. ) However, this is not cross-OS because the Windows separator is ; while Linux/Mac is :
If the .env PYTHONPATH solves your issue, then great. But due to the caveats, I ended up using the pylint --init-hook method because it only affects pylint, and it can be cross-OS:
.vscode/settings.json
"python.linting.pylintArgs": [
"--init-hook",
"import sys, os; sys.path.insert(0, os.path.join('src', 'foo'))"
]