I'm on a Jupyter Notebook server (v4.2.2) with Python 3.4.2 and
I want to use the global name __file__, because the notebook will be cloned from other users and in one section I have to run:
def __init__(self, trainingSamplesFolder='samples', maskFolder='masks'):
self.trainingSamplesFolder = self.__getAbsPath(trainingSamplesFolder)
self.maskFolder = self.__getAbsPath(maskFolder)
def __getAbsPath(self, path):
if os.path.isabs(path):
return path
else:
return os.path.join(os.path.dirname(__file__), path)
The __getAbsPath(self, path) checks if a path param is a relative or absolute path and returns the absolute version of path. So I can use the returned path safely later.
But I get the error
NameError: name '__file__' is not defined
I searched for this error online and found the "solution" that I should better use sys.argv[0], but print(sys.argv[0]) returns
/usr/local/lib/python3.4/dist-packages/ipykernel/__main__.py
But the correct notebook location should be /home/ubuntu/notebooks/.
Thanks for the reference How do I get the current IPython Notebook name from Martijn Pieters (comments) the last answer (not accepted) fits perfect for my needs:
print(os.getcwd())
/home/ubuntu/notebooks
If you want to get path of the directory in which your script is running, I would highly recommend using,
os.path.abspath('')
Advantages
It works from Jupyter Notebook
It work from REPL
It doesn't require Python 3.4's pathlib
Please note that one scenario where __file__ has advantage is when you are invoking python from directory A but running script in directory B. In that case above as well as most other methods will return A, not B. However for Jupyter notbook, you always get folder for .ipyn file instead of the directory from where you launched jupyter notebook.
__file__ might not be available for you, but you can get current folder in which your notebook is located in different way, actually.
There are traces in global variables, if you will call globals() you will see that there is an element with the key _dh, that might help you. Here how I managed to load the data.csv file that is located in the same folder as my notebook:
import os
current_folder = globals()['_dh'][0]
# Calculating path to the input data
data_location = os.path.join(current_folder,'data.csv')
In modern Python (v3.4+) we can use pathlib to get the notebook's directory:
from pathlib import Path
cwd = Path().resolve()
# cwd == PosixPath('/path/to/this/jupyter/ipynb/file's/directory/')
# or this way, thanks #NunoAndré:
cwd = Path.cwd()
# cwd == PosixPath('/path/to/this/jupyter/ipynb/file's/directory/')
Update
#ShitalShah I cannot reproduce the error you are reporting. Jupyter notebook seems to work fine, regardless of the current working directory the application was started.
Example: file ~/dir1/dir2/untitled.ipynb and Jupyter notebook started in ~/dir1:
Jupyter notebook started in ~/dir1/dir2:
It's not possible to get the path to the notebook. You may find a way to get it that only works in one environment (eg os.getcwd()), but it won't necessarily work if the notebook is loaded in a different way.
Instead, try to write the notebook so that it doesn't need to know its own path. If doing something like getting the pwd, then be sure to fail fast / print an error if this doesn't work, vs. just silently trying to continue on.
See also: https://github.com/ipython/ipython/issues/10123
I'm new to python, but this works for me.
You can get os.path.dirname(__file__) equivalence by:
sys.path[0]
On new version of python and notebook __file__ works... If you have older versions, to get __file__ you can use
import inspect
from pathlib import Path
module_path = Path(inspect.getframeinfo(inspect.currentframe()).filename).resolve()
But... This is much slower... On the other hand, it will not return currrent working directory, but path to module, even if module is imported from elsewhere.
Related
I was earlier using console to run scripts which worked fine. Now that I wish to shift the entire process to crontab, I m facing ModuleNotFoundError:. The structure follows as I run a main file from parent directory, which imports another .py script from different location. To tackle the path error issue I have used os.chdir(path) which seems to work when I give print(os.getcwd()) but fails to find the required scripts in this new location. Can someone please help me with this. I m new to scripting.
#TIA
cwd = os.getcwd()
try:
os.chdir(cwd + 'pathToOtherScripts')
print('#\tGet current directory:',os.getcwd())
import SomeScript as SS
I get ModuleNotFoundError: at the parent directory. This error doesn't happen when I run the same script through console.
So I builded a python package localy:
cgi#cgires:~$ pip list | grep mads
madscgi 0.1.0
Its nice! Afterwards I can use it in Jupyter Notebook, in iPython Shell, in Python Shell and even in python scripts outside the modules code. So it works as expected 100% outside the modules code:
Thats nice, but next I want to import code from one builded module (inside the package) into another python file (inside the package). Lets name it import_test.py and try it out:
So it fails if it is getting executed in the directory, where the package is build from. And it looks like, that the python interpreter is taking the parent directory (with the same name like the module) and this is failing.
Is is possible to enforce the usage of the installed pip-package?
As #MisterMiyagi pointed out, the problem was, that there were an upper folder which had the same name as the module.
Here: mads_cons is the upper folder from import_test.py. Therefore, the upper folder is getting imported instead of the via pip installed module. Thats it.
The file you want to import should either be in the same folder or referred to with the absolute path of it.
If that doesn't suit you, you can call sys.path
import sys
sys.path
You can keep your file in any of the directories sys.path returns.
Smart would be, if you keep the file inside.
......../site-packages/
I know imports are a common headache but none of the other solutions worked for me.
I'm getting a ModuleNotFoundError when trying to import functions from a module inside the parent folder of my script. Here is the folder structure.
python_tools
command_line
cli_script.py
utils.py
cli_script.py:
from python_tools.utils import foo
def bar():
foo()
if __name__ == "__main__":
bar()
utils.py:
def foo():
return x
def foo2():
return y
This works in PyCharm when I hit 'Run', but I get the aforementioned error when running it from the command line with python3 cli_script.py.
I've also tried python3 -m cli_script.py which is the same error:
ModuleNotFoundError: No module named 'python_tools'
From the other answers I gather that this has something to do with my sys.path. They mostly suggest modifying it in some way in the script, but that seems hacky / non-pythonic.
The idea is to have a module with a bunch of functions which are then imported recipe-style into a bunch of tools that are segregated into different directories. All of these tools will be scripts (if _name__ == _main__).
Please let me know if there's a better directory structure or if this isn't possible without hacky workarounds.
Thank you!
First of all, this will actually work once you pip install your package, even with as little as the --editable option intended for local testing. Because the Python interpreter will then be able to find your package, python_tools, on the module search path and take it from there.
I do not know about the PyCharm magic that seems to be going on. But, yes, you could add the parent folder "manually" to the module search path. For example, like so:
import sys
sys.path.append('..')
That would usually do it. If the relative path .. doesn't cut it, or seems too "hacky", you could also get to the absolute path of your script from the pre-defined __file__ attribute and using functions from os.path or methods from pathlib (and converting to str in order to append to sys.path).
The better directory structure might be to just have utils.py in the same folder as your command line scripts, or in a subfolder, as a module inside a package. That's because the directory of the script, when directly executed by the interpreter, is always on the module search path.
All of John's solutions work and I basically went with the last one, but want to formalize the issue I was experiencing (and now understand better).
Python's sys.path is built based on the execution context. Since I was executing the script from different directories, imports were relative to that context.
I fixed the issue by hardcoding the relative import by adding:
import os, sys
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
to the top of the cli_script.py. This is the most portable solution.
A more elegant solution is to pip install utils.py so it will always be on sys.path and can be imported by any module/script.
You could also add a PYTHONPATH environment variable to your bash profile, but I think that solution is the worst of all worlds.
Chris Yeh's incredible write-up helped me wrap my head around this, see Case #4.
its a simple program which outputs the full file path using a given path that does not include the root. But It just prints out the given path. why?
operating system Ubutu18.04 Lts, IDE Pycharm , python 3.7
I've tried installing pathlib and path.py but it still does not work.
from os.path import abspath, relpath
x = '/python/100 exercises/24.py'
print(abspath(x))
The expected output is
/home/tasif/Documents/python/100 exercises/24.py
actual result is
/python/100 exercises/24.py
By putting a / at the beginning of your path, your system is interpreting x as a root system path already. Instead, write your path as x = 'python/100 exercises/24.py' and try again.
I think this is true but I could be corrected: You need to run it as os.path.abspath(x) otherwise the script does not know where to get that function from and is doing nothing but printing your string with a non-function ran on it.
You should also rename your directory to 100_exercises/ or something more posix friendly.
I want to add a permanent PYTHONPATH using Jupyter Notebook to be able to access the data from a particular directory or folder. I have read that we could use JUPYTER_PATH for it.
Could someone tell me a step wise instruction on how to do it. I am new to it and the documentation was not very clear.
For example sake lets say my path is as follows:
C:\ENG\Fin_trade\ION
For a script that needs to reference your directory, you can do the following.
Say you have the file foo.py containing the class Foo in your directory C:\ENG\Fin_trade\ION
import os
import sys
new_path = r'C:\ENG\Fin_trade\ION'
sys.path.append(new_path)
import foo
Foo()