Import py file in another directory in Jupyter notebook - python-3.x

Question

My question is related to this. I am using Python 3.6 in Jupyter Notebook. My project directory is /user/project. In this directory I'm building a number of models and each has its own folder. However, there is a common functions.py file with functions that I want to use across all models. So I want to keep the functions.py file in /user/project but be able to call it from an .ipynb file in /user/project/model1, /user/project/model2, etc... How can I do this?

Answer 1

There is no simple way to import python files in another directory.
This is unrelated to the jupyter notebook
Here are 3 solutions to your problem
You can add the directory containing the file you want to import to your path and then import the file like this:
import sys
sys.path.insert(0, '/path/to/application/app/folder')
import file
You can create a local module by having an empty __init__.py file in the folder you want to import. There are some weird rules regarding the folder hierarchy that you have to take into consideration.
You can create a module for the file you wish to import and install it globally.

Answer 2

Assuming you have a folder name Jupyter and you wish to import modules (employee) from another folder named nn_webserver.
visualizing it:
do this:
import sys
import os
module_path = os.path.abspath(os.path.join('..'))
if module_path not in sys.path:
sys.path.append(module_path+"\\nn_webserver")
from employee import motivation_to_work
see additional information here from #metakermit

Answer 3

I've been thinking about this problem because I don't like the sys.path.append() answers. A solution I propose uses the built-in Jupyter magic command to change the current working directory. Assuming you have this file structure:
project
├── model1
| └── notebook1.ipynb
├── model2
| └── notebook2.ipynb
└── functions.py
Whether you wanted to import functions from notebook1.ipynb or notebook2.ipynb, you could simply add a cell with the following line before the cell that has your package imports:
%cd ..
This changes the current working directory to the parent directory of the notebook, which then adds the path of the functions module to the default locations that Python will check for packages. To import functions:
import functions
This would work similarly if you had multiple modules in the same package directory that you wanted to import:
project
├── model1
| └── notebook1.ipynb
├── model2
| └── notebook2.ipynb
└── package
├── functions1.py
└── functions2.py
You can import both modules functions1 and functions2 from package like this:
from package import functions1, functions2
EDIT: As pointed out below, the local imports will no longer work if the cell containing the magic command is run more than once (the current working directory will be changed to the directory above at each rerun of the command). To prevent this from happening, the %cd .. command should be in its own cell (not in the same cell as the imports) at the top of the notebook and before the imports so it won't be run multiple times. Restarting the kernel and running all cells will reset the current working directory however will still return the desired imports/results.

Answer 4

I've solved this problem in the past by creating a symbolic link in the directory where the Jupyter notebook is located to the library it wants to load, so that python behaves as if the module is in the correct path. So for the example above, you would run the following command once per directory inside a Jupyter cell:
!ln -s /user/project/functions.py functions.py
and then you could import with
import functions
Note: I've only tried this on Linux and Mac Os, so I can't vouch for Windows.

Answer 5

I would suggest to install functions.py as a package in your virtual environment. There are some benefits of this:
You can access functions.py file from any iPython notebook located in any place, but at the given environment (kernel).
Once you changed any function in functions.py file, you don't need to reload your iPython notebook again and again. It will automatically reload every change.
This is the way how it can be done:
Create setup.py file (https://docs.python.org/2/distutils/setupscript.html) in your project folder
Activate your virtual environment, go to your project location, and use this command pip install -e .
Then, in your iPython notebook:
%load_ext autoreload
%autoreload 1
%aimport yourproject.functions
from functions import *
That's it!

Answer 6

In addition to the answer from adhg, I recommend using Pathlib, for compatibility between Linux/Windows/WSL paths formats:
Assuming the following folder structure:
.
├── work
| ├── notebook.ipynb
| └── my_python_file.py
├── py
| ├──modules
| | ├──__init__.py # empty
| | └──preparations.py
| ├──__init__.py # empty
| └── tools.py
├──.git
└── README.md
To load tools.py or preparations.py in my_python_file.py (or in notebook notebook.ipynb):
import sys
from pathlib import Path
# in jupyter (lab / notebook), based on notebook path
module_path = str(Path.cwd().parents[0] / "py")
# in standard python
module_path = str(Path.cwd(__file__).parents[0] / "py")
if module_path not in sys.path:
sys.path.append(module_path)
from modules import preparations
import tools
...

Answer 7

Found myself in the same exact situation as the OP, going to create several notebooks hence the wish to organise them in different subfolders
Tried this that seems to do what I need and seems cleaner to me
import os
os.chdir(os.path.dirname(os.path.dirname(os.getcwd())))
My function being two levels above so nested two os.path.dirname (with different folder structure could be only one or more)
Just implemented it and working fine, and btw I'm using JupyterLab started... two levels above where the function resides