I am using an Automation project with following structure
C:\Project_Name\Scripts
Under script folder, I have my python automation modules, libraries and other required data folders.
But I want this project not only restricted to C: drive. I should seamlessly able to execute by scripts from any drive, any parent directory of "Project_Name" folder.
I have tried the following. But it is always not true if I try to trigger my test from jenkins.
os.path.join(os.path.dirname(os.path.abspath(__file__)))
Think about this tree:
Project
|
|--Scripts
| |
| |--__init__.py
| |--use_text.py
| |--text.txt
|
|--run_me.py
and imagine we have to get text.txt path in use_text.py
Step 1. in run_me.py get the path of project directory, and save it as a environ using os. (example:)
import os
from pathlib import Path
path = Path(__file__).parent # replace it with your preferred library's version
os.environ['PROJECT_BASE_DIR'] = str(path)
from Scripts.use_text import use
use()
now in use_text.py
import os
text_path = 'Scripts/text.txt'
project_path = os.getenv('PROJECT_BASE_DIR')
text_real_path = os.path.join(project_path, text_path)
def use():
# do whatever you want with text_real_path
I got a way to get the real path of the file no matter where the project directory been placed.
import os
os.path.join((os.path.realpath(__file__)).split("<Base Directory Name>")[0], "<relative path till the file>")
this worked as well, with simple path.
Related
My project structure
mt-kart
|
--> src/data_kart
| |
| -->apis
| |
| -->__init__.py
| |
| -->main.py
--> tests
|
-->__init__.py
|
-->conftest.py
|
-->test_others.py
conftest.py has the following line.
from data_kart.main import fastapi_app
When run pytest from command line from project root(mt-kart). I get the following error
ModuleNotFoundError No module named 'data_kart'
How to make data_kart visible for conftest.py .
Option 1
Use relative imports:
from ..src.data_kart.main import fastapi_app
fastapi_app()
Please note, the above requires running conftest.py outside the project's directory, like this:
python -m mt-kart.tests.conftest
Option 2
Use the below in conftest.py (ref):
import sys
import os
# getting the name of the directory where the this file is present.
current = os.path.dirname(os.path.realpath(__file__))
# Getting the parent directory name where the current directory is present.
parent = os.path.dirname(current)
# adding the parent directory to the sys.path.
sys.path.append(parent)
# import the module
from src.data_kart.main import fastapi_app
# call the function
fastapi_app()
And if it happens for a module being outside the parent directory of tests, then you can get the parent of the parent direcory and add that one to the sys.path. For example:
parentparent = os.path.dirname(parent)
sys.path.append(parentparent)
Option 3
Same as Option 2, but shorter.
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from src.data_kart.main import fastapi_app
fastapi_app()
Option 4
Options 2 and 3 would work for smaller projects, where only a couple of files need to be updated with the code above. However, for larger projects, you may consider including the directory containing your package directory in PYTHONPATH, as described here (Option 3).
I am using Python 3.9.2 with sphinx-build 3.5.2. I have created a project with the following directory and file structure
core_utilities
|_core_utilities
| |_ __init__.py
| |_read_files.py
|_docs
|_ sphinx
|_Makefile
|_conf.pg
|_source
| |_conf.py
| |_index.rst
| |_read_files.rst
|_build
The read_files.py file contains the following code. NOTE: I simplified this example so it would not have distracting information. Within this code their is one class that contains one member function to read a text file, look for a key word and read the variable to the right of the keyword as a numpy.float32 variable. The stand-alone function written to read in a csv file with specific data types and save it to a pandas dataframe.
# Import packages here
import os
import sys
import numpy as np
import pandas as pd
from typing import List
class ReadTextFileKeywords:
def __init__(self, file_name: str):
self.file_name = file_name
if not os.path.isfile(file_name):
sys.exit('{}{}{}'.format('FATAL ERROR: ', file_name, ' does not exist'))
# ----------------------------------------------------------------------------
def read_double(self, key_words: str) -> np.float64:
values = self.read_sentence(key_words)
values = values.split()
return np.float64(values[0])
# ================================================================================
# ================================================================================
def read_csv_columns_by_headers(file_name: str, headers: List[str],
data_type: List[type],
if not os.path.isfile(file_name):
sys.exit('{}{}{}'.format('FATAL ERROR: ', file_name, ' does not exist'))
dat = dict(zip(headers, data_type))
df = pd.read_csv(file_name, usecols=headers, dtype=dat, skiprows=skip)
return df
The conf.py file as the following information in it.
# Configuration file for the Sphinx documentation builder.
#
# This file only contains a selection of the most common options. For a full
# list see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html
# -- Path setup --------------------------------------------------------------
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
import os
import sys
sys.path.insert(0, os.path.abspath('../../../core_utilities'))
# -- Project information -----------------------------------------------------
project = 'Core Utilities'
copyright = 'my copyright'
author = 'my name'
# The full version, including alpha/beta/rc tags
release = '0.1.0'
# -- General configuration ---------------------------------------------------
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = ['sphinx.ext.todo', 'sphinx.ext.viewcode', 'sphinx.ext.autodoc',
'sphinx.ext.autosummary', 'sphinx.ext.githubpages']
autodoc_member_order = 'groupwise'
autodoc_default_flags = ['members', 'show-inheritance']
autosummary_generate = True
autodock_mock_imports = ['numpy']
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = []
# -- Options for HTML output -------------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = 'nature'
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
The index.rst file has the following information.
. Core Utilities documentation master file, created by
sphinx-quickstart
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to Core Utilities's documentation!
==========================================
.. toctree::
:maxdepth: 2
:caption: Contents:
read_files
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
and the read_files.rst file has the following information
**********
read_files
**********
The ``read_files`` module contains methods and classes that allow a user to read different
types of files in different ways. Within the module a user will find functionality that
allows them to read text files and csv files by columns and keywords. This module
also contains functions to read sqlite databases, xml files, json files, yaml files,
and html files
.. autoclass:: test.ReadTextFileKeywords
:members:
.. autofunction:: test.read_csv_columns_by_headers
In addition, I am using a virtual environment that is active when I try to compile this with command line. I run the following command to compile html files with sphinx.
sphinx-build -b html source build
When I run the above command it fails with the following error.
WARNING: autodoc: failed to import class 'ReadTextFileKeywords' from module 'read_files'; the following exception was raised; No module named pandas.
If I delete the line from pandas import pd and then delete the function read_csv_columns_by_headers along with the call to the function in the read_files.rst file, everything compiles fine. It appears that for some reason sphinx is able to find numpy, but it for some reason does not seem to recognize pandas, although both exist in the virtual environment and were loaded with a pip3 install statement. Does anyone know why sphinx is able to find other modules, but not pandas?
Add pandas to autodoc
autodoc_mock_imports = ['numpy','pandas']
I have the following directory structure
code
|-----|--experiments
| |---v0
| |---agcnn.py
|-----|--pn
| |---bag
| |--run.py
run.py uses a module inside the agcnn.py. However when I try the import as following:
from experiments.v0.agccn import AG
it gives a relative import error saying no module. I'm using VScode and the the working dir is taken care by Vscode as root which is /code/ in this case.
How can I change the settings to accommodate the import ?
I have some users on my PC and I try to create a Python code to open %temp% folder, but the problem is that it works under my account only. When I use the same code on a different account it does not work on the same PC.
My folder path >> C:\Users\MyAccount\AppData\Local\Temp <<,
the problem error with this user 'MyAccount'
This is my code:
import webbrowser
webbrowser.open('C:\Users\MyAccount\AppData\Local\Temp')
I need to pass the correct userFolder to my code to work with.
Example:
my account the path >> **C:\Users\MyAccount\AppData\Local\Temp**
on different account >> C:\Users\ **?** \AppData\Local\Temp
**?** = it should be the name of the user.
Could you please advise me?
If pathlib is an option (comes with Python 3.4+) you can use
from pathlib import Path
Path.home() / 'AppData' / 'Local' / 'Temp'
if not, try
from os import path
path.expanduser('~/AppData/Local/Temp')
How do I load a python module, that is not built in. I'm trying to create a plugin system for a small project im working on. How do I load those "plugins" into python? And, instaed of calling "import module", use a string to reference the module.
Have a look at importlib
Option 1: Import an arbitrary file in an arbiatrary path
Assume there's a module at /path/to/my/custom/module.py containing the following contents:
# /path/to/my/custom/module.py
test_var = 'hello'
def test_func():
print(test_var)
We can import this module using the following code:
import importlib.machinery
myfile = '/path/to/my/custom/module.py'
sfl = importlib.machinery.SourceFileLoader('mymod', myfile)
mymod = sfl.load_module()
The module is imported and assigned to the variable mymod. We can then access the module's contents as:
mymod.test_var
# prints 'hello' to the console
mymod.test_func()
# also prints 'hello' to the console
Option 2: Import a module from a package
Use importlib.import_module
For example, if you want to import settings from a settings.py file in your application root folder, you could use
_settings = importlib.import_module('settings')
The popular task queue package Celery uses this a lot, rather than giving you code examples here, please check out their git repository