including folder and files inside a Python package - python-3.x

I'm trying to make a python package and I have most of the things already setup by when I try to install the library from Github here, it installs everything except for the folder called champs and it's files
This is my File directory structure
LeagueYue
champs
-Lname_num.json
-Lname_Uname.json
-num_Uname.json
-__init__.py
-champion_files.py
-external.py
-match.py
-rank.py
-status.py
-summoner.py
-requirements.txt
-setup.py
All the files are installed except for the folder and the files inside champs

As this question answers:
There are 2 ways to add the static files:
1) Include_package_data=True + MANIFEST.in
A MANIFEST.in file in the same directory of setup.py, that looks like this:
include src/static/*
include src/Potato/*.txt
2) Package_data in setup.py
package_data = {
'static': ['*'],
'Potato': ['*.txt']
}
Specify the files inside the setup.py.

Two of the files could probably be derived at runtime from num_Uname.json, but that's fine.
I do not yet see a data_files directive in https://github.com/CharmingMother/LeagueLib/blob/async/setup.py
Thomas Cokelaer suggests using an expression like
datafiles = [(datadir, list(glob.glob(os.path.join(datadir, '*'))))]
and then
setup(
...
    data_files = datafiles,
)
in http://thomas-cokelaer.info/blog/2012/03/how-to-embedded-data-files-in-python-using-setuptools/
In your case this could be as simple as:
data_files = [('', ['champs/num_Uname.json'])],
Martin Thoma explains you should access them using filepath = pkg_resources.resource_filename(__name__, path)
in How to read a (static) file from inside a Python package?
When I Read The Fine Manual, this setup.cfg alternative surfaces:
[options.data_files]
...
data = data/img/logo.png, data/svg/icon.svg
suggesting a line like . = champs/num_Uname.json or champs = num_Uname.json

Related

Adding non-python files to colcon build

Building my workspace with colcon, some OSM files which are found in a directory "OSM" in a sub_package in the workspace are not found in built space. So when I go to the install space, the files are not there. I am not sure how to do this and if I should put it in the setup.py.
I tried putting this in the setup.py file in arguments of setup():
setup(
name=package_name,
version='0.0.0',
packages=[package_name, submodules, osm],
data_files=[
('share/ament_index/resource_index/packages',
['resource/' + package_name]),
('share/' + package_name, ['package.xml']),
(os.path.join('share', package_name), glob('launch/*.launch.py')),
('.package_name/sub_package', glob('OSM_folder/*.osm')),
],
.
.
.
) # close setup()
but it did not work.
I am using ROS2 Galactic.
Directory structure:
package_name
┃
┣━━━━setup.py
┣━━━━package.xml
┣━━━━resource/
┣━━━━launch/
┗━━━━package_name
┗━sub_package_name
┗━OSM
┣━__init__.py
┗━some_osm_files.osm
I have the OSM directory in the built workspace but it has only the init.py file
I solved it. I am not sure if this is the right way of doing this or is there like another better/proper way or not, but here we go.
In the setup.py file, I added the line
(os.path.join('lib/python3.8/site-packages/package_name/sub_package/OSM'),glob(package_name+'/sub_package_name/OSM/*.osm')), in the data_files variable.
The first part of the new line which is os.path.join('lib/python3.8/site-packages/package_name/sub_package_name/OSM') determines the new location of the files in the install folder after building the workspace.
The second part which is glob(package_name+'/sub_package_name/OSM/*.osm') determines the files original location in the project workspace.
So the result is that it takes the files from the location mentioned in the second part and puts them in the location mentioned in the first part.
The resulting block is:
setup(
name=package_name,
version='0.0.0',
packages=[package_name, submodules, osm],
data_files=[
('share/ament_index/resource_index/packages',
['resource/' + package_name]),
('share/' + package_name, ['package.xml']),
(os.path.join('share', package_name), glob('launch/*.launch.py')),
(os.path.join('lib/python3.8/site-packages/package_name/sub_package_name/OSM'), glob(package_name+'/sub_package_name/OSM/*.osm')),
],
.
.
.
)

define path to SConstruct root

My folder structure is:
\gitclone\SConstruct
\gitclone\level1\level2\SConscript.3dn
Now in SConscript.3dn I am creating empty folder: e.Execute(Mkdir('#/testDir'))
How to define the file path to create the folder in \gitclone\ (where root SConstruct file is)? I was reading manual, but somehow I can not manage it.
I have found solution here, so my code is:
e['SCONS_ROOT'] = Dir('#')
e.Execute(Mkdir('${SCONS_ROOT.abspath}/win_b64/code/bin/testDir'))

how to force SCons to build with relative paths?

I have the following project structure:
/project
/src_dirs
/src_dir_1
/...
/include_dirs
/inc_dir_1
/...
/output
SConstruct
/Sconscripts
lib1_sconscript
lib2_sconscript
/...
objects/
/...
libs/
/...
The build process invoked from the output directory. so all the paths in the sconscripts, are relative to output directory. my sconscript files are auto generated. as you can see below, the paths to source files and to include files are relative paths. this is a demo sconscript file:
Import('libaEnv')
includes = ['../../project/include_dirs/inc_dir_1/be/cmn/inc', '../../project/include_dirs/inc_dir_1/be/be/cmn/api_inc']
directives = ['-CC','-c','-g']
code_switches = ['FAST_RUNG_MLK', 'EN_PRIORITY']
listDefines = code_switches
linkLibs = []
crt = libaEnv.Object(target='objects/crt.o', source='../../project/src_dirs/src_dir_1/crt.c', CPPDEFINES=listDefines, CPPFLAGS=directives, CPPPATH=includes)
ert = libaEnv.Object(target='objects/ert.o', source='../../project/src_dirs/src_dir_1/ert.c', CPPDEFINES=listDefines, CPPFLAGS=directives, CPPPATH=includes)
urt = libaEnv.Object(target='objects/urt.o', source='../../project/src_dirs/src_dir_1/urt.c', CPPDEFINES=listDefines, CPPFLAGS=directives, CPPPATH=includes)
liba = libaEnv.Library (target='libs/liba.a', source=[crt,ert,urt])
Return('liba')
I have seen that scons invokes the compiler with absolute paths to the source and the headers files. i have seen this by running scons with -verbose (i have also validate this by printing the command line in Action.py in the spawn method).
My scons version is: SCons 2.5.1 and i am running with python 2.7.
How can i force scons to invoke the compiler with relative paths only ?
One approach here is you can use top-relative addressing, as in:
includes = ['#project/include_dirs/inc_dir_1/be/cmn/inc', '#project/include_dirs/inc_dir_1/be/be/cmn/api_inc']
This still gets you absolute paths, but they'll be computed based on a valid starting point (# = "the directory containing the top-level SConsctruct", thus top-relative)

scons fails to resolve relative source path

I have the following project structure:
/prj
SConstruct
/src
/app
/lib1
/lib2
/...
'/prj/src/lib1' structure:
/lib1
/src
/test
SConscript
'lib1/SConscript':
SConscript('test/test1/SConscript',
exports = 'env',
variant_dir = '#build/release/lib1/test',
duplicate = 0)
and, finally, 'test' directory:
/test
/common
helpers.cpp
/test1
SConscript
main.cpp
In 'test/test1/SConscript' sources specified as:
Sources = ['../common/helpers.cpp', 'main.cpp']
Result:
scons: *** [build/release/lib1/common/helpers.o]
Source `build/release/lib1/common/helpers.cpp' not found,
needed by target `build/release/lib1/common/helpers.o'
As can be seen the problem is that scons tries to find out the source file 'helpers.cpp' in build directory, not in source one.
Some research shows that the problem raised when source file path begins with '../'. When all sources defined underneath 'SConscript' file all is Ok.
Scons v2.5.1 and v3.0.1 demonstrate the same behavior.
What I did wrong? I've found this answer where the author advised:
You could use ../test.cpp as filename
but I doing exactly the same. Is such scons behavior intended or this is a bug?
Your "code" in "lib1/SConscript":
SConscript('test/test1/SConscript',
exports = 'env',
variant_dir = '#build/release/lib1/test',
duplicate = 0)
uses the name of the given "test/test1/SConscript" and implicitly links the folder "test/test1" as "variant folder" (=variant_dir) to the target directory "#build/release/lib1/test". So if a required file can't be found in "#build/release/lib1/test", SCons tries to do an alternative "lookup" in "test/test1/". But this link is not automatically set for the "common" folder as well, and that's why the lookup of the "helpers.cpp" fails. This is the intended behaviour and correct in SCons.
A solution for your current problem would be to move the "test/test1/SConscript" one folder level higher, include the sources there as "common/helpers.cpp" and "test/main.cpp" and call this new SConscript from "lib1/SConscript" as:
SConscript('test/SConscript',
exports = 'env',
variant_dir = '#build/release/lib1/test',
duplicate = 0)

1 liner load config package in Python 3

I desesperatly searching for 1 liner load config package in Python 3...
In python 2, there is :
https://pypi.python.org/pypi/config/0.3.7
from config import Config; cfg = Config(file('D://config.py', 'r'))
There is
https://pypi.python.org/pypi/loadconfig
but it does not seem loading file...
Main goal is to install this package in all the python envs and load
one big config file for all envs.
Use case:
You have a single piece of python, you want to run on several hardware/OS and python envs (conda,...) without changing the code...
Method 1:
1) Define a generic config files for all Envs needed.
2) At RunTime, determine the environnment (linux, username, ) and
define the absolute path root repository directory with config file.
3) Single piece of code, running in different envs without changing the code, the envs...
From similar functionnalities than config
https://pypi.python.org/pypi/config/0.3.7
Security is not guaranted
config.py should not contains harmful python code.
from attrdict import AttrDict; f= open('USER_ROOT/config.py'); cfg = AttrDict(eval(f.read())); f.close()
cfg.key1
cfg.mykey2
cfg.mykey4

Resources